Skip to content

index-tool.py inconsistent handling of 'Too many collections, maximum allowed 100000' errors #191

Description

@dbonser

index-tool.py does not handle 'Too many collections, maximum allowed 100000' errors consistently.

Here is the metadata file contents:

{
    "indexes":[
        {
            "v":2,
            "key":{"code":1},
            "name":"code_insensitive",
            "background":true,
            "collation":{
                "locale":"en_US",
                "caseLevel":false,
                "caseFirst":"off",
                "strength":1,
                "numericOrdering":false,
                "alternate":"non-ignorable",
                "maxVariable":"punct",
                "normalization":false,
                "backwards":false,
                "version":"57.1"
            },
            "ns":"db.collection"
        },
        {
            "v":2,
            "key":{"_id":1},
            "name":"_id_",
            "ns":"db.collection"
        }
    ]
}

When running without --skip-incompatible, the script fails as soon as this error is encountered:

$ python3 documentdb_index_tool.py --restore-indexes --dir dump --uri <uri>
2026-06-29 15:54:15,792: Connecting to instance using provided URI
2026-06-29 15:54:15,793: You appear to be connected to a DocumentDB cluster. For more information regarding feature compatibility and support please visit https://www.mongodb.com/supportability/documentdb
2026-06-29 15:54:15,817:   .. successfully connected
Traceback (most recent call last):
  File "/home/ec2-user/environment/amazon-documentdb-tools/index-tool/migrationtools/documentdb_index_tool.py", line 674, in <module>
    main()
  File "/home/ec2-user/environment/amazon-documentdb-tools/index-tool/migrationtools/documentdb_index_tool.py", line 670, in main
    indextool.run()
  File "/home/ec2-user/environment/amazon-documentdb-tools/index-tool/migrationtools/documentdb_index_tool.py", line 600, in run
    self._restore_indexes(connection, metadata_to_restore)
  File "/home/ec2-user/environment/amazon-documentdb-tools/index-tool/migrationtools/documentdb_index_tool.py", line 538, in _restore_indexes
    collection.create_index(keys_to_create,**index_options)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/pymongo/synchronous/collection.py", line 2358, in create_index
    return (self._create_indexes([index], session, **cmd_options))[0]
  File "/home/ec2-user/.local/lib/python3.9/site-packages/pymongo/_csot.py", line 119, in csot_wrapper
    return func(self, *args, **kwargs)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/pymongo/synchronous/collection.py", line 2232, in _create_indexes
    self._command(
  File "/home/ec2-user/.local/lib/python3.9/site-packages/pymongo/synchronous/collection.py", line 621, in _command
    return conn.command(
  File "/home/ec2-user/.local/lib/python3.9/site-packages/pymongo/synchronous/helpers.py", line 45, in inner
    return func(*args, **kwargs)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/pymongo/synchronous/pool.py", line 538, in command
    return command(
  File "/home/ec2-user/.local/lib/python3.9/site-packages/pymongo/synchronous/network.py", line 218, in command
    helpers_shared._check_command_response(
  File "/home/ec2-user/.local/lib/python3.9/site-packages/pymongo/helpers_shared.py", line 247, in _check_command_response
    raise OperationFailure(errmsg, code, response, max_wire_version)
pymongo.errors.OperationFailure: Too many collections, maximum allowed 100000, full error: {'ok': 0.0, 'operationTime': Timestamp(1782748455, 1), 'code': 100005, 'errmsg': 'Too many collections, maximum allowed 100000'}

When running with --skip-incompatible, the 'Too many collections, maximum allowed 100000' error is effectively ignored as it is being treated as a compatibilty-related error:

$ python3 documentdb_index_tool.py --restore-indexes --skip-incompatible --dir dump --uri <uri>
2026-06-29 16:42:34,133: Connecting to instance using provided URI
2026-06-29 16:42:34,133: You appear to be connected to a DocumentDB cluster. For more information regarding feature compatibility and support please visit https://www.mongodb.com/supportability/documentdb
2026-06-29 16:42:34,161:   .. successfully connected
2026-06-29 16:42:34,201: database111.collection: skipping index code_insensitive because create_index failed: Too many collections, maximum allowed 100000, full error: {'ok': 0.0, 'operationTime': Timestamp(1782751354, 1), 'code': 100005, 'errmsg': 'Too many collections, maximum allowed 100000'}
2026-06-29 16:42:34,239: database111.collection: skipping index _id_ because create_index failed: Too many collections, maximum allowed 100000, full error: {'ok': 0.0, 'operationTime': Timestamp(1782751354, 1), 'code': 100005, 'errmsg': 'Too many collections, maximum allowed 100000'}
2026-06-29 16:42:34,278: database111.collection2: skipping index _id_ because create_index failed: Too many collections, maximum allowed 100000, full error: {'ok': 0.0, 'operationTime': Timestamp(1782751354, 1), 'code': 100005, 'errmsg': 'Too many collections, maximum allowed 100000'}

Executing the script with --skip-incompatible should stop execution when the 'Too many collections, maximum allowed 100000' error is encountered.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions