20ce979 implemented the function of automatically converting to sha256 when the key is larger than 511 bytes. But it also makes it impossible to use lmdb.prefix to retrieve these normalized keys.
In order to balance the prefix function and the normalization function while meeting the 511-byte limit, I suggest adding an option to control whether to retain part of the prefix.
Specific implementation:
If Preserve Prefix is turned off, keep it consistent with the status quo to ensure backward compatibility.
If Preserve Prefix is turned on:
- MAX_KEY_SIZE is reduced to 511-64=447. Of course, another constant should be used in the specific implementation.
- Calculate the sha256 of the complete KEY.
- Extract the 477-byte prefix of the current KEY.
- Concatenate the intercepted prefix and sha256 into a new key, as the normalized key.
Currently, I have implemented a layer of encapsulation when calling this library. If necessary, I am willing to submit a pull request to implement this function.
20ce979 implemented the function of automatically converting to sha256 when the key is larger than 511 bytes. But it also makes it impossible to use lmdb.prefix to retrieve these normalized keys.
In order to balance the prefix function and the normalization function while meeting the 511-byte limit, I suggest adding an option to control whether to retain part of the prefix.
Specific implementation:
If Preserve Prefix is turned off, keep it consistent with the status quo to ensure backward compatibility.
If Preserve Prefix is turned on:
Currently, I have implemented a layer of encapsulation when calling this library. If necessary, I am willing to submit a pull request to implement this function.