Skip to content

fix bugs#145

Merged
ifduyue merged 14 commits intomasterfrom
duyue/fix-bugs
Apr 26, 2026
Merged

fix bugs#145
ifduyue merged 14 commits intomasterfrom
duyue/fix-bugs

Conversation

@ifduyue
Copy link
Copy Markdown
Owner

@ifduyue ifduyue commented Apr 26, 2026

No description provided.

ifduyue added 10 commits April 27, 2026 02:05
When XXH32_createState() returns NULL, the self object allocated by
PyObject_New was leaked. Free it with PyObject_Del before returning
NULL.
- Fix memory leak when XXH3_createState() fails (same pattern as XXH32/XXH64)
- Explicitly initialize self->seed = 0 for consistency with PYXXH3_128_new
  and to not rely on zero-initialized memory from PyObject_New
XXH3_128 produces a 128-bit digest, which is 16 bytes, not 8.
PyModuleDef.m_slots is PyModuleDef_Slot * (non-const), so declaring
slots as const causes a -Wincompatible-pointer-types-discards-qualifiers
warning. Remove the const qualifier.
@codspeed-hq
Copy link
Copy Markdown
Contributor

codspeed-hq Bot commented Apr 26, 2026

Merging this PR will not alter performance

✅ 96 untouched benchmarks


Comparing duyue/fix-bugs (45f7e73) with master (2bcd375)

Open in CodSpeed

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements memory cleanup using PyObject_Del during xxhash state creation failures, corrects the PYXXH3_128 digest documentation, and modifies module slot definitions. The reviewer suggests calling PyErr_NoMemory() when state creation fails to ensure the Python C API correctly signals an out-of-memory error instead of a SystemError.

Comment thread src/_xxhash.c Outdated
}

if ((self->xxhash_state = XXH32_createState()) == NULL) {
PyObject_Del(self);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When XXH32_createState() fails, it returns NULL without setting a Python exception. In Python's C API, returning NULL from a function without an exception set triggers a SystemError. You should call PyErr_NoMemory() to properly signal the out-of-memory condition.

        PyObject_Del(self);
        PyErr_NoMemory();

Comment thread src/_xxhash.c Outdated
}

if ((p->xxhash_state = XXH32_createState()) == NULL) {
PyObject_Del(p);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When XXH32_createState() fails, it returns NULL without setting a Python exception. In Python's C API, returning NULL from a function without an exception set triggers a SystemError. You should call PyErr_NoMemory() to properly signal the out-of-memory condition.

        PyObject_Del(p);
        PyErr_NoMemory();

Comment thread src/_xxhash.c Outdated
}

if ((self->xxhash_state = XXH64_createState()) == NULL) {
PyObject_Del(self);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When XXH64_createState() fails, it returns NULL without setting a Python exception. In Python's C API, returning NULL from a function without an exception set triggers a SystemError. You should call PyErr_NoMemory() to properly signal the out-of-memory condition.

        PyObject_Del(self);
        PyErr_NoMemory();

Comment thread src/_xxhash.c Outdated
}

if ((p->xxhash_state = XXH64_createState()) == NULL) {
PyObject_Del(p);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When XXH64_createState() fails, it returns NULL without setting a Python exception. In Python's C API, returning NULL from a function without an exception set triggers a SystemError. You should call PyErr_NoMemory() to properly signal the out-of-memory condition.

        PyObject_Del(p);
        PyErr_NoMemory();

Comment thread src/_xxhash.c Outdated
}

if ((self->xxhash_state = XXH3_createState()) == NULL) {
PyObject_Del(self);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When XXH3_createState() fails, it returns NULL without setting a Python exception. In Python's C API, returning NULL from a function without an exception set triggers a SystemError. You should call PyErr_NoMemory() to properly signal the out-of-memory condition.

        PyObject_Del(self);
        PyErr_NoMemory();

Comment thread src/_xxhash.c Outdated
}

if ((p->xxhash_state = XXH3_createState()) == NULL) {
PyObject_Del(p);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When XXH3_createState() fails, it returns NULL without setting a Python exception. In Python's C API, returning NULL from a function without an exception set triggers a SystemError. You should call PyErr_NoMemory() to properly signal the out-of-memory condition.

        PyObject_Del(p);
        PyErr_NoMemory();

Comment thread src/_xxhash.c Outdated
}

if ((self->xxhash_state = XXH3_createState()) == NULL) {
PyObject_Del(self);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When XXH3_createState() fails, it returns NULL without setting a Python exception. In Python's C API, returning NULL from a function without an exception set triggers a SystemError. You should call PyErr_NoMemory() to properly signal the out-of-memory condition.

        PyObject_Del(self);
        PyErr_NoMemory();

Comment thread src/_xxhash.c Outdated
}

if ((p->xxhash_state = XXH3_createState()) == NULL) {
PyObject_Del(p);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When XXH3_createState() fails, it returns NULL without setting a Python exception. In Python's C API, returning NULL from a function without an exception set triggers a SystemError. You should call PyErr_NoMemory() to properly signal the out-of-memory condition.

        PyObject_Del(p);
        PyErr_NoMemory();

Returning NULL from a Python C API function without setting an exception
causes CPython to raise a SystemError. When XXH*_createState() returns
NULL (out of memory), we now call PyErr_NoMemory() before returning NULL
in all 8 _new and _copy functions.
@ifduyue
Copy link
Copy Markdown
Owner Author

ifduyue commented Apr 26, 2026

/gemini review

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request improves error handling in src/_xxhash.c by adding memory cleanup and reporting when state creation fails. It also corrects the documentation for the 128-bit digest size, ensures the seed is initialized in PYXXH3_64_new, and modifies the module slots definition. The review feedback highlights a consistent improvement opportunity: using Py_DECREF instead of PyObject_Del in failure paths to ensure that Python's reference counting and object deallocation are handled correctly.

Comment thread src/_xxhash.c Outdated
Comment on lines 412 to 414
PyObject_Del(self);
PyErr_NoMemory();
return NULL;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using PyObject_Del directly on a newly created object bypasses the standard reference counting mechanism. While it frees the memory, it doesn't handle the type reference count correctly if the type is a heap type or if Python's internal logic expects a full deallocation cycle. It is better to use Py_DECREF(self), which will trigger the object's tp_dealloc (PYXXH32_dealloc). Since self->xxhash_state is set to NULL on failure, the destructor is safe to call.

        Py_DECREF(self);
        return PyErr_NoMemory();

Comment thread src/_xxhash.c Outdated
Comment on lines 536 to 538
PyObject_Del(p);
PyErr_NoMemory();
return NULL;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Similar to the new method, use Py_DECREF(p) instead of PyObject_Del(p) to ensure proper cleanup and adherence to Python's reference counting protocols.

        Py_DECREF(p);
        return PyErr_NoMemory();

Comment thread src/_xxhash.c Outdated
Comment on lines 715 to 717
PyObject_Del(self);
PyErr_NoMemory();
return NULL;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Use Py_DECREF(self) to correctly handle the destruction of the object and maintain consistent memory management practices.

        Py_DECREF(self);
        return PyErr_NoMemory();

Comment thread src/_xxhash.c Outdated
Comment on lines 839 to 841
PyObject_Del(p);
PyErr_NoMemory();
return NULL;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Use Py_DECREF(p) for proper cleanup on allocation failure.

        Py_DECREF(p);
        return PyErr_NoMemory();

Comment thread src/_xxhash.c Outdated
Comment on lines 1017 to 1019
PyObject_Del(self);
PyErr_NoMemory();
return NULL;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Use Py_DECREF(self) to ensure the object is properly cleaned up via its deallocator.

        Py_DECREF(self);
        return PyErr_NoMemory();

Comment thread src/_xxhash.c Outdated
Comment on lines 1145 to 1147
PyObject_Del(p);
PyErr_NoMemory();
return NULL;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Use Py_DECREF(p) instead of PyObject_Del(p) for consistent reference counting.

        Py_DECREF(p);
        return PyErr_NoMemory();

Comment thread src/_xxhash.c Outdated
Comment on lines 1331 to 1333
PyObject_Del(self);
PyErr_NoMemory();
return NULL;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Use Py_DECREF(self) to correctly handle the object's lifecycle on failure.

        Py_DECREF(self);
        return PyErr_NoMemory();

Comment thread src/_xxhash.c Outdated
Comment on lines 1472 to 1474
PyObject_Del(p);
PyErr_NoMemory();
return NULL;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Use Py_DECREF(p) for proper cleanup.

        Py_DECREF(p);
        return PyErr_NoMemory();

ifduyue added 2 commits April 27, 2026 02:22
Py_DECREF is the idiomatic CPython pattern for cleaning up a newly
allocated object in tp_new/tp_copy error paths — it calls tp_dealloc
which properly integrates with Python's refcounting and GC.

To make this safe, all four dealloc functions now NULL-check
self->xxhash_state before calling XXH*_freeState(), so they handle
partially-initialized objects where state creation failed.
@ifduyue
Copy link
Copy Markdown
Owner Author

ifduyue commented Apr 26, 2026

/gemini review

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request improves memory management and error handling in the C extension by adding Py_DECREF and PyErr_NoMemory calls during state creation failures, along with null checks in deallocators. It also corrects the xxh3_128 digest documentation and increments the package version. Review feedback highlights the need for consistent seed initialization and state resets in the tp_new handlers for xxh32 and xxh64 to ensure objects are in a valid state even if init is bypassed.

Comment thread src/_xxhash.c
Comment on lines +413 to +414
Py_DECREF(self);
PyErr_NoMemory();
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

While fixing the memory leak and setting the error indicator is correct, the xxh32 object is still left in an inconsistent state if __init__ is not called (e.g., when subclassing without calling super().__init__ or using __new__ directly). For consistency with the xxh3_64 and xxh3_128 implementations in this file, you should also initialize self->seed = 0 and call XXH32_reset(self->xxhash_state, 0) after the state is successfully created.

Comment thread src/_xxhash.c
Comment on lines +717 to +718
Py_DECREF(self);
PyErr_NoMemory();
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Similar to the xxh32 implementation, the xxh64 object should have its seed initialized to 0 and its state reset using XXH64_reset(self->xxhash_state, 0) in the tp_new handler. This ensures the object is in a valid state even if __init__ is bypassed, which is important for methods like copy() or update() that might be called on an uninitialized object.

For consistency with XXH3_64 and XXH3_128 implementations, tp_new now
initializes self->seed = 0 and resets the internal state. This ensures
the object is in a valid state even if __init__ is bypassed (e.g. via
__new__ directly or subclassing without calling super().__init__).
@ifduyue
Copy link
Copy Markdown
Owner Author

ifduyue commented Apr 26, 2026

/gemini review

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request enhances the C extension's stability by implementing proper reference counting and error handling during state allocation failures. Key changes include adding null checks in deallocators to prevent potential crashes, updating the XXH3_128 docstring to correctly specify a 16-byte digest, and adjusting the module slots' type to avoid compiler warnings. The version has also been updated to 3.8.0.dev3. I have no feedback to provide.

@ifduyue ifduyue merged commit ce1cb9f into master Apr 26, 2026
43 checks passed
@ifduyue ifduyue deleted the duyue/fix-bugs branch April 26, 2026 18:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant