Fatal error in dbm.gdbm

[serhiy-storchaka]

BPO	22035
Nosy	@vstinner, @serhiy-storchaka
Files	dbm_gdbm_fatal_error.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2014-07-22.10:20:40.004>
labels = ['extension-modules', 'type-crash']
title = 'Fatal error in dbm.gdbm'
updated_at = <Date 2019-03-15.22:15:58.161>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2019-03-15.22:15:58.161>
actor = 'BreamoreBoy'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Extension Modules']
creation = <Date 2014-07-22.10:20:40.004>
creator = 'serhiy.storchaka'
dependencies = []
files = ['36031']
hgrepos = []
issue_num = 22035
keywords = ['patch', 'needs review']
message_count = 7.0
messages = ['223658', '236005', '236006', '239688', '239689', '239773', '239778']
nosy_count = 2.0
nosy_names = ['vstinner', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue22035'
versions = ['Python 2.7', 'Python 3.4', 'Python 3.5']

Linked PRs

• gh-66234: Add flag to disable the use of mmap in dbm.gnu #135005
• Revert "gh-66234: Add flag to disable the use of mmap in dbm.gnu (GH-135005)" #136989

[serhiy-storchaka]

It is possible to crash Python by breaking opened gdbm database.

>>> import _gdbm as dbm
>>> db = dbm.open('x.db', 'n')
>>> open('x.db', 'wb').close()
>>> db[b'a'] = b'b'
gdbm fatal: read error

Proposed patch tries to convert fatal gdbm into regular exception or in Python fatal error (which at least produces traceback).

>>> import _gdbm as dbm
>>> db = dbm.open('x.db', 'n')
>>> open('x.db', 'wb').close()
>>> db[b'a'] = b'b'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
_gdbm.error: gdbm fatal: read error

[BreamoreBoy]

Would somebody please review Serhiy's patch.

[serhiy-storchaka]

Oh, Mark, please stop shaking up bug tracker.

[vstinner]

I would prefer to avoid setgmp/longjmp, it's kind of a hack. It's maybe more a design issue in the gdbm library to report errors.

I proposed a generic signal handler using setjmp/longjmp to convert SIGSEGV to regular Python exceptions, but it was rejected: issue bpo-3999.

[vstinner]

Oh, Mark, please stop shaking up bug tracker.

I agree: please stop posting useless messages, and review patches instead.

[serhiy-storchaka]

The patch for bpo-3999 was rejected because Python internal state may be corrupted when the SIGSEGV signal is raised. This is not the case of this issue. gdbm fatal function is called when Python is in consistent state. So we free to use any Python C-API. But internal state of concrete GDBM_FILE may be corrupted, so we shouldn't use it after handling fatal error. This cause a leak, but I think that a leak with an exception is better than just a crash.

May be different type of exception should be raised. May be we need FatalError that inherits from BaseException.

Other external libraries used by the stdlib also can crash, and perhaps crashes can be converted to exceptions. This issue is only first in the series.

[vstinner]

2015-04-01 10:39 GMT+02:00 Serhiy Storchaka <report@bugs.python.org>:

Other external libraries used by the stdlib also can crash, and perhaps crashes can be converted to exceptions. This issue is only first in the series.

I don't think that it's a good practice to try to workaround bugs. IMO
it's better to modify libraries directly to allow users of the library
to handle correctly errors.

[furkanonder]

The issue still exists; I can reproduce it on Python 3.15

$ ./python
Python 3.15.0a0 (heads/main:91b48868a8, May 25 2025, 18:45:04) [GCC 15.1.1 20250425] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import _gdbm as dbm
>>> db = dbm.open('x.db', 'n')
>>> open('x.db', 'wb').close()
>>> db[b'a'] = b'b'
[1]    163083 bus error (core dumped)  ./python

[serhiy-storchaka]

The fatal_func parameter is deprecated now, so this way should no longer be used. It is expected that new gdbm versions better handle errors and do not need such function.

The tests, added in dbm_gdbm_fatal_error.patch, fail with unpatched code because it raises different exceptions or do not raise exceptions. But they do not crash. The original example still crashes. The difference is that it uses empty database. Changing the tests to use empty database makes them crashing.

We can only hope that crashes will be fixed in upstream.

[serhiy-storchaka]

I have found that using the GDBM_NOMMAP flag prevents these crashes. But it can impact performance and/or memory consumption. I think we cannot expect more.

We can add support for that flag.