Skip to content

Fix KeyError in insert_all when pk column is missing from a single record#745

Open
ATOM00blue wants to merge 1 commit into
simonw:mainfrom
ATOM00blue:fix-insert-all-pk-not-in-records
Open

Fix KeyError in insert_all when pk column is missing from a single record#745
ATOM00blue wants to merge 1 commit into
simonw:mainfrom
ATOM00blue:fix-insert-all-pk-not-in-records

Conversation

@ATOM00blue
Copy link
Copy Markdown

@ATOM00blue ATOM00blue commented May 22, 2026

Fixes #732

The bug

insert_all() with a pk= column that isn't present in the records raised a KeyError — but only when inserting exactly one row. Every other row count succeeded:

import sqlite_utils

for n in [0, 1, 2, 3, 10]:
    db = sqlite_utils.Database(memory=True)
    db.conn.execute("CREATE TABLE t (a TEXT, b INT, PRIMARY KEY (a, b))")
    rows = [{"a": f"x{i}", "b": i} for i in range(n)]
    try:
        db.table("t").insert_all(rows, pk="not_a_column", alter=True)
        print(f"n={n}: ok")
    except KeyError as e:
        print(f"n={n}: KeyError {e}")
n=0: ok
n=1: KeyError 'not_a_column'
n=2: ok
n=3: ok
n=10: ok

Root cause

The single-record branch that populates last_pk reads the pk value straight out of the just-inserted row (row[pk]). When the named pk column doesn't actually exist on the table, that lookup throws. The multi-record path doesn't go through this branch, hence the inconsistency.

Fix

Check that the named pk column(s) are present on the row before reading them, and fall back to the rowid otherwise — the same fallback already used when no pk is given. Single-row inserts now behave like every other row count.

Testing

Added test_insert_all_pk_not_in_records, parametrized over 0/1/2/3/10 rows, which fails on main (KeyError for the single-row case) and passes with this change. Full test suite, black, flake8 and mypy all pass locally.


📚 Documentation preview 📚: https://sqlite-utils--745.org.readthedocs.build/en/745/

…cord

When a pk= column was named that isn't present in the data (and so isn't
created on the table), inserting exactly one record raised a KeyError while
inserting any other number of records did not. The single-record branch that
populates last_pk read the column straight out of the inserted row.

Check that the named pk column(s) actually exist on the row first and fall
back to the rowid otherwise, matching the behaviour for multiple records.

Closes simonw#732
Copilot AI review requested due to automatic review settings May 22, 2026 01:14
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR fixes inconsistent insert_all() behavior when a pk= column is specified but does not exist in the table/records, aligning single-row inserts with multi-row behavior (no KeyError; fall back to rowid).

Changes:

  • Add regression test covering 0..N inserts when pk column is missing.
  • Update insert_all() logic for computing last_pk to tolerate missing PK columns and fall back to last_rowid.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
tests/test_create.py Adds parametrized regression coverage for missing pk column behavior across row counts
sqlite_utils/db.py Adjusts last_pk derivation to handle missing PK columns and fall back to rowid

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread sqlite_utils/db.py
if hash_id
else ([pk] if isinstance(pk, str) else list(pk))
)
if all(col in row for col in pk_cols):
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Creating tables with missing primary keys works inconsistently

2 participants