Pydantic.PrivateAttr `default` and `default_factory` are ignored by SQLModel

[JLHasson]

First Check

• I added a very descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn't find it.
• I searched the SQLModel documentation, with the integrated search.
• I already searched in Google "How to X in SQLModel" and didn't find any information.
• I already read and followed all the tutorial in the docs and didn't find an answer.
• I already checked if it is not related to SQLModel but to Pydantic.
• I already checked if it is not related to SQLModel but to SQLAlchemy.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from typing import Optional
from pydantic import BaseModel, PrivateAttr

from sqlmodel import Field, Session, SQLModel, create_engine, select


class Hero(SQLModel, table=True):
    id: Optional[int] = Field(default=None, primary_key=True)
    _name: str = PrivateAttr(default=None)  # This field is not committed to the db
    secret_name: str


class HeroPydantic(BaseModel):
    _name: str = PrivateAttr(default=None)
    secret_name: str


sqlite_file_name = "database.db"
sqlite_url = f"sqlite:///{sqlite_file_name}"

engine = create_engine(sqlite_url, echo=True)

if __name__ == "__main__":
    SQLModel.metadata.create_all(engine)

    hero_1 = Hero(secret_name="Dive Wilson")
    print("Hero:", hero_1)
    print(hero_1._name)
    print(hero_1._name is None, type(hero_1._name))

    hero_2 = HeroPydantic(secret_name="Lance")
    print(hero_2)
    print(hero_2._name)
    print(hero_2._name is None, type(hero_2._name))

    with Session(engine) as session:
        session.add(hero_1)
        session.commit()
        statement = select(Hero)
        results = session.exec(statement)
        for hero in results:
            print(hero)
            print(hero_2._name)
            print(hero_2._name is None, type(hero_2._name))

Description

As far as I can tell SQLModel is ignoring the default and default_factory parameters of pydantic.PrivateAttr. The example I've given above reproduces on my system. The output can be seen here:

Hero: id=None secret_name='Dive Wilson'

False <class 'pydantic.fields.ModelPrivateAttr'>
secret_name='Lance'
None
True <class 'NoneType'>
2021-10-28 12:17:30,129 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2021-10-28 12:17:30,131 INFO sqlalchemy.engine.Engine INSERT INTO hero (secret_name) VALUES (?)
2021-10-28 12:17:30,131 INFO sqlalchemy.engine.Engine [generated in 0.00015s] ('Dive Wilson',)
2021-10-28 12:17:30,131 INFO sqlalchemy.engine.Engine COMMIT
2021-10-28 12:17:30,143 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2021-10-28 12:17:30,144 INFO sqlalchemy.engine.Engine SELECT hero.id, hero.secret_name
FROM hero
2021-10-28 12:17:30,144 INFO sqlalchemy.engine.Engine [no key 0.00010s] ()
secret_name='Dive Wilson' id=1

False <class 'pydantic.fields.ModelPrivateAttr'>
2021-10-28 12:17:30,144 INFO sqlalchemy.engine.Engine ROLLBACK

As you can see the field is not set to None, and instead is an empty instance of pydantic.fields.ModelPrivateAttr.

Operating System

macOS

Operating System Details

No response

SQLModel Version

0.0.4

Python Version

3.9.5

Additional Context

No response

[ChielWH]

Tried some other options for achieving this, with a pydantic model as the PrivateAttr. The solution as shown in #147 doens't work either. Interestingly, running this in a notebook yields the expected result for B3.

Environment:
Ubuntu 18.04.6 LTS (GNU/Linux 5.4.0-87-generic x86_64)
Python 3.8.12
pydantic==1.9.0
sqlmodel==0.0.6
ipykernel==6.9.0

from typing import Optional
from pydantic import BaseModel, PrivateAttr
from sqlmodel import Field, SQLModel

class A(BaseModel):
    some_attr: int = 2

class B1(SQLModel, table=True):
    _a: A = PrivateAttr(default_factory=A)
    id: Optional[int] = Field(default=None, primary_key=True)

class B2(SQLModel, table=True):
    _a: A = PrivateAttr(default=A())
    id: Optional[int] = Field(default=None, primary_key=True)

class B3(SQLModel, table=True):
    _a: A = PrivateAttr()
    id: Optional[int] = Field(default=None, primary_key=True)

    # from https://pydantic-docs.helpmanual.io/usage/models/#private-model-attributes
    def __init__(self, **data):
        super().__init__(**data)
        self._a = A()

class B4(SQLModel, table=True):
    _a: A = PrivateAttr()
    id: Optional[int] = Field(default=None, primary_key=True)

    @property
    def a(self):
        return self._a

for B in [B1, B2, B3, B4]:
    b = B()
    try:
        print(b._a.some_attr)
    except Exception as e:
        print(e)

Result in plain python:

'ModelPrivateAttr' object has no attribute 'some_attr'
'ModelPrivateAttr' object has no attribute 'some_attr'
'ModelPrivateAttr' object has no attribute 'some_attr'
'ModelPrivateAttr' object has no attribute 'some_attr'

Result in a notebook:

'ModelPrivateAttr' object has no attribute 'some_attr'
'ModelPrivateAttr' object has no attribute 'some_attr'
2
'ModelPrivateAttr' object has no attribute 'some_attr'

[Corfucinas]

Any solution for this? I just ran into the same problem

[alexisgaziello]

It appears that we are missing private attribute initialization. I have raised a PR to fix this.

[Corfucinas]

Great! I ended up adding 8 columns to my SQL table because of this bug.., finally able to clean up after this merge

[swan-alexchen]

same issue here

[robinelvin]

I have a similar related issue. I have a PrivateAttr field on my model so it is not persisted with a public getter for clarity. If I construct the model myself the getter works but if I retrieve from the database with select then accessing the getter raises:

  File "/Volumes/Crucial X8/Projects/Portal/.venv/lib/python3.12/site-packages/pydantic/main.py", line 807, in __getattr__
    return self.__pydantic_private__[item]  # type: ignore
           │    │                    └ '_anonymous'
           │    └ <member '__pydantic_private__' of 'BaseModel' objects>
           └ User(id=1, username='testuser')

TypeError: 'NoneType' object is not subscriptable

The model

class User(SQLModel, table=True):
    __tablename__ = "user"

    id: int | None = Field(default=None, primary_key=True, gt=0)
    username: str = Field(index=True, max_length=32)
    email: str
    password: str = Field(repr=False)
    superuser: bool = Field(default=False)

    _anonymous: bool = PrivateAttr(default=False)

    @property
    def is_anonymous(self) -> bool:
        return self._anonymous

If I change my repository method to the following it works fine. Obviously not a viable solution going forward but serves to prove the point.

    def find_by_username(self, username: str) -> User | None:
        stmt = select(User).where(User.username == username)
        result = self.session.exec(stmt)
        r = result.one_or_none()
        u = User(**r.model_dump())
        return u

[gravesee]

I have a similar related issue. I have a PrivateAttr field on my model so it is not persisted with a public getter for clarity. If I construct the model myself the getter works but if I retrieve from the database with select then accessing the getter raises:

  File "/Volumes/Crucial X8/Projects/Portal/.venv/lib/python3.12/site-packages/pydantic/main.py", line 807, in __getattr__
    return self.__pydantic_private__[item]  # type: ignore
           │    │                    └ '_anonymous'
           │    └ <member '__pydantic_private__' of 'BaseModel' objects>
           └ User(id=1, username='testuser')

TypeError: 'NoneType' object is not subscriptable

The model

class User(SQLModel, table=True):
    __tablename__ = "user"

    id: int | None = Field(default=None, primary_key=True, gt=0)
    username: str = Field(index=True, max_length=32)
    email: str
    password: str = Field(repr=False)
    superuser: bool = Field(default=False)

    _anonymous: bool = PrivateAttr(default=False)

    @property
    def is_anonymous(self) -> bool:
        return self._anonymous

If I change my repository method to the following it works fine. Obviously not a viable solution going forward but serves to prove the point.

    def find_by_username(self, username: str) -> User | None:
        stmt = select(User).where(User.username == username)
        result = self.session.exec(stmt)
        r = result.one_or_none()
        u = User(**r.model_dump())
        return u

Furthermore, if you update the object and add it to the session sqlmodel considers it a new instance entirely and you will see a unique constraint error for the duplicate ID.

[sarth6]

I have a similar related issue. I have a PrivateAttr field on my model so it is not persisted with a public getter for clarity. If I construct the model myself the getter works but if I retrieve from the database with select then accessing the getter raises:

File "/Volumes/Crucial X8/Projects/Portal/.venv/lib/python3.12/site-packages/pydantic/main.py", line 807, in getattr
return self.pydantic_private[item] # type: ignore
│ │ └ '_anonymous'
│ └ <member 'pydantic_private' of 'BaseModel' objects>
└ User(id=1, username='testuser')

TypeError: 'NoneType' object is not subscriptable
The model

class User(SQLModel, table=True):
tablename = "user"

id: int | None = Field(default=None, primary_key=True, gt=0)
username: str = Field(index=True, max_length=32)
email: str
password: str = Field(repr=False)
superuser: bool = Field(default=False)

_anonymous: bool = PrivateAttr(default=False)

@property
def is_anonymous(self) -> bool:
    return self._anonymous

If I change my repository method to the following it works fine. Obviously not a viable solution going forward but serves to prove the point.

def find_by_username(self, username: str) -> User | None:
    stmt = select(User).where(User.username == username)
    result = self.session.exec(stmt)
    r = result.one_or_none()
    u = User(**r.model_dump())
    return u

I've worked around this with an event listener

from pydantic._internal._model_construction import init_private_attributes
from sqlalchemy import event
from sqlmodel import SQLModel

@event.listens_for(SQLModel, "load", propagate=True)
def receive_load(target: Any, context: Any) -> None:
    init_private_attributes(target, None)