Describe the bug
When using a ForeignKeyField with to_field pointing to a non-primary key field (e.g., app_id which is a UUID/CharField), Tortoise ORM fails to correctly resolve the column for lookups.
Instead of using the specified to_field (e.g., app_id), the ORM incorrectly defaults to the model's primary key name (e.g., id) and generates SQL using the ${fk_name}_id pattern. This causes the generated query to filter by the wrong column or even use the wrong data type (e.g., passing a UUID string to an integer-based ID column), leading to runtime type errors or incorrect results.
To Reproduce
Steps to reproduce:
-
Define a model (Application) with a primary key (id) and a secondary unique field (app_id).
-
Define a second model (TableB) with a ForeignKeyField pointing to Application, setting to_field="app_id".
-
Perform a filter on TableB using an Application instance.
Minimal code snippet:
class Application(Model):
id = fields.IntField(pk=True) # Default auto-increment PK
app_id = fields.CharField(unique=True, max_length=50) # The custom field we want to link to
class TableB(Model):
app = fields.ForeignKeyField(
"models.Application",
to_field="app_id",
db_constraint=False
)
# Execution
app_instance = await Application.get(app_id="550e8400-e29b-41d4-a716-446655440000")
# The query incorrectly generates: ... WHERE app_id = 1 (or tries to use the internal pk)
# instead of: ... WHERE app_app_id = '550e8400-e29b-41d4-a716-446655440000'
await TableB.filter(app=app_instance)
Expected behavior
The ORM should respect the to_field configuration. When filtering by a related model instance, it should resolve the link using the column mapping associated with to_field(app_app_id in the example above) rather than defaulting to ${fk_name}_id.
Additional context
This issue effectively makes using to_field for non-primary key associations impossible for query lookups.
It appears that the query builder incorrectly assumes that the related field is always the Primary Key of the target model, ignoring the to_field metadata in the relationship.
Tortoise ORM version: 1.1.7
Technical Analysis of the Bug
My initial investigation points to the _get_actual_filter_params method as a likely culprit, where the logic seems to prioritize pk over the explicitly defined to_field.
Specifically, when processing a ForeignKeyField or OneToOneField, the ORM uses the following logic to resolve the filter value:
|
def _get_actual_filter_params( |
|
self, resolve_context: ResolveContext, key: str, value: Table | FilterInfoDict |
|
) -> tuple[str, Any]: |
|
filter_key = key |
|
if ( |
|
key in resolve_context.model._meta.fk_fields |
|
or key in resolve_context.model._meta.o2o_fields |
|
): |
|
field_object = resolve_context.model._meta.fields_map[key] |
|
filter_key = cast(str, field_object.source_field) |
|
filter_value = getattr(value, "pk", value) |
|
elif key in resolve_context.model._meta.m2m_fields: |
|
filter_value = getattr(value, "pk", value) |
|
elif ( |
|
key.split("__")[0] in resolve_context.model._meta.fetch_fields |
|
or key in resolve_context.custom_filters |
|
or key in resolve_context.model._meta.filters |
|
): |
|
filter_value = value |
|
else: |
|
allowed = sorted( |
|
resolve_context.model._meta.fields |
|
| resolve_context.model._meta.fetch_fields |
|
| set(resolve_context.custom_filters) |
|
) |
|
raise FieldError(f"Unknown filter param '{key}'. Allowed base values are {allowed}") |
|
|
|
if isinstance(filter_value, Expression): |
|
filter_value = filter_value.resolve(resolve_context).term |
|
|
|
return filter_key, filter_value |
The logic flaw:
- The filter_value is hardcoded to retrieve
value.pk.
- It fails to check if the ForeignKeyField has a custom
to_field attribute defined.
When to_field is specified (e.g., pointing to a UUID column), the ORM ignores this configuration and incorrectly extracts the primary key (id) instead, leading to type mismatches and invalid SQL filters.
Suggested Fix:
The implementation should inspect the field_object for a to_field attribute and use that to resolve the value from the value instance, falling back to .pk only if to_field is not defined.
Describe the bug
When using a ForeignKeyField with
to_fieldpointing to a non-primary key field (e.g., app_id which is a UUID/CharField), Tortoise ORM fails to correctly resolve the column for lookups.Instead of using the specified
to_field(e.g., app_id), the ORM incorrectly defaults to the model's primary key name (e.g., id) and generates SQL using the${fk_name}_idpattern. This causes the generated query to filter by the wrong column or even use the wrong data type (e.g., passing a UUID string to an integer-based ID column), leading to runtime type errors or incorrect results.To Reproduce
Steps to reproduce:
Define a model (Application) with a primary key (id) and a secondary unique field (app_id).
Define a second model (TableB) with a ForeignKeyField pointing to Application, setting
to_field="app_id".Perform a filter on TableB using an Application instance.
Minimal code snippet:
Expected behavior
The ORM should respect the
to_fieldconfiguration. When filtering by a related model instance, it should resolve the link using the column mapping associated withto_field(app_app_id in the example above) rather than defaulting to${fk_name}_id.Additional context
This issue effectively makes using to_field for non-primary key associations impossible for query lookups.
It appears that the query builder incorrectly assumes that the related field is always the Primary Key of the target model, ignoring the
to_fieldmetadata in the relationship.Tortoise ORM version: 1.1.7
Technical Analysis of the Bug
My initial investigation points to the
_get_actual_filter_paramsmethod as a likely culprit, where the logic seems to prioritizepkover the explicitly definedto_field.Specifically, when processing a ForeignKeyField or OneToOneField, the ORM uses the following logic to resolve the filter value:
tortoise-orm/tortoise/expressions.py
Lines 432 to 462 in bc4f971
The logic flaw:
value.pk.to_fieldattribute defined.When
to_fieldis specified (e.g., pointing to a UUID column), the ORM ignores this configuration and incorrectly extracts the primary key (id) instead, leading to type mismatches and invalid SQL filters.Suggested Fix:
The implementation should inspect the field_object for a
to_fieldattribute and use that to resolve the value from the value instance, falling back to .pk only ifto_fieldis not defined.