Default validators break self-references

[nursix]

DAL.lazy_define_table of a Table with a self-reference (foreign key to itself) will not work with default validators, and crash with an AttributeError.

The chain of effect:

• Field.bind tries to update the default validator - which has not been set before because the table is not yet defined
• get_default_validator tries to resolve the referenced table through Field.referenced_table/field
• Field.referenced_field crashes with an AttributeError when attempting to access self._db[tablename], which should not be possible given that it has just passed the test whether tablename is in self._db

Workarounds:
An explicit requires parameter on the self-reference side-steps the problem, as does setting lazy_tables=False on the DAL instance.

Analysis:
It is not immediately obvious why Field.referenced_field would pass the check whether tablename is in self._db, and then crash in the very next line in the attempt to access the exact same key. Maybe I am missing a layer of recursion here?

[nursix]

This is even more confusing considering the fact that it works with lazy_tables=False, as it takes exactly the same path.

...but maybe that's the key to the problem.

[nursix]

This is even more confusing considering the fact that it works with lazy_tables=False, as it takes exactly the same path.

...but maybe that's the key to the problem.

...quite so: the first call to define_table would add the table name to self.tables, and __contains__ only tests for that, but __getattr__ would attempt to lazy_define_table, which fails because the table name has already been removed (via pop) from self._LAZY_TABLES at that point.

Thus, if Field.referenced_field resp. Field.referenced_table could detect a self-reference and indicate that for get_default_validator via the return value, the latter could use that to instantiate an IS_IN_DB with a string for the primary key instead of a Field instance - and then it should work. Alternatively, a separate function could be written to detect a self-reference, before calling referenced_table.

[nursix]

The following patch fixes the problem:

diff --git a/pydal/objects.py b/pydal/objects.py
index 3b70461f..cb89ab1d 100644
--- a/pydal/objects.py
+++ b/pydal/objects.py
@@ -2542,7 +2542,13 @@ class Field(Expression, Serializable):
         if not self._db or tablename not in self._db:
             # The table being referenced is not defined yet
             return None
-        table = self._db[tablename]
+        try:
+            table = self._db[tablename]
+        except (KeyError, AttributeError):
+            if tablename == self._tablename:
+                table = self._table
+            else:
+                raise
         return table[fieldname] if fieldname else table._id
 
     def referenced_table(self):

...but I couldn't say whether this is the correct approach:

If tablename is in self._db, but not in self._db.dict, then it is defined but not instantiated yet, so we get a KeyError or AttributeError when we attempt to access it.

However, if in this case the name of the referenced table is the same as the one the field is bound to, then we can (I think safely) assume that self._table is intended - and use that. Otherwise, the exception is justified as before.

Will submit this as a PR to get it validated.

[nursix]

It might be valid to check for self._tablename first before attempting to access the table via self._db - which would remove the try-block. However, it would add a string comparison for all cases, which would give False most of the time, so adding unnecessary overhead to the critical path. Therefore I would keep the try-block (if that fix is correct at all).

[leonelcamara]

I had this fixed here....

https://github.com/web2py/pydal/pull/749/files

And it was reverted. Maybe make sure to add tests for this.