Do not deserialize trigger_kwargs when loading serialized DAGs#66002
Do not deserialize trigger_kwargs when loading serialized DAGs#66002amoghrajesh wants to merge 3 commits intoapache:mainfrom
Conversation
|
Just one test left! |
|
The changes to the
The raw JSON has Encoding enum instances as dict keys — that's how Python 3.10+ Python 3.10: On Python ≤3.10, str(Encoding.TYPE) returns "Encoding.TYPE" (the enum repr) instead of "__type" (its value), mangling the keys so the Triggerer cannot read them back. Adding |
Was generative AI tooling used to co-author this PR?
After #55068 was merged, inconsequentially,
_decode_start_trigger_argswas deserializingtrigger_kwargsandnext_kwargswhen loading a serialized DAG. These fields hold the serialized trigger state that only the Triggerer needs — when it picks up a deferred task, inflates the kwargs, and instantiates the trigger class.The Scheduler and API Server load serialized DAGs but never touch these values; deserializing them there is wasted work.
This PR removes the deserialization and keeps
trigger_kwargsandnext_kwargsas raw JSON onStartTriggerArgs. The Triggerer reads trigger kwargs directly from the Trigger DB row (not through DAG deserialization), so its path is unaffected:airflow-core/src/airflow/models/trigger.py#L170-L179What changes?
Why the changes to enum.py
#66002 (comment)
Scheduler
Before:
After:
The
Trigger.__init__callsserde.serialize()on whatever it receives, so deserializing then re-serializing was pure waste.API Server
Before: loads serialized DAG →
_decode_start_trigger_args()which deserializestrigger_kwargsinto Python objects → never used, thrown away.After: same but stays raw JSON. No functional difference, just skips the work.
Triggerer
Before and after:
The triggerer also loads the serialized DAG at https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/jobs/triggerer_job_runner.py#L717-L735 to check start_from_trigger. So it does go through
_decode_start_trigger_args()too, but it does not use trigger_kwargs from that — it readstrigger.encrypted_kwargsfrom the DB row directly below. So the fix is safe for the triggerer as well.{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.