Conditional ignoring of directories from parsing in airflow

[manoj121999]

Problem

.airflowignore currently works as a deny-list only.
While it supports ignoring specific files or directories, it does not support
conditional or allow-list based directory traversal.

Because Airflow walks directories before applying ignore rules, patterns like:

• “Ignore all folders except X”
• “Parse only this folder under a repo”
• “Ignore unless under ”

This becomes a real limitation in when requirements comes to where:

• DAGs live under one folder
• Shared code (utils, operators, libs) must coexist
• Restructuring the repo is not always possible

---

Proposed Enhancement

Add support for conditional ignore / allow-list semantics, for example:

if dags are under specific folder , we need to allow that and rest of the folders can be ignored.

[potiuk]

Look https://airflow.apache.org/docs/apache-airflow/3.1.7/core-concepts/dags.html#airflowignore -> the glob syntax actually allows you to negate patterns (in Airflow 3) - similar to .gitignore and effectively implement allowlist. We use it in airflow's .dockerignore for example https://github.com/apache/airflow/blob/main/.dockerignore and you can use similar patterns.

This is standard way how you imlement in glob .*ignore files.