Skip to content

27_cli_implementation_guide.rst

Latest commit

 

History

History
144 lines (111 loc) · 6.12 KB

27_cli_implementation_guide.rst

File metadata and controls

144 lines (111 loc) · 6.12 KB

CLI Implementation Guide

This document describes the direction for implementing new CLI functionality in Apache Airflow.

Table of Contents

Overview

Airflow ships two CLIs:

  • airflow (airflow-core) — bundled with the core distribution. Hosts both legacy remote commands (being rewired internally) and admin/local commands that have no Public API equivalent.
  • airflowctl (airflow-ctl) — a standalone CLI distributed separately that talks to a running Airflow instance exclusively through the Public (Core) API.

Following AIP-94 (tracked via GitHub Projects #570 and #571), CLI work follows two rules:

  1. New commands that are achievable via the Public API are added to airflowctl only. Adding the same command to the airflow CLI as well is discouraged — it duplicates maintenance surface without user benefit.
  2. Existing airflow CLI remote commands stay in place (so users keep running airflow dags list, airflow pools get, …) but are rewired internally to call the Public API via the airflowctl HTTP client instead of accessing the metadata database directly.

Both rules enforce RBAC, remove direct database exposure for remote operations, and eliminate duplicate code paths. This builds on AIP-81, which introduced the distinction between local and remote commands.

Decision Table

Scenario Where it goes Notes
New command, achievable via Public API airflowctl only Do not add to the airflow CLI unless strongly needed in core
New command, not achievable via Public API airflow CLI (admin/local) Admin/local commands only — see below
Existing airflow CLI command, achievable via Public API airflow CLI → delegates to the airflowctl HTTP client Rewire; no direct DB access, no SQLAlchemy/session usage
Existing airflow CLI command, not achievable via Public API airflow CLI, unchanged Stays as a pure airflow-core implementation

"Not achievable via the Public API" means the operation has no API representation and is inherently admin/local in nature — database shell, schema migrations, process management, or deployment configuration that requires direct infrastructure access.

Adding a New Command (Public API achievable)

Add the command to airflowctl only. Do not also add it to the airflow CLI unless there is a strong reason it must live in core (e.g., it is tightly coupled to a local process or a deployment concern with no API representation).

Source location: airflow-ctl/src/airflowctl/ctl/commands/

HTTP client and operations: airflow-ctl/src/airflowctl/api/ (client.py, operations.py).

  1. Add the command under the appropriate group module in airflow-ctl/src/airflowctl/ctl/commands/.

  2. Call the Public API through the airflowctl HTTP client (airflowctl.api.client) and the operations layer in airflowctl.api.operations. Do not import airflow-core models or touch the metadata database.

  3. If the required API endpoint does not exist yet, add it first (see Adding API Endpoints).

  4. Add tests under airflow-ctl/tests/.

  5. Run integration tests with:

    breeze testing airflow-ctl-integration-test

Rewiring an Existing airflow CLI Command

Use this when an existing airflow CLI remote command still talks to the database directly and needs to go through the Public API instead. The user-facing command name and arguments stay the same.

Source location: airflow-core/src/airflow/cli/

  1. Replace direct database access with calls through the airflowctl HTTP client (airflowctl.api.client / airflowctl.api.operations). Remove SQLAlchemy model imports and session-based helpers from the command.
  2. If the required API endpoint does not exist yet, add it first (see Adding API Endpoints).
  3. Update tests under airflow-core/tests/cli/ to mock or exercise the HTTP client instead of the database.

Adding an Admin/Local Command (no Public API equivalent)

Use this only when the operation cannot reasonably be exposed through the Public API — typically database shell, schema migrations, process management, or deployment-time configuration.

Source location: airflow-core/src/airflow/cli/

  1. Add the command to an appropriate admin group (e.g., db, config).
  2. Add (admin only) to the help string so users know the command requires direct infrastructure access.
  3. Add tests under airflow-core/tests/cli/.

References