feat: add openlineage support

Author: razvan

CHANGELOG.md

@@ -4,6 +4,7 @@
 
 ### Added
 
+- Support for emitting [OpenLineage](https://openlineage.io/) lineage events via `spec.clusterConfig.openLineage`, either with an inline connection or a reference to an `OpenLineageConnection`, with optional TLS and Static-provider API-key authentication. Requires the `apache-airflow-providers-openlineage` provider to be installed in the product image ([#XXXX]).
 - BREAKING: Add required CLI argument and env var to set the image repository used to construct final product image names: `IMAGE_REPOSITORY` (`--image-repository`), eg. `oci.example.org/my/namespace` ([#784]).
 - Support for passing CAs to GitSync ([#750]).
 - Support setting `clientAuthenticationMethod` for OIDC authentication. The value is passed through to the Flask-AppBuilder config as `token_endpoint_auth_method` ([#765]).
@@ -36,6 +37,7 @@
 - Allow OPA integration with arbitrary user role when using Airflow 3. Previously, this only worked for users with Admin role ([#800])
 
 [#750]: https://github.com/stackabletech/airflow-operator/pull/750
+[#XXXX]: https://github.com/stackabletech/airflow-operator/pull/XXXX
 [#765]: https://github.com/stackabletech/airflow-operator/pull/765
 [#767]: https://github.com/stackabletech/airflow-operator/pull/767
 [#770]: https://github.com/stackabletech/airflow-operator/pull/770

Cargo.lock

@@ -33,5 +33,6 @@ tokio = { version = "1.52", features = ["full"] }
 tracing = "0.1"
 
 [patch."https://github.com/stackabletech/operator-rs.git"]
-# stackable-operator = { git = "https://github.com/stackabletech//operator-rs.git", branch = "main" }
+# TODO: Switch back to a release tag once operator-rs#1250 (crd::openlineage) has merged.
+stackable-operator = { git = "https://github.com/stackabletech//operator-rs.git", branch = "feat/openlineage-crd" }
 # stackable-operator = { path = "../operator-rs/crates/stackable-operator" }

Cargo.nix

@@ -133,6 +133,17 @@ rules:
       - get
       - list
       - watch
+  # Resolve OpenLineage connections referenced by spec.clusterConfig.openLineage.
+  # Watched (list + watch) to re-reconcile when an OpenLineageConnection changes,
+  # and fetched individually (get) during reconciliation to dereference a `reference`.
+  - apiGroups:
+      - openlineage.stackable.tech
+    resources:
+      - openlineageconnections
+    verbs:
+      - get
+      - list
+      - watch
   # Publish Kubernetes events for reconciliation activity
   - apiGroups:
       - events.k8s.io