@@ -60,6 +60,23 @@ Follow these steps when fulfilling Dataform-related requests:
6060 * ** If ` workflow_settings.yaml ` IS found** :
6161 * Run ` dataform compile <PROJECT_DIR> ` to compile the pipeline and get
6262 an overview of existing files and the DAG.
63+ - Once the repository is located or initialized, check if
64+ ` .df-credentials.json ` is present in the Dataform project directory. If
65+ absent, ask the user to run ` dataform init-creds ` to create the credentials
66+ file. If the user cannot initialize the credentials, write the
67+ ` .df-credentials.json ` file manually, following the format below. Replace
68+ ` <PROJECT_ID> ` with a Google Cloud project for billing (e.g., obtained via
69+ ` gcloud config get-value project ` ) and ` <LOCATION> ` with the appropriate
70+ region (e.g., obtained via ` gcloud config get compute/region ` or defaulting
71+ to ` us-central1 ` if unspecified).
72+
73+ ``` json
74+ {
75+ "projectId" : " <PROJECT_ID>" ,
76+ "location" : " <LOCATION>"
77+ }
78+ ```
79+
6380- Use the compiled graph as the **source of truth** for existing assets.
6481
6582### 2. Gather Information
@@ -119,28 +136,23 @@ For non-trivial requests, create a clear specification before implementation:
119136### 6. Validate & Compile
120137
121138- Run `dataform compile` to catch syntax and dependency errors.
122- - Check if ` .df-credentials.json ` is present in the Dataform project
123- directory.
124-
125- - ** If present:** Run ` dataform run --dry-run ` for validation.
126- - ** If absent:** Ask the user to run ` dataform init-creds ` to create the
127- credentials file.
128- - ** If credentials cannot be initialized:** Fall back to using `dataform
129- compile` , manual SQL inspection, and ` bq query --dry_run` for
130- validation.
131-
132- > [ !IMPORTANT]
133- >
134- > If ` dataform run --dry-run ` fails, inspect the error message. If the
135- > failure is ONLY due to "Table not found" errors for nodes defined
136- > within the current Dataform project (which occurs when upstream
137- > dependencies haven't been materialized in BigQuery), then this
138- > specific error may be ignored. If the dry run fails for ANY other
139- > reason (such as SQL syntax errors, permission errors, or references to
140- > tables not defined in the project), these errors MUST be addressed. If
141- > only "Not found" errors for unmaterialized project tables are present,
142- > rely on ` dataform compile ` , manual SQL inspection, and `bq query
143- > --dry_run` for verification.
139+ - If `.df-credentials.json` is successfully set up (from Step 1), run
140+ `dataform run --dry-run` for validation.
141+ - If `.df-credentials.json` could not be initialized, fall back to using
142+ `dataform compile`, manual SQL inspection, and `bq query --dry_run` for
143+ validation.
144+
145+ > [!IMPORTANT]
146+ >
147+ > If `dataform run --dry-run` fails, inspect the error message. If the
148+ > failure is ONLY due to "Table not found" errors for nodes defined within
149+ > the current Dataform project (which occurs when upstream dependencies
150+ > haven't been materialized in BigQuery), then this specific error may be
151+ > ignored. If the dry run fails for ANY other reason (such as SQL syntax
152+ > errors, permission errors, or references to tables not defined in the
153+ > project), these errors MUST be addressed. If only "Not found" errors for
154+ > unmaterialized project tables are present, rely on `dataform compile`,
155+ > manual SQL inspection, and `bq query --dry_run` for verification.
144156
145157- Validate SQL logic of changed nodes and fix any errors.
146158
@@ -153,20 +165,19 @@ For non-trivial requests, create a clear specification before implementation:
153165
154166- Repeat steps 5–6 until the request is fully satisfied.
155167
156- ## Credentials for ` dataform run `
168+ ## Credentials for `dataform run` and `dataform run --dry-run`
157169
158170The command `dataform run` executes your Dataform pipeline in BigQuery but
159171requires credentials to be set up in a `.df-credentials.json` file in your
160172project directory.
161173
162- Generate pipeline code and ensure it compiles via ` dataform compile ` . Use
163- ` dataform run --dry-run ` for validation only if the ` .df-credentials.json ` file
164- is present. MUST NOT execute a real ` dataform run ` without explicit user
165- request.
174+ Generate pipeline code and ensure it compiles via `dataform compile`. Validate
175+ the pipeline using `dataform run --dry-run` once the `.df-credentials.json` file
176+ is successfully created (as instructed in the Understand the Current State
177+ step). MUST NOT execute a real `dataform run` without explicit user request.
166178
167- If ` .df-credentials.json ` is missing, instruct the user to run `dataform
168- init-creds` in their terminal. If they are unable to get the credentials
169- initialized, fall back on other methods of validation, such as `dataform
179+ If `.df-credentials.json` could not be initialized via `dataform init-creds` or
180+ manual creation, fall back on other methods of validation, such as `dataform
170181compile`, manual SQL inspection, and `bq query --dry_run`.
171182
172183## Incremental / Append Operations
0 commit comments