docs: clarify BigQuery project-level permissions for query history and Hex integration#2285
docs: clarify BigQuery project-level permissions for query history and Hex integration#2285arbiv wants to merge 5 commits into
Conversation
…d Hex integration - Restructure grant instructions to separate project-level IAM roles (Resource Viewer, Metadata Viewer) from dataset-level access (Data Viewer) - Add explicit note that bigquery.jobs.listAll requires project-level grant - Update permissions-and-security snippet with BigQuery-specific role details - Add prerequisites section to Hex integration page explaining query history dependency - Document multi-project setup for Hex when using a separate execution project Co-Authored-By: Yosef Arbiv <yosef.arbiv@gmail.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
There was a problem hiding this comment.
🚩 Pre-existing images in grant_user_access_on_dataset_level.mdx use external hosting and lack Frame wrappers
The three images in docs/snippets/dwh/bigquery/grant_user_access_on_dataset_level.mdx:40-43, 47-49, 57-59 use Cloudinary URLs (external hosting) and are not wrapped in <Frame> components. Both of these violate rules in docs/skills.md:198-209 ("All images must be wrapped in a <Frame> with a caption. No exceptions." and "Store images as local files under docs/pics/."). These are pre-existing issues not introduced by this PR — the images are in unchanged context lines — so they are not reported as PR bugs, but they should be cleaned up in a follow-up.
Was this helpful? React with 👍 or 👎 to provide feedback.
| For BigQuery, this translates to: | ||
| - **BigQuery Data Viewer** on the Elementary dataset (for reading Elementary schema data). | ||
| - **BigQuery Metadata Viewer** and **BigQuery Resource Viewer** at the **project level** (for reading `INFORMATION_SCHEMA.JOBS` query history and table metadata). These project-level roles are required for BI integrations like Hex that rely on query history. |
There was a problem hiding this comment.
🔴 BigQuery-specific permissions text incorrectly shown on Snowflake, Redshift, Postgres, ClickHouse, and Athena setup pages
BigQuery-specific role information is added to a shared permissions snippet (docs/snippets/cloud/integrations/permissions-and-security.mdx:8-10) that is rendered on every data-warehouse setup page, so Snowflake, Redshift, Postgres, ClickHouse, and Athena users see irrelevant BigQuery role names.
Impact: Users setting up non-BigQuery warehouses see confusing, inapplicable BigQuery permission instructions on their setup page.
Full mechanism and affected pages
The snippet permissions-and-security.mdx is imported and rendered by:
snippets/cloud/integrations/create-user-operation-snowflake.mdx:19(Snowflake)snippets/cloud/integrations/create-user-operation.mdx:14(used by Redshift viasnippets/cloud/integrations/redshift.mdx:1, Postgres viasnippets/cloud/integrations/postgres.mdx:1, ClickHouse viasnippets/cloud/integrations/clickhouse.mdx:1, Athena viasnippets/cloud/integrations/athena.mdx:1)snippets/cloud/integrations/bigquery.mdx:26(BigQuery — the only one where this content is appropriate)
The newly added lines read:
For BigQuery, this translates to:
- **BigQuery Data Viewer** on the Elementary dataset…
- **BigQuery Metadata Viewer** and **BigQuery Resource Viewer** at the **project level**…
This content should either be removed from the shared snippet and placed only in the BigQuery-specific page, or the snippet should conditionally render it only for BigQuery.
Prompt for agents
The permissions-and-security.mdx snippet is shared across all DWH integration pages (BigQuery, Snowflake, Redshift, Postgres, ClickHouse, Athena). The PR added BigQuery-specific role information (lines 8-10) to this shared snippet, which now incorrectly appears on all non-BigQuery setup pages.
To fix: Remove the BigQuery-specific lines ("For BigQuery, this translates to:" and the two bullet points about BigQuery Data Viewer / Metadata Viewer / Resource Viewer) from permissions-and-security.mdx. Instead, add this BigQuery-specific content directly to the BigQuery setup page — for example in docs/snippets/cloud/integrations/bigquery.mdx after the <PermissionsAndSecurity /> component call, or in a BigQuery-specific snippet. This way non-BigQuery DWH pages won't show irrelevant BigQuery role information.
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
Fixed in 5504bb6 — moved the BigQuery-specific role details out of the shared snippet and into snippets/cloud/integrations/bigquery.mdx (which is only rendered on the BigQuery page). The shared permissions-and-security.mdx now only contains the generic warehouse-agnostic text.
|
|
||
| The Hex integration relies on BigQuery query history to discover which tables Hex projects query. For the integration to work, Elementary's service account must have **read access to `INFORMATION_SCHEMA.JOBS`** in the BigQuery project(s) where Hex executes queries. | ||
|
|
||
| This requires the **BigQuery Resource Viewer** role (`roles/bigquery.resourceViewer`) granted at the **project level**. See [BigQuery permissions](/cloud/integrations/dwh/bigquery) for detailed setup instructions. |
There was a problem hiding this comment.
🟡 Hex prerequisites section lists only one of the two required BigQuery roles, contradicting the same page and other docs
The Hex prerequisites mention only the "BigQuery Resource Viewer" role (docs/cloud/integrations/bi/hex.mdx:13) but omit the also-required "BigQuery Metadata Viewer" role, so users following only this section may not grant sufficient permissions.
Impact: Users may grant incomplete permissions and then find the Hex integration does not discover any exposures.
Inconsistency details
Line 13 of hex.mdx says:
This requires the BigQuery Resource Viewer role (
roles/bigquery.resourceViewer) granted at the project level.
But the same page at line 37 says:
assign the BigQuery Resource Viewer and BigQuery Metadata Viewer roles
And every other doc in this PR consistently requires both roles:
docs/snippets/cloud/integrations/permissions-and-security.mdx:10docs/snippets/dwh/bigquery/grant_user_access_on_dataset_level.mdx:6
Line 13 should mention both roles to be consistent.
| This requires the **BigQuery Resource Viewer** role (`roles/bigquery.resourceViewer`) granted at the **project level**. See [BigQuery permissions](/cloud/integrations/dwh/bigquery) for detailed setup instructions. | |
| This requires the **BigQuery Metadata Viewer** (`roles/bigquery.metadataViewer`) and **BigQuery Resource Viewer** (`roles/bigquery.resourceViewer`) roles granted at the **project level**. See [BigQuery permissions](/cloud/integrations/dwh/bigquery) for detailed setup instructions. |
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
Fixed in 5504bb6 — updated to mention both BigQuery Metadata Viewer and BigQuery Resource Viewer consistently.
- Remove BigQuery-specific role details from shared permissions-and-security.mdx (was incorrectly shown on Snowflake, Redshift, Postgres, ClickHouse, Athena pages) - Add the BigQuery role details to bigquery.mdx instead (only shown on BigQuery page) - Fix Hex prerequisites to mention both required roles (Metadata Viewer + Resource Viewer) Co-Authored-By: Yosef Arbiv <yosef.arbiv@gmail.com>
- Rewrite prerequisites to reference query history generically with links to BigQuery, Snowflake, and Databricks permissions pages - Rename 'Using a different BigQuery project' to 'Hex queries on a different warehouse project' with generic language Co-Authored-By: Yosef Arbiv <yosef.arbiv@gmail.com>
Use 'environment' instead of 'project' since not all warehouses use the term 'project' (e.g., Snowflake has accounts, Databricks has workspaces). Co-Authored-By: Yosef Arbiv <yosef.arbiv@gmail.com>
Hex query history filters only exist for BigQuery and Snowflake. Removed Databricks from the prerequisites links and added explicit 'supported on BigQuery and Snowflake' statement. Co-Authored-By: Yosef Arbiv <yosef.arbiv@gmail.com>
1 participant
Summary
Fixes documentation gaps that caused customers to miss granting project-level IAM roles needed for query history access (specifically
bigquery.jobs.listAll), which in turn breaks BI integrations like Hex.Problem: The existing grant instructions showed the dataset-level sharing UI (three-dots → Share on a dataset) but BigQuery Resource Viewer and Metadata Viewer must be granted at the project level via IAM & Admin. Customers following the docs literally would miss these project-level roles, resulting in
INFORMATION_SCHEMA.JOBSaccess errors and broken Hex integration.Changes:
grant_user_access_on_dataset_level.mdx— restructured into two clear steps:<Note>explaining why project-level is requiredpermissions-and-security.mdx— kept warehouse-agnostic (shared across all DWH pages); BigQuery-specific role details moved to the BigQuery-only page.bigquery.mdx— added BigQuery-specific role breakdown (Data Viewer, Metadata Viewer, Resource Viewer) directly after the shared permissions snippet, so it only renders on the BigQuery setup page.hex.mdx— added (DWH-agnostic):Screenshots
BigQuery page — Grant service user access (new structure with Note callout)
BigQuery page — Step 1 (project-level) and Step 2 (dataset-level) instructions
BigQuery page — Permissions and security section with BigQuery-specific roles
Hex page — DWH-agnostic prerequisites with links to warehouse permissions
Hex page — "Hex queries on a different warehouse project" section
Link to Devin session: https://app.devin.ai/sessions/0da02a44dd534e4a909ae504d8eb6a19
Requested by: @arbiv