v0.8.0

The Security Analysis Tool 0.8.0 release adds 11 new security checks, brings per-user identity to the Permissions Analysis app, expands secret-scanning coverage, and ships a refreshed documentation site.

What's new

11 new security checks across all five SAT categories:

• Network Security: workspace IP access list enforcement (NS-11), Context-Based Ingress policy (NS-12), account console
  IP access list (NS-13), and a live-behavior egress control test from compute (NS-14).
• Identity & Access: PAT token creation restricted to admins (IA-8), service principal client secrets not stale (IA-9 —
  AWS and GCP).
• Data Protection: legacy DBFS root and mounts (DP-10), SQL warehouse results download (DP-11), DBFS file browser
  (DP-13).
• Governance: jobs run as service principal (GOV-42), jobs not granting CAN_MANAGE to non-admin principals (GOV-45).
• Informational: Git repository allowlist configured (INFO-42).

NS-14 — first live-behavior check. Most SAT checks inspect configuration. NS-14 actually probes outbound destinations
from the SAT driver compute to verify egress controls block public access. On a properly locked-down workspace, NS-14
reports "egress blocked" — that's the pass state.

Permissions Analysis app — per-user identity. The interactive permissions analysis app now runs every query as the
calling user via on-behalf-of-user (OBO) authentication. Unity Catalog enforces the user's grants directly on the analysis
tables. Includes per-request audit logging and a friendly banner when grants are missing — no more SQL errors in the UI.

Expanded secret scanning. Cluster init scripts referenced via FUSE mount paths are now scanned alongside
spark_env_vars. File-type secrets in shell scripts are detected. The DetectorName field on findings is now consistently
populated. TruffleHog is pinned to a tagged release for supply-chain reproducibility.

Documentation refreshed. New Upgrade
Guide, "What's new in 0.8.0" callout
on the Functionality page, and expanded troubleshooting for common upgrade scenarios.

Other improvements

• Serverless reliability — SAT jobs and the Permissions Analysis app pin to Environment v5 for predictable behavior across
  DABS and Terraform installs.
• The accounts bootstrap only fetches network configurations for SAT-enabled workspaces, reducing unnecessary API calls on
  accounts with many non-SAT workspaces.
• Faster install on serverless via a lazy msal import.

Cloud-specific note

IA-9 is evaluated on AWS and GCP only. Azure service principals are typically managed in Microsoft Entra ID; their OAuth
credentials live there, not in Databricks. The check is disabled for Azure to avoid misleading results.

Upgrading from 0.7.x

Re-deploy via your usual install path (./install.sh or terraform apply). See the Upgrade
Guide for behavior changes —
particularly the one-time consent re-prompt for the Permissions Analysis app and the dashboard check-count delta as new
findings surface.

Full changelog: v0.7.0...v0.8.0

v0.7.0

Release/0.7.0 core (#311)

* Fix typos in configs and notebook docstrings

- Fix GOV-20 check name: "Unity Sharing" → "Unity Catalog"
- Fix "metasores" → "metastores" in GOV-20 logic description
- Fix "cofiguration" → "configuration" in 5 manual check descriptions
  (DP-4, IA-1, IA-2, IA-3, INFO-7)
- Fix "configruation" → "configuration" in initialize.py and
  sat_checks_config.py docstrings

* Fix "recepient" → "recipient" typo in delta sharing code

- Rename functions: get_sharing_recepients_list, get_sharing_recepient,
  get_sharing_recepient_permissions → recipient spelling
- Rename intermediate table: delta_sharing_recepients_list →
  delta_sharing_recipients_list (safe: table is in intermediate schema
  which is dropped after each SAT run)
- Update all callers in workspace_bootstrap, workspace_analysis, and tests

* Fix typos in security checks CSV and workspace analysis notebook

- configs/security_best_practices.csv: fix "custer"→"cluster" (GOV-9) and "diplayName"→"displayName" (INFO-6)
- notebooks/Includes/workspace_analysis.py: fix "uc_metasore*" → "uc_metastore*" in local variable and inner function names

* Revert non-typo changes swept into previous commit

Restores CHANGELOG.md and configs/sat_dasf_mapping.csv to their
release/0.7.0_core state, and removes the unintended backup file.
This branch should only contain the two typo fixes:
  - security_best_practices.csv: custer→cluster, diplayName→displayName
  - workspace_analysis.py: uc_metasore*→uc_metastore*

* Fix 2 CSV typos cleanly (no extra changes)

Restores security_best_practices.csv to its pre-ac66c3d state and
re-applies only the 2 intended typo fixes:
  - GOV-9: 'custer' → 'cluster' in recommendation field
  - INFO-6: 'diplayName' → 'displayName' in logic field

* Bump SDK to 0.1.41 and rebuild wheel

- Increment __version__ in setup.py: 0.1.40 → 0.1.41
- Replace lib/dbl_sat_sdk-0.1.40-*.whl with newly built 0.1.41 wheel
- Update SDK_VERSION in install_sat_sdk.py to match

* feat(SFE-4539): add UC schema/table/column comments for Genie

Adds apply_schema_comments() to common.py with data-verified descriptions
for all 12 tables and ~110 columns in the security_analysis schema.
Called from initialize.py after load_sat_dasf_mapping() so comments are
applied on every SAT run.

- Covers 9 SAT core tables + 3 BrickHound tables (BH wrapped in try/except)
- Single quotes escaped as '' in SQL literals
- Idempotent: safe to re-run on existing deployments

* docs: add CLAUDE.md to repo and schema comment/validation rules

- Add CLAUDE.md to version control (force-add, was previously gitignored)
- Add Schema Comment Sync rule: keep apply_schema_comments() in sync
  whenever tables/columns are added, removed, or renamed
- Add security_best_practices.csv uniqueness validation rule: check that
  id and check_id are both unique before every CSV commit
- Add Pre-Commit Typo Check rule using codespell
- Update security_best_practices table/column comments to explicitly
  document that id and check_id are unique identifiers

* chore: allow CLAUDE.md to be tracked by git

Remove CLAUDE.md from .gitignore so it is versioned alongside the codebase.

* refactor(SFE-4539): co-locate table/column comments with table creation

Move all UC Genie column/table comments from the monolithic
apply_schema_comments() into the function that creates each table,
so documentation lives next to the DDL that defines it.

- Each create_*() in common.py now calls _set_table_comment +
  _set_column_comments immediately after its CREATE TABLE DDL
- readBestPracticesConfigsFile() and load_sat_dasf_mapping() set
  comments inside their saveAsTable blocks
- apply_schema_comments() deleted; callers in initialize.py removed
- Schema-level COMMENT ON SCHEMA moved into create_schema()
- BrickHound: get_vertex_schema() and get_edge_schema() in schema.py
  get inline COMMENT clauses on every column
- New GraphSchema.get_metadata_schema() DDL with inline column comments
  called in permission_analysis_data_collection.py before the first
  brickhound_collection_metadata saveAsTable, fixing the timing bug
  where BrickHound table comments were silently skipped at init time

* fix(SFE-4539): guard GraphSchema call when brickhound not installed

When brickhound is not installed the try/except ImportError block leaves
GraphSchema undefined, causing a NameError at the get_metadata_schema() call
added in the SFE-4539 refactor.  Fix: set GraphSchema = None in the except
branch and guard the spark.sql() call with `if GraphSchema is not None`.

When brickhound IS installed the UC column comments are applied as before.
When it is not installed the metadata table is still created via saveAsTable
without column comments, restoring the original works-with-or-without contract.

* feat(SFE-4539): add UC table/column comments for brickhound tables

Replace unreliable GraphSchema DDL approach with explicit ALTER TABLE
ALTER COLUMN COMMENT calls after each saveAsTable, mirroring the SAT
pattern used in common.py. Comments now apply regardless of whether
brickhound is installed and work on pre-existing tables.

Tables covered: brickhound_vertices (14 cols), brickhound_edges (8 cols),
brickhound_collection_metadata (10 cols).

* feat(SFE-4548): add INFO-42 Git repository allowlist check

- Add check id=113, check_id=INFO-42 to security_best_practices.csv
- Implement enableProjectsAllowList rule in workspace_settings.py
- Add DASF-52 mapping to sat_dasf_mapping.csv
- Append Phase 5 (bugs) and Phase 6 (new check backlog) to sat_checks_audit.md

* feat(SFE-4548): include Git URL allowlist entries in INFO-42 check details

When enableProjectsAllowList passes, also return the projectsAllowList
comma-separated URL prefixes in additional details so reviewers can verify
the allowlist is properly scoped. Add projectsAllowList to ws_keymap and
expand the SQL/rule function to fetch both keys. Rebuild wheel 0.1.41.

* feat(SFE-4548): add /add-sat-check Claude skill and expand checks audit doc

- Add .claude/commands/add-sat-check.md: project-level skill that guides
  end-to-end implementation of a new SAT security check (CSV → SDK →
  notebook check block → DASF mapping → validations)
- Expand docs/sat_checks_audit.md with INFO-42 additional-details section
  and Phase 7 planned checks (NS-12, IA-10, GOV-44, NS-13)

* fix: correct typo 'respones' -> 'responses' in notebook header comments

* chore: remove sat_checks_audit.md from branch

* feat(SFE-4549): remove 24 unrelated checks and self-assessment functionality

- Remove 24 checks from security_best_practices.csv: DP-4, GOV-1/6/7/8/9/13/23/24/26, IA-1/2/3/7, INFO-1/2/4/7/12/13/14/17, NS-1/2
- Remove corresponding DASF mapping entries and self_assessment_checks.yaml
- Delete self-assessment notebook (Setup/9) and its two functions in sat_checks_config.py
- Remove schema fields: object_storage_encrypted, vpc_peering_done, table_access_control_enabled, sso_enabled, scim_enabled from account_workspaces DDL, CSV schema, drivers, setup notebooks, and config utilities
- Remove all check implementations from workspace_analysis.py and workspace_settings.py

* feat(SFE-4549): remove legacy Databricks SQL API EOL endpoints and bump SDK to 0.1.42

Remove deprecated /api/2.0/sql/alerts, /sql/queries, /sql/config/warehouses,
/preview/sql/permissions, and /preview/sql/data_sources usages. Dead-code
bootstrap calls for dbsql_workspaceconfig and dbsql_alerts are dropped from
workspace_bootstrap.py. configure_alerts_template.py now uses the warehouse ID
directly as data_source_id. SDK version bumped 0.1.41 -> 0.1.42 with rebuilt wheel.

* feat(SFE-4549): remove alerts feature and drop alert column from schema

- Delete notebooks/Setup/6. configure_alerts_template.py (deprecated SQL API endpoints at EOL)
- Remove dangling references to notebooks 6 and 9 from security_analysis_initializer.py
- Drop alert column from configs/security_best_practices.csv (all 55 rows had alert=0)
- Remove alert from security_best_practices Delta table schema in common.py (schema_list, DDL, select, column comments)
- Remove alert widget and UPDATE SQL field from sat_checks_config.py

* fix(SFE-4549): NS-9 — treat all DRY_RUN modes and unknown enforcement as violations

Only ENFORCED passes. Selective dry-run (non-empty product filter) now returns
DRY_RUN_SELECTIVE violation instead of passing. Unknown/missing enforcement_mode
now returns UNKNOWN_ENFORCEMENT_MODE violation instead of passing.

* fix: escape single quotes in schema comment helpers to prevent SQL parse errors

_set_table_comment and _set_column_comments now escape single quotes in comment
strings before interpolating into SQL. Also removed embedded single-quoted
examples from the additional_details column comment that triggered the error.

* fix: correct SQL single-quote escaping in comment helpers and insertIntoInfoTable

- _set_table_comment/_set_column_comments: use standard SQL '' escaping instead
  of backslash escaping (which is unreliable in Spark SQL)
- insertIntoInfoTable: escape name and category before SQL interpolation
  (was already escaping jsonstr but not the other string fields)

* fix: remove pre-escaped single quotes from schema comment strings

Comment strings passed to _set_table_comment/_set_column_comments were using
SQL-style '' escaping manually, which the helper then doubled again to '''',
causing PARSE_SYNTAX_ERROR. Replaced all ''word'' patterns with plain text.
Affected: account_info.category, account_workspaces table comment,
sat_dasf_mapping table and dasf_control_id column comments.

* removing manual config text from dashboard

* widget name

* chore: bump sat_version to 0.7.0 in initialize.py

---------

Co-authored-by: shdzhang <39942190+shdzhang@users.noreply.github.com>
Co-authored-by: Kiran Anand <16294307+kiran-anand@users.noreply.github.com>
Co-authored-by: Shreel Shah <shreelshah12@gmail.com>

v0.6.0

[0.6.0]

• SAT Permissions Analysis — New graph-based permissions analysis tool integrated into SAT. Collects all Databricks objects and permissions across account and workspaces. Includes a Databricks web app.
• Serverless Egress Control security checks — New checks NS-9 evaluating workspace network policies and serverless egress controls across AWS, Azure, and GCP.
• GOV-37: Disable Legacy Features check — New account-level security check detecting whether legacy features are disabled.
• Cluster Config Secrets Scanning — Extended the secrets scanner to scan cluster environment variables for hardcoded secrets using TruffleHog. Results unified with notebook scanning in the dashboard.
• Government / Staging cloud support — Accounts console authentication now supports gov cloud, DoD, and staging environments.
• Centralized SDK distribution — SAT SDK wheel now stored in lib/ directory for reliable notebook installation across all compute types.

v0.5.0

Key Updates:

• Fully revamped Lakeview SAT dashboards
• New accounts console IP allow list check
• Brand new Secret Scanning feature for Databricks Notebooks
• Updated documentation and clarification instructions
• Bug fixes

v0.4.0

Release notes:

• Simplified GCP setup with Service principle ( similar to AWS, no more key files for OIDC etc)
• Improved library with REST API pagination
• AWS Gov Cloud support
• SAT tagging for deployed jobs

Bug fixes

• Azure diagnostics logs check issue fix
• Azure account API checks fix
• Improved documentation

v0.3.4

Release notes:

• Improved Lakeview dashboard for SAT reports (legacy dashboard is no longer supported)
• Enhanced diagnostic notebooks
• Improved serverless performance
• Added current SAT version to TF branch and diagnostic logs
• Updated documentation and clarification instructions
• Updated TF files with clarification instructions
• Bug fixes

v0.3.3

Release notes

• Support SAT on Serverless compute deployment option with DABs and TF
• Account level Lakeview dashboard and improved Lakeview dashboard for workspaces.
• Updated documentation and clarification instructions
• Updated TF files with clarification instructions
• Self diagnostics to detect misconfigurations included in the workflow to detect configuration errors.
• Bug fixes.

v0.3.2

Release notes

• Simplified deployment with DABs and TF
• Improved setup instructions and documentation
• Removed dependency on PAT
• UC and hive support
• New Lakeview dashboard report
• Self diagnostics to detect setup misconfigurations
• 10 new checks (ex: System table audit logs, Model endpoints, Restrict workspace admins)
• Mapped to Security Best Practices docs (Thanks to Andy) to SAT
• Removed stale checks (to align with product updates)
• Many bug fixes (including the issues caused due to product drift) (Thanks to Laxmi for surfacing them at customer)
• 3 SSPMs adopted SAT inspired security checks

v0.3.1

Release Notes

• Feature/sfe 3348 testing and fixes for msx
• Feature/sfe 3041 azure sub doc
• Force clusters to use SINGLE_USER as Security mode

Full Changelog: v0.3.0...v0.3.1

v0.3.0

Release Notes

• Removal of PAT dependency
• Configurable schema name
• Optional proxy
• Disabled old checks
• Clarified a few mutually exclusive checks
• Documentation updates
• Bug fix related to best practice manual disablement and override

Full Changelog: v0.2.0...v0.3.0