Update based on new templates and add a script for future updates (#93)

This updates to the latest templates, adds
`lakeflow-pipelines-`{`python`,`sql`}`, and a script for future updates
under `scripts/`.

Reviewers, please review the script + whether references to personal
addresses like `lennart.kats@databricks.com` are properly scrubbed.

Author: lennartkats-db

.ruff.toml

@@ -5,4 +5,5 @@ exclude = [
     "default_python/*",
     "default_sql/*",
     "mlops_stacks/*",
+    "lakeflow_pipelines_python/*",
 ]

contrib/data_engineering/README.md

@@ -1,20 +1,6 @@
 # data_engineering
 
-The 'data_engineering' project was generated by using the contrib/data-engineering template.
-
-Learn more about this template here:
-
-https://github.com/databricks/bundle-examples/tree/main/contrib/templates/data-engineering
-
-You can re-create this bundle by running the following commands:
-
-```
-$ databricks bundle init https://github.com/databricks/bundle-examples --template-dir contrib/templates/data-engineering
-# (answer prompts, call the project data_engineering)
-$ cd data_engineering
-$ uv run add-asset
-# (select etl-pipeline)
-```
+The 'data_engineering' project was generated by using the data-engineering template.
 
 ## Setup
 

contrib/data_engineering/assets/etl_pipeline/__init__.py

@@ -26,21 +26,21 @@ targets:
     workspace:
       host: https://company.databricks.com
     variables:
-      catalog: main
+      catalog: catalog
       schema: ${workspace.current_user.short_name}
       notifications: []
   prod:
     mode: production
     workspace:
       host: https://company.databricks.com
-      # We explicitly specify /Workspace/Users/user@databricks.com to make sure we only have a single copy.
-      root_path: /Workspace/Users/user@databricks.com/.bundle/${bundle.name}/${bundle.target}
+      # We explicitly specify /Workspace/Users/user@company.com to make sure we only have a single copy.
+      root_path: /Workspace/Users/user@company.com/.bundle/${bundle.name}/${bundle.target}
     permissions:
-      - user_name: user@databricks.com
+      - user_name: user@company.com
         level: CAN_MANAGE
     run_as:
-      user_name: user@databricks.com
+      user_name: user@company.com
     variables:
-      catalog: main
+      catalog: catalog
       schema: default
-      notifications: [user@databricks.com]
+      notifications: [user@company.com]