Skip to content

DCM Part 4: DCM Projects for Tasks quickstart guide#3210

Merged
sfc-gh-ridasafdar merged 13 commits intoSnowflake-Labs:masterfrom
sfc-gh-yostrinsky:dcm-tasks-refactor
May 5, 2026
Merged

DCM Part 4: DCM Projects for Tasks quickstart guide#3210
sfc-gh-ridasafdar merged 13 commits intoSnowflake-Labs:masterfrom
sfc-gh-yostrinsky:dcm-tasks-refactor

Conversation

@sfc-gh-yostrinsky
Copy link
Copy Markdown
Contributor

Covers task graphs, finalizer notifications, DMF quality gates, serverless alerts — all managed via DCM Plan & Deploy.

.... Generated with Cortex Code

Covers task graphs, finalizer notifications, DMF quality gates,
serverless alerts — all managed via DCM Plan & Deploy.

.... Generated with [Cortex Code](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code)

Co-Authored-By: Cortex Code <noreply@snowflake.com>
@github-actions
Copy link
Copy Markdown
Contributor

🔗 Staging Preview Ready

Quickstart: dcm-projects-for-tasks
Language: en

👉 Preview URL: https://publish-p57963-e462098.adobeaemcloud.com/en/developers/guides/dcm-projects-for-tasks-70a3f9c825cad13635da0c3b27ee90c60f04b9aa

Note: Images are uploaded asynchronously. If images are missing, wait a few minutes and refresh.


Generated by GitHub Actions

@github-actions
Copy link
Copy Markdown
Contributor

🔗 Staging Preview Ready

Quickstart: dcm-projects-for-tasks
Language: en

👉 Preview URL: https://publish-p57963-e462098.adobeaemcloud.com/en/developers/guides/dcm-projects-for-tasks-aa440fcf7cf5bdc57f075f02fd46d04865e99883

Note: Images are uploaded asynchronously. If images are missing, wait a few minutes and refresh.


Generated by GitHub Actions

- Add OVERLAP_POLICY mention on root task and in feature tour
- Add EXECUTE TASK USING CONFIG tip in post-deploy section
- TASK_9 row explains retries inherited from root task
- Switch finalizer description from HTML to plain-text JSON
- Replace custom UDMF + Python UDF with 3 system DMFs
- Remove unused alert_schedule from manifest snippet
- Add OVERLAP_POLICY and USING CONFIG to Related Resources

.... Generated with [Cortex Code](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code)

Co-Authored-By: Cortex Code <noreply@snowflake.com>
@github-actions
Copy link
Copy Markdown
Contributor

🔗 Staging Preview Ready

Quickstart: dcm-projects-for-tasks
Language: en

👉 Preview URL: https://publish-p57963-e462098.adobeaemcloud.com/en/developers/guides/dcm-projects-for-tasks-1515be4ebbe4e254441526381a5ed9936297d695

Note: Images are uploaded asynchronously. If images are missing, wait a few minutes and refresh.


Generated by GitHub Actions

…on table

- Add EXECUTE MANAGED ALERT grant to pre-deploy snippet
- Add email verification section (step 8)
- Fix INFORMATION_SCHEMA to fully-qualified in alert snippet
- Add callout explaining dual email config (manifest vs alert)
- Replace single inbox paragraph with two-email notification table
- Add EXECUTE ALERT mention for on-demand testing
- Insert select_project.png, plan_results.png, tasks_dag.png assets

.... Generated with [Cortex Code](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code)

Co-Authored-By: Cortex Code <noreply@snowflake.com>
@github-actions
Copy link
Copy Markdown
Contributor

🔗 Staging Preview Ready

Quickstart: dcm-projects-for-tasks
Language: en

👉 Preview URL: https://publish-p57963-e462098.adobeaemcloud.com/en/developers/guides/dcm-projects-for-tasks-782c43e930721df20d4a40bd1eeb1e99db8d2f56

Note: Images are uploaded asynchronously. If images are missing, wait a few minutes and refresh.


Generated by GitHub Actions

Copy link
Copy Markdown
Collaborator

@sfc-gh-jsommerfeld sfc-gh-jsommerfeld left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice job! I see you pimped my original DEMO DAG even more :)

please remove the PrPr callouts. procedure, target-state and alert are coming to PuPr with 10.15 this week

Comment thread site/sfguides/src/dcm-projects-for-tasks/dcm-projects-for-tasks.md Outdated
Comment thread site/sfguides/src/dcm-projects-for-tasks/dcm-projects-for-tasks.md Outdated
Comment thread site/sfguides/src/dcm-projects-for-tasks/dcm-projects-for-tasks.md Outdated
Comment thread site/sfguides/src/dcm-projects-for-tasks/dcm-projects-for-tasks.md Outdated
Comment thread site/sfguides/src/dcm-projects-for-tasks/dcm-projects-for-tasks.md
Comment thread site/sfguides/src/dcm-projects-for-tasks/dcm-projects-for-tasks.md Outdated
Comment thread site/sfguides/src/dcm-projects-for-tasks/dcm-projects-for-tasks.md Outdated
<!-- ------------------------ -->
## Post-Deploy: Stream, Alert, and a Manual Run

Streams and alerts are not yet supported as DCM `DEFINE` statements. Those pieces live in `scripts/02_post_deploy.sql`. (The DMF attachments themselves are already in DCM — see `expectations.sql` above.)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

→ "Streams are not yet supported..."

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will they be supported by 10.15 as well?

Comment thread site/sfguides/src/dcm-projects-for-tasks/dcm-projects-for-tasks.md Outdated
Comment thread site/sfguides/src/dcm-projects-for-tasks/dcm-projects-for-tasks.md
- Add Charlie Hammond and Jan Sommerfeld as authors
- Add platform category
- Rewrite DAG intro (explain "acyclic" in-line)
- Soften language: "becomes much easier to maintain as a DCM Project"
- Remove LIMITEDACCESS from doc links, drop Private Preview callout
- Expand STARTED | SUSPENDED explanation as DCM-only DEFINE TASK extension
- Rename DCM project to DCM_TASKS_PROJECT_DEV (avoid clash)
- Remove "early-access" from DEFINE PROCEDURE description
- Move failed-task alert into DCM project as DEFINE ALERT with STARTED
- Update Post-Deploy section title and description (streams only)
- Update conclusion bullet to reflect DCM-managed alert
- Replace manual DROP cleanup with EXECUTE DCM PROJECT PURGE
- Add What's Next CTA to advanced DCM docs

.... Generated with [Cortex Code](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code)

Co-Authored-By: Cortex Code <noreply@snowflake.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 1, 2026

🔗 Staging Preview Ready

Quickstart: dcm-projects-for-tasks
Language: en

👉 Preview URL: https://publish-p57963-e462098.adobeaemcloud.com/en/developers/guides/dcm-projects-for-tasks-cbfbd829cbaceaa033d93e3e7a861817cf2e66b5

Note: Images are uploaded asynchronously. If images are missing, wait a few minutes and refresh.


Generated by GitHub Actions

@sfc-gh-yostrinsky sfc-gh-yostrinsky changed the title [Draft] Add Part 4: DCM Projects for Tasks quickstart guide DCM Part 4: DCM Projects for Tasks quickstart guide May 1, 2026
@sfc-gh-yostrinsky
Copy link
Copy Markdown
Contributor Author

@sfc-gh-jsommerfeld I pushed a new version, but let's wait for TARGET STATE to be public before publishing this. In the meantime, tell me if you have any feedback on the language

.... Generated with [Cortex Code](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code)

Co-Authored-By: Cortex Code <noreply@snowflake.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 1, 2026

🔗 Staging Preview Ready

Quickstart: dcm-projects-for-tasks
Language: en

👉 Preview URL: https://publish-p57963-e462098.adobeaemcloud.com/en/developers/guides/dcm-projects-for-tasks-7f0902c320a868e3ce3f644c5dc4e635ade8e977

Note: Images are uploaded asynchronously. If images are missing, wait a few minutes and refresh.


Generated by GitHub Actions


In the [Get Started with Snowflake DCM Projects](https://www.snowflake.com/en/developers/guides/get-started-snowflake-dcm-projects/), [Build Data Pipelines with Snowflake DCM Projects](https://www.snowflake.com/en/developers/guides/build-data-pipelines-with-snowflake-dcm-projects/), and [DCM Projects for Dynamic Tables](https://www.snowflake.com/en/developers/guides/dcm-projects-for-dynamic-tables/) guides, you learned how DCM Projects manage Snowflake infrastructure declaratively — from a single project up to dynamic-table pipelines that evolve without full recomputation.

In this guide, you'll focus on **Tasks and task graphs** — the orchestration layer that drives scheduled and event-driven work across your pipeline.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tasks and Task graphs

You'll define a fifteen-task demo graph that shows off every interesting Task feature in one place:

- **Root task with retries, overlap policy, and graph-level config** pushed into children via `SYSTEM$GET_TASK_GRAPH_CONFIG`
- **Finalizer task** that emails a plain-text JSON summary of every graph run
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

even when it failed mid-way

- **Serverless task** and **multi-predecessor task** patterns
- **Stream-conditional** and **return-value-conditional** child tasks
- **Failing task with retries** + dependent that gets skipped
- **DCM-managed target state** — a task deployed as `SUSPENDED` that never runs
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer to focus on the positive here 😉 target-state allows to deploy Tasks STARTED.

If you don't already have one handy for running DCM itself:

```sql
CREATE WAREHOUSE IF NOT EXISTS dcm_wh
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we can skip this one, if the Tasks run on the WH_4 below?

- DEFINE ALERT stays in sources/definitions/alerts.sql (without STARTED)
- Alert deploys suspended; 02_post_deploy.sql resumes it once
- Clarify alert vs finalizer as complementary (per Jan's review)
- Update conclusion bullet to say 'task in the database' (broader scope)

.... Generated with [Cortex Code](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code)

Co-Authored-By: Cortex Code <noreply@snowflake.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 5, 2026

🔗 Staging Preview Ready

Quickstart: dcm-projects-for-tasks
Language: en

👉 Preview URL: https://publish-p57963-e462098.adobeaemcloud.com/en/developers/guides/dcm-projects-for-tasks-1bbe75cf31aeb8ab06fc5bca881210e5684add4a

Note: Images are uploaded asynchronously. If images are missing, wait a few minutes and refresh.


Generated by GitHub Actions

.... Generated with [Cortex Code](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code)

Co-Authored-By: Cortex Code <noreply@snowflake.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 5, 2026

🔗 Staging Preview Ready

Quickstart: dcm-projects-for-tasks
Language: en

👉 Preview URL: https://publish-p57963-e462098.adobeaemcloud.com/en/developers/guides/dcm-projects-for-tasks-f3d38b5826d0d62aa99deff1f20664244b709dbc

Note: Images are uploaded asynchronously. If images are missing, wait a few minutes and refresh.


Generated by GitHub Actions

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 5, 2026

🔗 Staging Preview Ready

Quickstart: dcm-projects-for-tasks
Language: en

👉 Preview URL: https://publish-p57963-e462098.adobeaemcloud.com/en/developers/guides/dcm-projects-for-tasks-43a0fd8a3f31d23bfc9db6e2a9e51da6559c5dc5

Note: Images are uploaded asynchronously. If images are missing, wait a few minutes and refresh.


Generated by GitHub Actions

… are now PuPr)

Addresses Jan's review comment: 'please remove. they are both promoted to PuPr this week'.

.... Generated with [Cortex Code](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code)

Co-Authored-By: Cortex Code <noreply@snowflake.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 5, 2026

🔗 Staging Preview Ready

Quickstart: dcm-projects-for-tasks
Language: en

👉 Preview URL: https://publish-p57963-e462098.adobeaemcloud.com/en/developers/guides/dcm-projects-for-tasks-2f02d6b26bd0fb159c1f23552f462b51f8ab4980

Note: Images are uploaded asynchronously. If images are missing, wait a few minutes and refresh.


Generated by GitHub Actions

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 5, 2026

🔗 Staging Preview Ready

Quickstart: dcm-projects-for-tasks
Language: en

👉 Preview URL: https://publish-p57963-e462098.adobeaemcloud.com/en/developers/guides/dcm-projects-for-tasks-c23a94db68b36c86b5d4c21f2ae685742cedf034

Note: Images are uploaded asynchronously. If images are missing, wait a few minutes and refresh.


Generated by GitHub Actions

…rget-state)

These three edits were lost during an earlier stash-pop cycle:
- "Tasks and task graphs" -> "Tasks and Task graphs" (line 17)
- Finalizer bullet: "... even when it failed mid-way"
- Positive target-state bullet framing

.... Generated with [Cortex Code](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code)

Co-Authored-By: Cortex Code <noreply@snowflake.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 5, 2026

🔗 Staging Preview Ready

Quickstart: dcm-projects-for-tasks
Language: en

👉 Preview URL: https://publish-p57963-e462098.adobeaemcloud.com/en/developers/guides/dcm-projects-for-tasks-0519d02f3e27ec40a0f03e7c89b67bb907c66f25

Note: Images are uploaded asynchronously. If images are missing, wait a few minutes and refresh.


Generated by GitHub Actions

@sfc-gh-yostrinsky sfc-gh-yostrinsky marked this pull request as ready for review May 5, 2026 20:56
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 5, 2026

🔗 Staging Preview Ready

Quickstart: dcm-projects-for-tasks
Language: en

👉 Preview URL: https://publish-p57963-e462098.adobeaemcloud.com/en/developers/guides/dcm-projects-for-tasks-6e3631800b1399117ce5389ad5ac89d8465a3aa1

Note: Images are uploaded asynchronously. If images are missing, wait a few minutes and refresh.


Generated by GitHub Actions

@sfc-gh-ridasafdar sfc-gh-ridasafdar merged commit 1a4d570 into Snowflake-Labs:master May 5, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants