Skip to content

Create Bronze-to-Silver Pipeline Template for Snowflake #1221

@terzioglub

Description

@terzioglub

Create a bronze-to-silver pipeline template using Snowflake as the destination. This template will use either frankfurter or chess as the data source (since they don't require credentials) and showcase a complete ingestr -> transformation workflow with Snowflake. This template should be able to initialized with bruin init template-name
more about bruin init on : https://getbruin.com/docs/bruin/commands/init.html

Template Structure

Bronze Layer (Ingestr Asset):

  • Raw data ingestion using ingestr from frankfurter or chess API
  • No transformations, just data extraction to Snowflake

Silver Layer (SQL Asset):

  • Aggregates and cleans bronze data from Snowflake tables
  • Demonstrates typical transformation patterns

Implementation

Create templates/bronze-silver-snowflake/ with:

  • Bronze asset: assets/bronze_raw_data.asset.yml (ingestr from frankfurter/chess)
  • Silver asset: assets/silver_aggregated.sql (Snowflake SQL transformation)

Acceptance Criteria

  • Template created in templates/bronze-silver-snowflake/
  • Bronze layer uses ingestr asset to extract data from frankfurter or chess (no credentials needed)
  • Silver layer aggregates bronze data with Snowflake SQL transformations
  • README with Snowflake setup instructions
  • Works with bruin init bronze-silver-snowflake
  • bruin validate passes successfully
  • Assets have meaningful quality checks (column checks, custom checks)
  • bruin run executes the full pipeline successfully

Reference Templates

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions