Skip to content

overview.mdx

Latest commit

 

History

History
166 lines (118 loc) · 5.88 KB

overview.mdx

File metadata and controls

166 lines (118 loc) · 5.88 KB
title API Reference Overview

The Elementary Python SDK provides a simple API for sending data quality information to Elementary Cloud. This page provides an overview of the API structure and endpoints.

Getting Your API Credentials

Before initializing the client, you need to obtain your API credentials from Elementary Cloud.

Generate an Access Token

You can generate tokens directly from the Elementary UI:

  1. Go to User → Personal Tokens or Account → Account Tokens
  2. Click Generate token
  3. (Optional) Add a name/description for the token
  4. Copy the token and store it securely — it is shown only once

Security

  • User tokens are user-scoped bearer tokens and inherit your workspace permissions
  • Account tokens are account-scoped bearer tokens with "Can View" permissions
  • Treat tokens like passwords — do not share or commit them to version control
  • Keep them secret, rotate regularly, and revoke immediately if compromised

For more details, see the MCP Setup Guide which uses the same token generation process.

Client Initialization

The SDK uses ElementaryCloudClient to send data to Elementary Cloud:

from elementary_python_sdk.core.cloud.cloud_client import ElementaryCloudClient

client = ElementaryCloudClient(project_id, api_key, url)

Where:

  • project_id is your Python project identifier (chosen by you, used to deduplicate and identify reported assets)
  • api_key is your API token (generated from the steps above)
  • url is the full SDK ingest endpoint URL (the Elementary team will provide you with this URL): {base_url}/sdk-ingest/{env_id}/batch
    • Example: https://app.elementary-data.com/sdk-ingest/a6b2425d-36e2-4e13-8458-9825688ca1f2/batch

Test Context

Tests are run within an elementary_test_context which automatically captures test results:

from elementary_python_sdk.core.tests import elementary_test_context
from elementary_python_sdk.core.types.asset import TableAsset

asset = TableAsset(name="users", database_name="prod", schema_name="public", table_name="users")

with elementary_test_context(asset=asset) as ctx:
    # Run your tests here
    # Results are automatically captured in ctx
    client.send_to_cloud(ctx)

raise_on_error

By default, elementary_test_context uses raise_on_error=False. This means that if a decorated test (or something inside the context) raises an exception, the SDK captures it and records an ERROR execution so you can still send results to Elementary Cloud without crashing your pipeline.

If you prefer fail-fast behavior (for example in CI), pass raise_on_error=True to re-raise exceptions after they are recorded:

with elementary_test_context(asset=asset, raise_on_error=True) as ctx:
    run_my_tests(df)

Test Decorators

The SDK provides decorators to define tests:

  • @boolean_test - For tests that return True/False (pass/fail)
  • @expected_range - For tests that return numeric values within a range
  • @expected_values - For tests that return values matching a list of expected values
  • @row_count - For tests that return a Sized object (DataFrame, list, etc.) to check row count

Context Manager Approach

You can also use context managers for inline tests:

with elementary_test_context(asset=asset) as ctx:
    # Using context managers
    with ctx.boolean_test(name="my_test", description="Inline test") as my_bool_test:
        my_bool_test.assert_value(my_test_function())
    
    with ctx.expected_values_test(
        name="country_count",
        expected=[2, 3],
        allow_none=True,
        metadata={"my_metadata_field": "my_metadata_value"},
    ) as my_expected_values_test:
        # This will fail
        my_expected_values_test.assert_value(5)
        # This will pass
        my_expected_values_test.assert_value(3)
    
    with ctx.expected_range_test(
        name="age_range",
        min=18,
        max=50,
    ) as my_range_test:
        my_range_test.assert_value(25.5)
    
    with ctx.row_count_test(
        name="row_count",
        min=1,
        max=1000,
    ) as my_row_count_test:
        my_row_count_test.assert_value(users_df)  # Passes DataFrame, list, etc.

Supported Objects

The SDK supports reporting table assets and test results.

Register tables and views in your data warehouse Define data quality tests using decorators

Sending Results

After running tests in a context, send results to Elementary Cloud:

client.send_to_cloud(ctx)

This automatically batches all test results from the context and sends them in a single request.

Error Handling

The SDK handles errors automatically, but you can wrap calls in try-except blocks:

try:
    client.send_to_cloud(ctx)
except Exception as e:
    print(f"Error sending results: {e}")

Best Practices

  • Run multiple tests in one context - All tests in a single elementary_test_context are automatically batched
  • Use descriptive test names - Clear names help identify tests in the Elementary UI
  • Include asset metadata - Add descriptions, owners, tags, and dependencies to assets
All tests run within a single `elementary_test_context` are automatically batched and sent together.

Next Steps