Skip to content

README.md

Latest commit

 

History

History
156 lines (102 loc) · 4.43 KB

README.md

File metadata and controls

156 lines (102 loc) · 4.43 KB

DataCrunch Python SDK

The official DataCrunch.io Python SDK.

The SDK's documentation is available on ReadTheDocs

DataCrunch's Public API documentation is available here.

Getting Started - Using the SDK:

  • Install:

    # via pip
pip3 install datacrunch

# via uv
uv add datacrunch

  • Generate your client credentials - instructions in the public API docs.

  • Add your client id and client secret to an environment variable (don't want it to be hardcoded):

    Linux (bash):

    export DATACRUNCH_CLIENT_ID=YOUR_ID_HERE
export DATACRUNCH_CLIENT_SECRET=YOUR_SECRET_HERE

  • To enable sending inference requests from SDK you must generate an inference key - Instructions on inference authorization

  • Add your inference key to an environment variable

    Linux (bash):

    export DATACRUNCH_INFERENCE_KEY=YOUR_API_KEY_HERE

    Other platforms: https://en.wikipedia.org/wiki/Environment_variable

  • Example for creating a new instance:

    import os
from datacrunch import DataCrunchClient

# Get credentials from environment variables
CLIENT_ID = os.environ.get('DATACRUNCH_CLIENT_ID')
CLIENT_SECRET = os.environ['DATACRUNCH_CLIENT_SECRET']

# Create datcrunch client
datacrunch = DataCrunchClient(CLIENT_ID, CLIENT_SECRET)

# Get all SSH keys
ssh_keys = [key.id for key in datacrunch.ssh_keys.get()]

# Create a new instance
instance = datacrunch.instances.create(instance_type='1V100.6V',
                                      image='ubuntu-24.04-cuda-12.8-open-docker',
                                      ssh_key_ids=ssh_keys,
                                      hostname='example',
                                      description='example instance')

# Delete instance
datacrunch.instances.action(instance.id, datacrunch.constants.instance_actions.DELETE)

    More examples can be found in the /examples folder or in the documentation.

Development

Set up the local development environment

Prerequisite: install uv.

Clone the repository, create local environment and install dependencies:

git clone git@github.com:DataCrunch-io/datacrunch-python.git
cd datacrunch-python
uv sync

Run Tests

  • Execute all tests

    uv run pytest

  • Execute a single test file

    uv run pytest tests/unit_tests/test_file.py

Local Manual Testing

Create a file in the root directory of the project:

# example.py
from datacrunch.datacrunch import DataCrunchClient

CLIENT_SECRET = 'secret'
CLIENT_ID = 'your-id'

# Create datacrunch client
datacrunch = DataCrunchClient(CLIENT_ID, CLIENT_SECRET, base_url='http://localhost:3001/v1')

Run it:

uv run python example.py

Generating the documentation

If added a new service, create a documentation template under api/services for that service.

cd docs
make html

Code style

# Lint
uv run ruff check

# Format code
uv run ruff format

Contact

You can contact us here, or open an issue in the repo.