Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,15 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Set up Python
uses: actions/setup-python@v5
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.11'

- name: Install uv
uses: astral-sh/setup-uv@v4
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0

- name: Install dependencies
run: uv sync --extra dev
Expand All @@ -32,7 +32,7 @@ jobs:
run: uv run pytest tests/ -m "not integration" -v --cov=src --cov-report=xml

- name: Upload coverage report
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: always()
with:
name: coverage-report
Expand All @@ -43,15 +43,15 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Set up Python
uses: actions/setup-python@v5
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.11'

- name: Install uv
uses: astral-sh/setup-uv@v4
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0

- name: Install dependencies
run: uv sync --extra dev
Expand All @@ -67,15 +67,15 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Set up Python
uses: actions/setup-python@v5
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.11'

- name: Install uv
uses: astral-sh/setup-uv@v4
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0

- name: Install dependencies
run: uv sync --extra dev
Expand Down
35 changes: 35 additions & 0 deletions .github/workflows/dockerhub-description.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
name: Docker Hub Description

# Syncs the Docker Hub repository overview + short description from
# docs/dockerhub-overview.md. Decoupled from releases so the page updates
# whenever the overview changes (or on demand), not just when a version ships.
on:
push:
branches: [main]
paths:
- docs/dockerhub-overview.md
- .github/workflows/dockerhub-description.yml
workflow_dispatch:

permissions:
contents: read

jobs:
sync:
name: Sync Docker Hub description
runs-on: ubuntu-latest
# Requires the DOCKERHUB_USERNAME and DOCKERHUB_TOKEN repo secrets (token
# needs Read & Write). If Docker Hub rejects a PAT for the description API,
# supply an account password secret instead.
steps:
- name: Checkout code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Sync Docker Hub description
uses: peter-evans/dockerhub-description@1b9a80c056b620d92cedb9d9b5a223409c68ddfa # v5.0.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
repository: lunarcommand/audio-refinery
short-description: "GPU audio pipeline: separation, diarization, WhisperX transcription, sentiment — HTTP service"
readme-filepath: ./docs/dockerhub-overview.md
28 changes: 14 additions & 14 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,15 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Set up Python
uses: actions/setup-python@v5
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.11'

- name: Install uv
uses: astral-sh/setup-uv@v4
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0

- name: Install dependencies
run: uv sync --extra dev
Expand All @@ -37,10 +37,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Set up Python
uses: actions/setup-python@v5
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.11'

Expand All @@ -51,7 +51,7 @@ jobs:
run: python -m build

- name: Upload artifacts
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: dist
path: dist/
Expand All @@ -64,16 +64,16 @@ jobs:
contents: write
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Download artifacts
uses: actions/download-artifact@v4
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: dist
path: dist/

- name: Create Release
uses: softprops/action-gh-release@v1
uses: softprops/action-gh-release@b4309332981a82ec1c5618f44dd2e27cc8bfbfda # v3.0.0
with:
files: dist/*
draft: false
Expand All @@ -90,7 +90,7 @@ jobs:
# this job fails at the login step; the GitHub release above still succeeds.
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Free disk space
# The CUDA image is ~10GB built; standard runners need room cleared.
Expand All @@ -100,25 +100,25 @@ jobs:
df -h

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0

- name: Login to Docker Hub
uses: docker/login-action@v3
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Compute image tags
id: meta
uses: docker/metadata-action@v5
uses: docker/metadata-action@c299e40c65443455700f0fdfc63efafe5b349051 # v5.10.0
with:
images: lunarcommand/audio-refinery
tags: |
type=semver,pattern={{version}}
type=raw,value=latest,enable=${{ !contains(github.ref, '-') }}

- name: Build and push
uses: docker/build-push-action@v6
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6.19.2
with:
context: .
push: true
Expand Down
21 changes: 10 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,17 +88,16 @@ CLI users should also review the [scratch directory](docs/cli.md#scratch-directo

Both entry points are thin callers around one shared pipeline core:

```
CLI (audio-refinery) Service (HTTP API)
one-shot / batch over a directory async jobs, URI-in / URI-out
\ /
\ /
v v
┌───────────────────────────────────────┐
│ core pipeline │
│ separate → diarize → transcribe → … │
│ (Demucs, Pyannote, WhisperX) │
└───────────────────────────────────────┘
```mermaid
flowchart TD
CLI["CLI — audio-refinery<br/>one-shot / batch over a directory"]
SVC["Service — HTTP API<br/>async jobs, URI-in / URI-out"]
CLI --> CORE
SVC --> CORE
subgraph CORE [core pipeline]
direction LR
SEP[separate] --> DIA[diarize] --> TRX[transcribe] --> SEN["sentiment (optional)"]
end
```

The CLI loads models per invocation; the service loads them once at container
Expand Down
19 changes: 10 additions & 9 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -217,15 +217,16 @@ from the first unprocessed file. `--no-resume` forces full reprocessing.
The pipeline core (`run_pipeline()` and the stage modules) is driven by two thin
callers that share it unchanged:

```
CLI (audio-refinery) Service (HTTP API)
one-shot / batch over a directory async jobs, URI-in / URI-out
\ /
v v
┌───────────────────────────────────────┐
│ core pipeline (separate → diarize │
│ → transcribe → optional sentiment) │
└───────────────────────────────────────┘
```mermaid
flowchart TD
CLI["CLI — audio-refinery<br/>one-shot / batch over a directory"]
SVC["Service — HTTP API<br/>async jobs, URI-in / URI-out"]
CLI --> CORE
SVC --> CORE
subgraph CORE [core pipeline]
direction LR
SEP[separate] --> DIA[diarize] --> TRX[transcribe] --> SEN["sentiment (optional)"]
end
```

- **CLI mode** loads models per invocation and processes a local directory. See
Expand Down
48 changes: 48 additions & 0 deletions docs/dockerhub-overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Audio Refinery

GPU-accelerated audio processing pipeline — vocal separation (Demucs), speaker
diarization (Pyannote), transcription (WhisperX), and sentiment analysis —
packaged as a long-lived HTTP service. Point it at audio by URI and get back
speaker-attributed transcripts with word-level timestamps as JSON. Models load
once at container startup and stay resident across jobs.

## Quick start

```bash
docker run --gpus all -p 8000:8000 \
-e REFINERY_API_KEYS=your-secret-key \
-e HF_TOKEN=hf_your_token \
lunarcommand/audio-refinery:latest
```

Wait for `GET /health` to return `200` (model warmup, ~10–30 s), then
`POST /transcribe` a batch of jobs. Full API and environment reference:
[Service Guide](https://github.com/LunarCommand/audio-refinery/blob/main/docs/service.md).

## Requirements

- NVIDIA GPU — **24 GB VRAM recommended** (holds all models resident)
- NVIDIA driver ≥ 525.85.12 and the NVIDIA Container Toolkit on the host
- A HuggingFace token for the gated Pyannote models (`HF_TOKEN`)

## Tags

- `latest` — most recent stable release
- `X.Y.Z` — pinned release (e.g. `0.2.0`)

## Key environment variables

| Variable | Purpose |
|----------|---------|
| `REFINERY_API_KEYS` | **required** — bearer-token allowlist |
| `HF_TOKEN` | **required** — HuggingFace token for diarization |
| `REFINERY_WHISPER_MODEL` | Whisper variant (default `large-v3`) |
| `REFINERY_SCRATCH_DIR` | per-job scratch; mount a tmpfs for throughput |

Full environment reference, CLI usage, and architecture:
**https://github.com/LunarCommand/audio-refinery**

## License

MIT. Pyannote model weights are gated under separate HuggingFace terms — verify
your account's accepted terms cover your use case.