Skip to content

Commit 5927b48

Browse files
dklibanclaude
dkliban
and
claude
authored
feat: add dev container with all Pulp services for AI agent development (#1062)
Adds a self-contained development container (dev-container/) that runs PostgreSQL 16, Redis, and all three Pulp services (api, content, worker) managed by supervisord. Includes helper scripts for patch management, service restarts, and testing. A GitHub Action builds and pushes the image to GHCR on every branch push. The /workspace volume follows alcove conventions for shared access between the Skiff container and the dev environment. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent e090b3e commit 5927b48

11 files changed

Lines changed: 577 additions & 0 deletions

File tree

‎.CLAUDE.md‎

Lines changed: 178 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,178 @@
1+
# Dev Container: hosted-pulp-dev-env
2+
3+
A self-contained development container with PostgreSQL, Redis, and all Pulp services running locally. Built automatically on every push to any branch.
4+
5+
## Image Location
6+
7+
```
8+
ghcr.io/pulp/hosted-pulp-dev-env:<branch-name>
9+
```
10+
11+
Branch names are sanitized for Docker tags: `/` is replaced with `-` (e.g., branch `feature/foo` produces tag `feature-foo`).
12+
13+
## Running the Container
14+
15+
Mount the shared workspace volume at `/workspace` per alcove conventions:
16+
17+
```bash
18+
docker run -d \
19+
-v workspace:/workspace \
20+
-p 24817:24817 \
21+
-p 24816:24816 \
22+
--name pulp-dev \
23+
ghcr.io/pulp/hosted-pulp-dev-env:main
24+
```
25+
26+
If the container finds a pulp-service checkout at `/workspace/pulp-service`, it installs it in development mode (`pip install -e`) at startup. Because `PULP_GUNICORN_RELOAD` is enabled by default, Gunicorn watches for Python file changes and automatically reloads -- you do not need to run `pulp-restart` after editing source files in most cases. See "Service Management" below for when a manual restart is required.
27+
28+
## Services
29+
30+
| Service | Port | Description |
31+
|------------- |-------|------------------------------------------|
32+
| pulp-api | 24817 | Django REST API (Gunicorn WSGI) |
33+
| pulp-content | 24816 | Async content delivery (Gunicorn aiohttp)|
34+
| pulp-worker | -- | Celery background task worker |
35+
| PostgreSQL | 5432 | Database (local, trust auth) |
36+
| Redis | 6379 | Cache and Celery broker |
37+
38+
All five services are managed by supervisord. The `pulp-restart` command only restarts Pulp services (api, content, worker), not PostgreSQL or Redis.
39+
40+
## Configuration
41+
42+
The dev container has domain support enabled (`DOMAIN_ENABLED=True`) and token authentication disabled (`TOKEN_AUTH_DISABLED=True`). This matches the hosted Pulp production configuration.
43+
44+
### Environment Variables
45+
46+
Override these at `docker run` time with `-e`:
47+
48+
| Variable | Default | Description |
49+
|----------|---------|-------------|
50+
| `PULP_DEFAULT_ADMIN_PASSWORD` | `password` | Admin user password set during initialization |
51+
| `PULP_API_WORKERS` | `2` | Number of Gunicorn workers for the API |
52+
| `PULP_CONTENT_WORKERS` | `2` | Number of Gunicorn workers for the content app |
53+
| `PULP_WORKERS` | `2` | Number of Celery worker processes |
54+
| `PULP_GUNICORN_TIMEOUT` | `90` | Gunicorn request timeout in seconds |
55+
| `PULP_GUNICORN_RELOAD` | `true` | Auto-reload Gunicorn on Python file changes |
56+
57+
## Commands
58+
59+
### Patch Management
60+
61+
Apply a patch to pulpcore's installed site-packages:
62+
63+
```bash
64+
pulp-add-patch /path/to/my-fix.patch
65+
```
66+
67+
Reverse a previously applied patch:
68+
69+
```bash
70+
pulp-remove-patch /path/to/my-fix.patch
71+
```
72+
73+
Patches target `/usr/local/lib/pulp/lib/python${PYTHON_VERSION}/site-packages` (default Python version: 3.11). After adding or removing a patch, you must restart services manually because patched files are in site-packages, not in the dev-installed editable source that Gunicorn watches:
74+
75+
```bash
76+
pulp-restart
77+
```
78+
79+
### Service Management
80+
81+
Restart all Pulp services:
82+
83+
```bash
84+
pulp-restart
85+
```
86+
87+
Restart a specific service:
88+
89+
```bash
90+
pulp-restart api
91+
pulp-restart content
92+
pulp-restart worker
93+
```
94+
95+
**When to restart manually:** You need `pulp-restart` after applying or removing patches, changing settings in `/etc/pulp/settings.py`, or modifying Celery task definitions (the worker does not auto-reload). For regular Python source edits with dev mode installed, Gunicorn auto-reload handles API and content restarts automatically.
96+
97+
Check service status (includes PostgreSQL and Redis):
98+
99+
```bash
100+
supervisorctl status
101+
```
102+
103+
View service logs:
104+
105+
```bash
106+
# stderr logs (application errors, Django output)
107+
tail -f /var/log/pulp/pulp-api-stderr.log
108+
tail -f /var/log/pulp/pulp-content-stderr.log
109+
tail -f /var/log/pulp/pulp-worker-stderr.log
110+
111+
# stdout logs (access logs, startup messages)
112+
tail -f /var/log/pulp/pulp-api-stdout.log
113+
tail -f /var/log/pulp/pulp-content-stdout.log
114+
tail -f /var/log/pulp/pulp-worker-stdout.log
115+
116+
# infrastructure logs
117+
tail -f /var/log/pulp/postgresql-stderr.log
118+
tail -f /var/log/pulp/redis-stderr.log
119+
```
120+
121+
### Running Tests
122+
123+
Run all functional tests (requires pulp-service source in workspace):
124+
125+
```bash
126+
pulp-test
127+
```
128+
129+
Run a specific test file or test:
130+
131+
```bash
132+
pulp-test pulp_service/pulp_service/tests/functional/test_authentication.py
133+
pulp-test pulp_service/pulp_service/tests/functional/test_authentication.py::TestClass::test_method
134+
```
135+
136+
By default, `pulp-test` looks for tests in `/workspace/pulp-service`. Override the source location by setting `PULP_SERVICE_DIR`:
137+
138+
```bash
139+
PULP_SERVICE_DIR=/workspace/my-fork pulp-test
140+
```
141+
142+
If the test directory is not found under `PULP_SERVICE_DIR`, the script falls back to `/tmp/pulp_service/pulp_service/tests`.
143+
144+
### Database Access
145+
146+
Connect to the local PostgreSQL database:
147+
148+
```bash
149+
runuser -u pulp -- psql -d pulp
150+
```
151+
152+
Run Django management commands:
153+
154+
```bash
155+
runuser -u pulp -- pulpcore-manager showmigrations
156+
runuser -u pulp -- pulpcore-manager shell
157+
```
158+
159+
## Admin Credentials
160+
161+
- **Username:** admin
162+
- **Password:** password (override with `PULP_DEFAULT_ADMIN_PASSWORD` env var at container start)
163+
164+
## Alcove Integration
165+
166+
When used with alcove, the `/workspace` volume is automatically shared between the Skiff container (running Claude Code) and this dev container. Repositories are cloned into `/workspace/<name>/`. The dev container image is declared in the alcove agent definition via `dev_container.image`.
167+
168+
## Architecture
169+
170+
The container mirrors the production three-service model:
171+
172+
1. **pulp-api** -- Gunicorn WSGI serving the Django REST API
173+
2. **pulp-content** -- Gunicorn with aiohttp worker for async content delivery
174+
3. **pulp-worker** -- Celery worker for background task processing
175+
176+
PostgreSQL and Redis run locally inside the same container (managed by supervisord) rather than as external services.
177+
178+
All production patches from `images/assets/patches/` are pre-applied at build time. The container is based on UBI9 (Red Hat Universal Base Image).

‎.dockerignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,4 @@
22
!/LICENSE
33
!/images
44
!/pulp_service
5+
!/dev-container

‎.github/workflows/build-dev-container.yml‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
name: Build Dev Container
2+
3+
on:
4+
push:
5+
branches:
6+
- '**'
7+
8+
permissions:
9+
contents: read
10+
packages: write
11+
12+
jobs:
13+
build-and-push:
14+
runs-on: ubuntu-latest
15+
env:
16+
REGISTRY: ghcr.io
17+
steps:
18+
- name: Checkout
19+
uses: actions/checkout@v4
20+
21+
- name: Set up Docker Buildx
22+
uses: docker/setup-buildx-action@v3
23+
24+
- name: Log in to GitHub Container Registry
25+
uses: docker/login-action@v3
26+
with:
27+
registry: ghcr.io
28+
username: ${{ github.actor }}
29+
password: ${{ secrets.GITHUB_TOKEN }}
30+
31+
- name: Extract branch name
32+
id: branch
33+
env:
34+
REF_NAME: ${{ github.ref_name }}
35+
run: |
36+
# Sanitize branch name for Docker tag (replace / with -)
37+
TAG=$(echo "$REF_NAME" | sed 's|/|-|g')
38+
echo "tag=$TAG" >> "$GITHUB_OUTPUT"
39+
40+
- name: Build and push
41+
uses: docker/build-push-action@v6
42+
with:
43+
context: .
44+
file: dev-container/Dockerfile
45+
push: true
46+
tags: ghcr.io/${{ github.repository_owner }}/hosted-pulp-dev-env:${{ steps.branch.outputs.tag }}
47+
cache-from: type=gha
48+
cache-to: type=gha,mode=max

‎dev-container/Dockerfile‎

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
FROM registry.access.redhat.com/ubi9/ubi
2+
3+
ARG PYTHON_VERSION=3.11
4+
ENV PYTHONUNBUFFERED=0
5+
ENV DJANGO_SETTINGS_MODULE=pulpcore.app.settings
6+
ENV PULP_SETTINGS=/etc/pulp/settings.py
7+
ENV PULP_GUNICORN_TIMEOUT=90
8+
ENV PULP_API_WORKERS=2
9+
ENV PULP_CONTENT_WORKERS=2
10+
ENV PULP_GUNICORN_RELOAD=true
11+
ENV PULP_WORKERS=2
12+
ENV PULP_HTTPS=false
13+
ENV PULP_STATIC_ROOT=/var/lib/operator/static/
14+
ENV PYTHON_VERSION=${PYTHON_VERSION}
15+
ENV PATH="/usr/local/lib/pulp/bin:${PATH}"
16+
COPY images/repos.d/centos9-crb.repo /etc/yum.repos.d/
17+
COPY images/repos.d/centos9-appstream.repo /etc/yum.repos.d/
18+
19+
RUN dnf -y install dnf-plugins-core && \
20+
dnf -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm && \
21+
dnf -y update
22+
23+
RUN dnf -y install \
24+
python${PYTHON_VERSION} python${PYTHON_VERSION}-cryptography \
25+
python${PYTHON_VERSION}-devel python${PYTHON_VERSION}-pip \
26+
openssl openssl-devel \
27+
wget git \
28+
lsof procps-ng \
29+
python${PYTHON_VERSION}-psycopg2 \
30+
redhat-rpm-config gcc \
31+
glibc-langpack-en \
32+
python${PYTHON_VERSION}-setuptools \
33+
swig \
34+
ostree-libs ostree --allowerasing --nobest && \
35+
dnf -y install patch jq zstd vim-minimal less findutils
36+
37+
RUN dnf -y install https://download.postgresql.org/pub/repos/yum/reporpms/EL-9-x86_64/pgdg-redhat-repo-latest.noarch.rpm && \
38+
dnf -y module disable postgresql && \
39+
dnf -y install postgresql16-server postgresql16 postgresql16-contrib && \
40+
dnf -y install redis && \
41+
dnf clean all
42+
43+
ENV PATH="/usr/pgsql-16/bin:${PATH}"
44+
ENV PGDATA=/var/lib/pgsql/16/data
45+
46+
RUN python${PYTHON_VERSION} -m venv --system-site-packages /usr/local/lib/pulp
47+
48+
RUN pip install --upgrade pip setuptools wheel && \
49+
pip install \
50+
rhsm setproctitle \
51+
"gunicorn>=22.0,<25.1.0" \
52+
python-nginx \
53+
"django-storages[boto3,azure]>=1.12.2" \
54+
"requests[use_chardet_on_py3]" \
55+
importlib-metadata watchtower \
56+
supervisor \
57+
&& rm -rf /root/.cache/pip
58+
59+
COPY pulp_service/ /tmp/pulp_service
60+
RUN pip install /tmp/pulp_service && rm -rf /root/.cache/pip
61+
62+
RUN groupadd -g 700 --system pulp && \
63+
useradd -d /var/lib/pulp --system -u 700 -g pulp pulp
64+
65+
RUN mkdir -p /etc/pulp/certs \
66+
/etc/ssl/pulp \
67+
/var/lib/operator/static \
68+
/var/lib/pgsql \
69+
/var/lib/pulp/assets \
70+
/var/lib/pulp/media \
71+
/var/lib/pulp/scripts \
72+
/var/lib/pulp/tmp \
73+
/workspace \
74+
/var/log/pulp
75+
76+
RUN chown pulp:pulp -R /var/lib/pulp /var/lib/operator/static /var/log/pulp
77+
78+
COPY images/assets/route_paths.py /usr/bin/route_paths.py
79+
COPY images/assets/wait_on_postgres.py /usr/bin/wait_on_postgres.py
80+
COPY images/assets/wait_on_database_migrations.sh /usr/bin/wait_on_database_migrations.sh
81+
COPY images/assets/set_init_password.sh /usr/bin/set_init_password.sh
82+
COPY images/assets/add_signing_service.sh /usr/bin/add_signing_service.sh
83+
COPY images/assets/pulp-api /usr/bin/pulp-api
84+
COPY images/assets/pulp-content /usr/bin/pulp-content
85+
COPY images/assets/pulp-resource-manager /usr/bin/pulp-resource-manager
86+
COPY images/assets/pulp-worker /usr/bin/pulp-worker
87+
COPY images/assets/log_middleware.py /usr/bin/log_middleware.py
88+
89+
USER pulp:pulp
90+
RUN PULP_STATIC_ROOT=/var/lib/operator/static/ PULP_CONTENT_ORIGIN=localhost \
91+
pulpcore-manager collectstatic --clear --noinput --link
92+
USER root:root
93+
94+
RUN ln -sf /usr/local/lib/pulp/bin/pulpcore-manager /usr/local/bin/pulpcore-manager
95+
96+
RUN chmod 2775 /var/lib/pulp/{scripts,media,tmp,assets} && \
97+
chown :root /var/lib/pulp/{scripts,media,tmp,assets}
98+
99+
COPY images/assets/patches/ /tmp/patches/
100+
RUN for patch_file in /tmp/patches/*.patch; do \
101+
echo "Applying $(basename $patch_file)..." && \
102+
patch -p1 -d /usr/local/lib/pulp/lib/python${PYTHON_VERSION}/site-packages < "$patch_file" || \
103+
echo "WARNING: Failed to apply $(basename $patch_file)"; \
104+
done && rm -rf /tmp/patches
105+
106+
RUN openssl rand -base64 32 > /etc/pulp/certs/database_fields.symmetric.key && \
107+
chown pulp:pulp /etc/pulp/certs/database_fields.symmetric.key && \
108+
chmod 600 /etc/pulp/certs/database_fields.symmetric.key
109+
110+
RUN mkdir -p /var/run/postgresql /var/lib/pgsql/16/data && \
111+
chown -R postgres:postgres /var/run/postgresql /var/lib/pgsql/16
112+
113+
RUN runuser -l postgres -c "/usr/pgsql-16/bin/initdb -D /var/lib/pgsql/16/data" && \
114+
echo "local all all trust" > /var/lib/pgsql/16/data/pg_hba.conf && \
115+
echo "host all all 127.0.0.1/32 trust" >> /var/lib/pgsql/16/data/pg_hba.conf && \
116+
echo "host all all ::1/128 trust" >> /var/lib/pgsql/16/data/pg_hba.conf
117+
118+
COPY dev-container/settings.py /etc/pulp/settings.py
119+
COPY dev-container/supervisord.conf /etc/supervisord.conf
120+
COPY dev-container/entrypoint.sh /entrypoint.sh
121+
COPY dev-container/scripts/ /usr/local/bin/
122+
RUN chmod +x /entrypoint.sh /usr/local/bin/pulp-*
123+
124+
VOLUME ["/workspace"]
125+
EXPOSE 24817 24816
126+
127+
ENTRYPOINT ["/entrypoint.sh"]

0 commit comments

Comments
 (0)