This guide will get you up and running with Tidy3D Python client development using Docker. The Docker workflow provides an isolated, reproducible development environment without affecting your system packages.
Quick Links
The Docker development environment provides:
- Isolation: Clean environment separate from your system
- Reproducibility: Everyone uses the same base environment
- Simplicity: No need to manage system dependencies
- Flexibility: Easy to reset and start fresh
What's included:
- Python 3.11 with
uvpackage manager - Poetry for dependency management
- AWS CLI for CodeArtifact access
- Neovim, git, pandoc for documentation
- Jupyter Lab for notebook development
- Operating System: Ubuntu 22.04+, macOS, or Windows with WSL2
- Docker Engine: Installed and running
- Git: With SSH keys configured for GitHub
- Disk Space: ~2GB for Docker image + container
# Check Docker is installed and running
docker --version
docker ps
# Check Git SSH access
ssh -T git@github.comIf you don't have Docker installed:
Ubuntu
# Install Docker Engine (NOT from snap)
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
# Add your user to docker group
sudo usermod -aG docker $USER
# Reboot to apply group changes
sudo reboot
# Verify installation
docker run hello-worldSee: Docker Ubuntu Install Guide
macOS
Download and install Docker Desktop from docker.com
Windows
Install Docker Desktop with WSL2 backend from docker.com
- GitHub: Access to flexcompute/tidy3d
- Tidy3D Account: Sign up at tidy3d.simulation.cloud
- API Key: Get yours from account page
This guide is for anyone setting up the development environment for the first time or migrating from a version after 2.9.0 when the dev.Dockerfile image was enabled.
# Clone the repository and navigate into it
git clone https://github.com/flexcompute/tidy3d.git
cd tidy3d
# Checkout develop branch or your feature branch
git checkout developBuild the development Docker image (~5 minutes):
# Build the development Docker image
docker build -t tidy3d-python-client-dev -f dev.Dockerfile .The image includes:
- Debian base with Python 3.11
uvfor fast package installation- Poetry for dependency management
- AWS CLI, git, pandoc, neovim
flexdaemonuser (UID 1000, GID 1000)
Create a persistent container with your local code mounted:
# Create a persistent container with your local code mounted
# This also maps port 8888 for tools like Jupyter Lab
docker container create --name=tidy3d_python_client_dev \
--userns=host \
-p 8888:8888 \
-v .:/home/flexdaemon/tidy3d \
tidy3d-python-client-devFlags explained:
--name: Container name for easy reference--userns=host: Use host user namespace (prevents permission issues)-p 8888:8888: Expose port 8888 for Jupyter Lab-v .:/home/flexdaemon/tidy3d: Mount current directory into container
# Start the container
docker start tidy3d_python_client_dev# Open an interactive shell inside the running container
docker exec -it tidy3d_python_client_dev /bin/bashYou're now inside the container at /home/flexdaemon!
Inside the container, create a virtual environment and install the package:
# (Inside Container) Navigate to the mounted tidy3d directory
cd tidy3d
# (Inside Container) Create a virtual environment and activate it
uv venv -p 3.11
source .venv/bin/activate
# (Inside Container) Install the project in editable mode with dev dependencies
uv pip install -e .[dev]Note
The virtual environment is created inside the mounted directory, so it persists between container restarts.
Configure your API key and verify everything is working:
# (Inside Container) Configure your API key
tidy3d configure --apikey=YOUR_API_KEY_HERE
# (Inside Container) Test the configuration
python -c "import tidy3d.web as web; web.test()"
# (Inside Container) Run tests
pytest tests/ -vIf tests pass, you're ready to develop!
Start the container (if not running):
# On host docker start tidy3d_python_client_dev docker exec -it tidy3d_python_client_dev bash
Activate virtual environment (inside container):
cd tidy3d source .venv/bin/activate
Make changes to the code on your host machine using your favorite editor (VSCode, PyCharm, etc.)
Changes are immediately reflected in the container since the directory is mounted.
Run tests (inside container):
# Run all tests pytest # Run specific test file pytest tests/test_simulation.py # Run with verbose output pytest -v -s
Commit changes (on host or in container):
git add . git commit -m "Your descriptive message" git push origin your-feature-branch
You can open multiple terminals into the same container:
# Terminal 1 (host)
docker exec -it tidy3d_python_client_dev bash
# Terminal 2 (host)
docker exec -it tidy3d_python_client_dev bash
# Terminal 3 (host)
docker exec -it tidy3d_python_client_dev bashEach terminal shares the same filesystem and running processes.
# Inside container with venv activated
# Run all tests
pytest
# Run tests in parallel (faster)
pytest -n auto
# Run specific test module
pytest tests/test_components.py
# Run tests matching pattern
pytest -k "test_simulation"
# Run with coverage
pytest --cov=tidy3d --cov-report=html
# View coverage: python -m http.server 8000 -d htmlcov# Inside container with venv activated
# Format code with ruff
ruff format .
# Check for linting issues
ruff check .
# Auto-fix linting issues
ruff check --fix .
# Run type checking
mypy tidy3d# Inside container with venv activated
cd docs
# Build HTML documentation
make clean
make html
# View the docs
cd _build/html
python -m http.server 8000
# Open http://localhost:8000 in your host browser# Start container
docker start tidy3d_python_client_dev
# Stop container (preserves state)
docker stop tidy3d_python_client_dev
# Restart container
docker restart tidy3d_python_client_dev# Check if container is running
docker ps
# Check all containers (including stopped)
docker ps -a
# View container logs
docker logs tidy3d_python_client_dev
# Follow container logs (live)
docker logs -f tidy3d_python_client_dev# Stop and remove container
docker stop tidy3d_python_client_dev
docker container rm tidy3d_python_client_dev
# Recreate container (preserves code on host)
docker container create --name=tidy3d_python_client_dev \
--userns=host \
-p 8888:8888 \
-v .:/home/flexdaemon/tidy3d \
tidy3d-python-client-devNote
Your code changes are safe! They're stored on your host in the mounted directory.
If dev.Dockerfile changes or you want to update base dependencies:
# Rebuild the image
docker build -t tidy3d-python-client-dev -f dev.Dockerfile .
# Remove old container and create new one
docker stop tidy3d_python_client_dev
docker container rm tidy3d_python_client_dev
docker container create --name=tidy3d_python_client_dev \
--userns=host \
-p 8888:8888 \
-v .:/home/flexdaemon/tidy3d \
tidy3d-python-client-devStart Jupyter Lab inside the container to work with notebooks:
# Inside container with venv activated
cd tidy3d
jupyter lab . --ip=0.0.0.0 --port=8888Jupyter Lab will print a URL like:
http://127.0.0.1:8888/lab?token=abc123def456...
Copy this complete URL and paste it into your web browser (like Firefox) on your host machine to begin your session.
Note
Port 8888 is forwarded to your host via the -p 8888:8888 flag used when creating the container.
To run GitHub Actions locally, install the act extension:
gh extension install nektos/gh-actYou can then run remote-tests locally using your dev container image:
gh act --pull=false \
-W .github/workflows/tidy3d-python-client-tests.yml \
-P "ubuntu-latest=tidy3d-python-client-dev:latest" \
--input remote_tests=true \
"workflow_dispatch"Note
The image name must match the one you built: tidy3d-python-client-dev:latest
Attach VSCode to the running container for a full IDE experience:
Install Extensions
In VSCode, install:
- Remote - Containers (ms-vscode-remote.remote-containers)
- Docker (ms-azuretools.vscode-docker)
Attach to Container
Start your container:
docker start tidy3d_python_client_dev
In VSCode:
- Open Command Palette (Cmd/Ctrl+Shift+P)
- Type "Remote-Containers: Attach to Running Container"
- Select
tidy3d_python_client_dev
VSCode will open a new window connected to the container
Open folder:
/home/flexdaemon/tidy3dSelect Python interpreter:
.venv/bin/python
Now you can:
- Edit code with full IntelliSense
- Run tests from VSCode test explorer
- Debug Python code with breakpoints
- Use integrated terminal (already inside container)
Note
Who needs this?
CodeArtifact hosts internal packages like tidy3d-extras needed for:
- Integration tests in CI/CD
- Backend-dependent development work
- Full-stack Tidy3D development
External contributors: You can skip this entire section. The open-source tidy3d client works without CodeArtifact.
Warning
Security Policy: We use AWS SSO for authentication, which requires daily login for security compliance. This is a simple 2-step process per development session:
aws sso login- Opens browser for authentication (~30 seconds)poetry aws-login- Automatic token management
Q: Why can't we use PyPI?
A: Internal packages like tidy3d-extras contain proprietary code that cannot be published publicly.
Q: Why SSO with daily expiration?
A: Flexcompute security policy requires time-limited credentials. Long-lived tokens pose security risks.
Q: What if I don't want to deal with this?
A: Use the Docker development environment, which matches our CI setup exactly. Or if you're only working on the open-source client, you don't need CodeArtifact at all.
The development container includes the poetry-codeartifact-login plugin for simplified authentication.
Note
For non-Docker users: If you're not using the Docker container, you'll need to install the plugin first:
# Install the Poetry plugin (one-time, outside Docker)
poetry self add poetry-codeartifact-login
# Also ensure AWS CLI v2 is installed on your system
aws --version # Should show version 2.xDocker users can skip this step - the plugin is pre-installed in the dev container.
One-Time Setup
Inside the container (or on your local machine if not using Docker), configure AWS SSO (only needed once):
# Configure AWS SSO
aws configure ssoEnter the following when prompted:
| Prompt | Value |
|---|---|
| SSO session name | prod |
| SSO start URL | https://d-9067bfae6e.awsapps.com/start/# |
| SSO region | us-east-1 |
| Account | 625554095313 |
| Role | codeartifact-readonly |
| Profile name | prod |
Daily Authentication (each development session)
Each day when you start development, run these two commands:
# Step 1: Login to AWS SSO (opens browser on host for authentication)
aws sso login --profile prod
# Step 2: Configure Poetry with CodeArtifact
poetry aws-login codeartifact --profile prod
# That's it! Now you can install internal packages
poetry install -E extrasNote
How it works: The poetry aws-login command automatically:
- Generates a short-lived CodeArtifact authentication token
- Configures Poetry to use the token for the
codeartifactsource - Stores credentials securely (no manual token passing required)
The plugin handles all token management internally, so you never see or copy tokens manually.
To avoid re-configuring AWS SSO each time you recreate the container, mount your AWS credentials:
# Recreate container with AWS credentials mounted
docker container create --name=tidy3d_python_client_dev \
--userns=host \
-p 8888:8888 \
-v .:/home/flexdaemon/tidy3d \
-v ~/.aws:/home/flexdaemon/.aws \
tidy3d-python-client-devThis way, you only need to run aws configure sso once on your host machine.
If you prefer pip, uv, or other package managers instead of Poetry, you'll need to configure
CodeArtifact authentication yourself.
The officially supported development path uses Poetry. For other package managers:
- See AWS CodeArtifact pip documentation
- Use
aws codeartifact login --tool pipfor manual setup - You are responsible for configuration and troubleshooting
Alternatively, use the Docker development environment which has everything pre-configured.
Container Management:
# Build image
docker build -t tidy3d-python-client-dev -f dev.Dockerfile .
# Create container
docker container create --name=tidy3d_python_client_dev \
--userns=host -p 8888:8888 \
-v .:/home/flexdaemon/tidy3d \
tidy3d-python-client-dev
# Start/stop container
docker start tidy3d_python_client_dev
docker stop tidy3d_python_client_dev
# Enter container
docker exec -it tidy3d_python_client_dev bashInside Container:
# Setup environment
cd tidy3d
uv venv -p 3.11
source .venv/bin/activate
uv pip install -e .[dev]
# Testing
pytest # All tests
pytest -n auto # Parallel tests
pytest tests/test_file.py # Specific file
# Code quality
ruff format . # Format code
ruff check . # Lint code
mypy tidy3d # Type check
# Documentation
cd docs && make html # Build docs
# Jupyter
jupyter lab . --ip=0.0.0.0 # Start Jupyter Lab
# CodeArtifact (internal developers only)
aws sso login --profile prod # Daily: Login to AWS
poetry aws-login codeartifact --profile prod # Daily: Configure PoetryHost: ~/flexcompute/tidy3d/ ├── tidy3d/ # Source code (mounted to container) ├── tests/ # Test suite (mounted) ├── docs/ # Documentation (mounted) ├── .venv/ # Virtual env (created inside, persists) ├── pyproject.toml # Poetry config (mounted) └── dev.Dockerfile # Docker image definition Container: /home/flexdaemon/ ├── tidy3d/ # Mounted from host │ ├── .venv/ # Your virtual environment │ ├── tidy3d/ # Package source │ └── tests/ # Tests └── .aws/ # AWS credentials (if mounted)
Problem: docker: permission denied
Solution: Add your user to the docker group:
sudo usermod -aG docker $USER
# Logout and login (or reboot)Problem: Container exits immediately
Solution: Check logs for errors:
docker logs tidy3d_python_client_dev
# Try running interactively to debug
docker run --rm -it tidy3d-python-client-dev bashProblem: Port 8888 already in use
Solution: Use a different port:
docker container create --name=tidy3d_python_client_dev \
--userns=host \
-p 8889:8888 \
-v .:/home/flexdaemon/tidy3d \
tidy3d-python-client-dev
# Access Jupyter at http://localhost:8889Problem: Cannot edit files, permission denied
Solution: Ensure container uses host user namespace:
# Container must be created with --userns=host flag
docker container create --name=tidy3d_python_client_dev \
--userns=host \
-v .:/home/flexdaemon/tidy3d \
tidy3d-python-client-devProblem: flexdaemon user has wrong UID
Solution: The dev.Dockerfile creates user with UID 1000. If your host user has a different UID, files may have permission issues. Check your host UID:
# On host
id -u # Should output 1000 for best compatibility
# If different, you can modify the Dockerfile or change file ownershipProblem: uv venv command not found
Solution: uv is pre-installed in the image. Verify:
which uv
uv --version
# If missing, rebuild the imageProblem: Virtual environment activation fails
Solution: Make sure you're in the right directory:
cd /home/flexdaemon/tidy3d
ls .venv # Should exist after 'uv venv'
source .venv/bin/activateProblem: Tests fail with ModuleNotFoundError
Solution: Reinstall in editable mode:
# Inside container with venv activated
cd /home/flexdaemon/tidy3d
uv pip install -e .[dev]Problem: Import errors for optional dependencies
Solution: Install all extras:
uv pip install -e .[dev,jax,vtk,trimesh,gdstk]Problem: poetry install -E extras fails with "Unable to find tidy3d-extras" or authentication error
Solution: Verify SSO session is active and re-authenticate:
# Check if SSO session is valid
aws sts get-caller-identity --profile prod
# If expired or fails, re-authenticate (2-step process)
aws sso login --profile prod
poetry aws-login codeartifact --profile prod
# Retry installation
poetry install -E extrasProblem: aws sso login doesn't work, times out, or fails
Solution: This is an AWS account access issue. Contact IT support or your team lead to verify:
- Your AWS account is active
- You have access to the correct AWS organization
- Your SSO credentials are correct
- You have the
codeartifact-readonlyrole assigned
Problem: poetry aws-login command not found
Solution: The plugin is pre-installed in the dev container. If missing:
# Check if plugin is installed
poetry self show plugins
# Reinstall if needed
poetry self add poetry-codeartifact-loginProblem: I use pip/uv, not Poetry. How do I authenticate?
Solution: The officially supported path is Poetry with the poetry aws-login command.
For pip/uv users, you must self-support using AWS documentation.
Option 1 (for pip users):
aws sso login --profile prod
aws codeartifact login --tool pip \
--repository pypi-releases \
--domain flexcompute \
--domain-owner 625554095313 \
--region us-east-1 \
--profile prodOption 2 (not recommended):
CODEARTIFACT_AUTH_TOKEN=`aws codeartifact get-authorization-token \
--domain flexcompute \
--domain-owner 625554095313 \
--query authorizationToken \
--output text \
--profile prod`
pip config set site.extra-index-url \
https://aws:$CODEARTIFACT_AUTH_TOKEN@flexcompute-625554095313.d.codeartifact.us-east-1.amazonaws.com/pypi/pypi-releases/simple/Note: Tokens expire after 12 hours, requiring daily re-authentication.
Alternatively, use the Docker development environment.
Problem: poetry lock or poetry update hangs
Solution: Clear Poetry cache and retry:
# Clear Poetry cache
poetry cache clear pypi --all
poetry cache clear codeartifact --all
# Retry operation
poetry update --lockProblem: Cannot access Jupyter Lab from host browser
Solution: Ensure Jupyter is binding to 0.0.0.0:
# Inside container
jupyter lab . --ip=0.0.0.0 --port=8888
# Port must be forwarded when creating container
# Check with: docker inspect tidy3d_python_client_dev | grep PortBindingsProblem: Jupyter kernel dies or doesn't start
Solution: Check virtual environment has ipykernel:
uv pip install ipykernel
python -m ipykernel install --user --name=tidy3dAfter completing this guide, you should have:
✅ Docker development container running ✅ Python environment set up with dependencies ✅ Tests passing successfully ✅ (Optional) Jupyter Lab accessible ✅ (Optional) VSCode attached to container ✅ (Optional) AWS CodeArtifact configured
Continue learning:
- :doc:`usage` - Advanced development workflows
- :doc:`documentation` - Building and writing documentation
- :doc:`recommendations` - Best practices and code style
- :doc:`release/index` - Release process and versioning
More Docker resources:
Welcome to Docker-based Tidy3D development!