Skip to content

Commit 04ff6b9

Browse files
travisjneumanclaude
andcommitted
feat: add 8 module-specific architecture diagrams (Mermaid)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 9996a59 commit 04ff6b9

File tree

8 files changed

+1117
-0
lines changed

8 files changed

+1117
-0
lines changed
Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,157 @@
1+
# Async Event Loop Deep Dive — Diagrams
2+
3+
[<- Back to Diagram Index](../../guides/DIAGRAM_INDEX.md)
4+
5+
## Overview
6+
7+
These diagrams go deeper into how Python's asyncio event loop schedules coroutines, manages task lifecycles, and coordinates concurrent operations with `gather()` and `wait()`.
8+
9+
## Event Loop Internals
10+
11+
The event loop maintains queues of ready and waiting tasks. Each iteration ("tick") of the loop runs all ready callbacks, checks for completed I/O, and moves newly-ready tasks into the run queue.
12+
13+
```mermaid
14+
flowchart TD
15+
TICK["Event Loop Tick"] --> READY{"Ready queue<br/>has tasks?"}
16+
READY -->|"Yes"| RUN["Run next callback<br/>(one at a time)"]
17+
RUN --> CHECK_MORE{"More ready<br/>callbacks?"}
18+
CHECK_MORE -->|"Yes"| RUN
19+
CHECK_MORE -->|"No"| POLL["Poll for I/O events<br/>(select/epoll/kqueue)"]
20+
READY -->|"No"| POLL
21+
22+
POLL --> IO_DONE{"I/O completed?"}
23+
IO_DONE -->|"Yes"| WAKE["Move completed tasks<br/>to ready queue"]
24+
IO_DONE -->|"No"| TIMERS["Check timers<br/>(sleep, timeout)"]
25+
WAKE --> TIMERS
26+
27+
TIMERS --> EXPIRED{"Timers expired?"}
28+
EXPIRED -->|"Yes"| SCHEDULE["Schedule timer<br/>callbacks as ready"]
29+
EXPIRED -->|"No"| TICK
30+
SCHEDULE --> TICK
31+
32+
style TICK fill:#cc5de8,stroke:#9c36b5,color:#fff
33+
style RUN fill:#51cf66,stroke:#27ae60,color:#fff
34+
style POLL fill:#4a9eff,stroke:#2670c2,color:#fff
35+
style WAKE fill:#ffd43b,stroke:#f59f00,color:#000
36+
```
37+
38+
**Key points:**
39+
- The loop runs one callback at a time (single-threaded concurrency, not parallelism)
40+
- I/O polling is where the loop waits for network responses, file reads, etc.
41+
- Timers handle `asyncio.sleep()` and timeout deadlines
42+
- A "tick" is one full cycle: run ready tasks, poll I/O, check timers, repeat
43+
44+
## Task Lifecycle States
45+
46+
A coroutine goes through several states from creation to completion. Understanding these states helps you debug hanging tasks and cancellation behavior.
47+
48+
```mermaid
49+
stateDiagram-v2
50+
[*] --> Coroutine: async def my_func()
51+
Coroutine --> Task: asyncio.create_task(my_func())
52+
Task --> Pending: Scheduled on event loop
53+
Pending --> Running: Loop picks this task
54+
Running --> Awaiting: Hits await expression
55+
Awaiting --> Pending: Awaited result ready
56+
Running --> Done: Return value
57+
Running --> Cancelled: task.cancel() called
58+
Awaiting --> Cancelled: task.cancel() called
59+
Pending --> Cancelled: task.cancel() called
60+
Done --> [*]: result = task.result()
61+
Cancelled --> [*]: raises CancelledError
62+
63+
note right of Coroutine: Calling async def<br/>returns a coroutine object<br/>(does NOT start running)
64+
note right of Task: create_task() wraps<br/>the coroutine and<br/>schedules it
65+
note left of Cancelled: CancelledError is raised<br/>inside the coroutine at<br/>the current await point
66+
```
67+
68+
**Key points:**
69+
- Calling an `async def` function returns a coroutine object but does NOT start executing it
70+
- `create_task()` wraps the coroutine in a Task and schedules it on the loop
71+
- Cancellation raises `CancelledError` at the coroutine's current `await` point
72+
- A task is "done" when it returns, raises an exception, or is cancelled
73+
74+
## gather() vs wait() vs TaskGroup
75+
76+
Three ways to run multiple coroutines concurrently. Each has different error handling and completion semantics.
77+
78+
```mermaid
79+
flowchart TD
80+
subgraph GATHER ["asyncio.gather(*coros)"]
81+
G_START["Start all tasks<br/>concurrently"]
82+
G_WAIT["Wait for ALL<br/>to complete"]
83+
G_RESULT["Returns list of results<br/>in original order"]
84+
G_ERR["If one fails:<br/>cancels others (return_exceptions=False)<br/>or returns exception in list (=True)"]
85+
G_START --> G_WAIT --> G_RESULT
86+
G_WAIT --> G_ERR
87+
end
88+
89+
subgraph WAIT ["asyncio.wait(tasks, ...)"]
90+
W_START["Start all tasks<br/>concurrently"]
91+
W_RETURN["Returns two sets:<br/>(done, pending)"]
92+
W_MODE["Control when it returns:<br/>FIRST_COMPLETED<br/>FIRST_EXCEPTION<br/>ALL_COMPLETED"]
93+
W_START --> W_RETURN
94+
W_RETURN --> W_MODE
95+
end
96+
97+
subgraph TASKGROUP ["asyncio.TaskGroup() — Python 3.11+"]
98+
TG_START["async with TaskGroup() as tg:<br/> tg.create_task(coro1)<br/> tg.create_task(coro2)"]
99+
TG_WAIT["Waits at end of<br/>async with block"]
100+
TG_ERR["If one fails:<br/>cancels all others,<br/>raises ExceptionGroup"]
101+
TG_START --> TG_WAIT --> TG_ERR
102+
end
103+
104+
style GATHER fill:#51cf66,stroke:#27ae60,color:#fff
105+
style WAIT fill:#4a9eff,stroke:#2670c2,color:#fff
106+
style TASKGROUP fill:#cc5de8,stroke:#9c36b5,color:#fff
107+
```
108+
109+
**Key points:**
110+
- `gather()` is simplest: run tasks, get results in order. Best for "do all of these and give me all results"
111+
- `wait()` gives you fine-grained control: react to the first completion or first failure
112+
- `TaskGroup` (Python 3.11+) is the modern approach with structured concurrency and clean cancellation
113+
- `gather(return_exceptions=True)` collects exceptions as values instead of propagating them
114+
115+
## Sequence: Timeout and Cancellation
116+
117+
What happens when a task exceeds a deadline. This shows the mechanics of `asyncio.wait_for()` and how cancellation propagates.
118+
119+
```mermaid
120+
sequenceDiagram
121+
participant Main as Main Coroutine
122+
participant Loop as Event Loop
123+
participant Task as Slow Task
124+
participant Timer as Timeout Timer
125+
126+
Main->>Loop: asyncio.wait_for(slow_task(), timeout=2.0)
127+
Loop->>Task: Start slow_task()
128+
Loop->>Timer: Set timer for 2.0 seconds
129+
130+
Note over Task: Working...<br/>await aiohttp.get(slow_url)
131+
Note over Loop: Loop continues<br/>running other tasks
132+
133+
Timer-->>Loop: 2.0 seconds elapsed!
134+
Loop->>Task: task.cancel()
135+
Note over Task: CancelledError raised<br/>at current await
136+
137+
alt Task has try/finally
138+
Note over Task: finally: cleanup()
139+
Task-->>Loop: Cleanup complete
140+
else No cleanup
141+
Task-->>Loop: CancelledError propagates
142+
end
143+
144+
Loop-->>Main: raises asyncio.TimeoutError
145+
Note over Main: Handle timeout gracefully
146+
```
147+
148+
**Key points:**
149+
- `wait_for()` wraps a coroutine with a deadline and cancels it if the deadline passes
150+
- Cancellation gives the task a chance to clean up in `try/finally` blocks
151+
- The caller receives `TimeoutError`, not `CancelledError`
152+
- Always use timeouts for network operations to prevent tasks from hanging forever
153+
154+
---
155+
156+
| [Back to Diagram Index](../../guides/DIAGRAM_INDEX.md) |
157+
|:---:|
Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
# CI/CD Pipeline — Diagrams
2+
3+
[<- Back to Diagram Index](../../guides/DIAGRAM_INDEX.md)
4+
5+
## Overview
6+
7+
These diagrams show how continuous integration and continuous deployment pipelines automate code quality checks, testing, and deployment using GitHub Actions as the primary example.
8+
9+
## Full CI/CD Pipeline Flow
10+
11+
A CI/CD pipeline runs automatically on every push or pull request. Each stage acts as a gate: if one fails, later stages do not run, preventing broken code from reaching production.
12+
13+
```mermaid
14+
flowchart LR
15+
PUSH["git push / PR opened"] --> LINT["Lint & Format<br/>ruff check .<br/>black --check ."]
16+
LINT -->|"Pass"| TYPE["Type Check<br/>mypy src/"]
17+
TYPE -->|"Pass"| TEST["Run Tests<br/>pytest --cov"]
18+
TEST -->|"Pass"| BUILD["Build<br/>docker build -t app ."]
19+
BUILD -->|"Pass"| DEPLOY_STAGING["Deploy to Staging<br/>Automatic"]
20+
DEPLOY_STAGING --> SMOKE["Smoke Tests<br/>Hit /health endpoint"]
21+
SMOKE -->|"Pass"| GATE{"Manual Approval?"}
22+
GATE -->|"Approved"| DEPLOY_PROD["Deploy to Production"]
23+
GATE -->|"Skip"| DONE["Pipeline Complete"]
24+
DEPLOY_PROD --> DONE
25+
26+
LINT -->|"Fail"| STOP1["Pipeline Stops<br/>Fix lint errors"]
27+
TYPE -->|"Fail"| STOP2["Pipeline Stops<br/>Fix type errors"]
28+
TEST -->|"Fail"| STOP3["Pipeline Stops<br/>Fix failing tests"]
29+
BUILD -->|"Fail"| STOP4["Pipeline Stops<br/>Fix build errors"]
30+
31+
style PUSH fill:#cc5de8,stroke:#9c36b5,color:#fff
32+
style LINT fill:#ffd43b,stroke:#f59f00,color:#000
33+
style TYPE fill:#ffd43b,stroke:#f59f00,color:#000
34+
style TEST fill:#4a9eff,stroke:#2670c2,color:#fff
35+
style BUILD fill:#ff922b,stroke:#e8590c,color:#fff
36+
style DEPLOY_STAGING fill:#51cf66,stroke:#27ae60,color:#fff
37+
style DEPLOY_PROD fill:#51cf66,stroke:#27ae60,color:#fff
38+
style STOP1 fill:#ff6b6b,stroke:#c92a2a,color:#fff
39+
style STOP2 fill:#ff6b6b,stroke:#c92a2a,color:#fff
40+
style STOP3 fill:#ff6b6b,stroke:#c92a2a,color:#fff
41+
style STOP4 fill:#ff6b6b,stroke:#c92a2a,color:#fff
42+
```
43+
44+
**Key points:**
45+
- Fast checks (lint, format) run first so you get quick feedback on simple mistakes
46+
- Each stage is a gate: failures stop the pipeline early, saving compute time
47+
- Staging deployment happens automatically; production may require manual approval
48+
- Smoke tests verify the deployed app actually starts and responds
49+
50+
## GitHub Actions Workflow Structure
51+
52+
A GitHub Actions workflow is a YAML file in `.github/workflows/`. It defines triggers, jobs, and steps. Jobs run in parallel by default; use `needs` to create dependencies.
53+
54+
```mermaid
55+
flowchart TD
56+
subgraph TRIGGER ["Triggers (on:)"]
57+
PUSH_T["push:<br/>branches: [main]"]
58+
PR_T["pull_request:<br/>branches: [main]"]
59+
end
60+
61+
subgraph JOB_LINT ["Job: lint"]
62+
LINT_1["runs-on: ubuntu-latest"]
63+
LINT_2["Step: checkout code"]
64+
LINT_3["Step: setup-python"]
65+
LINT_4["Step: pip install ruff"]
66+
LINT_5["Step: ruff check ."]
67+
LINT_1 --> LINT_2 --> LINT_3 --> LINT_4 --> LINT_5
68+
end
69+
70+
subgraph JOB_TEST ["Job: test"]
71+
TEST_1["runs-on: ubuntu-latest"]
72+
TEST_M["strategy: matrix<br/>python: [3.11, 3.12, 3.13]"]
73+
TEST_2["Step: checkout code"]
74+
TEST_3["Step: setup-python (matrix)"]
75+
TEST_4["Step: pip install -r requirements.txt"]
76+
TEST_5["Step: pytest --cov"]
77+
TEST_1 --> TEST_M --> TEST_2 --> TEST_3 --> TEST_4 --> TEST_5
78+
end
79+
80+
subgraph JOB_DEPLOY ["Job: deploy"]
81+
DEP_1["runs-on: ubuntu-latest"]
82+
DEP_2["Step: checkout code"]
83+
DEP_3["Step: deploy to Railway/Render"]
84+
DEP_1 --> DEP_2 --> DEP_3
85+
end
86+
87+
PUSH_T --> JOB_LINT
88+
PR_T --> JOB_LINT
89+
PUSH_T --> JOB_TEST
90+
PR_T --> JOB_TEST
91+
JOB_LINT -->|"needs: lint"| JOB_DEPLOY
92+
JOB_TEST -->|"needs: test"| JOB_DEPLOY
93+
94+
style TRIGGER fill:#cc5de8,stroke:#9c36b5,color:#fff
95+
style JOB_LINT fill:#ffd43b,stroke:#f59f00,color:#000
96+
style JOB_TEST fill:#4a9eff,stroke:#2670c2,color:#fff
97+
style JOB_DEPLOY fill:#51cf66,stroke:#27ae60,color:#fff
98+
```
99+
100+
**Key points:**
101+
- Jobs run in parallel by default: `lint` and `test` start at the same time
102+
- `needs:` creates dependencies: `deploy` waits for both `lint` and `test` to pass
103+
- Matrix strategy runs the same steps across multiple Python versions simultaneously
104+
- Each job gets a fresh virtual machine: no state leaks between jobs
105+
106+
## Sequence: Pull Request Lifecycle
107+
108+
The full lifecycle from opening a PR to merging, showing how CI checks integrate with code review.
109+
110+
```mermaid
111+
sequenceDiagram
112+
participant Dev as Developer
113+
participant GH as GitHub
114+
participant CI as GitHub Actions
115+
participant Rev as Reviewer
116+
117+
Dev->>GH: Open Pull Request
118+
GH->>CI: Trigger workflow (on: pull_request)
119+
120+
par Parallel Jobs
121+
CI->>CI: Job: lint (ruff, black)
122+
CI->>CI: Job: test (pytest, 3 Python versions)
123+
end
124+
125+
alt All checks pass
126+
CI-->>GH: Status: All checks passed
127+
GH-->>Dev: Green checkmarks
128+
Dev->>Rev: Request review
129+
Rev->>GH: Review changes
130+
Rev-->>Dev: Approve / Request changes
131+
132+
alt Approved
133+
Dev->>GH: Merge PR
134+
GH->>CI: Trigger deploy workflow (on: push to main)
135+
CI->>CI: Build and deploy to production
136+
CI-->>GH: Deployment successful
137+
end
138+
else Some checks fail
139+
CI-->>GH: Status: Checks failed
140+
GH-->>Dev: Red X marks
141+
Dev->>Dev: Fix issues locally
142+
Dev->>GH: Push fix commits
143+
GH->>CI: Re-trigger workflow
144+
end
145+
```
146+
147+
**Key points:**
148+
- CI runs automatically when a PR is opened and on every subsequent push to the PR branch
149+
- Merge is blocked until all required checks pass (configurable in branch protection rules)
150+
- Deploying on merge to `main` means every merged PR goes to production automatically
151+
- Failed checks give immediate feedback: fix and push again to re-trigger
152+
153+
---
154+
155+
| [Back to Diagram Index](../../guides/DIAGRAM_INDEX.md) |
156+
|:---:|

0 commit comments

Comments
 (0)