Skip to content

Commit 05db189

Browse files
committed
Add Phase 5 content: Performance, Memory, and Distributed Computing
Chapter 10: Performance Optimization - Introduction to performance metrics (FLOPS, bandwidth, roofline) - Devito optimization architecture (loop blocking, SIMD, OpenMP) - GPU computing with Devito (platforms, memory management) - Performance analysis and profiling Chapter 11: Memory Management and Wavefield Storage - Memory challenges in wave propagation - Snapshotting with ConditionalDimension - Checkpointing strategies (Revolve algorithm, pyrevolve) - I/O strategies (disk, compression, HDF5) Chapter 12: Distributed Computing and Scalability - Embarrassingly parallel problems and shot-parallel workflows - Domain decomposition with MPI - Task-based parallelism with Dask - Hybrid approaches (MPI + threading, cloud) All implementations use explicit Devito API without convenience classes. New files: - chapters/performance/performance.qmd - chapters/memory/memory.qmd - chapters/distributed/distributed.qmd - src/performance/benchmark.py (benchmarking utilities) - src/memory/snapshotting.py (ConditionalDimension patterns) - src/distributed/dask_utils.py (shot-parallel workflows) - tests/test_performance.py (20 tests) - tests/test_memory.py (22 tests, 2 skip without h5py) - tests/test_distributed.py (26 tests, skip without dask) Total tests: 541 (40 new tests for Phase 5) PDF builds successfully.
1 parent b9dc387 commit 05db189

17 files changed

Lines changed: 6368 additions & 24 deletions

File tree

_quarto.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,9 @@ book:
2323
- chapters/systems/index.qmd
2424
- chapters/highorder/index.qmd
2525
- chapters/adjoint/index.qmd
26+
- chapters/memory/index.qmd
27+
- chapters/distributed/index.qmd
28+
- chapters/performance/index.qmd
2629
- part: "Appendices"
2730
chapters:
2831
- chapters/appendices/formulas/index.qmd
@@ -177,6 +180,9 @@ src_elliptic: "https://github.com/devitocodes/devito_book/tree/devito/src/ellipt
177180
src_systems: "https://github.com/devitocodes/devito_book/tree/devito/src/systems"
178181
src_highorder: "https://github.com/devitocodes/devito_book/tree/devito/src/highorder"
179182
src_adjoint: "https://github.com/devitocodes/devito_book/tree/devito/src/adjoint"
183+
src_memory: "https://github.com/devitocodes/devito_book/tree/devito/src/memory"
184+
src_distributed: "https://github.com/devitocodes/devito_book/tree/devito/src/distributed"
185+
src_performance: "https://github.com/devitocodes/devito_book/tree/devito/src/performance"
180186
src_formulas: "https://github.com/devitocodes/devito_book/tree/devito/src/formulas"
181187
src_softeng2: "https://github.com/devitocodes/devito_book/tree/devito/src/softeng2"
182188

book-roadmap.md

Lines changed: 35 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -20,12 +20,12 @@ Extend *Finite Difference Computing with PDEs* with content from `devito_repo/ex
2020
| **Phase 1** | ✅ Complete | 62 | `ab9b38c0` |
2121
| **Phase 2** | ✅ Complete | 73 | `b9e017b3` |
2222
| **Phase 3** | ✅ Complete | 91 | `04accab7` |
23-
| **Phase 4** | ✅ Complete | 90 | pending |
24-
| **Phase 5** | 🔲 Not started | - | - |
23+
| **Phase 4** | ✅ Complete | 90 | `b9dc387d` |
24+
| **Phase 5** | ✅ Complete | 40 | pending |
2525
| **Phase 6** | 🔲 Not started | - | - |
2626
| **Phase 7** | 🔲 Not started | - | - |
2727

28-
**Total tests: 501**
28+
**Total tests: 541**
2929

3030
---
3131

@@ -285,50 +285,61 @@ op = Operator([stencil] + src_term + rec_term)
285285

286286
---
287287

288-
## Phase 5: Performance and Scalability
288+
## Phase 5: Performance and Scalability ✅ COMPLETE
289289

290290
**Effort**: Medium | **Value**: High | **Source**: Ready notebooks
291291

292292
Practical content for anyone running real simulations.
293293

294-
### 5.1 Chapter 15: GPU Computing
294+
### 5.1 Chapter 10: Performance Optimization ✅
295295

296-
**Source**: `performance/01_gpu.ipynb`
296+
**Source**: `performance/01_gpu.ipynb`, `performance/00_overview.ipynb`
297297

298-
- Devito GPU backends
299-
- Memory management on GPU
300-
- Performance comparison
298+
**Sections**:
299+
- 10.1 Introduction to Performance (FLOPS, bandwidth, roofline)
300+
- 10.2 Devito Optimization Architecture (loop blocking, SIMD, OpenMP)
301+
- 10.3 GPU Computing with Devito (platforms, memory management)
302+
- 10.4 Performance Analysis (profiling, bottlenecks)
303+
- 10.5 Exercises
301304

302-
### 5.2 Chapter 16: Memory Management and I/O
305+
### 5.2 Chapter 11: Memory Management and I/O
303306

304-
**Source**: `08_snapshotting.ipynb`, `12_time_blocking.ipynb`
307+
**Source**: `08_snapshotting.ipynb`, `05_conditional_dimension.ipynb`
305308

306309
**Sections**:
307-
- 16.1 Wavefield Storage Strategies
308-
- 16.2 Snapshotting with ConditionalDimension
309-
- 16.3 Time Blocking and Compression
310+
- 11.1 Memory Challenges in Wave Propagation
311+
- 11.2 Snapshotting with ConditionalDimension
312+
- 11.3 Checkpointing Strategies (Revolve algorithm, pyrevolve)
313+
- 11.4 I/O Strategies (disk, compression, HDF5)
314+
- 11.5 Exercises
310315

311316
**Key pattern**:
312317
```python
313318
time_sub = ConditionalDimension('t_sub', parent=grid.time_dim, factor=10)
314319
usave = TimeFunction(name='usave', grid=grid, save=nsnaps, time_dim=time_sub)
315320
```
316321

317-
### 5.3 Chapter 17: Distributed Computing with Dask
322+
### 5.3 Chapter 12: Distributed Computing with Dask
318323

319324
**Source**: `04_dask.ipynb`
320325

321-
- Dask cluster integration
322-
- Shot-parallel FWI
323-
- Scaling to HPC
326+
**Sections**:
327+
- 12.1 Introduction to Parallel Computing
328+
- 12.2 Domain Decomposition with MPI
329+
- 12.3 Task-Based Parallelism with Dask (shot-parallel FWI)
330+
- 12.4 Hybrid Approaches (MPI + threading, cloud)
331+
- 12.5 Exercises
324332

325333
**Deliverables**:
326-
- [ ] `chapters/performance/gpu.qmd`
327-
- [ ] `chapters/performance/memory.qmd`
328-
- [ ] `chapters/performance/distributed.qmd`
329-
- [ ] `src/performance/snapshotting.py`
330-
- [ ] `src/performance/checkpointing.py`
331-
- [ ] Tests for performance patterns
334+
- [x] `chapters/performance/performance.qmd` (Chapter 10)
335+
- [x] `chapters/memory/memory.qmd` (Chapter 11)
336+
- [x] `chapters/distributed/distributed.qmd` (Chapter 12)
337+
- [x] `src/performance/benchmark.py`
338+
- [x] `src/memory/snapshotting.py`
339+
- [x] `src/distributed/dask_utils.py`
340+
- [x] `tests/test_performance.py` (20 tests)
341+
- [x] `tests/test_memory.py` (22 tests, 2 skip without h5py)
342+
- [x] `tests/test_distributed.py` (26 tests, all skip without dask)
332343

333344
---
334345

0 commit comments

Comments
 (0)