Commit 20f45ae
committed
Refactors to a simplified, single-process image processing model
Revamps the image processing pipeline from a distributed, asynchronous, queue-based architecture to a synchronous, direct file processing approach.
This significantly simplifies the codebase by:
- Removing complex components for batching, worker pools, and message queues.
- Streamlining ONNX model loading and inference, removing dedicated batch processing at the model level.
- Consolidating input file discovery from `walkdir` to glob patterns.
- Reducing external dependencies such as `tokio`, `rayon`, `async-trait`, and `serde`.
- Simplifying configuration options and error handling, removing distributed-specific settings and retry logic.
The model now processes images sequentially within a single thread, utilizing `fast_image_resize` and `imageops-kit` for robust image manipulation. This improves maintainability and reduces operational overhead.1 parent bb027f6 commit 20f45ae
16 files changed
Lines changed: 273 additions & 3459 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
23 | | - | |
24 | | - | |
| 23 | + | |
25 | 24 | | |
26 | 25 | | |
27 | 26 | | |
| |||
31 | 30 | | |
32 | 31 | | |
33 | 32 | | |
34 | | - | |
35 | | - | |
36 | 33 | | |
| 34 | + | |
| 35 | + | |
37 | 36 | | |
38 | 37 | | |
39 | 38 | | |
40 | 39 | | |
41 | 40 | | |
42 | 41 | | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | 42 | | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | 43 | | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
60 | | - | |
61 | | - | |
62 | | - | |
63 | | - | |
64 | | - | |
65 | 44 | | |
66 | 45 | | |
0 commit comments