You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Streams allow concurrent execution of GPU operations. By default, all Quadrants kernels launch on the default stream, which serializes everything. With streams, you can run multiple top-level for loops in parallel.
4
+
5
+
## Supported platforms
6
+
7
+
| Backend | Supported |
8
+
|---------|-----------|
9
+
| CUDA | Yes |
10
+
| AMDGPU | Yes |
11
+
| CPU | No-op |
12
+
| Metal | No-op |
13
+
| Vulkan | No-op |
14
+
15
+
On backends without native stream support, stream operations are no-ops and for loops run serially. Code using streams is portable across all backends — it will run without modifications, but serially.
16
+
17
+
## Stream parallelism
18
+
19
+
Inside a `@qd.kernel`, each `with qd.stream_parallel():` block runs on its own GPU stream.
20
+
21
+
```python
22
+
import quadrants as qd
23
+
24
+
qd.init(arch=qd.cuda)
25
+
26
+
N =1024
27
+
a = qd.field(qd.f32, shape=(N,))
28
+
b = qd.field(qd.f32, shape=(N,))
29
+
c = qd.field(qd.f32, shape=(N,))
30
+
31
+
@qd.kernel
32
+
defcompute_ab():
33
+
with qd.stream_parallel():
34
+
for i inrange(N):
35
+
a[i] = compute_a(i)
36
+
with qd.stream_parallel():
37
+
for j inrange(N):
38
+
b[j] = compute_b(j)
39
+
40
+
@qd.kernel
41
+
defcombine():
42
+
for i inrange(N):
43
+
c[i] = a[i] + b[i]
44
+
45
+
compute_ab() # the two stream_parallel blocks run concurrently
46
+
combine() # runs after compute_ab() returns — a[] and b[] are ready
47
+
```
48
+
49
+
Consecutive `with qd.stream_parallel():` blocks run concurrently. Multiple for loops within a single block share a stream and run serially on it. All streams are synchronized before the kernel returns.
50
+
51
+
### Restrictions
52
+
53
+
- All top-level statements in a kernel must be either all `stream_parallel` blocks or all regular statements. Mixing the two at the top level is a compile-time error.
54
+
- Nesting `stream_parallel` blocks is not supported.
55
+
56
+
## Explicit streams
57
+
58
+
For cases that require manual control — such as launching separate kernels on different streams or interoperating with PyTorch — you can create and manage streams directly.
59
+
60
+
### Creating and using streams
61
+
62
+
Any `@qd.kernel` function accepts a special `qd_stream` keyword argument — you do not need to declare it in the kernel signature. The `@qd.kernel` decorator handles it automatically.
63
+
64
+
```python
65
+
@qd.kernel
66
+
defmy_kernel():
67
+
for i inrange(N):
68
+
a[i] = i
69
+
70
+
s1 = qd.create_stream()
71
+
s2 = qd.create_stream()
72
+
73
+
my_kernel(qd_stream=s1)
74
+
my_kernel(qd_stream=s2)
75
+
76
+
s1.synchronize()
77
+
s2.synchronize()
78
+
79
+
s1.destroy()
80
+
s2.destroy()
81
+
```
82
+
83
+
Kernels on different streams may execute concurrently. Call `synchronize()` to block until all work on a stream completes.
84
+
85
+
### Events
86
+
87
+
Events let you express dependencies between streams without full synchronization.
88
+
89
+
```python
90
+
s1 = qd.create_stream()
91
+
s2 = qd.create_stream()
92
+
93
+
@qd.kernel
94
+
defproduce():
95
+
for i inrange(N):
96
+
a[i] =10.0
97
+
98
+
@qd.kernel
99
+
defconsume():
100
+
for i inrange(N):
101
+
b[i] = a[i]
102
+
103
+
produce(qd_stream=s1)
104
+
105
+
e = qd.create_event()
106
+
e.record(s1) # record when s1 finishes produce()
107
+
e.wait(qd_stream=s2) # s2 waits for that event before proceeding
108
+
109
+
consume(qd_stream=s2) # safe to read a[] — produce() is guaranteed complete
110
+
s2.synchronize()
111
+
112
+
e.destroy()
113
+
s1.destroy()
114
+
s2.destroy()
115
+
```
116
+
117
+
`e.record(stream)` captures the point in `stream`'s execution. `e.wait(qd_stream=stream)` makes `stream` wait until the recorded point is reached. If `qd_stream` is omitted, the default stream waits.
118
+
119
+
### Context managers
120
+
121
+
Streams and events support `with` blocks for automatic cleanup:
122
+
123
+
```python
124
+
with qd.create_stream() as s:
125
+
some_func1(qd_stream=s)
126
+
# s.destroy() called automatically — waits for in-flight work
127
+
```
128
+
129
+
## Synchronization notes
130
+
131
+
-**`qd.sync()` only waits on the default stream.** It does not drain explicit streams. Call `stream.synchronize()` on each stream you need to wait for.
132
+
-**No automatic synchronization with explicit streams.** When using explicit streams, you are responsible for inserting events or `synchronize()` calls when one stream's output is another stream's input. `stream_parallel` handles this automatically.
133
+
134
+
## Limitations
135
+
136
+
-**Not compatible with graphs.** Do not pass `qd_stream` to a kernel decorated with `graph=True` (if you do, a `RuntimeError` will be raised).
137
+
-**Not compatible with autodiff.** Do not pass `qd_stream` to a kernel that uses reverse-mode or forward-mode differentiation, or inside a `qd.ad.Tape` context (if you do, a `RuntimeError` will be raised).
0 commit comments