You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+2-311Lines changed: 2 additions & 311 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -44,321 +44,12 @@ pip install tinyio
44
44
45
45
## Documentation
46
46
47
-
### Loops
48
-
49
-
Create a loop with `tinyio.Loop()`. It has a single method, `.run(coro)`, which consumes a coroutine, and which returns the output of that coroutine.
50
-
51
-
Coroutines can `yield` four possible things:
52
-
53
-
- `yield`: yield nothing, this just pauses and gives other coroutines a chance to run.
54
-
- `yield coro`: wait on a single coroutine, in which case we'll resume with the output of that coroutine once it is available.
55
-
- `yield [coro1, coro2, coro3]`: wait on multiple coroutines by putting them in a list, and resume with a list of outputs once all have completed. This is what `asyncio` calls a 'gather' or 'TaskGroup', and what `trio` calls a 'nursery'.
56
-
- `yield {coro1, coro2, coro3}`: schedule one or more coroutines but do not wait on their result - they will run independently in the background.
57
-
58
-
If you `yield` on the same coroutine multiple times (e.g. in a diamond dependency pattern) then the coroutine will be scheduled once, and on completion all dependees will receive its output. (You can even do this if the coroutine has already finished: `yield` on it to retrieve its output.)
59
-
60
-
### Threading
61
-
62
-
Blocking functions can be ran in threads using `tinyio.run_in_thread(fn, *args, **kwargs)`, which gives a coroutine you can `yield` on. Example:
63
-
64
-
```python
65
-
import time, tinyio
66
-
67
-
def slow_blocking_add_one(x: int) -> int:
68
-
time.sleep(1)
69
-
return x + 1
70
-
71
-
def foo(x: int):
72
-
out = yield [tinyio.run_in_thread(slow_blocking_add_one, x) for _ in range(3)]
73
-
return out
74
-
75
-
loop = tinyio.Loop()
76
-
out = loop.run(foo(x=1)) # runs in one second, not three
77
-
assert out == [2, 2, 2]
78
-
```
79
-
80
-
### Sleeping
81
-
82
-
This is `tinyio.sleep(delay_in_seconds)`, which is a coroutine you can `yield` on.
83
-
84
-
### Error propagation
85
-
86
-
If any coroutine raises an error, then:
87
-
88
-
1. All coroutines across the entire loop will have `tinyio.CancelledError` raised in them (from whatever `yield` point they are currently waiting at).
89
-
2. Any functions ran in threads via `tinyio.run_in_thread` will also have `tinyio.CancelledError` raised in the thread.
90
-
3. The original error is raised out of `loop.run(...)`. This behaviour can be configured (e.g. to collect errors into a `BaseExceptionGroup`) by setting `loop.run(..., exception_group=None/False/True)`.
91
-
92
-
This gives every coroutine a chance to shut down gracefully. Debuggers like [`patdb`](https://github.com/patrick-kidger/patdb) offer the ability to navigate across exceptions in an exception group, allowing you to inspect the state of all coroutines that were related to the error.
93
-
94
-
## Further documentation
95
-
96
-
<details><summary><h3>Synchronisation</h3> (Click to expand)</summary>
97
-
98
-
We ship batteries-included with the usual collection of standard operations for synchronisation.
99
-
100
-
```python
101
-
tinyio.as_completed tinyio.Semaphore
102
-
tinyio.Barrier tinyio.ThreadPool
103
-
tinyio.Event tinyio.timeout
104
-
tinyio.Lock tinyio.TimeoutError
105
-
```
106
-
107
-
---
108
-
109
-
-`tinyio.as_completed({coro1, coro2, ...})`
110
-
111
-
This schedules multiple coroutines in the background (like `yield {coro1, coro2, ...}`), and then offers their results in the order they complete.
112
-
113
-
This is iterated over in the following way, using its `.done()` and `.get()` methods:
This has a single method `barrier.wait()`, which is a coroutine you can `yield` on. Once `value` many coroutines have yielded on this method then it will unblock.
126
-
127
-
---
128
-
129
-
-`tinyio.Event()`
130
-
131
-
This is a wrapper around a boolean flag, initialised with`False`.
132
-
This has the following methods:
133
-
134
-
-`.is_set()`: return the value of the flag.
135
-
-`.set()`: set the flag to `True`.
136
-
-`.clear()`: set the flag to `False`.
137
-
-`.wait(timeout_in_seconds=None)`, which is a coroutine you can `yield` on. This will unblock if the internal flag is`True`orif`timeout_in_seconds` seconds pass. (Typically the former is accomplished by calling `.set()`from another coroutine orfrom a thread.)
138
-
139
-
---
140
-
141
-
-`tinyio.Lock()`
142
-
143
-
This is just a convenience for`tinyio.Semaphore(value=1)`, see below.
144
-
145
-
---
146
-
147
-
-`tinyio.Semaphore(value)`
148
-
149
-
This manages an internal counter that is initialised at `value`, is decremented when entering a region, and incremented when exiting. This blocks if this counter is at zero. In this way, at most `value` coroutines may acquire the semaphore at a time.
150
-
151
-
This is used as:
152
-
```python
153
-
semaphore = Semaphore(value)
154
-
155
-
...
156
-
157
-
with (yield semaphore()):
158
-
...
159
-
```
160
-
161
-
---
162
-
163
-
-`tinyio.timeout(coro, timeout_in_seconds)`
164
-
165
-
This is a coroutine you can `yield` on, used as`output, success = yield tinyio.timeout(coro, timeout_in_seconds)`.
166
-
167
-
This runs `coro`for at most `timeout_in_seconds`. If it succeeds in that time then the pair `(output, True)`is returned . Else this will return`(None, False)`, and`coro` will be halted by raising `tinyio.TimeoutError` inside it.
168
-
169
-
---
170
-
171
-
-`tinyio.ThreadPool(max_threads)`
172
-
173
-
This is equivalent to making multiple `tinyio.run_in_thread` calls, but will limit the number of threads to at most `max_threads`. Additional work after that will block until a thread becomes available.
174
-
175
-
This has two methods:
176
-
177
-
-`.run_in_thread(fn, *args, **kwargs)`, which is a coroutine you can `yield` on. This is equivalent to `yield tinyio.run_in_thread(fn, *args, **kwargs)`.
178
-
-`.map(fn, xs)`, which is a coroutine you can `yield` on. This is equivalent to `yield [tinyio.run_in_thread(fn, x) for x in xs]`.
179
-
180
-
---
181
-
182
-
</details>
183
-
184
-
<details><summary><h3>Asynchronous context managers</h3> (Click to expand)</summary>
185
-
186
-
You can use the following pattern to implement context managers with asynchronous entry:
187
-
188
-
```python
189
-
def my_coro():
190
-
with (yield my_context_manager(x=5)) as val:
191
-
print(f"Got val {val}")
192
-
```
193
-
where:
194
-
```python
195
-
def my_context_manager(x):
196
-
print("Initialising...")
197
-
yield tinyio.sleep(1)
198
-
print("Initialised")
199
-
return make_context_manager(x)
200
-
201
-
@contextlib.contextmanager
202
-
def make_context_manager(x):
203
-
try:
204
-
yield x
205
-
finally:
206
-
print("Cleaning up")
207
-
```
208
-
209
-
This isn't anything fancier than just using a coroutine that returns a regular `with`-compatible context manager. See `tinyio.Semaphore` for an example of this pattern.
210
-
211
-
</details>
212
-
213
-
<details><summary><h3>Asynchronous iterators</h3> (Click to expand)</summary>
214
-
215
-
You can use the following pattern to implement asychronous iterators:
216
-
217
-
```python
218
-
def my_coro():
219
-
for x in slow_range(5):
220
-
x = yield x
221
-
print(f"Got {x}")
222
-
```
223
-
where:
224
-
```python
225
-
def slow_range(x): # this function is an iterator-of-coroutines
226
-
for i inrange(x):
227
-
yield slow_range_i(i) # this `yield` statement is seen by the `for` loop
228
-
229
-
def slow_range_i(i): # this function is a coroutine
230
-
yield tinyio.sleep(1) # this `yield` statement is seen by the `tinyio.Loop()`
231
-
return i
232
-
```
233
-
234
-
Here we just have `yield` being used in a couple of different ways that you're already used to:
235
-
-as a regular Python generator/iterator;
236
-
-as a `tinyio` coroutine.
237
-
238
-
For an example of this, see `tinyio.as_completed`.
239
-
240
-
</details>
241
-
242
-
<details><summary><h3>Integration with`asyncio`and`trio`</h3> (Click to expand)</summary>
243
-
244
-
We have support for putting `trio` event loops within `asyncio`/`trio` event loops, or vice-versa.
245
-
246
-
```python
247
-
tinyio.to_asyncio tinyio.to_trio
248
-
tinyio.from_asyncio tinyio.from_trio
249
-
```
250
-
251
-
---
252
-
253
-
-`tinyio.to_asyncio(coro, exception_group=None)`
254
-
255
-
This converts a `tinyio` coroutine into an `asyncio` coroutine.
256
-
257
-
For example:
258
-
```python
259
-
def add_one(x):
260
-
yield tinyio.sleep(1)
261
-
return x +1
262
-
263
-
asyncdef foo(x):
264
-
y = await tinyio.to_asyncio(add_one(x))
265
-
return y
266
-
267
-
asyncio.run(foo(3))
268
-
```
269
-
270
-
---
271
-
272
-
-`tinyio.from_asyncio(coro)`
273
-
274
-
This converts an `asyncio` coroutine into a `tinyio` coroutine.
275
-
276
-
>WARNING!
277
-
> This works by running the entire `asyncio` portion in a separate thread. This may lead to surprises if the `asyncio`and non-`asyncio` portions interact in non-threadsafe ways.
278
-
279
-
For example:
280
-
```python
281
-
asyncdefadd_one(x):
282
-
await asyncio.sleep(1)
283
-
return x +1
284
-
285
-
deffoo(x):
286
-
y =yield tinyio.from_asyncio(add_one(x))
287
-
return y
288
-
289
-
tinyio.Loop().run(foo(3))
290
-
```
291
-
292
-
---
293
-
294
-
-`tinyio.to_trio(coro, exception_group=None)`
295
-
296
-
This converts a `tinyio` coroutine into an `trio` coroutine.
297
-
298
-
For example:
299
-
```python
300
-
defadd_one(x):
301
-
yield tinyio.sleep(1)
302
-
return x +1
303
-
304
-
asyncdeffoo(x):
305
-
y =await tinyio.to_trio(add_one(x))
306
-
return y
307
-
308
-
trio.run(foo, 3)
309
-
```
310
-
311
-
---
312
-
313
-
-`tinyio.from_trio(coro)`
314
-
315
-
This converts an `trio` coroutine into a `tinyio` coroutine.
316
-
317
-
For example:
318
-
```python
319
-
asyncdefadd_one(x):
320
-
await trio.sleep(1)
321
-
return x +1
322
-
323
-
deffoo(x):
324
-
y =yield tinyio.from_trio(add_one(x))
325
-
return y
326
-
327
-
tinyio.Loop().run(foo(3))
328
-
```
329
-
330
-
---
331
-
332
-
</details>
333
-
334
-
<details><summary><h3>Isolating some coroutines without crashing the others</h3> (Click to expand)</summary>
335
-
336
-
If you would like to run a coroutine (and transitively, all the coroutines it `yield`s) without crashing the entire event loop, then you can use `tinyio.isolate`. This will run the coroutine in a nested `tinyio.Loop` – so that it crashing will only affect everything in that nested loop – and then return the result or the error.
# at this point then `someone_elses_buggy_code`, `some_coroutine` and
351
-
#`another_coroutine` have been cancelled, but `main` is still running.
352
-
```
353
-
354
-
</details>
47
+
Available at [https://docs.kidger.site/tinyio](https://docs.kidger.site/tinyio).
355
48
356
49
## FAQ
357
50
358
-
(Click to expand)
359
-
360
51
<details>
361
-
<summary>Why <code>yield</code> - why not <code>await</code> like is normally seen for coroutines?</summary>
52
+
<summary>Why <code>yield</code> — why not <code>await</code> like is normally seen for coroutines?</summary>
362
53
<br>
363
54
364
55
The reason is that `await` does not offer a suspension point to an event loop (it just calls `__await__` and maybe *that* offers a suspension point), so if we wanted to use that syntax then we'd need to replace `yield coro` with something like `await tinyio.Task(coro)`. The traditional syntax is not worth the extra class.
0 commit comments