Skip to content

Commit b9a1c46

Browse files
committed
Fix all the code blocks in the markdown files
1 parent 88690f1 commit b9a1c46

10 files changed

Lines changed: 486 additions & 294 deletions

docs/User/Explanatory/unified-abstractions.md

Lines changed: 63 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -79,11 +79,15 @@ API.
7979
This simple console echo program shows the simplicity of Streamly API
8080
and its similarity with the list API:
8181

82-
```haskell
82+
```haskell unshared
83+
import Data.Function ((&))
84+
import qualified Streamly.Data.Stream as Stream
85+
import qualified Streamly.Data.Fold as Fold
86+
8387
echo =
8488
Stream.repeatM getLine
8589
& Stream.mapM putStrLn
86-
& Stream.drain
90+
& Stream.fold Fold.drain
8791
```
8892

8993
Streamly uses dual representation for streams. On top it uses Scott
@@ -105,20 +109,35 @@ programmers use for what you call nested loops in imperative
105109
programming. List transformers are the basic implementations for
106110
non-determinism.
107111

108-
The stream monad in Streamly is a list-transformer with behavior similar
109-
to the list monad. It provides the functionality provided by `list-t` or
110-
`logict` packages for free. Here is an example of nested looping using
111-
the serial stream monad:
112+
You can use `Streamly.Data.Stream.MkType` to create a custom type and specify
113+
the behaviour of non-determinism. The basic `concatMap` would result in a
114+
list-transformer with behavior similar to the list monad. It provides the
115+
functionality provided by `list-t` or `logict` packages for free. Here is an
116+
example of nested looping using the new type:
112117

113118
```haskell
114-
import qualified Streamly.Prelude as S
115-
116-
loops = do
117-
x <- Stream.fromFoldable [1,2]
118-
y <- Stream.fromFoldable [3,4]
119-
Stream.fromEffect $ print (x, y)
119+
{-# LANGUAGE TemplateHaskell #-}
120+
{-# LANGUAGE UndecidableInstances #-}
121+
122+
import Streamly.Data.Stream.Prelude (Stream, MonadAsync)
123+
import qualified Streamly.Data.Stream.Prelude as Stream
124+
import qualified Streamly.Data.StreamK as StreamK
125+
import Streamly.Data.Stream.MkType
126+
127+
bindSerial :: Monad m => Stream m a -> (a -> Stream m b) -> Stream m b
128+
bindSerial = flip Stream.concatMap
129+
$(mkCrossType "CrossT" "bindSerial" False)
130+
131+
loopsCross :: Stream IO ()
132+
loopsCross = unCrossT $ do
133+
x <- CrossT $ Stream.fromList [1,2]
134+
y <- CrossT $ Stream.fromList [3,4]
135+
CrossT $ Stream.fromEffect $ print (x, y)
120136
```
121137

138+
`Streamly.Data.Stream.CrossStream` and `Streamly.Data.StreamK.CrossStreamK`
139+
provide the same functionality as `CrossT` type created above.
140+
122141
Moreover, the list transformer in Streamly can be concurrent. The Scott
123142
encoding of streams also avoids the quadratic performance issue of
124143
`logict`.
@@ -142,12 +161,18 @@ concurrent round-robin scheduling of streams.
142161

143162
The monad instance provides a convenient way to combine streams in
144163
different ways. It is just non-determinism with different flavors of
145-
scheduling behavior. The example from the previous section can be run
146-
with interleaved scheduling behavior as follows, without changing the
147-
code at all:
164+
scheduling behavior. The scheduling behavior can be controlled as follows.
148165

149166
```haskell
150-
main = Stream.drain $ Stream.fromWserial loops
167+
bindWSerialK = StreamK.bindWith StreamK.interleave
168+
bindWSerial a f = StreamK.toStream $ bindWSerialK (StreamK.fromStream a) (StreamK.fromStream . f)
169+
$(mkCrossType "WSerialT" "bindWSerial" True)
170+
171+
loopsWSerial :: Stream IO ()
172+
loopsWSerial = unWSerialT $ do
173+
x <- WSerialT $ Stream.fromList [1,2]
174+
y <- WSerialT $ Stream.fromList [3,4]
175+
WSerialT $ Stream.fromEffect $ print (x, y)
151176
```
152177

153178
Scheduling is fundamental to expressing many common programming problems
@@ -159,6 +184,9 @@ Please see [mini kanren implementation using streamly](https://github.com/compos
159184

160185
The same combinators that are used for serial streams e.g. 'unfoldrM',
161186
'replicateM', 'repeatM' work concurrently when used at the appropriate type.
187+
188+
There are several concurrent combinators in streamly which are prefixed with
189+
`par`. These combinators fit into the default composition very cleanly.
162190
It allows concurrent programs to be written declaratively and composed
163191
idiomatically. They are not much different than serial programs. See
164192
[Streamly vs async](/docs/User/HowTo/streamly-vs-async.md)
@@ -168,18 +196,33 @@ Streamly provides concurrent scheduling and looping similar to to
168196
[OpenMP](https://en.wikipedia.org/wiki/OpenMP) and
169197
[Cilk](https://en.wikipedia.org/wiki/Cilk) but with a more declarative
170198
style. The list transformer example can be run with concurrent
171-
execution of loop iterations as follows, without changing the code at
172-
all:
199+
execution of loop iterations as follows:
173200

174201
```haskell
175-
main = Stream.drain $ Stream.fromAhead loops
202+
bindAhead :: MonadAsync m => Stream m a -> (a -> Stream m b) -> Stream m b
203+
bindAhead = flip (Stream.parConcatMap (Stream.ordered True))
204+
$(mkCrossType "AheadT" "bindAhead" True)
205+
206+
loopsAhead :: Stream IO ()
207+
loopsAhead = unAheadT $ do
208+
x <- AheadT $ Stream.fromList [1,2]
209+
y <- AheadT $ Stream.fromList [3,4]
210+
AheadT $ Stream.fromEffect $ print (x, y)
176211
```
177212

178213
And interleaving with concurrent execution of the loop iterations can be
179214
written like this:
180215

181216
```haskell
182-
main = Stream.drain $ Stream.fromWAsync loops
217+
bindWAsync :: MonadAsync m => Stream m a -> (a -> Stream m b) -> Stream m b
218+
bindWAsync = flip (Stream.parConcatMap (Stream.interleaved True))
219+
$(mkCrossType "WAsyncT" "bindWAsync" True)
220+
221+
loopsWAsync :: Stream IO ()
222+
loopsWAsync = unWAsyncT $ do
223+
x <- WAsyncT $ Stream.fromList [1,2]
224+
y <- WAsyncT $ Stream.fromList [3,4]
225+
WAsyncT $ Stream.fromEffect $ print (x, y)
183226
```
184227

185228
All this comes with no change in the streaming APIs.

docs/User/HowTo/faq.md

Lines changed: 22 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -13,41 +13,45 @@ together to create a single transformed stream.
1313

1414
Distributing a value to a stream of consumers concurrently:
1515

16-
```haskell ghci
17-
{-# LANGUAGE FlexibleContexts #-}
18-
19-
import Data.Function ((&))
20-
import qualified Streamly.Data.Fold as Fold
21-
import qualified Streamly.Data.Stream.Prelude as Stream
16+
```haskell docspec
17+
>>> :set -XFlexibleContexts
18+
>>> import Data.Function ((&))
19+
>>> import qualified Streamly.Data.Fold as Fold
20+
>>> import qualified Streamly.Data.Stream.Prelude as Stream
2221

22+
>>> :{
2323
f1 x =
2424
Stream.fromList [return . (+ 1), return . (+ 2)] -- Stream of functions
2525
& fmap ($ x) -- Stream of lazy actions
2626
& Stream.parSequence (Stream.ordered True) -- Evaluate concurrently
2727
& Stream.fold Fold.toList -- Fold to list
28+
:}
2829
```
2930

3031
Use `parZipWith` to zip streams concurrently. Here, we zip three singleton
3132
streams:
3233

33-
```haskell ghci
34-
34+
```haskell docspec
35+
>>> :{
3536
f2 x =
3637
let app = Stream.parZipWith id ($)
3738
in (,,)
3839
`fmap` Stream.fromEffect (return $ show x)
3940
`app` Stream.fromEffect (return $ x + 1)
4041
`app` Stream.fromEffect (return $ fromIntegral x / 2)
4142
& Stream.fold Fold.one
43+
:}
4244
```
4345

4446
Applying a function concurrently to your input stream:
4547

46-
```haskell ghci
48+
```haskell docspec
49+
>>> :{
4750
g f xs =
4851
Stream.fromList xs
4952
& Stream.parMapM (Stream.ordered True) f
5053
& Stream.fold Fold.toList
54+
:}
5155
```
5256

5357
You can now use the concurrent map to pipe each element through multiple
@@ -66,22 +70,25 @@ create a zip Applicative newtype so that you can use the `Applicative`
6670
instance.
6771

6872
```haskell
69-
{-# LANGUAGE UndecidableInstances #-}
70-
{-# LANGUAGE FlexibleInstances #-}
71-
{-# LANGUAGE MultiParamTypeClasses #-}
7273
{-# LANGUAGE TemplateHaskell #-}
74+
{-# LANGUAGE UndecidableInstances #-}
7375

74-
import Streamly.Internal.Data.Stream.TypeGen
76+
import Control.Monad.Trans.Control (MonadBaseControl)
77+
import Streamly.Data.Stream.Prelude (MonadAsync, Stream)
78+
import qualified Streamly.Data.Stream.Prelude as Stream
79+
import Streamly.Data.Stream.MkType
7580

76-
app = parZipWith id ($)
77-
$(mkZippingType "ZipConcurrent" "app" True)
81+
app :: MonadAsync m => Stream m (a -> b) -> Stream m a -> Stream m b
82+
app = Stream.parZipWith id ($)
83+
$(mkZipType "ZipConcurrent" "app" True)
7884
```
7985

8086
## Sliding Window
8187

8288
The `createOfLast` fold can be used to create a stream of sliding windows.
8389

8490
```haskell docspec
91+
>>> :set -Wno-deprecations
8592
>>> import qualified Streamly.Data.Array as Array
8693
>>> :{
8794
Stream.fromList [1,2,3,4,5::Int]

docs/User/HowTo/streamly-vs-async.md

Lines changed: 37 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -48,11 +48,12 @@ stream folding operations can also be used, see the docs for more details.
4848
Use the following imports to run the snippets shown below:
4949

5050
```haskell
51-
import Streamly
52-
import Streamly.Prelude ((|:))
53-
import qualified Streamly.Prelude as S
51+
import qualified Streamly.Data.Stream.Prelude as S
52+
import qualified Streamly.Data.Fold as F
5453
import qualified Data.Text as Text
5554
import Control.Concurrent (threadDelay)
55+
import Control.Exception (Exception, SomeException)
56+
import Control.Monad.Catch (throwM, try)
5657
```
5758

5859
Let us simulate a URL fetch with a delay of `n` seconds using the following
@@ -71,7 +72,7 @@ You can run any number of actions concurrently. For example, to fetch two URLs
7172
concurrently:
7273

7374
```haskell
74-
urls <- S.toList $ fromParallel $ getURL 2 |: getURL 1 |: S.nil
75+
getUrlsConcurrently = S.parSequence id $ S.fromList [getURL 2, getURL 1]
7576
```
7677

7778
This would return the results in their arrival order i.e. first 1 and then 2.
@@ -81,26 +82,25 @@ concurrently, and even though URL 1 arrives before URL 2 the results will
8182
return 2 first and then 1.
8283

8384
```haskell
84-
urls <- S.toList $ fromAhead $ getURL 2 |: getURL 1 |: S.nil
85+
getUrlsOrdered = S.parSequence (S.ordered True) $ S.fromList [getURL 2, getURL 1]
8586
```
8687

8788
### concurrently_
8889

8990
Use `drain` instead of `toList` to run the actions but ignore the results:
9091

9192
```haskell
92-
S.drain $ fromParallel $ getURL 1 |: getURL 2 |: S.nil
93+
drainUrlsConcurrently = S.fold F.drain $ S.parMapM id getURL $ S.fromList [1, 2]
9394
```
9495

95-
### Concurrent Applicative
96+
### Concurrent Zipping
9697

9798
If the actions that you are executing result in different output types you can
98-
use applicative zip to collect the results or to directly apply them to a
99-
function:
99+
use zip to collect the results or to directly apply them to a function:
100100

101101
```haskell
102-
tuples <- S.toList $ fromZipAsync $
103-
(,) <$> S.fromEffect (getURLString 1) <*> S.fromEffect (getURLText 2)
102+
concurrentZipping =
103+
S.parZipWith id (,) (S.fromEffect (getURLString 1)) (S.fromEffect (getURLText 2))
104104
```
105105

106106
### race
@@ -114,14 +114,14 @@ We can run multiple actions concurrently and take the first result that
114114
arrives:
115115

116116
```haskell
117-
urls <- S.toList $ S.take 1 $ fromParallel $ getURL 1 |: getURL 2 |: S.nil
117+
fastest = S.toList $ S.take 1 $ S.parSequence id $ S.fromList [getURL 1, getURL 2]
118118
```
119119

120120
After the first result arrives, the rest of the actions are canceled
121121
automatically. In general, we can take first `n` results as they arrive:
122122

123123
```haskell
124-
urls <- S.toList $ S.take 2 $ fromParallel $ getURL 1 |: getURL 2 |: S.nil
124+
fastestN n = S.toList $ S.take n $ S.parSequence id $ S.fromList [getURL 1, getURL 2]
125125
```
126126

127127
#### `race` Using Exceptions
@@ -133,54 +133,38 @@ exception to communicate the result. As soon as the first result arrives all
133133
other actions will be canceled, for example:
134134

135135
```haskell
136-
data Result = Result String deriving Show
137-
instance Exception Result
138-
139-
main = do
140-
url <- try $ S.drain $ fromParallel $
141-
(getURL 2 >>= throwM . Result)
142-
|: (getURL 1 >>= throwM . Result)
143-
|: S.nil
144-
case url of
145-
Left (e :: SomeException) -> print e
146-
Right _ -> undefined
136+
data Result = Result String deriving Show
137+
instance Exception Result
138+
139+
raceUsingExceptions = do
140+
url <- try $ S.fold F.drain $ S.parSequence id $ S.fromList
141+
[ (getURL 2 >>= throwM . Result)
142+
, (getURL 1 >>= throwM . Result)
143+
]
144+
case url of
145+
Left (e :: SomeException) -> print e
146+
Right _ -> undefined
147147
```
148148

149149
### mapConcurrently
150150

151-
There are many ways to map concurrently on a container and collect the results:
152-
153-
You can create a concurrent stream from a `Foldable` container of monadic
154-
actions:
155-
156-
```haskell
157-
urls <- S.toList $ fromAhead $ S.fromFoldableM $ fmap getURL [1..3]
158-
```
159-
160-
You can first convert a `Foldable` into a stream and then map an action on the
161-
stream concurrently:
151+
There are many ways to map concurrently on a container and collect the results.
152+
The recommended way is to first convert itinto a stream and then map an action
153+
on the stream concurrently:
162154

163155
```haskell
164-
urls <- S.toList $ fromAhead $ S.mapM getURL $ foldMap return [1..3]
165-
```
166-
167-
You can map a monadic action to a `Foldable` container to convert it into a
168-
stream and at the same time fold it:
169-
170-
```haskell
171-
urls <- S.toList $ fromAhead $ foldMap (S.fromEffect . getURL) [1..3]
156+
mapConcurrently = S.toList $ S.parMapM id getURL $ S.fromList [1..3]
172157
```
173158

174159
### replicateConcurrently
175160

176161
Streamly has not just the equivalent of `replicateConcurrently` which is
177-
`replicateM` but many more ways to generate concurrent streams, for example,
178-
`|:`, `unfoldrM`, `repeatM`, `iterateM`, `fromFoldableM` etc. See the
179-
[Streamly.Prelude](https://hackage.haskell.org/package/streamly/docs/Streamly-Prelude.html)
180-
module documentation for more details.
162+
`parReplicateM` but many more ways to generate concurrent streams, for example,
163+
`parRepeatM`, `parBuffered`, etc. See the `Streamly.Data.Stream.Prelude` module
164+
documentation for more details.
181165

182166
```haskell
183-
xs <- S.toList $ fromParallel $ S.replicateM 2 $ getURL 1
167+
replicateConcurrently = S.toList $ S.parReplicateM id 2 $ getURL 1
184168
```
185169

186170
### Functor
@@ -191,14 +175,15 @@ concurrently.
191175
To map serially just use `fmap`:
192176

193177
```haskell
194-
xs <- S.toList $ fromParallel $ fmap (+1) $ return 1 |: return 2 |: S.nil
178+
serialMap = S.toList $ fmap (+1) (S.fromList [1, 2] :: S.Stream IO Int)
179+
195180
```
196181

197-
To map a monadic action concurrently on all elements of the stream use `mapM`:
182+
To map a monadic action concurrently on all elements of the stream use `parMapM`:
198183

199184
```haskell
200-
xs <- S.toList $ fromParallel $ S.mapM (\x -> return (x + 1))
201-
$ return 1 |: return 2 |: S.nil
185+
concurrentMap =
186+
S.toList $ S.parMapM id (\x -> pure (x + 1) :: IO Int) $ S.fromList [1, 2]
202187
```
203188

204189
### Semigroup

0 commit comments

Comments
 (0)