Skip to content

Commit 283e5a9

Browse files
authored
Merge pull request #51 from mbarbin/use-mdexp
Use mdexp
2 parents 781fe9b + acaa68a commit 283e5a9

42 files changed

Lines changed: 5344 additions & 2084 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/ci.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,12 @@ jobs:
3939
# janestreet-bleeding: https://github.com/janestreet/opam-repository.git
4040
# janestreet-bleeding-external: https://github.com/janestreet/opam-repository.git#external-packages
4141

42+
- name: Setup mdexp
43+
uses: mbarbin/mdexp-actions/setup-mdexp@42da13e622de9559da363ef5906ffde63a982efd # v1.0.0-alpha.1
44+
with:
45+
mdexp-version: 0.0.20260315
46+
mdexp-digest: sha256:f4fc53bcaa50c9dd979b968804c38322b9b7e6aa699d9d6d2d1f101965332018
47+
4248
- name: Install dependencies
4349
run: opam install . --deps-only --with-doc --with-test --with-dev-setup
4450

Lines changed: 138 additions & 93 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,28 @@
11
# Lookup Strategy
22

3-
In this part of the documentation, we will set aside typing considerations and focus on the strategy used to search for a trait at runtime within a provider construct.
3+
In this part of the documentation, we will set aside typing considerations
4+
and focus on the strategy used to search for a trait at runtime within a
5+
provider construct.
46

5-
We will describe the currently implemented strategy and discuss possible trade-offs.
7+
We will describe the currently implemented strategy and discuss possible
8+
trade-offs.
69

710
## Runtime representation
811

9-
Under the hood, a provider is an array where each cell is a binding. This binding is a pair that associates a trait with its implementation.
12+
Under the hood, a provider is an array where each cell is a binding. This
13+
binding is a pair that associates a trait with its implementation.
1014

11-
To simplify, we will consider traits as integers and implementations as strings. This makes the examples easier to read.
15+
To simplify, we will consider traits as integers and implementations as
16+
strings. This makes the examples easier to read.
1217

1318
```ocaml
1419
type impl = string
15-
1620
type trait = int
17-
1821
type provider = (trait * impl) array
1922
```
2023

21-
We are interested in defining lookup strategies — functions that return the implementation bound to a trait if the provider implements it.
24+
We are interested in defining lookup strategies — functions that return the
25+
implementation bound to a trait if the provider implements it.
2226

2327
```ocaml
2428
type lookup_strategy = provider -> trait -> impl option
@@ -31,30 +35,38 @@ type lookup_strategy = provider -> trait -> impl option
3135
One possible lookup strategy is a linear scan of the array. For example:
3236

3337
```ocaml
34-
open! Base
35-
36-
let linear_scan : lookup_strategy = fun provider trait ->
37-
Array.find_map provider ~f:(fun (t, impl) ->
38-
Option.some_if (Int.equal t trait) impl)
38+
let linear_scan : lookup_strategy =
39+
fun provider trait ->
40+
Array.find_map provider ~f:(fun (t, impl) -> Option.some_if (Int.equal t trait) impl)
3941
;;
40-
```
4142
42-
```ocaml
43-
# linear_scan [| 134, "Hello" ; 7, "World" |] 13 ;;
44-
- : impl option/2 = None
43+
let print_dyn dyn = Stdlib.Format.printf "%a@." Pp.to_fmt (Dyn.pp dyn)
44+
45+
let test scan provider trait =
46+
print_dyn (scan provider trait |> Dyn.option Dyn.string);
47+
()
48+
;;
4549
46-
# linear_scan [| 134, "Hello" ; 7, "World" |] 134 ;;
47-
- : impl option/2 = Some "Hello"
50+
let%expect_test "linear_scan" =
51+
test linear_scan [| 134, "Hello"; 7, "World" |] 13;
52+
[%expect {| None |}];
53+
test linear_scan [| 134, "Hello"; 7, "World" |] 134;
54+
[%expect {| Some "Hello" |}];
55+
()
56+
;;
4857
```
4958

50-
This strategy is simple and does not require any specific ordering within the bindings. It runs in `O(n)`, where `n` is the number of traits. When `n` is small, this is likely to be very fast, if not the fastest implementation.
59+
This strategy is simple and does not require any specific ordering within the
60+
bindings. It runs in `O(n)`, where `n` is the number of traits. When `n` is
61+
small, this is likely to be very fast, if not the fastest implementation.
5162

5263
### Binary Search
5364

5465
This is a slight twist on the linear scan.
5566

5667
```ocaml
57-
let binary_search : lookup_strategy = fun provider trait ->
68+
let binary_search : lookup_strategy =
69+
fun provider trait ->
5870
Binary_search.binary_search
5971
provider
6072
~length:Array.length
@@ -66,49 +78,65 @@ let binary_search : lookup_strategy = fun provider trait ->
6678
;;
6779
```
6880

69-
In this strategy, the bindings need to be ordered in increasing order according to their traits. Typically, a provider is constructed once and then passed around and used many times during its lifetime. This approach lends itself well to the idea of performing a bit more computation upfront to achieve a systematic speedup during subsequent use.
81+
In this strategy, the bindings need to be ordered in increasing order
82+
according to their traits. Typically, a provider is constructed once and then
83+
passed around and used many times during its lifetime. This approach lends
84+
itself well to the idea of performing a bit more computation upfront to
85+
achieve a systematic speedup during subsequent use.
7086

7187
```ocaml
7288
let make_provider bindings =
73-
bindings
74-
|> List.sort ~compare:(fun (i, _) (j, _) -> Int.compare i j)
75-
|> Array.of_list
89+
bindings |> List.sort ~compare:(fun (i, _) (j, _) -> Int.compare i j) |> Array.of_list
7690
;;
7791
78-
let provider =
79-
make_provider
80-
[ 99, "Hello" ; 7, "World" ; 134, "Foo" ; 17, "Bar" ]
92+
let provider = make_provider [ 99, "Hello"; 7, "World"; 134, "Foo"; 17, "Bar" ]
93+
94+
let provider_to_dyn a =
95+
Dyn.array (fun (trait, impl) -> Dyn.Tuple [ Dyn.int trait; Dyn.string impl ]) a
8196
;;
82-
```
83-
```ocaml
84-
# provider ;;
85-
- : (trait * impl) array =
86-
[|(7, "World"); (17, "Bar"); (99, "Hello"); (134, "Foo")|]
8797
88-
# binary_search provider 13 ;;
89-
- : impl option/2 = None
98+
let%expect_test "sorted provider" =
99+
print_dyn (provider |> provider_to_dyn);
100+
[%expect {| [| (7, "World"); (17, "Bar"); (99, "Hello"); (134, "Foo") |] |}];
101+
()
102+
;;
90103
91-
# binary_search provider 99 ;;
92-
- : impl option/2 = Some "Hello"
104+
let%expect_test "binary_search" =
105+
test binary_search provider 13;
106+
[%expect {| None |}];
107+
test binary_search provider 99;
108+
[%expect {| Some "Hello" |}];
109+
()
110+
;;
93111
```
94112

95113
### Cache
96114

97-
If we are frequently searching for the same trait multiple times in a row, we might benefit from using a cache.
115+
If we are frequently searching for the same trait multiple times in a row, we
116+
might benefit from using a cache.
98117

99-
When we started implementing the library, we were unsure whether a cache would be useful. We decided to implement a simple caching mechanism as a proof of concept. This approach allowed us to evaluate whether we needed to change the representation early on, avoiding potential incompatible changes in later versions.
118+
When we started implementing the library, we were unsure whether a cache
119+
would be useful. We decided to implement a simple caching mechanism as a
120+
proof of concept. This approach allowed us to evaluate whether we needed to
121+
change the representation early on, avoiding potential incompatible changes
122+
in later versions.
100123

101-
The strategy we implemented uses the first cell (index 0) of the array to store the most recently accessed trait. The rest of the array (from index 1 to n) is kept sorted by trait and is searched using a binary search.
124+
The strategy we implemented uses the first cell (index 0) of the array to
125+
store the most recently accessed trait. The rest of the array (from index 1
126+
to n) is kept sorted by trait and is searched using a binary search.
102127

103128
```ocaml
104-
let binary_search_with_cache : lookup_strategy = fun provider trait ->
129+
let binary_search_with_cache : lookup_strategy =
130+
fun provider trait ->
105131
if Array.length provider = 0
106-
then None
107-
else
132+
then None [@coverage off]
133+
else (
108134
let cache = provider.(0) in
109135
if Int.equal (fst cache) trait
110-
then (print_endline "Hitting the cache!"; Some (snd cache))
111-
else
136+
then (
137+
Stdlib.print_endline "Hitting the cache!";
138+
Some (snd cache))
139+
else (
112140
match
113141
Binary_search.binary_search
114142
provider
@@ -122,82 +150,99 @@ let binary_search_with_cache : lookup_strategy = fun provider trait ->
122150
| None -> None
123151
| Some i ->
124152
let impl = snd provider.(i) in
125-
provider.(0) <- (trait, impl);
126-
Some impl
153+
provider.(0) <- trait, impl;
154+
Some impl))
127155
;;
128156
129157
let make_provider_with_cache bindings =
130-
match
131-
bindings
132-
|> List.sort ~compare:(fun (i, _) (j, _) -> Int.compare i j)
133-
with
134-
| [] -> [||]
158+
match bindings |> List.sort ~compare:(fun (i, _) (j, _) -> Int.compare i j) with
159+
| [] -> [||] [@coverage off]
135160
| hd :: tl ->
136161
(* Initialize the cache arbitrarily with the smallest trait. *)
137162
Array.of_list (hd :: hd :: tl)
138163
;;
139-
```
140164
141-
```ocaml
142-
let provider =
143-
make_provider_with_cache
144-
[ 99, "Hello" ; 7, "World" ; 134, "Foo" ; 17, "Bar" ]
165+
let provider = make_provider_with_cache [ 99, "Hello"; 7, "World"; 134, "Foo"; 17, "Bar" ]
166+
167+
let%expect_test "binary_search_with_cache" =
168+
print_dyn (provider |> provider_to_dyn);
169+
[%expect
170+
{|
171+
[| (7, "World")
172+
; (7, "World")
173+
; (17, "Bar")
174+
; (99, "Hello")
175+
; (134, "Foo")
176+
|]
177+
|}];
178+
test binary_search_with_cache provider 13;
179+
[%expect {| None |}];
180+
test binary_search_with_cache provider 99;
181+
[%expect {| Some "Hello" |}];
182+
print_dyn (provider |> provider_to_dyn);
183+
[%expect
184+
{|
185+
[| (99, "Hello")
186+
; (7, "World")
187+
; (17, "Bar")
188+
; (99, "Hello")
189+
; (134, "Foo")
190+
|]
191+
|}];
192+
test binary_search_with_cache provider 99;
193+
[%expect
194+
{|
195+
Hitting the cache!
196+
Some "Hello"
197+
|}];
198+
()
145199
;;
146200
```
147-
```ocaml
148-
# provider ;;
149-
- : (trait * impl) array =
150-
[|(7, "World"); (7, "World"); (17, "Bar"); (99, "Hello"); (134, "Foo")|]
151-
152-
# binary_search_with_cache provider 13 ;;
153-
- : impl option/2 = None
154-
155-
# binary_search_with_cache provider 99 ;;
156-
- : impl option/2 = Some "Hello"
157-
158-
# provider ;;
159-
- : (trait * impl) array =
160-
[|(99, "Hello"); (7, "World"); (17, "Bar"); (99, "Hello"); (134, "Foo")|]
161-
162-
# binary_search_with_cache provider 99 ;;
163-
Hitting the cache!
164-
165-
- : impl option/2 = Some "Hello"
166-
```
167201

168202
## Future Plans
169203

170-
We are not entirely convinced that the cache is useful. It is difficult to determine its utility without more information about how the library is actually used. Additionally, we envision that:
204+
We are not entirely convinced that the cache is useful. It is difficult to
205+
determine its utility without more information about how the library is
206+
actually used. Additionally, we envision that:
171207

172-
- Defaulting to a linear scan may be faster overall in cases where the number of traits is small.
173-
- The cache introduces complexity if the provider is accessed by multiple domains in parallel. In such cases, you may prefer to disable it entirely.
208+
- Defaulting to a linear scan may be faster overall in cases where the number
209+
of traits is small.
210+
- The cache introduces complexity if the provider is accessed by multiple
211+
domains in parallel. In such cases, you may prefer to disable it entirely.
174212

175-
Therefore, our future plans may include tweaking the lookup strategy, conducting benchmarks, and providing more control to the user.
213+
Therefore, our future plans may include tweaking the lookup strategy,
214+
conducting benchmarks, and providing more control to the user.
176215

177216
We are not concerned about breaking changes because:
178217

179-
1. Building a provider is done through a function call to which we could add optional parameters, such as `~enable_cache:bool`.
218+
1. Building a provider is done through a function call to which we could add
219+
optional parameters, such as `~enable_cache:bool`.
180220

181-
2. There exists a simple criterion to determine whether a provider has a cache location at position 0:
221+
2. There exists a simple criterion to determine whether a provider has a
222+
cache location at position 0:
182223

183224
```ocaml
184225
let uses_a_cache provider =
185-
Array.length provider >= 2
186-
&& fst provider.(0) >= fst provider.(1)
226+
Array.length provider >= 2 && fst provider.(0) >= fst provider.(1)
187227
;;
188228
189-
let bindings = [ 99, "Hello" ; 7, "World" ; 134, "Foo" ; 17, "Bar" ]
190-
```
191-
```ocaml
192-
# uses_a_cache (make_provider bindings) ;;
193-
- : bool = false
229+
let bindings = [ 99, "Hello"; 7, "World"; 134, "Foo"; 17, "Bar" ]
194230
195-
# uses_a_cache (make_provider_with_cache bindings) ;;
196-
- : bool = true
231+
let%expect_test "uses_a_cache" =
232+
print_dyn (uses_a_cache (make_provider bindings) |> Dyn.bool);
233+
[%expect {| false |}];
234+
print_dyn (uses_a_cache (make_provider_with_cache bindings) |> Dyn.bool);
235+
[%expect {| true |}]
236+
;;
197237
```
198238

199-
This demonstrates how we can recognize whether caching was enabled at provider creation time and act accordingly.
239+
This demonstrates how we can recognize whether caching was enabled at
240+
provider creation time and act accordingly.
200241

201242
## Conclusion
202243

203-
In this documentation, we've explained the runtime representation of a provider and discussed different lookup strategies for searching traits. We've documented the current strategy, which uses a binary search and a simple cache, and concluded by offering some ideas for future improvements while maintaining good compatibility with the existing code.
244+
In this documentation, we've explained the runtime representation of a
245+
provider and discussed different lookup strategies for searching traits.
246+
We've documented the current strategy, which uses a binary search and a
247+
simple cache, and concluded by offering some ideas for future improvements
248+
while maintaining good compatibility with the existing code.
Lines changed: 28 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,29 @@
1-
(mdx
1+
(library
2+
(name provider_doc_lookup_strategy)
23
(package provider-dev)
3-
(deps
4-
(package provider)
5-
(glob_files *.txt))
6-
(preludes prelude.txt))
4+
(inline_tests)
5+
(flags :standard -w +a-4-40-41-42-44-45-48-66 -warn-error +a -open Base)
6+
(libraries base dyn pp)
7+
(instrumentation
8+
(backend bisect_ppx))
9+
(lint
10+
(pps ppx_js_style -allow-let-operators -check-doc-comments))
11+
(preprocess
12+
(pps ppx_expect)))
13+
14+
(rule
15+
(package provider-dev)
16+
(enabled_if %{bin-available:mdexp})
17+
(target lookup_strategy.md.gen)
18+
(deps lookup_strategy.ml)
19+
(action
20+
(with-stdout-to
21+
%{target}
22+
(run mdexp pp %{deps}))))
23+
24+
(rule
25+
(package provider-dev)
26+
(enabled_if %{bin-available:mdexp})
27+
(alias runtest)
28+
(action
29+
(diff README.md lookup_strategy.md.gen)))

0 commit comments

Comments
 (0)