Skip to content

Commit 0f50daa

Browse files
committed
Updated readme and docs (mostly info about 'get_model_from_name' function)
1 parent 298b636 commit 0f50daa

3 files changed

Lines changed: 28 additions & 2 deletions

File tree

README.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,15 @@ In order to reproduce the above example you can run the following command:
5151
thunder benchmark phikon break_his knn
5252
```
5353

54+
### Extracting embeddings with any supported foundation model (API Usage)
55+
We also provide a [`get_model_from_name`](https://mics-lab.github.io/thunder/api/#thunder.models.get_model_from_name) function through our API to extract embeddings using any foundation model we support on your own data. Below is an example if you want to get the Pytorch callable, transforms and function to extract embeddings for `uni2h`:
56+
57+
```python
58+
from thunder.models import get_model_from_name
59+
60+
model, transform, get_embeddings = get_model_from_name("uni2h", device="cuda")
61+
```
62+
5463
## Installing thunder
5564

5665
Code tested with Python 3.10. To replicate, you can create the following conda environment and activate it,

docs/get_model.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
In addition to benchmarking, we also provide a [`get_model_from_name`](https://mics-lab.github.io/thunder/api/#thunder.models.get_model_from_name) function through our API to extract embeddings using any foundation model we support on your own data. It returns a Pytorch model callable, associated transforms (to apply to images to extract embeddings from) and a function to get embeddings from a batch of transformed images, the model callable, and the type of embeddings (pooled or spatial) to return.
2+
3+
Below is an example if you want to get the Pytorch callable, transforms and function to extract embeddings for `uni2h`:
4+
5+
```python
6+
from thunder.models import get_model_from_name
7+
8+
model, transform, get_embeddings = get_model_from_name("uni2h", device="cuda")
9+
```
10+
11+
You can then extract the embedding of a PIL image `im` as follows:
12+
13+
```python
14+
pooled_emb = get_embeddings(transform(im), model, pooled_emb=True)
15+
spatial_emb = get_embeddings(transform(im), model, pooled_emb=False)
16+
```

mkdocs.yml

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,11 +10,12 @@ nav:
1010
- Home: index.md
1111
- Tutorials:
1212
- Getting Started: getting_started.md
13-
- Benchmark a Custom Model: custom_model.md
13+
- Benchmarking a Custom Model: custom_model.md
1414
- Overriding Configurations: custom_config.md
15+
- Using foundation models on your data: get_model.md
1516
- API: api.md
1617

17-
theme:
18+
theme:
1819
name: material
1920
logo: logo.svg
2021
favicon: logo.svg

0 commit comments

Comments
 (0)