You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/hpc/08_ml_ai_hpc/03_llm_on_hpc.md
+9-9Lines changed: 9 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -76,14 +76,14 @@ You can find more information about using Singularity and Conda on our HPC syste
76
76
## Prepare script
77
77
Create a python script using the following code from sections 1-9 and save it in a file called `huggingface.py`:
78
78
79
-
1.Import necessary modules:
79
+
1. Import necessary modules:
80
80
```python
81
81
import torch
82
82
import numpy as np
83
83
from transformers import AutoTokenizer, AutoModel
84
84
```
85
85
86
-
1. Create a list of reviews:
86
+
1. Create a list of reviews:
87
87
```python
88
88
texts = ["How do I get a replacement Medicare card?",
89
89
"What is the monthly premium for Medicare Part B?",
@@ -93,43 +93,43 @@ Create a python script using the following code from sections 1-9 and save it in
93
93
"How do I sign up for Medicare Part B if I already have Part A?"]
94
94
```
95
95
96
-
1. Choose the model name from huggingface’s model hub and instantiate the model and tokenizer objectfor the given model. We are setting `output_hidden_states`as`True`as we want the output of the model to not only have loss, but also the embeddings for the sentences.
96
+
1. Choose the model name from huggingface’s model hub and instantiate the model and tokenizer object for the given model. We are setting `output_hidden_states` as `True` as we want the output of the model to not only have loss, but also the embeddings for the sentences.
model = AutoModel.from_pretrained(model_name, output_hidden_states=True)
101
101
```
102
102
103
-
1. Create the ids to be used in the model using the tokenizer object. We set the return_tensors as “pt” as we want to return the pytorch tensor of the ids:
103
+
1. Create the ids to be used in the model using the tokenizer object. We set the return_tensors as “pt” as we want to return the pytorch tensor of the ids:
1. Set the device to cuda, and move the model and the tokenizer to cuda as well. Since, we will be extracting embeddings, we will only be performing a forward pass of the model and hence we will set the model to validation mode using `eval()`:
108
+
1. Set the device to cuda, and move the model and the tokenizer to cuda as well. Since, we will be extracting embeddings, we will only be performing a forward pass of the model and hence we will set the model to validation mode using `eval()`:
109
109
```python
110
110
device = 'cuda' if torch.cuda.is_available() else 'cpu'
111
111
model.to(device)
112
112
ids = ids.to(device)
113
113
model.eval()
114
114
```
115
115
116
-
1. Performing the forward passand storing the output tuplein out:
116
+
1. Performing the forward pass and storing the output tuple in out:
117
117
```python
118
118
with torch.no_grad():
119
119
out = model(**ids)
120
120
```
121
121
122
-
1. Extracting the embeddings of each review from the last layer:
122
+
1. Extracting the embeddings of each review from the last layer:
123
123
```python
124
124
last_hidden_states = out.last_hidden_state
125
125
```
126
126
127
-
1. For the purpose of classification, we are extracting the CLS token which is the first embedding in the embedding listfor each review:
127
+
1. For the purpose of classification, we are extracting the CLS token which is the first embedding in the embedding list for each review:
128
128
```python
129
129
sentence_embedding = last_hidden_states[:, 0, :]
130
130
```
131
131
132
-
1. We can check the shape of the final sentence embeddings forall the reviews. The output should look like `torch.Size([6, 768])`, where 6is the batch size as we input6 reviews as shown in step `2b`, and768is the embedding size of the RoBERTa model used.
132
+
1. We can check the shape of the final sentence embeddings for all the reviews. The output should look like `torch.Size([6, 768])`, where 6 is the batch size as we input 6 reviews as shown in step `2b`, and 768 is the embedding size of the RoBERTa model used.
133
133
```python
134
134
print("Shape of the batch embedding: {}".format(sentence_embedding.shape))
0 commit comments