Skip to content

Commit 5834612

Browse files
committed
Fix all .md files
1 parent 0de0140 commit 5834612

11 files changed

Lines changed: 24 additions & 24 deletions

docs/genai/05_how_to_guides/01_temperature.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
11
# Temperature
22

3-
Generating text (or images) from LLMs is inherently probabilistic. However, as an end user you have many parameters at your disposal to tweak the behavior of LLMs. Of these, temperature is the most commonly used. Broadly, it controls the randonmess of the generated text. A lower temperature produces more deterministic outputs, while a higher temperature produces more random "creative" output. For a more comprehensive explanation on this topic, refer to the following:
3+
Generating text (or images) from LLMs is inherently probabilistic. However, as an end user you have many parameters at your disposal to tweak the behavior of LLMs. Of these, temperature is the most commonly used. Broadly, it controls the randomness of the generated text. A lower temperature produces more deterministic outputs, while a higher temperature produces more random "creative" output. For a more comprehensive explanation on this topic, refer to the following:
44
- [How to generate text: using different decoding methods for language generation with Transformers](https://huggingface.co/blog/how-to-generate)
55
- [What is LLM Temperature?](https://www.ibm.com/think/topics/llm-temperature)
66

77
:::tip
88

9-
The effect of temperature is probabalistic, so you might need to run the script repeatedly to obtain a representative sample of generated text from the LLM.
9+
The effect of temperature is probabilistic, so you might need to run the script repeatedly to obtain a representative sample of generated text from the LLM.
1010

1111
:::
1212

docs/hpc/01_getting_started/02_getting_and_renewing_an_account.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88

99
[hpc account request form link for courses]: https://docs.google.com/forms/d/e/1FAIpQLSdehngqL1xso-YV6MOhplKNwxXjASHYnDtM_5THB3P2vrDKcg/viewform?usp=sf_link
1010

11-
This section deals with the eligibility for getting HPC accounts and the process to create new ones, renew existing oaccountsThis section deals with the eligibility for getting HPC accounts, the process to create new accounts, renew existing ones and touches on access policies after graduation fom NYU and access for non-NYU researchers.
11+
This section deals with the eligibility for getting HPC accounts and the process to create new ones, renew existing oaccountsThis section deals with the eligibility for getting HPC accounts, the process to create new accounts, renew existing ones and touches on access policies after graduation from NYU and access for non-NYU researchers.
1212

1313
:::info
1414

@@ -61,7 +61,7 @@ HPC faculty sponsors are expected to:
6161

6262
- Respond promptly to account-related requests from HPC staff
6363

64-
Each year, your sponosred users must renew their account. You will need to approve the renewal by logging into the [NYU Identity Management service][nyu ims link]. We have a [walkthrogh of the approval process here](./03_walkthrough_approve_hpc_account_request.md)
64+
Each year, your sponsored users must renew their account. You will need to approve the renewal by logging into the [NYU Identity Management service][nyu ims link]. We have a [walkthrogh of the approval process here](./03_walkthrough_approve_hpc_account_request.md)
6565

6666
## Bulk HPC Accounts for Courses
6767

@@ -95,7 +95,7 @@ If you are not part of a collaboration, your access to cluster will end together
9595

9696
:::note
9797

98-
In order to request a new HPC account or renew an expired one, you need to be connected to the NYU VPN if you are working remotely, Please see [instructions on how to install and use the NYU VPN][nyu vpn link]. Linux clients are not officially supported, however we were able to successfully use openVPN client. Here are installation and connection instructions for a debian linux distribution with apt pacakge manager:
98+
In order to request a new HPC account or renew an expired one, you need to be connected to the NYU VPN if you are working remotely, Please see [instructions on how to install and use the NYU VPN][nyu vpn link]. Linux clients are not officially supported, however we were able to successfully use openVPN client. Here are installation and connection instructions for a debian linux distribution with apt package manager:
9999

100100
```sh
101101
apt-get install openconnect

docs/hpc/01_getting_started/06_coldfront_requesting_an_allocation_request.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,9 @@ If you scroll down a bit, you'll see "+Request Resource Allocation" button.
1111
!["PI request allocation" section](./static/PI_request_allocation.png)
1212

1313
After clicking "+Request Resource Allocation", you'll see a list of resources you can request for.
14-
Basically, you'll see general "Univeristy HPC" which is default for all and school-wise resources (e.g., Tandon(Genric)) based on a school that your project belongs to.
15-
Please select a resource and fill in justification to complete the allocation reqeust process.
16-
!["Univeristy HPC resource" section](./static/uni_resource.png)
14+
Basically, you'll see general "University HPC" which is default for all and school-wise resources (e.g., Tandon(Generic)) based on a school that your project belongs to.
15+
Please select a resource and fill in justification to complete the allocation request process.
16+
!["University HPC resource" section](./static/uni_resource.png)
1717
!["School-wise resource" section](./static/school_resource.png)
1818

1919
Now your allocation request is created! You'll see an allocation request with "New" status.

docs/hpc/03_storage/03_data_transfers.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ logout
3333
### Linux & Mac Tools
3434
#### scp and rsync
3535
:::warning
36-
Please use Data Transfer Nodes (DTNs) with these tools. While one can transfer data while on login nodes, it is considered a bad practice because it can degrate the node's performance.
36+
Please use Data Transfer Nodes (DTNs) with these tools. While one can transfer data while on login nodes, it is considered a bad practice because it can degrade the node's performance.
3737
:::
3838

3939
Sometimes these two tools are convenient for transferring small files. Using the DTNs does not require to set up an SSH tunnel; use the hostname `dtn.hpc.nyu.edu` for one-step copying. See below for examples of commands invoked on the command line on a laptop running a Unix-like operating system:

docs/hpc/03_storage/08_transferring_cloud_storage_data_with_rclone.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -344,7 +344,7 @@ Please enter 'q' and we're done with configuration.
344344
345345
### Step 4: Transfer
346346
:::warning
347-
Please be sure to perform data transters on a data transfer node (DTN). It can degrade performace for other users to perform transfers on other types of nodes. For more information please see [Data Transfers](./03_data_transfers.md)
347+
Please be sure to perform data transters on a data transfer node (DTN). It can degrade performance for other users to perform transfers on other types of nodes. For more information please see [Data Transfers](./03_data_transfers.md)
348348
:::
349349
350350
Sample commands:

docs/hpc/05_submitting_jobs/01_slurm_submitting_jobs.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -312,7 +312,7 @@ or as a command-line option to sbatch when you submit the job:
312312
### Options for running many similar jobs
313313

314314
- `-a, --array=indexes`
315-
- Submit an array of jobs with array ids as specified. Array ids can be specified as a numerical range, a comma-seperated list of numbers, or as some combination of the two. Each job instance will have an environment variable `SLURM_ARRAY_JOB_ID` and `SLURM_ARRAY_TASK_ID`. For example:
315+
- Submit an array of jobs with array ids as specified. Array ids can be specified as a numerical range, a comma-separated list of numbers, or as some combination of the two. Each job instance will have an environment variable `SLURM_ARRAY_JOB_ID` and `SLURM_ARRAY_TASK_ID`. For example:
316316
- `--array=1-11`, to start an array job with index from 1 to 11
317317
- `--array=1-7:2`, to submit an array job with index step size 2
318318
- `--array=1-9%4`, to submit an array job with simultaneously running job elements set to 4
@@ -489,7 +489,7 @@ cd /scratch/$USER/myambertest
489489
pmemd.cuda -O
490490
```
491491

492-
The demo Amber job should take ~2 minutes to finish once it starts runnning. When the job is done, several output files are generated. Check the one named "mdout", which has a section most relevant here:
492+
The demo Amber job should take ~2 minutes to finish once it starts running. When the job is done, several output files are generated. Check the one named "mdout", which has a section most relevant here:
493493

494494
```sh
495495
|--------------------- INFORMATION ----------------------
@@ -599,7 +599,7 @@ If you do not request resources you will get the default settings. If after some
599599
- Enable X forwarding, so programs using a GUI can be used during the session (provided you have X forwarding to your workstation set up)
600600
- To leave an interactive batch session, type `exit` at the command prompt
601601

602-
Certain tasks need user iteraction - such as debugging and some GUI-based applications. However the HPC clusters rely on batch job scheduling to efficiently allocate resources. Interactive batch jobs allow these apparently conflicting requirements to be met.
602+
Certain tasks need user interaction - such as debugging and some GUI-based applications. However the HPC clusters rely on batch job scheduling to efficiently allocate resources. Interactive batch jobs allow these apparently conflicting requirements to be met.
603603

604604
### Interactive Bash Job Examples
605605

docs/hpc/05_submitting_jobs/02_slurm_main_commands.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -112,7 +112,7 @@ man squeue # for more information
112112
| ------- | ----------- |
113113
| `--me` | Prints queued jobs for the current user |
114114
| `--user` | Prints queued jobs under a specific user, or a comma list of users |
115-
| `--job` | Specify a comma seperated list of job IDs to display |
115+
| `--job` | Specify a comma separated list of job IDs to display |
116116
| `--help` | Print a help message describing all options squeue |
117117

118118
## sacct

docs/hpc/05_submitting_jobs/03_slurm_tutorial.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ In a High Performance Computing Cluster, such as the NYU-IT HPC Greene cluster,
66

77
Linux operating system ( in our case Red Hat Enterprise Linux) runs on each of the nodes individually. The resources are shared among many users for their technical or scientific computing purposes.
88

9-
Slurm is a cluster software layer built on top of the interconnected nodes, aiming at orchestrating the nodes' computing activites, so that the cluster could be viewed as a unified, enhanced and scalable computing system by its users.
9+
Slurm is a cluster software layer built on top of the interconnected nodes, aiming at orchestrating the nodes' computing activities, so that the cluster could be viewed as a unified, enhanced and scalable computing system by its users.
1010

1111
In NYU HPC clusters the users coming from many departments with various disciplines and subjects, with their own computing projects, impose on us very diverse requirements regarding hardware, software resources, and processing parallelism. Users submit jobs, which compete for computing resources.
1212

@@ -24,7 +24,7 @@ For an overview of useful Slurm commands, please read [Slurm Main Commands](./02
2424

2525
## Software and Environment Modules
2626

27-
Lmod, an Environment Module system, is a tool for managing multiple versions and configurations of software packages and is used by many HPC centers around the world. With Environment Modules, software packages are installed away from the base system directories, and for each pacakge, an associated modulefile describes what must be altered in a user's shell environment - such as the $PATH environment variable - in order to use the software package. The modulefile also describes dependencies and conflicts between this software package and other packages and versions.
27+
Lmod, an Environment Module system, is a tool for managing multiple versions and configurations of software packages and is used by many HPC centers around the world. With Environment Modules, software packages are installed away from the base system directories, and for each package, an associated modulefile describes what must be altered in a user's shell environment - such as the $PATH environment variable - in order to use the software package. The modulefile also describes dependencies and conflicts between this software package and other packages and versions.
2828

2929
To use a given software package, you load the corresponding module. Unloading the module afterwards cleanly undoes the changes that loading the modules made to your environment, thus freeing you to use other software packages that might have conflicted with the first one.
3030

@@ -135,7 +135,7 @@ You can submit the job with the following command:
135135
sbatch myscript.sbatch
136136
```
137137

138-
The command will result in the job queuing as it awaits resources to become available (which varies on the number of other jobs being run on the cluster and the resources requested). You can see the status of yor jobs with the following command:
138+
The command will result in the job queuing as it awaits resources to become available (which varies on the number of other jobs being run on the cluster and the resources requested). You can see the status of your jobs with the following command:
139139

140140
```sh
141141
squeue --me

docs/hpc/06_tools_and_software/02_conda_environments.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -145,7 +145,7 @@ If you installed extra packages using pip (Python)
145145

146146
In this you can use
147147
```sh
148-
export PYTHONNOUSERSITE=True ## to ingnore packages in ~/.local/lib/python<version>
148+
export PYTHONNOUSERSITE=True ## to ignore packages in ~/.local/lib/python<version>
149149
# save
150150
conda list --export > conda_requirements.txt
151151
pip freeze > pip_requirements.txt
@@ -162,7 +162,7 @@ However, this may fail if your conda environment is created as a sub-directory o
162162

163163
Installed extra packages using install.packages? (R)
164164

165-
Usecase: You need packages not availalbe in conda channels, and want to use install.packages.
165+
Usecase: You need packages not available in conda channels, and want to use install.packages.
166166

167167
Command `conda list --export` will not include packages installed by "install.packages". So, do not use `conda install` at all. To have reproducibility in this case you need to use Conda and renv together, as described below
168168

docs/hpc/06_tools_and_software/07_sqlite_handling_large_structured_data.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ Storing your data in the [SQLite](https://www.sqlite.org/index.html) format allo
1010
### Some use-cases
1111

1212
- You think you need MySQL, PostreSQL, etc for your ML project. Usually you don't
13-
- You have to deal with hundreds of GB of table-stuctured data (or larger) and your script (for whatever reason) can't be made parallel.
13+
- You have to deal with hundreds of GB of table-structured data (or larger) and your script (for whatever reason) can't be made parallel.
1414
- You would request a lot of RAM and work with data slowly.
1515
:::warning
1616
This would be a waste of RAM.

0 commit comments

Comments
 (0)