diff --git a/docs/userguides/anvil/access.md b/docs/userguides/anvil/access.md index b0b8f28f..4c365d0e 100644 --- a/docs/userguides/anvil/access.md +++ b/docs/userguides/anvil/access.md @@ -2,8 +2,10 @@ tags: - Anvil - ACCESS + - NAIRR authors: - jin456 + - hkashgar search: boost: 2 draft: true @@ -13,13 +15,15 @@ draft: true ## Obtaining an Account -Anvil is an ACCESS computing resource. To use Anvil, you must first create an ACCESS account and request an allocation through the [ACCESS Allocation Request System](https://allocations.access-ci.org/). +Anvil is an ACCESS and NAIRR computing resource. To use Anvil, you must first create an ACCESS account and request an allocation through the [ACCESS Allocation Request System](https://allocations.access-ci.org/) or [NAIRR Pilot](https://nairrpilot.org/). !!! tip "New users to existing projects" - If you are joining an existing project, all you need to do is [login or create your ACCESS account](https://allocations.access-ci.org/) and the manager of your project can add you themselves! + If you are joining an existing project with an **[ACCESS allocation](#access)**, [login or create your ACCESS account](https://allocations.access-ci.org/) and the project manager can add you through the ACCESS portal. If your project has a **[NAIRR allocation](#nairr)**, you still need an [ACCESS account](https://allocations.access-ci.org/), but user management and allocation administration are handled by the project manager through the [NAIRR Pilot portal](https://submit-nairr.xras.org/).
+## ACCESS + ### What is ACCESS? [Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS)](https://access-ci.org) is an NSF-funded program that manages access to the national research cyberinfrastructure (CI) resources. Any researcher who seeks to use one of these CI resources must follow ACCESS processes to get onto these resources. @@ -28,11 +32,9 @@ Anvil is an ACCESS computing resource. To use Anvil, you must first create an AC ACCESS coordinates a diverse set of resources including Anvil and other traditional HPC resources suited for resource-intensive CPU workloads, modern accelerator-based systems (e.g., GPU), as well as cloud resources. **Anvil provides both CPU and GPU resources as part of ACCESS.** A comprehensive list of all the ACCESS-managed resources can be found here along with their descriptions and ideal workloads: [https://allocations.access-ci.org/resources](https://allocations.access-ci.org/resources). -
- -## Access to Anvil +### Access to Anvil through ACCESS program -### Applying for a New Project +#### Applying for a New Project 1. Sign up for an ACCESS account (if you don’t have one already) at [https://allocations.access-ci.org](https://allocations.access-ci.org). 2. Prepare an allocation request with details of your proposed computational workflows (science, software needs), resource requirements, and a short CV. See the individual “Preparing Your … Request” pages for details on what documents are required: [https://allocations.access-ci.org/prepare-requests](https://allocations.access-ci.org/prepare-requests). @@ -41,7 +43,7 @@ ACCESS coordinates a diverse set of resources including Anvil and other traditio !!! tip "Help with proposal" Interested parties may contact the [ACCESS support](https://support.access-ci.org/) for help with an Anvil proposal. -### Joining an Existing Project +#### Joining an Existing Project 1. Sign up for an ACCESS account (if you don’t have one already) at [https://allocations.access-ci.org](https://allocations.access-ci.org). 2. Have the manager of the project add you through their admin portal. That simple! @@ -55,7 +57,7 @@ ACCESS coordinates a diverse set of resources including Anvil and other traditio You will use your Anvil username when logging into Anvil systems via SSH or your ACCESS username when using Open OnDemand. -### Which ACCESS tier should I choose? +#### Which ACCESS tier should I choose? As you can gather from [https://allocations.access-ci.org/project-types](https://allocations.access-ci.org/project-types), there are **four different tiers in ACCESS**. Broadly, these tiers provide increasing computational resources with corresponding stringent documentation and resource justification requirements. Furthermore, while Explore and Discover tier requests are reviewed on a rolling basis as they are submitted, Accelerate requests will be reviewed monthly and Maximize will be reviewed twice a year. The review period reflects the level of resources provided, and Explore and Discover applications are generally reviewed within a week. An important point to note is that ACCESS does not award you time on a specific computational resource (except for the Maximize tier). Users are awarded a certain number of ACCESS credits which they then exchange for time on a particular resource. Here are some guidelines on how to choose between the tiers: @@ -64,7 +66,7 @@ As you can gather from [https://allocations.access-ci.org/project-types](https:/ 3. If you would like to run simulations across multiple resources to identify the one best suited for you, Discover will provide you with sufficient credits to exchange across multiple systems. 4. One way of determining the appropriate tier is to determine what the credits would translate to in terms of computational resources. The [exchange calculator](https://allocations.access-ci.org/exchange_calculator) can be used to calculate what a certain number of ACCESS credits translates to in terms of “core-hours” or “GPU-hours” or “node-hours” on an ACCESS resource. For example: the maximum 400,000 ACCESS credits that you may be awarded in the Explore tier translates to ~334,000 CPU core hours or ~6000 GPU hours on Anvil. Based on the scale of simulations you would like to run, you may need to choose one tier or the other. -### What else should I know? +#### What else should I know? 1. You may request a **separate** allocation for each of your research grants and the allocation can last the duration of the grant (except for the Maximize tier which only lasts for 12 months). Allocations that do not cite a grant will last for 12 months. 2. **Supplements are not allowed** for Explore, Discover, and Accelerate tiers. Instead, you will need to move to a different tier if you require more resources. @@ -74,6 +76,47 @@ As you can gather from [https://allocations.access-ci.org/project-types](https:/ 6. You will also need to go to the allocations page and add any users you would like to have access to these resources. Note that they will need to sign up for ACCESS accounts as well before you can add them. 7. For other questions you may have, take a look at [ACCESS policies](https://allocations.access-ci.org/allocations-policy). + +
+ +## NAIRR +### What is NAIRR? + +The [National Artificial Intelligence Research Resource (NAIRR)](https://nairrpilot.org/) is a shared national research infrastructure that connects U.S. researchers and educators to AI resources to advance research, discovery, and innovation. The NAIRR Pilot is led by the U.S. National Science Foundation in collaboration with federal agency partners and non-governmental organizations. + + +### Which Anvil resources are available via NAIRR? + +NAIRR users can access Anvil's CPU, GPU, and AI resources for AI/ML research: + +- **Anvil GPU**: Nodes with AMD EPYC™ 7763 CPUs and 4 NVIDIA A100 GPUs (40GB each) +- **Anvil AI**: Nodes with Intel Xeon Platinum 8468 CPUs and 4 NVIDIA H100 GPUs (80GB each) + +
+ + +### Access to Anvil through NAIRR program + +Anvil is also available as a resource through the [National AI Research Resource (NAIRR) Pilot](https://nairrpilot.org/). **Anvil GPU** (NVIDIA A100) and **Anvil AI** (NVIDIA H100) nodes are accessible through the NAIRR Pilot for AI research and research that employs AI. + +#### Applying for NAIRR Access + +To request access to Anvil through the NAIRR Pilot: + +1. Sign up for an ACCESS account (if you don’t have one already) at [https://allocations.access-ci.org](https://allocations.access-ci.org). +2. Visit the [NAIRR Pilot portal](https://submit-nairr.xras.org/login). +2. Create a new proposal request for start up there is 1 page required proposal and for research resource there is 3 page one. +3. Submit an allocation request under the appropriate opportunity (such as Research Resources or Start-Up Projects). +4. Select Anvil GPU or Anvil AI as the target resource for your AI research also justify in you proposal on how you will use the resource and why you need the specific GPU or CPU hours. + +For more details on available opportunities and application procedures, see [https://nairrpilot.org/](https://nairrpilot.org/). + +!!! note "NAIRR Acknowledgment" + When you publish research based on NAIRR Pilot-provided resources, please include the following acknowledgment: + + *"This research is supported by the National Artificial Intelligence Research Resource (NAIRR) Pilot and the Anvil supercomputer supported by the National Science Foundation (award NSF-OAC 2005632) at Purdue University."* + + ## Helpful Tips We will strive to ensure that Anvil serves as a valuable resource to the national research community. We hope that you the user will assist us by making note of the following: diff --git a/docs/userguides/anvil/getting-started.md b/docs/userguides/anvil/getting-started.md index 35bd6733..c9f11f43 100644 --- a/docs/userguides/anvil/getting-started.md +++ b/docs/userguides/anvil/getting-started.md @@ -3,6 +3,7 @@ tags: - Anvil authors: - dane + - hkashgar search: boost: 2 --- @@ -11,7 +12,9 @@ search: ## Start here -Before you can log in to Anvil, ensure you have: +Before you can log in to Anvil, ensure you have completed the steps for your allocation type: + +**For ACCESS allocations:** 1. Created your [ACCESS account](https://access-ci.org) 2. Applied for a project @@ -19,7 +22,14 @@ Before you can log in to Anvil, ensure you have: 4. [Transferred your credits to Anvil](https://allocations.access-ci.org/how-to#manage-an-explore-discover-or-accelerate-project) !!! warning "Anvil account creation" - Unless you have **transferred** credits onto Anvil, you will not have a valid Anvil username. This will result in errors that may look like: `failed to map user `. If you are not the PI of the project, ensure the PI has added you to the project and exchanged credits to Anvil. + For ACCESS allocations, unless you have **transferred** credits onto Anvil, you will not have a valid Anvil username. This will result in errors that may look like: `failed to map user `. If you are not the PI of the project, ensure the PI has added you to the project and exchanged credits to Anvil. For NAIRR allocations, your account is created automatically when your allocation is approved. + +**For NAIRR allocations:** + +1. Create your [ACCESS account](https://access-ci.org) +2. Apply for a project through the [NAIRR Pilot](https://nairrpilot.org/) +3. Your allocation on Anvil is approved (no credit transfer required with NAIRR—when your NAIRR proposal is accepted for Anvil resources, or when your transfer request to Anvil is accepted, your allocation is automatically available) +4. For more information on NAIRR allocation management, see the [NAIRR section](access.md#nairr). ## Logging in diff --git a/docs/userguides/anvil/overview.md b/docs/userguides/anvil/overview.md index 9ea0d34c..aac21980 100644 --- a/docs/userguides/anvil/overview.md +++ b/docs/userguides/anvil/overview.md @@ -3,6 +3,7 @@ tags: - Anvil authors: - jin456 + - hkashgar search: boost: 2 draft: false @@ -18,7 +19,7 @@ Anvil, which is funded by a $10 million award from the National Science Foundati The name "Anvil" reflects the Purdue Boilermakers' strength and workmanlike focus on producing results, and the Anvil supercomputer enables important discoveries across many different areas of science and engineering. Anvil also serves as an experiential learning laboratory for students to gain real-world experience using computing for their science, and for student interns to work with the Anvil team for construction and operation. We are training the research computing practitioners of the future. -Anvil is built in partnership with Dell and AMD and consists of 1,000 nodes with two 64-core AMD Epyc "Milan" processors each and delivers over 1 billion CPU core hours to ACCESS each year, with a peak performance of 5.3 petaflops. Anvil's nodes are interconnected with **100 Gbps Mellanox HDR InfiniBand**. The supercomputer ecosystem also includes 32 large memory nodes, each with 1 TB of RAM, and 16 nodes each with four **NVIDIA A100 Tensor Core GPUs** providing 1.5 PF of single-precision performance to support machine learning and artificial intelligence applications. +Anvil is built in partnership with Dell and AMD and consists of 1,000 nodes with two 64-core AMD Epyc "Milan" processors each and delivers over 1 billion CPU core hours to ACCESS each year, with a peak performance of 5.3 petaflops. Anvil's nodes are interconnected with **100 Gbps Mellanox HDR InfiniBand**. The supercomputer ecosystem also includes 32 large memory nodes, each with 1 TB of RAM, 16 nodes each with four **NVIDIA A100 Tensor Core GPUs**, and 21 nodes each with four **NVIDIA H100 Tensor Core GPUs** providing accelerated performance to support machine learning and artificial intelligence applications.
![anvil_glance](../../assets/images/userguides/anvil/anvil-glance.png){ width="800" } @@ -41,6 +42,7 @@ All Anvil nodes have 128 processor cores, 256 GB to 1 TB of RAM, and 100 Gbps In |A |1,000 |Two Milan CPUs @ 2.45GHz|128 |256GB | |B |32 |Two Milan CPUs @ 2.45GHz|128 |1TB | |G |16 |Two Milan CPUs @ 2.45GHz + Four NVIDIA A100 GPUs|128 |512GB | +|H |21 |Dual Intel Xeon Platinum 8468 CPUs + Four NVIDIA H100 GPUs|96 |1TB | Anvil nodes run CentOS 8 (Rocky Linux) and use Slurm (Simple Linux Utility for Resource Management) as the batch scheduler for resource and job management. The application of operating system patches will occur as security needs dictate. All nodes allow for unlimited stack usage, as well as unlimited core dump size (though disk space and server quotas may still be a limiting factor).