U/hkashgar/ug anvilfixes

docs/assets/data/breadcrumbs.json

@@ -165,6 +165,55 @@
     "Bell User Guide",
     "Frequently Asked Questions"
   ],
+  "/userguides/negishi/": [
+    "Home",
+    "Negishi User Guide"
+  ],
+  "/userguides/negishi/overview/": [
+    "Home",
+    "Negishi User Guide",
+    "Negishi Overview"
+  ],
+  "/userguides/negishi/biography/": [
+    "Home",
+    "Negishi User Guide",
+    "Biography of Negishi"
+  ],
+  "/userguides/negishi/accounts/": [
+    "Home",
+    "Negishi User Guide",
+    "Accounts"
+  ],
+  "/userguides/negishi/software/": [
+    "Home",
+    "Negishi User Guide",
+    "Software"
+  ],
+  "/userguides/negishi/run_jobs/": [
+    "Home",
+    "Negishi User Guide",
+    "Running Jobs"
+  ],
+  "/userguides/negishi/storage/": [
+    "Home",
+    "Negishi User Guide",
+    "File Storage and Transfer"
+  ],
+  "/userguides/negishi/gateway/": [
+    "Home",
+    "Negishi User Guide",
+    "Gateway (Open OnDemand)"
+  ],
+  "/userguides/negishi/compile/": [
+    "Home",
+    "Negishi User Guide",
+    "Compiling Source Code"
+  ],
+  "/userguides/negishii/faqs/": [
+    "Home",
+    "Negishi User Guide",
+    "Frequently Asked Questions"
+  ],
   "/userguides/gautschi/": [
     "Home",
     "Gautschi User Guide"
@@ -972,4 +1021,4 @@
     "Home",
     "FAQs"
   ]
-}
+}

docs/userguides/anvil/access.md

@@ -2,8 +2,10 @@
 tags:
   - Anvil
   - ACCESS
+  - NAIRR
 authors:
   - jin456
+  - hkashgar
 search:
   boost: 2
 draft: true
@@ -13,13 +15,15 @@ draft: true
 
 ## Obtaining an Account
 
-Anvil is an ACCESS computing resource. To use Anvil, you must first create an ACCESS account and request an allocation through the [ACCESS Allocation Request System](https://allocations.access-ci.org/).
+Anvil is an ACCESS an NAIRR computing resource. To use Anvil, you must first create an ACCESS account and request an allocation through the [ACCESS Allocation Request System](https://allocations.access-ci.org/) or [NAIRR Pilot](https://nairrpilot.org/).
 
 !!! tip "New users to existing projects"
-    If you are joining an existing project, all you need to do is [login or create your ACCESS account](https://allocations.access-ci.org/) and the manager of your project can add you themselves!
+    If you are joining an existing project with an **[ACCESS allocation](#access)**, [login or create your ACCESS account](https://allocations.access-ci.org/) and the project manager can add you through the ACCESS portal. If your project has a **[NAIRR allocation](#nairr)**, you still need an [ACCESS account](https://allocations.access-ci.org/), but user management and allocation administration are handled by the project manager through the [NAIRR Pilot portal](https://submit-nairr.xras.org/).
 
 <hr style="border: 3px solid">
 
+## ACCESS
+
 ### What is ACCESS?
 
 [Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS)](https://access-ci.org) is an NSF-funded program that manages access to the national research cyberinfrastructure (CI) resources. Any researcher who seeks to use one of these CI resources must follow ACCESS processes to get onto these resources.
@@ -28,11 +32,9 @@ Anvil is an ACCESS computing resource. To use Anvil, you must first create an AC
 
 ACCESS coordinates a diverse set of resources including Anvil and other traditional HPC resources suited for resource-intensive CPU workloads, modern accelerator-based systems (e.g., GPU), as well as cloud resources. **Anvil provides both CPU and GPU resources as part of ACCESS.** A comprehensive list of all the ACCESS-managed resources can be found here along with their descriptions and ideal workloads: [https://allocations.access-ci.org/resources](https://allocations.access-ci.org/resources).
 
-<hr style="border: 3px solid">
-
-## Access to Anvil
+### Access to Anvil through ACCESS program
 
-### Applying for a New Project
+#### Applying for a New Project
 
 1. Sign up for an ACCESS account (if you don’t have one already) at [https://allocations.access-ci.org](https://allocations.access-ci.org).
 2. Prepare an allocation request with details of your proposed computational workflows (science, software needs), resource requirements, and a short CV. See the individual “Preparing Your … Request” pages for details on what documents are required: [https://allocations.access-ci.org/prepare-requests](https://allocations.access-ci.org/prepare-requests).
@@ -41,7 +43,7 @@ ACCESS coordinates a diverse set of resources including Anvil and other traditio
 !!! tip "Help with proposal"
     Interested parties may contact the [ACCESS support](https://support.access-ci.org/) for help with an Anvil proposal.
 
-### Joining an Existing Project
+#### Joining an Existing Project
 
 1. Sign up for an ACCESS account (if you don’t have one already) at [https://allocations.access-ci.org](https://allocations.access-ci.org).
 2. Have the manager of the project add you through their admin portal. That simple!
@@ -55,7 +57,7 @@ ACCESS coordinates a diverse set of resources including Anvil and other traditio
 
 You will use your Anvil username when logging into Anvil systems via SSH or your ACCESS username when using Open OnDemand.
 
-### Which ACCESS tier should I choose?
+#### Which ACCESS tier should I choose?
 
 As you can gather from [https://allocations.access-ci.org/project-types](https://allocations.access-ci.org/project-types), there are **four different tiers in ACCESS**. Broadly, these tiers provide increasing computational resources with corresponding stringent documentation and resource justification requirements. Furthermore, while Explore and Discover tier requests are reviewed on a rolling basis as they are submitted, Accelerate requests will be reviewed monthly and Maximize will be reviewed twice a year. The review period reflects the level of resources provided, and Explore and Discover applications are generally reviewed within a week. An important point to note is that ACCESS does not award you time on a specific computational resource (except for the Maximize tier). Users are awarded a certain number of ACCESS credits which they then exchange for time on a particular resource. Here are some guidelines on how to choose between the tiers:
 
@@ -64,7 +66,7 @@ As you can gather from [https://allocations.access-ci.org/project-types](https:/
 3. If you would like to run simulations across multiple resources to identify the one best suited for you, Discover will provide you with sufficient credits to exchange across multiple systems.
 4. One way of determining the appropriate tier is to determine what the credits would translate to in terms of computational resources. The [exchange calculator](https://allocations.access-ci.org/exchange_calculator) can be used to calculate what a certain number of ACCESS credits translates to in terms of “core-hours” or “GPU-hours” or “node-hours” on an ACCESS resource. For example: the maximum 400,000 ACCESS credits that you may be awarded in the Explore tier translates to ~334,000 CPU core hours or ~6000 GPU hours on Anvil. Based on the scale of simulations you would like to run, you may need to choose one tier or the other.
 
-### What else should I know?
+#### What else should I know?
 
 1. You may request a **separate** allocation for each of your research grants and the allocation can last the duration of the grant (except for the Maximize tier which only lasts for 12 months). Allocations that do not cite a grant will last for 12 months.
 2. **Supplements are not allowed** for Explore, Discover, and Accelerate tiers. Instead, you will need to move to a different tier if you require more resources.
@@ -74,6 +76,47 @@ As you can gather from [https://allocations.access-ci.org/project-types](https:/
 6. You will also need to go to the allocations page and add any users you would like to have access to these resources. Note that they will need to sign up for ACCESS accounts as well before you can add them.
 7. For other questions you may have, take a look at [ACCESS policies](https://allocations.access-ci.org/allocations-policy).
 
+
+<hr style="border: 3px solid">
+
+## NAIRR
+### What is NAIRR?
+
+The [National Artificial Intelligence Research Resource (NAIRR)](https://nairrpilot.org/) is a shared national research infrastructure that connects U.S. researchers and educators to AI resources to advance research, discovery, and innovation. The NAIRR Pilot is led by the U.S. National Science Foundation in collaboration with federal agency partners and non-governmental organizations.
+
+
+### Which Anvil resources are available via NAIRR?
+
+NAIRR users can access Anvil's CPU, GPU, and AI resources for AI/ML research:
+
+- **Anvil GPU**: Nodes with AMD EPYC™ 7763 CPUs and 4 NVIDIA A100 GPUs (40GB each)
+- **Anvil AI**: Nodes with Intel Xeon Platinum 8468 CPUs and 4 NVIDIA H100 GPUs (80GB each)
+
+<hr style="border: 3px solid">
+
+
+### Access to Anvil through NAIRR program
+
+Anvil is also available as a resource through the [National AI Research Resource (NAIRR) Pilot](https://nairrpilot.org/). **Anvil GPU** (NVIDIA A100) and **Anvil AI** (NVIDIA H100) nodes are accessible through the NAIRR Pilot for AI research and research that employs AI.
+
+#### Applying for NAIRR Access
+
+To request access to Anvil through the NAIRR Pilot:
+
+1. Sign up for an ACCESS account (if you don’t have one already) at [https://allocations.access-ci.org](https://allocations.access-ci.org).
+2. Visit the [NAIRR Pilot portal](https://submit-nairr.xras.org/login).
+2. Create a new proposal request for start up there is 1 page required proposal and for research resource there is 3 page one.
+3. Submit an allocation request under the appropriate opportunity (such as Research Resources or Start-Up Projects).
+4. Select Anvil GPU or Anvil AI as the target resource for your AI research also justify in you proposal on how you will use the resource and why you need the specific GPU or CPU hours.
+
+For more details on available opportunities and application procedures, see [https://nairrpilot.org/](https://nairrpilot.org/).
+
+!!! note "NAIRR Acknowledgment"
+    When you publish research based on NAIRR Pilot-provided resources, please include the following acknowledgment:
+
+    *"This research is supported by the National Artificial Intelligence Research Resource (NAIRR) Pilot and the Anvil supercomputer supported by the National Science Foundation (award NSF-OAC 2005632) at Purdue University."*
+
+
 ## Helpful Tips
 
 We will strive to ensure that Anvil serves as a valuable resource to the national research community. We hope that you the user will assist us by making note of the following:

docs/userguides/anvil/getting-started.md

@@ -11,15 +11,24 @@ search:
 
 ## Start here
 
-Before you can log in to Anvil, ensure you have:
+Before you can log in to Anvil, ensure you have completed the steps for your allocation type:
+
+**For ACCESS allocations:**
 
 1. Created your [ACCESS account](https://access-ci.org)
 2. Applied for a project
 3. Received credits in ACCESS
 4. [Transferred your credits to Anvil](https://allocations.access-ci.org/how-to#manage-an-explore-discover-or-accelerate-project)
 
 !!! warning "Anvil account creation"
-    Unless you have **transferred** credits onto Anvil, you will not have a valid Anvil username. This will result in errors that may look like: `failed to map user <username@access-cli.com>`. If you are not the PI of the project, ensure the PI has added you to the project and exchanged credits to Anvil.
+    For ACCESS allocations, unless you have **transferred** credits onto Anvil, you will not have a valid Anvil username. This will result in errors that may look like: `failed to map user <username@access-cli.com>`. If you are not the PI of the project, ensure the PI has added you to the project and exchanged credits to Anvil. For NAIRR allocations, your account is created automatically when your allocation is approved.
+
+**For NAIRR allocations:**
+
+1. Create your [ACCESS account](https://access-ci.org)
+2. Apply for a project through the [NAIRR Pilot](https://nairrpilot.org/)
+3. Your allocation on Anvil is approved (no credit transfer required with NAIRR—when your NAIRR proposal is accepted for Anvil resources, or when your transfer request to Anvil is accepted, your allocation is automatically available)
+4. For more information on NAIRR allocation management, see the [NAIRR section](access.md#nairr).
 
 ## Logging in
 

docs/userguides/anvil/overview.md

@@ -3,6 +3,7 @@ tags:
   - Anvil
 authors:
   - jin456
+  - hkashgar
 search:
   boost: 2
 draft: false
@@ -18,7 +19,7 @@ Anvil, which is funded by a $10 million award from the National Science Foundati
 
 The name "Anvil" reflects the Purdue Boilermakers' strength and workmanlike focus on producing results, and the Anvil supercomputer enables important discoveries across many different areas of science and engineering. Anvil also serves as an experiential learning laboratory for students to gain real-world experience using computing for their science, and for student interns to work with the Anvil team for construction and operation. We are training the research computing practitioners of the future.
 
-Anvil is built in partnership with Dell and AMD and consists of 1,000 nodes with two 64-core AMD Epyc "Milan" processors each and delivers over 1 billion CPU core hours to ACCESS each year, with a peak performance of 5.3 petaflops. Anvil's nodes are interconnected with **100 Gbps Mellanox HDR InfiniBand**. The supercomputer ecosystem also includes 32 large memory nodes, each with 1 TB of RAM, and 16 nodes each with four **NVIDIA A100 Tensor Core GPUs** providing 1.5 PF of single-precision performance to support machine learning and artificial intelligence applications.
+Anvil is built in partnership with Dell and AMD and consists of 1,000 nodes with two 64-core AMD Epyc "Milan" processors each and delivers over 1 billion CPU core hours to ACCESS each year, with a peak performance of 5.3 petaflops. Anvil's nodes are interconnected with **100 Gbps Mellanox HDR InfiniBand**. The supercomputer ecosystem also includes 32 large memory nodes, each with 1 TB of RAM, 16 nodes each with four **NVIDIA A100 Tensor Core GPUs**, and 21 nodes each with four **NVIDIA H100 Tensor Core GPUs** providing accelerated performance to support machine learning and artificial intelligence applications.
 
 <figure markdown="span">
   ![anvil_glance](../../assets/images/userguides/anvil/anvil-glance.png){ width="800" }
@@ -41,6 +42,7 @@ All Anvil nodes have 128 processor cores, 256 GB to 1 TB of RAM, and 100 Gbps In
 |A          |1,000          |Two Milan CPUs @ 2.45GHz|128             |256GB           |
 |B          |32             |Two Milan CPUs @ 2.45GHz|128             |1TB           |
 |G          |16             |Two Milan CPUs @ 2.45GHz + Four NVIDIA A100 GPUs|128             |512GB           |
+|H          |21             |Dual Intel Xeon Platinum 8468 CPUs + Four NVIDIA H100 GPUs|96             |1TB           |
 
 
 Anvil nodes run CentOS 8 (Rocky Linux) and use Slurm (Simple Linux Utility for Resource Management) as the batch scheduler for resource and job management. The application of operating system patches will occur as security needs dictate. All nodes allow for unlimited stack usage, as well as unlimited core dump size (though disk space and server quotas may still be a limiting factor).

docs/userguides/negishi/accounts.md

@@ -0,0 +1,27 @@
+---
+tags:
+  - Negishi
+authors:
+  - hkashgar
+resource: negishi
+search:
+  boost: 2
+---
+
+
+{% set resource = "negishi" %}
+
+{{ accounts_md_snippet(resource) }}
+
+## SSH Keys
+{{ ssh_keys_snippet(resource) }}
+
+## SSH X11 Forwarding
+{{ ssh_x11_snippet(resource) }}
+
+## Thinlinc
+{{ thinlinc_snippet(resource) }}
+
+## Purchasing Nodes
+
+--8<-- "docs/snippets/purchase_nodes.md"

docs/userguides/negishi/biography.md

@@ -0,0 +1,17 @@
+---
+tags:
+  - Negishi
+authors:
+  - hkashgar
+search:
+  boost: 2
+---
+
+# Biography of Ei-ichi Negishi
+
+![Portrait of Ei-ichi Negishi](../../assets/images/userguides/negishi/bio_negishi.jpg){ align=right }
+
+
+Ei-ichi Negishi (1935-2021) was the Herbert C. Brown Distinguished Professor in the Department of Chemistry at Purdue. He came to Purdue in 1966 as a postdoctoral researcher in the lab of the Late Herbert C. Brown, and published 33 papers with Prof. Brown up through the time that Prof. Brown was awarded the Nobel Prize in Chemistry in 1979. With the award of the Nobel to Ei-ichi Negishi in 2010, Purdue has the rare distinction of a pair of Nobel Prize awards in two closely related areas. Professor Negishi’s Nobel Prize was awarded in recognition of his work on palladium-catalyzed cross-coupling chemistry (known world-– wide as the Negishi coupling). That work was described by the Nobel Foundation as "great art in a test tube". This is certainly appropriate as great scientists regard themselves as artists and explorers. The impact of that work was widespread, as it had been used in synthetic organic chemistry research worldwide, as well as in the commercial production of an array of pharmaceuticals and molecules used in the electronics industry. In recognition of and consistent with this idea, Ei-ichi and co-recipient Akira Suzuki were recently awarded Japan's highest cultural award, the "Order of Culture", bestowed in Nov. 2010 by the Emperor.
+
+Professor Negishi was a prolific researcher, with ~400 publications on an array of problems in synthetic organic chemistry, leading to numerous awards. To name just a few, the list includes the Chemical Society of Japan Award (1997), the American Chemical Society Award in Organometallic Chemistry (1998), the McCoy Award (1998), the Sigma Xi Award at Purdue (2003), the Nobel Prize in Chemistry (2010), the Order of Culture in Japan (2010), the American Chemical Society Award for Creative Work in Synthetic Organic Chemistry (2010), the Indiana Sagamore of the Wabash (2011) and the Purdue Order of the Griffin (2011). He was elected to the American Academy of Arts and Sciences in 2011. Professor Negishi was leading the Negishi-Brown Institute, which had continued his work on catalytic organic synthesis. Dr. Negishi was passionate about the prospects for catalytic approaches to the reduction of carbon dioxide to enable large scale production of useful products from this environmental waste product. It is very fitting that Purdue bestow an honorary doctorate degree on Professor Negishi, whose accomplishments and contributions will have a permanent impact on Purdue’s stature and global recognition.

docs/userguides/negishi/compile.md

@@ -0,0 +1,23 @@
+---
+tags:
+  - Negishi
+authors:
+  - hkashgar
+resource: Negishi
+search:
+  boost: 2
+---
+
+# Compiling Source Code
+
+Documentation on compiling source code on Negishi.
+
+## In This Section
+
+- [Compiling Serial Programs](compile/serial.md)
+- [Compiling MPI Programs](compile/mpi.md)
+- [Compiling OpenMP Programs](compile/openmp.md)
+- [Compiling Hybrid Programs](compile/hybrid.md)
+- [Intel MKL Library](compile/intel_mkl.md)
+
+[**Back to Negishi User Guide**](index.md)

docs/userguides/negishi/compile/hybrid.md

@@ -0,0 +1,13 @@
+---
+tags:
+  - Negishi
+authors:
+  - hkashgar
+resource: Negishi
+search:
+  boost: 2
+---
+
+--8<-- "docs/snippets/compile_hybrid.md"
+
+[**Back to the Compiling Source Code section**](../compile.md)

docs/userguides/negishi/compile/intel_mkl.md

@@ -0,0 +1,13 @@
+---
+tags:
+  - Negishi
+authors:
+  - hkashgar
+resource: Negishi
+search:
+  boost: 2
+---
+
+--8<-- "docs/snippets/compile_intel_mkl.md"
+
+[**Back to the Compiling Source Code section**](../compile.md)

docs/userguides/negishi/compile/mpi.md

@@ -0,0 +1,13 @@
+---
+tags:
+  - Negishi
+authors:
+  - hkashgar
+resource: Negishi
+search:
+  boost: 2
+---
+
+--8<-- "docs/snippets/compile_mpi.md"
+
+[**Back to the Compiling Source Code section**](../compile.md)

docs/userguides/negishi/compile/openmp.md

@@ -0,0 +1,13 @@
+---
+tags:
+  - Negishi
+authors:
+  - hkashgar
+resource: Negishi
+search:
+  boost: 2
+---
+
+--8<-- "docs/snippets/compile_openmp.md"
+
+[**Back to the Compiling Source Code section**](../compile.md)

docs/userguides/negishi/compile/serial.md

@@ -0,0 +1,13 @@
+---
+tags:
+  - Negishi
+authors:
+  - hkashgar
+resource: Negishi
+search:
+  boost: 2
+---
+
+--8<-- "docs/snippets/compile_serial.md"
+
+[**Back to the Compiling Source Code section**](../compile.md)