OOD Tutorials

docs/hpc/09_ood/02_CellACDC.mdx

@@ -0,0 +1,32 @@
+# Cell-ACDC in OOD
+
+[Cell-ACDC](https://cell-acdc.readthedocs.io) is a GUI-based Python framework for segmentation, tracking, cell cycle annotations and quantification of microscopy data.
+
+## Getting Started
+You can run Cell-ACDC in OOD by going to the URL [ood.hpc.nyu.edu](http://ood.hpc.nyu.edu) in your browser and selecting `Cell-ACDC` from the `Interactive Apps` pull-down menu at the top of the page.  Once you've used it and other interactive apps they'll show up on your home screen under the `Recently Used Apps` header.
+
+:::note
+Be aware that when you start from `Recently Used Apps` it will start with the same configuration that you used previously.  If you'd like to configure your Cell-ACDC session differently, you'll need to select it from the menu.
+:::
+
+## Configuration
+
+You can select the number of cores, amount of memory, and number of hours.
+
+![OOD Cell-ACDC Configuration](./static/ood_cellacdc_config.png)
+
+## Cell-ACDC running in OOD
+
+After you hit the `Launch` button you'll have to wait for the scheduler to find node(s) for you to run on:
+![OOD Cell-ACDC in queue](./static/ood_cellacdc_in_queue.png)
+
+Then you'll have a short wait for the Cell-ACDC itself to start up.<br />
+Once that happens you'll get one last page that will give you links to:
+-   open a terminal window on the compute node your Cell-ACDC session is running on
+-   go to the directory associated with your Session ID that stores output, config and other related files for your session
+-   make changes to compression and image qualtiy
+-   get a link that you can share that will allow others to view your Cell-ACDC session
+
+![Pre-launch Cell-ACDC OOD](./static/ood_cellacdc_prelaunch.png)
+
+Please click the `Launch Cell-ACDC` button and a Cell-ACDC window will open.

docs/hpc/09_ood/03_Dask.mdx

@@ -0,0 +1,113 @@
+# Dask in Jupyter Notebook in OOD
+
+[Dask](https://docs.dask.org/en/stable/) is a Python library for parallel and distributed computing.
+
+## Getting Started
+You can run Dask in a Jupyter Notebook in OOD by going to the URL [ood.hpc.nyu.edu](http://ood.hpc.nyu.edu) in your browser and selecting `DS-GA.1004 - Jupyter Dask` from the `Interactive Apps` pull-down menu at the top of the page.  Once you've used it and other interactive apps they'll show up on your home screen under the `Recently Used Apps` header.
+
+:::note
+Be aware that when you start from `Recently Used Apps` it will start with the same configuration that you used previously.  If you'd like to configure your Dask session differently, you'll need to select it from the menu.
+:::
+
+## Configuration
+
+You can select the Dask version, number of cores, amount of memory, root directory, number of hours, and optional Slurm options.
+
+![OOD Dask Configuration](./static/ood_dask_config.png)
+
+:::warning
+If you select to use `/home` as your root directory be careful not to go over your quota.  You can find your current usage with the `myquota` command.  Please see our [Storage documentation](../03_storage/01_intro_and_data_management.mdx) for details about your storage options.
+:::
+
+## Dask with Jupyter Notebook running in OOD
+
+After you hit the `Launch` button you'll have to wait for the scheduler to find node(s) for you to run on:
+![OOD Dask in queue](./static/ood_dask_in_queue.png)
+
+Then you'll have a short wait for Dask itself to start up.<br />
+Once that happens you'll get one last page that will give you links to:
+-   open a terminal window on the compute node your Dask session is running on
+-   go to the directory associated with your Session ID that stores output, config and other related files for your session
+
+![Pre-launch Dask OOD](./static/ood_dask_prelaunch.png)
+
+Please click the `Connect to Jupyter` button and a Jupyter window will open.
+
+## Dask Example
+
+Start a new Jupyter notebook with 4 cores, 16GB memory, and set your root directory to `/scratch`.  Enter the following code in the first cell and execute it by pressing the `Shift` and `Enter` keys at the same time.
+```python
+import os
+import pandas as pd
+import numpy as np
+import time
+
+# Create a directory for the large files
+output_dir = "tmp/large_data_files"
+os.makedirs(output_dir, exist_ok=True)
+
+num_files = 5  # Number of files to create
+rows_per_file = 10_000_000  # 10 million rows per file
+for i in range(num_files):
+    data = {
+        'col1': np.random.randint(0, 100, size=rows_per_file),
+        'value': np.random.rand(rows_per_file) * 100
+    }
+    df = pd.DataFrame(data)
+    df.to_csv(os.path.join(output_dir, f'data_{i}.csv'), index=False)
+print(f"{num_files} large CSV files created in '{output_dir}'.")
+
+import dask.dataframe as dd
+from dask.distributed import Client
+import time
+import os
+
+# Start a Dask client for distributed processing (optional but recommended)
+# This allows you to monitor the computation with the Dask dashboard
+client = Client(n_workers=4, threads_per_worker=2, memory_limit='16GB') # Adjust these as per your system resources
+print(client)
+
+# Load multiple CSV files into a Dask DataFrame
+# Dask will automatically partition and parallelize the reading of these files
+output_dir = '/scratch/rjy1/tmp/large_data_files'
+dask_df = dd.read_csv(os.path.join(output_dir, 'data_*.csv'))
+
+# Perform a calculation (e.g., calculate the mean of the 'value' column)
+# This operation will be parallelized across the available workers
+result_dask = dask_df['value'].mean()
+
+# Trigger the computation and measure the time
+start_time = time.time()
+computed_result_dask = result_dask.compute()
+end_time = time.time()
+
+print(f"Dask took {end_time - start_time} seconds to compute the mean across {num_files} files.")
+print(f"Result (Dask): {computed_result_dask}")
+
+import pandas as pd
+import time
+import os
+
+# Perform the same calculation sequentially with Pandas
+start_time_pandas = time.time()
+total_mean = 0
+total_count = 0
+for i in range(num_files):
+    df = pd.read_csv(os.path.join(output_dir, f'data_{i}.csv'))
+    total_mean += df['value'].sum()
+    total_count += len(df)
+computed_result_pandas = total_mean / total_count
+end_time_pandas = time.time()
+
+print(f"Pandas took {end_time_pandas - start_time_pandas} seconds to compute the mean across {num_files} files.")
+print(f"Result (Pandas): {computed_result_pandas}")
+```
+You should get output like:
+```
+5 large CSV files created in 'tmp/large_data_files'.
+<Client: 'tcp://127.0.0.1:45511' processes=4 threads=8, memory=59.60 GiB>
+Dask took 3.448112726211548 seconds to compute the mean across 5 files.
+Result (Dask): 50.010815178612596
+Pandas took 9.641847610473633 seconds to compute the mean across 5 files.
+Result (Pandas): 50.01081517861258
+```

docs/hpc/09_ood/04_Desktop.mdx

@@ -0,0 +1,32 @@
+# Desktop in OOD
+
+You can get a basic desktop interface to HPC resources.
+
+## Getting Started
+You can get a desktop in OOD by going to the URL [ood.hpc.nyu.edu](http://ood.hpc.nyu.edu) in your browser and selecting `Desktop` from the `Interactive Apps` pull-down menu at the top of the page.  Once you've used it and other interactive apps they'll show up on your home screen under the `Recently Used Apps` header.
+
+:::note
+Be aware that when you start from `Recently Used Apps` it will start with the same configuration that you used previously.  If you'd like to configure your Desktop session differently, you'll need to select it from the menu.
+:::
+
+## Configuration
+
+You can select the number of cores, amount of memory, GPU type (if any), number of hours, and optional Slurm options.
+
+![OOD Desktop Configuration](./static/ood_desktop_config.png)
+
+## Desktop running in OOD
+
+After you hit the `Launch` button you'll have to wait for the scheduler to find node(s) for you to run on:
+![OOD Desktop in queue](./static/ood_desktop_in_queue.png)
+
+Then you'll have a short wait for the Desktop itself to start up.<br />
+Once that happens you'll get one last page that will give you links to:
+-   open a terminal window on the compute node your Desktop session is running on
+-   go to the directory associated with your Session ID that stores output, config and other related files for your session
+-   make changes to compression and image qualtiy
+-   get a link that you can share that will allow others to view your Desktop session
+
+![Pre-launch Desktop OOD](./static/ood_desktop_prelaunch.png)
+
+Please click the `Launch Desktop` button and a Desktop window will open.

docs/hpc/09_ood/05_FileZilla.mdx

@@ -1,4 +1,4 @@
-# FileZilla
+# FileZilla in OOD
 
 As mentioned in [File Transfer Clients](../03_storage/03_data_transfers.md#file-transfer-clients), FileZilla is a client for transferring files.  It has been installed on OOD as a convenience, but be aware that you can use many of the other methods described in the [Data Transfers](../03_storage/03_data_transfers.md) section of our documentation for transferring your files on OOD, since they all use the same underlying filesystems.
 

docs/hpc/09_ood/07_igv.mdx → docs/hpc/09_ood/06_igv.mdx

@@ -1,4 +1,4 @@
-# Integrative Genomics Viewer (IGV) 
+# Integrative Genomics Viewer (IGV) in OOD
 
 The IGV is a high-performance, easy-to-use, interactive tool for the visual exploration of genomic data.
 
@@ -7,14 +7,12 @@ Please see the following links for details:
 -   [Tutorial Videos](https://www.youtube.com/channel/UCb5W5WqauDOwubZHb-IA_rA)
 
 ## Getting Started
-You can run IGV in OOD by going to the URL [ood.hpc.nyu.edu](http://ood.hpc.nyu.edu) in your browser and selecting `IGV` from the `Interactive Apps` pull-down menu at the top of the page.  As you can see below, once you've used it and other interactive apps they'll show up on your home screen under the `Recently Used Apps` header.
+You can run IGV in OOD by going to the URL [ood.hpc.nyu.edu](http://ood.hpc.nyu.edu) in your browser and selecting `IGV` from the `Interactive Apps` pull-down menu at the top of the page.  Once you've used it and other interactive apps they'll show up on your home screen under the `Recently Used Apps` header.
 
 :::note
 Be aware that when you start from `Recently Used Apps` it will start with the same configuration that you used previously.  If you'd like to configure your IGV session differently, you'll need to select it from the menu.
 :::
 
-![OOD Interactive Apps menu IGV](./static/ood_interactive_apps_igv.png)
-
 ## Configuration
 
 You can select the number or cores, amount of memory, amount of time, and optional Slurm options.

docs/hpc/09_ood/09_jbrowse_genome_browser.mdx → docs/hpc/09_ood/07_jbrowse_genome_browser.mdx

@@ -1,4 +1,4 @@
-# JBrowse Genome Browser
+# JBrowse Genome Browser in OOD
 
 [JBrowse](https://jbrowse.org/jbrowse1.html) is a web-based genome browser for visualizing genomic features in common file formats, such as variants (VCF), genes (GFF3, BigBed) and gene expression (BigWig), and sequence alignments (BAM, CRAM, and GFF3).
 

docs/hpc/09_ood/02_jupyter_with_conda_singularity.mdx → docs/hpc/09_ood/08_jupyter_with_conda_singularity.mdx

@@ -1,4 +1,4 @@
-# Jupyter with Conda/Singularity in OOD
+# Jupyter Notebook with Conda/Singularity in OOD
 
 ## OOD + Singularity + conda
 This page describes how to use your Singularity with conda environment in OOD GUI at Greene. 

docs/hpc/09_ood/03_matlab.mdx → docs/hpc/09_ood/09_matlab.mdx

@@ -1,14 +1,12 @@
 # Matlab in OOD
 
 ## Getting Started
-You can run Matlab in OOD by going to the URL [ood.hpc.nyu.edu](http://ood.hpc.nyu.edu) in your browser and selecting `MATLAB` from the `Interactive Apps` pull-down menu at the top of the page.  As you can see below, once you've used it and other interactive apps they'll show up on your home screen under the `Recently Used Apps` header.
+You can run Matlab in OOD by going to the URL [ood.hpc.nyu.edu](http://ood.hpc.nyu.edu) in your browser and selecting `MATLAB` from the `Interactive Apps` pull-down menu at the top of the page.  Once you've used it and other interactive apps they'll show up on your home screen under the `Recently Used Apps` header.
 
 :::note
 Be aware that when you start from `Recently Used Apps` it will start with the same configuration that you used previously.  If you'd like to configure your Matlab session differently, you'll need to select it from the menu.
 :::
 
-![OOD Interactive Apps menu](./static/ood_interactive_apps_matlab.png)
-
 ## Configuration
 
 You can select the version of Matlab to use, the number or cores, amount of memory, GPU type (if any), amount of time, and optional Slurm options.

docs/hpc/09_ood/10_NGLView.mdx

@@ -0,0 +1,44 @@
+# NGLView Jupyter Notebook in OOD
+
+## Getting Started
+You can run NGLView in a Jupyter Notebook in OOD by going to the URL [ood.hpc.nyu.edu](http://ood.hpc.nyu.edu) in your browser and selecting `NGLView Jupyter Notebook` from the `Interactive Apps` pull-down menu at the top of the page.  Once you've used it and other interactive apps they'll show up on your home screen under the `Recently Used Apps` header.
+
+:::note
+Be aware that when you start from `Recently Used Apps` it will start with the same configuration that you used previously.  If you'd like to configure your NGLView Jupyter Notebook session differently, you'll need to select it from the menu.
+:::
+
+## Configuration
+
+You can select the number or cores, amount of memory, GPU type (if any) and CUDA version, amount of time, optional Slurm options, and the root directory.
+
+![OOD NGLView Configuration](./static/ood_nglview_config.png)
+
+:::warning
+If you select home as the root directory, be careful not to go over your quota.  You can find your current usage with the `myquota` command.  Please see our [Storage documentation](../03_storage/01_intro_and_data_management.mdx) for details about your storage options.
+:::
+
+## NGLView Jupyter Notebook running in OOD
+
+After you hit the `Launch` button you'll have to wait for the scheduler to find you node(s) to run on:
+![OOD NGLView in queue](./static/ood_nglview_in_queue.png)
+
+Then you'll have a short wait for the Jupyter Notebook itself to start up.<br />
+Once that happens you'll get one last form that will allow you to:
+-   open a terminal window on the compute node your NGLView Jupyter Notebook session is running on
+-   go to the directory associated with your Session ID that stores output, config and other related files for your session
+
+![Pre-launch NGLView OOD](./static/ood_nglview_prelaunch.png)
+
+Then after you hit the `Connect to NGLView Jupyter` button you'll be able to use NGLView commands in a Jupyter Notebook.
+
+### NGLView Jupyter Notebook example
+
+Please enter the following into the first Jupyter Notebook cell and run it by hitting `shift`+`enter`:
+```python
+import nglview as nv
+view = nv.show_structure_file(nv.datafiles.PDB)
+view
+```
+
+You should get output like this:
+![OOD NGLView example](./static/ood_nglview_example.png)

docs/hpc/09_ood/04_RStudio.mdx → docs/hpc/09_ood/11_RStudio.mdx

@@ -2,14 +2,12 @@
 
 
 ## Getting Started
-You can run RStudio in OOD by going to the URL [ood.hpc.nyu.edu](http://ood.hpc.nyu.edu) in your browser and selecting `RStudio Server` from the `Interactive Apps` pull-down menu at the top of the page.  As you can see below, once you've used it and other interactive apps they'll show up on your home screen under the `Recently Used Apps` header.
+You can run RStudio in OOD by going to the URL [ood.hpc.nyu.edu](http://ood.hpc.nyu.edu) in your browser and selecting `RStudio Server` from the `Interactive Apps` pull-down menu at the top of the page.  Once you've used it and other interactive apps they'll show up on your home screen under the `Recently Used Apps` header.
 
 :::note
 Be aware that when you start from `Recently Used Apps` it will start with the same configuration that you used previously.  If you'd like to configure your RStudio session differently, you'll need to select it from the menu.
 :::
 
-![OOD Interactive Apps menu](./static/ood_interactive_apps_rstudio.png)
-
 ## Configuration
 
 You can select the version of R to use, the R package location, the number or cores, amount of memory, GPU type (if any), amount of time, and optional Slurm options.