You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Welcome to the NeMo-Run guides! This section provides comprehensive documentation on how to use NeMo-Run effectively for your machine learning experiments.
17
+
18
+
## Get Started
19
+
20
+
If you're new to NeMo-Run, we recommend starting with:
21
+
22
+
-**[Why Use NeMo-Run?](why-use-nemo-run.md)** - Understand the benefits and philosophy behind NeMo-Run.
23
+
-**[Configuration](configuration.md)** - Learn how to configure your ML tasks and experiments.
24
+
-**[Execution](execution.md)** - Discover how to run your experiments across different computing environments.
25
+
-**[Management](management.md)** - Master experiment tracking, reproducibility, and organization.
26
+
27
+
## Advanced Topics
28
+
29
+
For more advanced usage:
30
+
31
+
-**[Ray Integration](ray.md)** - Learn how to use NeMo-Run with Ray for distributed computing.
32
+
-**[CLI Reference](cli.md)** - Explore the command-line interface for NeMo-Run.
33
+
34
+
## Core Concepts
35
+
36
+
NeMo-Run is built around three core responsibilities:
37
+
38
+
1.**Configuration** - Define your ML experiments using a flexible, Pythonic configuration system.
39
+
2.**Execution** - Run your experiments seamlessly across local machines, Slurm clusters, cloud providers, and more.
40
+
3.**Management** - Track, reproduce, and organize your experiments with built-in experiment management.
41
+
42
+
Each guide dives deep into these concepts with practical examples and best practices. Choose a guide above to get started!
Copy file name to clipboardExpand all lines: docs/source/guides/management.md
+11-11Lines changed: 11 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,8 +1,8 @@
1
-
# Management
1
+
# Manage NeMo-Run
2
2
3
-
The central component for management of tasks in NeMo-Run is the `Experiment` class. It allows you to define, launch, and manage complex workflows consisting of multiple tasks. This guide provides an overview of the `Experiment` class, its methods, and how to use it effectively.
3
+
The central component for the management of tasks in NeMo-Run is the `Experiment` class. It allows you to define, launch, and manage complex workflows consisting of multiple tasks. This guide provides an overview of the `Experiment` class, its methods, and how to use it effectively.
4
4
5
-
## **Creating an Experiment**
5
+
## Create an Experiment
6
6
7
7
To create an experiment, you can instantiate the `Experiment` class by passing in a descriptive title:
8
8
@@ -14,11 +14,11 @@ When executed, it will automatically generate a unique experiment ID for you, wh
14
14
15
15
> [!NOTE] > `Experiment` is a context manager and `Experiment.add` and `Experiment.run` methods can currently only be used after entering the context manager.
16
16
17
-
## **Adding Tasks**
17
+
## Add Tasks
18
18
19
19
You can add tasks to an experiment using the `add` method. This method supports tasks of the following kind:
20
20
21
-
- A single task which is an instance of either `run.Partial` or `run.Script`, along with its executor.
21
+
- A single task, which is an instance of either `run.Partial` or `run.Script`, along with its executor.
22
22
23
23
```python
24
24
with exp:
@@ -50,7 +50,7 @@ with run.Experiment("dag-experiment", log_level="INFO") as exp:
50
50
)
51
51
```
52
52
53
-
## **Launching an Experiment**
53
+
## Launch an Experiment
54
54
55
55
Once you have added all tasks to an experiment, you can launch it using the `run` method. This method takes several optional arguments, including `detach`, `sequential`, and `tail_logs` and `direct`:
You can check the status of an experiment using the `status` method:
71
71
72
72
```python
73
73
exp.status()
74
74
```
75
75
76
-
This method will display information the status of each task in the experiment. The following is a sample output from the status of experiment in [hello_scripts.py](../../../examples/hello-world/hello_scripts.py):
76
+
This method will display information about the status of each task in the experiment. The following is a sample output from the status of experiment in [hello_scripts.py](../../../examples/hello-world/hello_scripts.py):
77
77
78
78
```bash
79
79
Experiment Status for experiment_with_scripts_1730761155
@@ -97,23 +97,23 @@ Task 2: simple.add.add_object
97
97
- Local Directory: /home/your_user/.nemo_run/experiments/experiment_with_scripts/experiment_with_scripts_1730761155/simple.add.add_object
98
98
```
99
99
100
-
## **Canceling a Task**
100
+
## Cancel a Task
101
101
102
102
You can cancel a task using the `cancel` method:
103
103
104
104
```python
105
105
exp.cancel("task_id")
106
106
```
107
107
108
-
## **Viewing Logs**
108
+
## View Logs
109
109
110
110
You can view the logs of a task using the `logs` method:
111
111
112
112
```python
113
113
exp.logs("task_id")
114
114
```
115
115
116
-
## **Experiment output**
116
+
## Review Experiment Output
117
117
118
118
Once an experiment is run, NeMo-Run displays information on ways to inspect and reproduce past experiments. This allows you to check logs, sync artifacts (in the future), cancel running tasks, and rerun an old experiment.
Copy file name to clipboardExpand all lines: docs/source/index.rst
+15-13Lines changed: 15 additions & 13 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,7 +3,7 @@
3
3
You can adapt this file completely to your liking, but it should at least
4
4
contain the root `toctree` directive.
5
5
6
-
NeMo-Run documentation
6
+
NeMo-Run Documentation
7
7
======================
8
8
9
9
NeMo-Run is a powerful tool designed to streamline the configuration, execution and management of Machine Learning experiments across various computing environments. NeMo Run has three core responsibilities:
@@ -18,23 +18,25 @@ This is also the typical order Nemo Run users will follow to setup and launch ex
18
18
.. toctree::
19
19
:maxdepth:1
20
20
21
-
guides/index.rst
21
+
guides/index
22
22
API Reference <api/nemo_run/index>
23
23
faqs
24
24
25
-
Installation
26
-
------------
25
+
Install the Project
26
+
-------------------
27
27
To install the project, use the following command:
You can also manually install Skypilot from https://skypilot.readthedocs.io/en/latest/getting-started/installation.html
40
42
@@ -50,7 +52,7 @@ Make sure you have `pip` installed and configured properly.
50
52
Tutorials
51
53
---------
52
54
53
-
The ``hello_world`` tutorial series provides a comprehensive introduction to NeMoRun, demonstrating its capabilities through a simple example. The tutorial covers:
55
+
The ``hello_world`` tutorial series provides a comprehensive introduction to NeMo-Run, demonstrating its capabilities through a simple example. The tutorial covers:
54
56
55
57
- Configuring Python functions using ``Partial`` and ``Config`` classes.
56
58
- Executing configured functions locally and on remote clusters.
@@ -59,6 +61,6 @@ The ``hello_world`` tutorial series provides a comprehensive introduction to NeM
0 commit comments