-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathindia-agentic-ai-open-hackathon-technical-guide.html
More file actions
2024 lines (1782 loc) · 79.9 KB
/
Copy pathindia-agentic-ai-open-hackathon-technical-guide.html
File metadata and controls
2024 lines (1782 loc) · 79.9 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>India Agentic AI Open Hackathon - Know-Before-You-Go Technical Guide</title>
<link rel="icon" href="https://www.nvidia.com/favicon.ico">
<style>
:root {
--bg: #0d1117;
--surface: #161b22;
--surface-2: #1f2630;
--surface-3: #111821;
--border: #30363d;
--text: #e6edf3;
--muted: #9aa4af;
--green: #76b900;
--blue: #58a6ff;
--yellow: #d29922;
--orange: #f0883e;
--red: #ff7b72;
--purple: #bc8cff;
}
* { box-sizing: border-box; }
html { scroll-behavior: smooth; }
body {
margin: 0;
background: var(--bg);
color: var(--text);
font-family: Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
font-size: 15px;
line-height: 1.65;
}
a { color: var(--blue); text-decoration: none; }
a:hover { text-decoration: underline; }
header {
border-bottom: 1px solid var(--border);
background:
radial-gradient(circle at 35% -20%, rgba(118, 185, 0, 0.16), transparent 42%),
linear-gradient(145deg, #0d1117 0%, #10190d 45%, #0d1117 100%);
padding: 38px 28px 30px;
}
.wrap {
width: min(1240px, calc(100% - 40px));
margin: 0 auto;
}
.badge {
display: inline-flex;
align-items: center;
gap: 8px;
border: 1px solid rgba(118, 185, 0, 0.35);
border-radius: 999px;
background: rgba(118, 185, 0, 0.1);
color: var(--green);
padding: 4px 12px;
font-size: 12px;
font-weight: 750;
letter-spacing: 0.04em;
text-transform: uppercase;
}
h1 {
margin: 16px 0 8px;
font-size: clamp(28px, 4vw, 44px);
line-height: 1.08;
}
.subtitle {
max-width: 980px;
color: var(--muted);
font-size: 17px;
margin: 0;
}
.meta {
display: flex;
flex-wrap: wrap;
gap: 10px;
margin-top: 18px;
}
.pill {
border: 1px solid var(--border);
border-radius: 999px;
background: rgba(255, 255, 255, 0.035);
color: var(--muted);
padding: 5px 12px;
font-size: 13px;
}
.tabbar {
position: sticky;
top: 0;
z-index: 10;
border-bottom: 1px solid var(--border);
background: rgba(13, 17, 23, 0.96);
backdrop-filter: blur(12px);
overflow-x: auto;
}
.tabs {
display: flex;
gap: 0;
min-width: max-content;
}
.tab {
border: 0;
border-bottom: 2px solid transparent;
background: transparent;
color: var(--muted);
cursor: pointer;
font: inherit;
font-size: 14px;
font-weight: 700;
padding: 14px 16px 12px;
white-space: nowrap;
}
.tab:hover { color: var(--text); }
.tab.active { color: var(--green); border-bottom-color: var(--green); }
.layout {
display: grid;
grid-template-columns: 270px minmax(0, 1fr);
gap: 28px;
padding: 32px 0 56px;
}
aside {
position: sticky;
top: 58px;
align-self: start;
max-height: calc(100vh - 76px);
overflow-y: auto;
border: 1px solid var(--border);
border-radius: 8px;
background: var(--surface);
padding: 16px;
}
aside strong {
display: block;
margin-bottom: 10px;
font-size: 12px;
letter-spacing: 0.06em;
text-transform: uppercase;
color: var(--muted);
}
aside a {
display: block;
border-left: 2px solid transparent;
color: var(--muted);
padding: 5px 8px;
font-size: 13px;
}
aside a:hover {
color: var(--text);
background: rgba(255, 255, 255, 0.03);
text-decoration: none;
}
main { min-width: 0; }
.panel { display: none; }
.panel.active { display: block; }
h2 {
margin: 38px 0 15px;
padding-bottom: 8px;
border-bottom: 1px solid var(--border);
font-size: 24px;
line-height: 1.25;
scroll-margin-top: 78px;
}
h2:first-child { margin-top: 0; }
h3 {
margin: 26px 0 10px;
font-size: 18px;
line-height: 1.3;
scroll-margin-top: 78px;
}
h4 {
margin: 20px 0 8px;
font-size: 15px;
scroll-margin-top: 78px;
}
p { margin: 0 0 14px; }
ul, ol { padding-left: 22px; margin: 0 0 15px; }
li { margin-bottom: 6px; }
code {
border: 1px solid var(--border);
border-radius: 5px;
background: var(--surface-2);
color: #ffb4b4;
padding: 1px 5px;
font-size: 0.92em;
}
.lead {
border: 1px solid rgba(118, 185, 0, 0.24);
border-radius: 8px;
background: rgba(118, 185, 0, 0.075);
color: var(--muted);
padding: 15px 16px;
margin: 16px 0 20px;
}
.warning {
border-left: 3px solid var(--orange);
border-radius: 0 8px 8px 0;
background: rgba(240, 136, 62, 0.08);
color: var(--muted);
padding: 13px 15px;
margin: 16px 0;
}
.success {
border-left: 3px solid var(--green);
border-radius: 0 8px 8px 0;
background: rgba(118, 185, 0, 0.08);
color: var(--muted);
padding: 13px 15px;
margin: 16px 0;
}
.grid {
display: grid;
grid-template-columns: repeat(3, minmax(0, 1fr));
gap: 14px;
margin: 16px 0;
}
.grid.two { grid-template-columns: repeat(2, minmax(0, 1fr)); }
.card {
border: 1px solid var(--border);
border-radius: 8px;
background: var(--surface);
padding: 15px;
}
.card h3, .card h4 { margin-top: 0; }
.card p:last-child, .card ul:last-child, .card ol:last-child { margin-bottom: 0; }
.code-wrap {
border: 1px solid var(--border);
border-radius: 8px;
background: var(--surface-3);
overflow: hidden;
margin: 16px 0;
}
.code-head {
display: flex;
align-items: center;
gap: 8px;
border-bottom: 1px solid var(--border);
background: var(--surface);
color: var(--muted);
padding: 8px 12px;
font-size: 12px;
font-weight: 700;
}
.copy {
margin-left: auto;
border: 1px solid rgba(88, 166, 255, 0.28);
border-radius: 5px;
background: rgba(88, 166, 255, 0.09);
color: var(--blue);
cursor: pointer;
font: inherit;
font-size: 11px;
font-weight: 700;
padding: 3px 9px;
}
pre {
margin: 0;
overflow-x: auto;
padding: 15px;
color: #d6deeb;
font-family: "SFMono-Regular", Consolas, "Liberation Mono", monospace;
font-size: 13px;
line-height: 1.55;
}
pre code {
border: 0;
border-radius: 0;
background: transparent;
color: inherit;
padding: 0;
font-size: inherit;
}
table {
width: 100%;
border-collapse: collapse;
margin: 16px 0;
font-size: 14px;
}
th, td {
border: 1px solid var(--border);
padding: 10px 12px;
vertical-align: top;
}
th {
background: var(--surface);
color: var(--text);
text-align: left;
}
td { color: var(--muted); }
.path {
border: 1px solid var(--border);
border-radius: 8px;
overflow: hidden;
margin: 16px 0;
}
.path div {
border-bottom: 1px solid var(--border);
background: var(--surface);
padding: 11px 13px;
}
.path div:last-child { border-bottom: 0; }
footer {
border-top: 1px solid var(--border);
color: var(--muted);
padding: 24px 0 36px;
font-size: 13px;
}
@media (max-width: 980px) {
.layout { grid-template-columns: 1fr; }
aside { position: static; max-height: none; }
.grid, .grid.two { grid-template-columns: 1fr; }
}
</style>
</head>
<body>
<header>
<div class="wrap">
<span class="badge">Know-Before-You-Go</span>
<h1>India Agentic AI Open Hackathon Technical Guide</h1>
<p class="subtitle">A hands-on technical companion for workshop participants. Use it during the workshop and hackathon to choose the right NVIDIA path, copy starter configs, connect SDKs, and turn your project idea into a working demo.</p>
<div class="meta">
<span class="pill">For eligible applicants and developer teams</span>
<span class="pill">No hands-on workshop required</span>
<span class="pill">Primary focus: Nemotron, NIM, NemoClaw, and NeMo</span>
</div>
</div>
</header>
<nav class="tabbar">
<div class="tabs wrap" role="tablist" aria-label="Guide sections">
<button class="tab active" data-tab="overview">Overview</button>
<button class="tab" data-tab="nim">Common: NIM + Nemotron</button>
<button class="tab" data-tab="a">Track A: Agentic Workflows</button>
<button class="tab" data-tab="b">Track B: Finetuning</button>
<button class="tab" data-tab="c">Track C: Synthetic Data</button>
<button class="tab" data-tab="checklists">Checklists</button>
<button class="tab" data-tab="links">Links</button>
</div>
</nav>
<div class="wrap layout">
<aside>
<strong>Guide map</strong>
<a href="#overview-goal">How to use this guide</a>
<a href="#overview-track-map">Track selection map</a>
<a href="#nim-setup">NIM setup</a>
<a href="#nim-skills">NVIDIA Skills</a>
<a href="#nim-chat">Chat API starter</a>
<a href="#nim-tools">Tool calling</a>
<a href="#a-nat">NeMo Agent Toolkit</a>
<a href="#a-nemoclaw">NemoClaw</a>
<a href="#b-bridge">Megatron-Bridge</a>
<a href="#b-rl">NeMo RL</a>
<a href="#c-designer">Data Designer</a>
<a href="#c-personas-india">India personas</a>
<a href="#c-curator">Curator</a>
<a href="#final-checklist">Final checklist</a>
</aside>
<main>
<section class="panel active" id="overview">
<h2 id="overview-goal">How To Use This Guide</h2>
<div class="lead">
This guide is intentionally more technical than the public workshop agenda. Use it as the developer companion during the workshop to set up your stack, pick a track, reuse starter snippets, and shape a build plan for the hackathon days.
</div>
<div class="grid">
<article class="card">
<h3>Workshop setup</h3>
<ul>
<li>Create NVIDIA Developer, build.nvidia.com, NGC, and Hugging Face accounts if needed.</li>
<li>Read the common NIM + Nemotron section.</li>
<li>List the NVIDIA verified agent skills and install the skills relevant to your track.</li>
<li>Pick a primary track and one backup track.</li>
<li>Capture one architecture sketch and one technical blocker question.</li>
</ul>
</article>
<article class="card">
<h3>During the workshop</h3>
<ul>
<li>Use tabs as reference material during demos.</li>
<li>Mark which code snippets your team will reuse.</li>
<li>Ask mentors where your project fits in the NVIDIA stack.</li>
<li>Decide what you will build on Day 0 and what you will defer.</li>
</ul>
</article>
<article class="card">
<h3>Before hackathon Day 0</h3>
<ul>
<li>Have a runnable repo with a license and README.</li>
<li>Prepare small test data and an evaluation prompt set.</li>
<li>Confirm your inference path: hosted NIM, self-hosted NIM, vLLM, or cluster.</li>
<li>Define a demo path that works even if the full system is not complete.</li>
</ul>
</article>
</div>
<h2 id="overview-track-map">Track Selection Map</h2>
<table>
<thead>
<tr>
<th>If your project is mainly...</th>
<th>Choose</th>
<th>Use these NVIDIA assets</th>
<th>Expected demo proof</th>
</tr>
</thead>
<tbody>
<tr>
<td>Connecting tools, APIs, documents, agents, and workflows into a working assistant or automation service.</td>
<td>Track A: Agentic Workflows</td>
<td>Nemotron, NIM, NVIDIA Agent Skills, NeMo Agent Toolkit, MCP, A2A, optional NemoClaw.</td>
<td>A traced workflow with tool calls, result synthesis, and at least one reliability/evaluation metric.</td>
</tr>
<tr>
<td>Changing model behavior for a domain, persona, language, or task using training or post-training.</td>
<td>Track B: Model Finetuning and Customisation</td>
<td>NeMo, Megatron-Bridge, SFT, LoRA/PEFT, NeMo RL, NIM deployment.</td>
<td>Base vs adapted comparison, training data sample, evaluation result, and deployment path.</td>
</tr>
<tr>
<td>Creating structured, high-quality training or evaluation data for low-resource, domain, or agentic tasks.</td>
<td>Track C: Synthetic Data Generation</td>
<td>Nemotron, Nemotron-Personas-India, NeMo Data Designer, NeMo Curator, LLM-as-judge, validators.</td>
<td>A generated dataset sample plus quality filters, deduplication, and export format.</td>
</tr>
</tbody>
</table>
<h2 id="overview-repo">Recommended Repo Structure</h2>
<div class="code-wrap">
<div class="code-head">text <button class="copy" onclick="copyCode(this)">Copy</button></div>
<pre><code>india-agentic-ai-hackathon/
README.md
LICENSE
.gitignore
docs/
architecture.md
demo-script.md
evaluation-plan.md
app/
api/
ui/
agents/
configs/
nim.env.example
nat/
training/
synthetic-data/
data/
samples/
eval/
notebooks/
scripts/
smoke_test.sh
run_demo.sh
results/
screenshots/
eval_report.md</code></pre>
</div>
<div class="success">
Minimum bar for a strong technical submission: a clear track, runnable demo path, evidence of NVIDIA stack usage, and a small but believable evaluation loop.
</div>
</section>
<section class="panel" id="nim">
<h2 id="nim-setup">Common Foundation: NIM + Nemotron</h2>
<p>NIM and Nemotron are the common layer across all three tracks. NIM gives you a production-style inference endpoint with an OpenAI-compatible API. Nemotron gives you a strong NVIDIA model family for reasoning, tool use, long-context work, and synthetic data generation.</p>
<h3>Accounts and keys</h3>
<div class="grid two">
<article class="card">
<h3>Hosted path</h3>
<ol>
<li>Sign in at <a href="https://build.nvidia.com/" target="_blank" rel="noreferrer">build.nvidia.com</a>.</li>
<li>Open the model page you want to test.</li>
<li>Create an API key and store it as <code>NVIDIA_API_KEY</code>.</li>
<li>Use <code>https://integrate.api.nvidia.com/v1</code> as the base URL.</li>
</ol>
</article>
<article class="card">
<h3>Self-hosted path</h3>
<ol>
<li>Prepare Docker and NVIDIA Container Toolkit.</li>
<li>Log in to <code>nvcr.io</code> with an NGC API key.</li>
<li>Pull/run the NIM container for your selected model.</li>
<li>Use your local container URL, usually <code>http://localhost:8000/v1</code>.</li>
</ol>
</article>
</div>
<div class="warning">
Keep API keys out of notebooks, screenshots, commits, and pitch decks. Commit only <code>.env.example</code>, never <code>.env</code>.
</div>
<h3>NIM path comparison</h3>
<table>
<thead>
<tr><th>Item</th><th>Hosted NIM</th><th>Self-hosted NIM</th></tr>
</thead>
<tbody>
<tr><td>Best for</td><td>Fast prototyping, laptops, early demos.</td><td>Data control, private infrastructure, lower network latency.</td></tr>
<tr><td>Local GPU</td><td>Not required.</td><td>Required for local deployment.</td></tr>
<tr><td>Key</td><td><code>NVIDIA_API_KEY</code> from build.nvidia.com.</td><td><code>NGC_API_KEY</code> for pulling containers and assets.</td></tr>
<tr><td>Base URL</td><td><code>https://integrate.api.nvidia.com/v1</code></td><td><code>http://localhost:8000/v1</code> or cluster endpoint.</td></tr>
<tr><td>Hackathon advice</td><td>Start here for all teams.</td><td>Move here when privacy, latency, or deployment realism matters.</td></tr>
</tbody>
</table>
<h3>Nemotron model selection</h3>
<table>
<thead>
<tr><th>Model</th><th>Use in hackathon</th><th>Rule of thumb</th></tr>
</thead>
<tbody>
<tr><td><code>nvidia/nemotron-3-nano-30b-a3b</code></td><td>Default starting model for agentic reasoning, tool calling, and SDG prototyping.</td><td>Start with Nano unless you have a clear reason not to.</td></tr>
<tr><td><code>nvidia/nemotron-3-super-120b-a12b</code></td><td>Higher-quality planning, heavier multi-agent workflows, and more complex data generation.</td><td>Upgrade when quality matters more than latency or cost.</td></tr>
<tr><td>Nemotron multimodal or VL variants</td><td>Optional extension for image, document, or visual reasoning projects.</td><td>Use only when the project actually needs multimodal inputs.</td></tr>
</tbody>
</table>
<h3>Starter environment</h3>
<div class="code-wrap">
<div class="code-head">bash <button class="copy" onclick="copyCode(this)">Copy</button></div>
<pre><code>mkdir nvidia-ai-starter
cd nvidia-ai-starter
python3 -m venv .venv
source .venv/bin/activate
pip install -U openai python-dotenv pandas rich
cat > .env.example <<'EOF'
NVIDIA_API_KEY=nvapi-your-key
NIM_BASE_URL=https://integrate.api.nvidia.com/v1
NIM_MODEL=nvidia/nemotron-3-nano-30b-a3b
EOF
cp .env.example .env
echo ".env" >> .gitignore</code></pre>
</div>
<h3 id="nim-skills">NVIDIA Verified Agent Skills</h3>
<p>NVIDIA Agent Skills are portable instruction sets that help coding agents use NVIDIA CUDA-X libraries, AI Blueprints, and platform tools correctly. Use them when you want your agentic coding assistant to follow product-specific NVIDIA patterns instead of relying only on generic model knowledge.</p>
<div class="grid two">
<article class="card">
<h3>Browse and install</h3>
<ol>
<li>Browse the public catalog at <a href="https://github.com/NVIDIA/skills" target="_blank" rel="noreferrer">github.com/NVIDIA/skills</a>.</li>
<li>List available skills before installing anything.</li>
<li>Install only the skills your team will use during the hackathon.</li>
<li>Reload or restart your agent so the new skill instructions are available.</li>
</ol>
</article>
<article class="card">
<h3>Good hackathon matches</h3>
<ul>
<li><code>cuOpt</code> for optimization, routing, logistics, and scheduling agents.</li>
<li><code>RAG Blueprint</code> for retrieval-augmented generation demos.</li>
<li><code>TensorRT-LLM</code> for inference optimization and deployment work.</li>
<li><code>NeMo RL</code>, <code>NeMo Gym</code>, or <code>Megatron</code> skills for Track B training and alignment projects.</li>
</ul>
</article>
</div>
<div class="code-wrap">
<div class="code-head">bash <button class="copy" onclick="copyCode(this)">Copy</button></div>
<pre><code># See the available NVIDIA-verified skills.
npx skills add nvidia/skills --list
# Interactive install flow.
npx skills add nvidia/skills
# Install one known skill for Codex without prompts.
npx skills add nvidia/skills \
--skill cuopt-numerical-optimization-api-python \
--agent codex \
--yes
# Install the same skill into multiple agent clients if your team uses more than one.
npx skills add nvidia/skills \
--skill cuopt-numerical-optimization-api-python \
--agent codex \
--agent claude-code \
--agent cursor \
--yes</code></pre>
</div>
<div class="warning">
Treat installed skills like project dependencies: pin what you use in your README, record the skill name in your demo notes, and avoid installing broad extras that your project never exercises.
</div>
<h3 id="nim-chat">Chat completion starter</h3>
<div class="code-wrap">
<div class="code-head">python <button class="copy" onclick="copyCode(this)">Copy</button></div>
<pre><code># scripts/nim_chat_smoke_test.py
import os
from dotenv import load_dotenv
from openai import OpenAI
load_dotenv()
client = OpenAI(
base_url=os.getenv("NIM_BASE_URL", "https://integrate.api.nvidia.com/v1"),
api_key=os.getenv("NVIDIA_API_KEY", "not-used"),
)
model = os.getenv("NIM_MODEL", "nvidia/nemotron-3-nano-30b-a3b")
response = client.chat.completions.create(
model=model,
messages=[
{"role": "system", "content": "You are a concise technical mentor for hackathon teams."},
{"role": "user", "content": "Suggest a Track A architecture for invoice processing."},
],
temperature=0.2,
max_tokens=600,
)
print(response.choices[0].message.content)</code></pre>
</div>
<h3>Streaming response</h3>
<div class="code-wrap">
<div class="code-head">python <button class="copy" onclick="copyCode(this)">Copy</button></div>
<pre><code>stream = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": "Create a demo script for a multilingual support agent."}],
stream=True,
temperature=0.3,
max_tokens=700,
)
for chunk in stream:
if chunk.choices and chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)
print()</code></pre>
</div>
<h3>Reasoning controls</h3>
<p>Use non-thinking mode for fast routing and tool loops. Use thinking mode for deeper planning, architecture comparison, and data quality review. Avoid combining experimental parallel reasoning modes with tool-calling loops unless the model documentation explicitly supports it.</p>
<div class="code-wrap">
<div class="code-head">python <button class="copy" onclick="copyCode(this)">Copy</button></div>
<pre><code># Fast concise mode
resp = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": "Draft a simple grievance-routing agent architecture."}],
temperature=0,
max_tokens=700,
extra_body={
"chat_template_kwargs": {
"enable_thinking": False
}
},
)
print(resp.choices[0].message.content)
# Deeper planning mode
resp = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": "Compare three architectures for a public-services assistant and choose one."}],
temperature=0,
max_tokens=1800,
extra_body={
"chat_template_kwargs": {
"enable_thinking": True
}
},
)
print(resp.choices[0].message.content)</code></pre>
</div>
<h3 id="nim-tools">Tool calling pattern</h3>
<p>Use tool calling when the model should decide when to call an application function, database lookup, policy engine, calculator, search API, or workflow step.</p>
<div class="code-wrap">
<div class="code-head">python <button class="copy" onclick="copyCode(this)">Copy</button></div>
<pre><code>import json
tools = [{
"type": "function",
"function": {
"name": "lookup_policy",
"description": "Look up a company policy by topic.",
"parameters": {
"type": "object",
"properties": {
"topic": {"type": "string", "description": "Policy topic, for example travel or reimbursement"}
},
"required": ["topic"],
},
},
}]
def lookup_policy(topic: str) -> str:
policies = {
"travel": "Flights require manager approval. Hotels must be under the city cap.",
"reimbursement": "Submit receipts within 30 days with project code and GST details.",
}
return policies.get(topic.lower(), "No policy found for that topic.")
messages = [{"role": "user", "content": "Can I expense a hotel for my Bangalore workshop trip?"}]
first = client.chat.completions.create(
model=model,
messages=messages,
tools=tools,
tool_choice="auto",
temperature=0,
)
assistant_message = first.choices[0].message
messages.append(assistant_message)
if assistant_message.tool_calls:
for call in assistant_message.tool_calls:
args = json.loads(call.function.arguments)
result = lookup_policy(**args)
messages.append({
"role": "tool",
"tool_call_id": call.id,
"content": result,
})
final = client.chat.completions.create(model=model, messages=messages, temperature=0.2)
print(final.choices[0].message.content)</code></pre>
</div>
<h3>Self-hosted NIM smoke tests</h3>
<div class="code-wrap">
<div class="code-head">bash <button class="copy" onclick="copyCode(this)">Copy</button></div>
<pre><code># Log in to NGC before pulling private/entitled containers.
docker login nvcr.io -u '$oauthtoken' -p "$NGC_API_KEY"
# Example pattern. Confirm the exact container path from the model page or NIM docs.
docker run --rm -it --gpus all \
-e NGC_API_KEY="$NGC_API_KEY" \
-v "$HOME/.cache/nim:/opt/nim/.cache" \
-p 8000:8000 \
nvcr.io/nim/nvidia/nemotron-3-nano-30b-a3b:latest
curl http://localhost:8000/v1/models
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "nvidia/nemotron-3-nano-30b-a3b",
"messages": [{"role": "user", "content": "Say hello from local NIM."}],
"max_tokens": 64
}'</code></pre>
</div>
<h3>Model-free NIM option</h3>
<p>Use model-free NIM only when you need to serve a supported custom or newer model path and understand the deployment requirements. For most hackathon teams, hosted NIM or a model-specific NIM container is simpler.</p>
<div class="code-wrap">
<div class="code-head">bash <button class="copy" onclick="copyCode(this)">Copy</button></div>
<pre><code>export LOCAL_NIM_CACHE="$HOME/.cache/nim"
mkdir -p "$LOCAL_NIM_CACHE"
export NIM_LLM_IMAGE="nvcr.io/nim/nvidia/model-free-nim:latest"
export NIM_MODEL_PATH="hf://meta-llama/Llama-3.1-8B-Instruct"
docker run --gpus all \
-e NIM_MODEL_PATH="$NIM_MODEL_PATH" \
-e HF_TOKEN="$HF_TOKEN" \
-v "$LOCAL_NIM_CACHE:/opt/nim/.cache" \
-p 8000:8000 \
"$NIM_LLM_IMAGE"</code></pre>
</div>
<h3>Common debugging checklist</h3>
<table>
<thead>
<tr><th>Symptom</th><th>Check</th><th>Fix</th></tr>
</thead>
<tbody>
<tr><td>401 or 403</td><td>API key, account access, endpoint URL.</td><td>Regenerate key, re-export env var, verify model access.</td></tr>
<tr><td>404 model not found</td><td>Model ID mismatch between hosted endpoint and self-hosted endpoint.</td><td>Call <code>/v1/models</code> and use returned model ID.</td></tr>
<tr><td>Slow first response</td><td>Container cold start, model download, cache miss.</td><td>Warm up endpoint before demos and cache models.</td></tr>
<tr><td>JSON/tool parse errors</td><td>Prompt too loose or schema too complex.</td><td>Simplify schema, set temperature low, validate arguments.</td></tr>
</tbody>
</table>
</section>
<section class="panel" id="a">
<h2 id="a-nat">Track A: Agentic Workflows</h2>
<p>Track A teams should aim to show a working agentic service, not just a chatbot. The service should plan, call tools, recover from failure, expose a usable interface, and report at least basic quality or latency metrics.</p>
<h3>Reference architecture</h3>
<div class="code-wrap">
<div class="code-head">text <button class="copy" onclick="copyCode(this)">Copy</button></div>
<pre><code>User / UI / API
-> NeMo Agent Toolkit workflow
-> Nemotron via NIM for reasoning
-> Tools:
- search / RAG
- databases
- business APIs
- document parsers
- calculators / validators
-> Optional MCP server/client boundary
-> Optional A2A specialist agent delegation
-> Trace, evaluation, final answer</code></pre>
</div>
<h3>Install NeMo Agent Toolkit</h3>
<div class="code-wrap">
<div class="code-head">bash <button class="copy" onclick="copyCode(this)">Copy</button></div>
<pre><code>mkdir track-a-agent
cd track-a-agent
python3 -m venv .venv
source .venv/bin/activate
pip install -U uv
uv pip install "nvidia-nat[langchain,mcp,a2a,eval]"
export NVIDIA_API_KEY=nvapi-your-key
nat --help
nat info components -t function</code></pre>
</div>
<h3>Minimal ReAct workflow config</h3>
<div class="code-wrap">
<div class="code-head">yaml <button class="copy" onclick="copyCode(this)">Copy</button></div>
<pre><code># configs/invoice_agent.yml
llms:
nim_llm:
_type: nim
model_name: nvidia/nemotron-3-nano-30b-a3b
api_key: ${NVIDIA_API_KEY}
base_url: ${NIM_BASE_URL:-https://integrate.api.nvidia.com/v1}
temperature: 0.0
max_tokens: 1024
functions:
current_datetime:
_type: current_datetime
wikipedia_search:
_type: wiki_search
max_results: 3
code_generator:
_type: code_generation
programming_language: Python
llm_name: nim_llm
description: "Generate Python helper code only when a code artifact is required."
workflow:
_type: react_agent
llm_name: nim_llm
tool_names:
- current_datetime
- wikipedia_search
- code_generator
verbose: true
parse_agent_response_max_retries: 2</code></pre>
</div>
<h3>Workflow config patterns</h3>
<table>
<thead>
<tr><th>Config block</th><th>What it controls</th></tr>
</thead>
<tbody>
<tr><td><code>llms</code></td><td>Model providers, model names, generation settings, and NIM base URLs.</td></tr>
<tr><td><code>functions</code></td><td>Built-in tools, custom Python tools, retrieval tools, calculators, and code tools.</td></tr>
<tr><td><code>function_groups</code></td><td>Grouped dynamic tools, especially MCP client tool groups.</td></tr>
<tr><td><code>workflow</code></td><td>The agent, router, sequential flow, or executor that binds model and tools.</td></tr>
<tr><td><code>eval</code></td><td>Datasets, evaluators, metrics, and output paths for quality checks.</td></tr>
<tr><td><code>general</code></td><td>Telemetry, logging, retries, object stores, and runtime-level settings.</td></tr>
</tbody>
</table>
<table>
<thead>
<tr><th>Workflow type</th><th>Use when</th></tr>
</thead>
<tbody>
<tr><td><code>react_agent</code></td><td>You need a first agent with explicit tool reasoning and easy logs.</td></tr>
<tr><td>Tool-calling workflow</td><td>You want cleaner structured tool calls and less free-form reasoning.</td></tr>
<tr><td>Router agent</td><td>You need to send India-specific tasks to specialist workflows.</td></tr>
<tr><td>Sequential workflow</td><td>You need deterministic steps such as extract, validate, retrieve, respond.</td></tr>
<tr><td>Parallel workflow</td><td>You want several tools or specialists to answer, then merge results.</td></tr>
</tbody>
</table>
<h3>Run, serve, and test</h3>
<div class="code-wrap">
<div class="code-head">bash <button class="copy" onclick="copyCode(this)">Copy</button></div>
<pre><code># Run once from CLI.
nat run --config_file configs/invoice_agent.yml \
--input "Design an enterprise invoice triage workflow and list the tools it needs."
# Serve as an HTTP API.
nat serve --config_file configs/invoice_agent.yml --host 0.0.0.0 --port 8000
# Call it from another terminal.
curl -X POST http://localhost:8000/generate \
-H "Content-Type: application/json" \
-d '{
"messages": [
{"role": "user", "content": "Check this invoice workflow for missing approval steps."}
]
}'</code></pre>
</div>
<h3>MCP server pattern</h3>
<p>Expose workflow functions as MCP tools when another client or agent needs to call them.</p>
<div class="code-wrap">
<div class="code-head">bash <button class="copy" onclick="copyCode(this)">Copy</button></div>
<pre><code># Publish functions from the workflow as MCP tools.
nat mcp serve --config_file configs/invoice_agent.yml --host 0.0.0.0 --port 9901
# Common smoke test: list tools from an MCP-capable client, then call one tool.
# Keep network and credential policies explicit if connecting enterprise systems.</code></pre>
</div>
<h3>MCP client pattern</h3>
<p>Use an MCP client group when your agent needs to consume tools from another local service, partner API wrapper, database adapter, or team-built tool server.</p>
<div class="code-wrap">
<div class="code-head">yaml <button class="copy" onclick="copyCode(this)">Copy</button></div>
<pre><code># configs/agent_with_mcp.yml
function_groups:
mcp_public_data:
_type: mcp_client
server:
transport: streamable-http
url: "http://localhost:9901/mcp"
include:
- search_schemes
- get_state_policy
tool_call_timeout: 30
reconnect_enabled: true
tool_overrides:
get_state_policy:
alias: india_state_policy_lookup
description: "Look up Indian state-level policy and program data."
workflow:
_type: react_agent
llm_name: nim_llm
tool_names:
- mcp_public_data</code></pre>
</div>
<div class="code-wrap">
<div class="code-head">bash <button class="copy" onclick="copyCode(this)">Copy</button></div>
<pre><code>nat serve --config_file configs/agent_with_mcp.yml --port 8000
curl -s http://localhost:8000/mcp/client/tool/list | jq</code></pre>
</div>
<h3>A2A server pattern</h3>
<p>Use A2A when a workflow should be discoverable as a specialist agent by other agents.</p>
<div class="code-wrap">
<div class="code-head">bash <button class="copy" onclick="copyCode(this)">Copy</button></div>
<pre><code>uv pip install "nvidia-nat[a2a]"