Skip to content

Commit a4696e9

Browse files
s-sajid-alis-sajid-ali
committed
genai updates
 modified: docs/genai/01_getting_started/01_intro.mdx new file: docs/genai/01_getting_started/02_portkey.mdx renamed: docs/genai/03_external_llms/static/portkey_arch.png -> docs/genai/01_getting_started/static/portkey_arch.png modified: docs/genai/02_onboarding/01_intro.mdx new file: docs/genai/02_onboarding/02_setup.mdx renamed: docs/genai/03_external_llms/01_llm_access.mdx -> docs/genai/02_onboarding/03_quickstart.mdx modified: docs/genai/02_onboarding/_category_.json renamed: docs/genai/03_external_llms/02_catalogue.md -> docs/genai/03_external_llms/01_catalogue.md renamed: docs/genai/03_external_llms/03_openwebui.md -> docs/genai/03_external_llms/02_openwebui.md
1 parent b5002d9 commit a4696e9

9 files changed

Lines changed: 55 additions & 25 deletions

File tree

‎docs/genai/01_getting_started/01_intro.mdx‎

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,11 @@ If you're looking to harness Generative AI for administrative or classroom use,
55
:::
66

77
Welcome to Pythia, the generative AI platform for research workflows. As part of the Pythia platform, the following capabilities are offered:
8-
- [Access to externally hosted LLMs](../03_external_llms/01_llm_access.mdx)
8+
- [Access to externally hosted LLMs](../01_getting_started/02_portkey.mdx)
99
- [HPC resources for fine tuning LLMs](../../hpc/08_ml_ai_hpc/04_llm_fine_tuning.md)
1010

11-
Please proceed to the next section to learn about how you could harness LLMs for your research!
12-
13-
1411
:::tip[Personal use]
1512
If you want to access NYU provided LLMs for personal use, proceed to https://gemini.google.com/app with your NYU credentials.
1613
:::
14+
15+
For an overview of the capabilities afforded by the LLM gateway, proceed to the next section.

‎docs/genai/01_getting_started/02_portkey.mdx‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Portkey
2+
3+
[Portkey](https://portkey.ai/docs/introduction/what-is-portkey) is an enterprise-grade LLM gateway running on-prem:
4+
5+
![Portkey Architecture](./static/portkey_arch.png)
6+
7+
Broadly it facilitates:
8+
- **Route requests** via configurations that can allow for fallbacks, load balance, etc
9+
- **Observability** via the control pane that displays your usage statistics and logs. You can retrieve your logs via an API.
10+
- **Prompt management** via the prompt playground. You can compare the response for a prompt across multiple LLMs, collaborate with your team and export your prompts.
11+
- **Guardrails** usage by allowing you to define them or integrate with third party guardrails.
12+
- **Agentic workflows** by integrating with various agentic frameworks like langchain, llamaindex, etc.
13+
- **Security & Governance** by allowing you to set budget and rate limits on the API keys created for your workspace
14+
15+
Proceed to the next section to learn more about the on-boarding process.

docs/genai/03_external_llms/static/portkey_arch.png renamed to docs/genai/01_getting_started/static/portkey_arch.png

File renamed without changes.

‎docs/genai/02_onboarding/01_intro.mdx‎

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,19 @@
11
# Getting access
2-
[workspace-request]: https://forms.gle/6DVA5dX4uPPrdX4Q9
2+
[research-workspace-request]: https://forms.gle/6DVA5dX4uPPrdX4Q9
3+
[member-onboarding-form]: https://forms.gle/wd3zcntnY94Q4LWq5
4+
5+
This section deals with the eligibility for getting access to Pythia. To learn more about affiliate access (students after graduation, collaborators from other institutions, please refer to [this section](../../hpc/01_getting_started/02_getting_and_renewing_an_account.mdx))
6+
7+
:::tip[workspaces]
8+
Access to Pythia is facilitated via workspaces. Beyond facilitating access, they also allow team members to collaboratively dvelop prompts.
9+
:::
310

4-
This section deals with the eligibility for getting access to Pythia.
511

612
## Who is eligible for to access Pythia?
7-
Access to Pythia is facilitated via workspaces. All full-time NYU faculty have the ability to request the creation of workspaces by filling out [this form][workspace-request]:
13+
All full-time NYU faculty have the ability to request the creation of workspaces by filling out [the research intake form][research-workspace-request]. Once we create the workspace, members need to submit the [member on-boarding form][member-onboarding-form] to gain access to the workspace. Please be sure to indicate the models you'd like access to from the [catalouge](../03_external_llms/01_catalogue.md)
14+
15+
16+
:::info
17+
Policies for access to non-research workflows (instruction/administrative) are being worked upon. For now, you can request the creation of workspaces for instruction/administraton uses via the same intake form for research [here][research-workspace-request].
18+
:::
19+

‎docs/genai/02_onboarding/02_setup.mdx‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Setup
2+
3+
## Accessing your workspace
4+
Login to Portkey at [`app.portkey.ai`](https://app.portkey.ai/login) with the `Single-sign-on` option using your NYU NetID. Once you're in, you'll be placed in the "Shared Workspace" by default. You can navigate to your workspace by selecting it from the workspace drop-down list from the top of the left sidebar.
5+
6+
## Virtual Keys
7+
As workspace member you will be able to access virtual keys by clicking on the "Virtual Keys" item on the left sidebar in the "AI Gateway" section. You will have access to virtual keys added by the RTS team, but as an end-user, you will not have the ability to add them.
8+
9+
## API Keys
10+
The first task we recommend that you perform is to create a "User" API key for yourself. Click on the "API Keys" item on the left sidebar. Once you hit the create button you'd be able to choose the type (User/Service), optional items like config and metadata. In addition, you are free to choose as many or as few permissions you'd provide the API key.
11+
12+
:::tip
13+
Having a "User" API key is a pre-requisite for using the [prompt playground](https://app.portkey.ai/playground) where you can collaboratively tune prompts and compare the responses from multiple LLMs.
14+
:::
15+
16+
:::info
17+
The only difference between user and service keys is that the logs with user key will have the NetID of the user as part of the metadata while the services will not. We recommend service keys to be used when you're creating end user applications like chatbots and stick to user keys otherwise.
18+
:::
19+
20+
You now have everything you need to send your first request to an LLM. The next section demonstrates an example.

docs/genai/03_external_llms/01_llm_access.mdx renamed to docs/genai/02_onboarding/03_quickstart.mdx

Lines changed: 1 addition & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,8 @@
1-
# Portkey
2-
3-
[Portkey](https://portkey.ai/docs/introduction/what-is-portkey) is an enterprise-grade LLM gateway running on-prem:
4-
5-
![Portkey Architecture](./static/portkey_arch.png)
6-
7-
Broadly it facilitates:
8-
- **Route requests** via configurations that can allow for fallbacks, load balance, etc
9-
- **Observability** via the control pane that displays your usage statistics and logs. You can retrieve your logs via an API.
10-
- **Prompt management** via the prompt playground. You can compare the response for a prompt across multiple LLMs, collaborate with your team and export your prompts.
11-
- **Guardrails** usage by allowing you to define them or integrate with third party guardrails.
12-
- **Agentic workflows** by integrating with various agentic frameworks like langchain, llamaindex, etc.
13-
- **Security & Governance** by allowing you to set budget and rate limits on the API keys created for your workspace
14-
1+
# Quickstart
152
:::tip[Gateway URL]
163
Whenever you instantiate a Portkey client, the `base_url` must be set to `base_url="https://ai-gateway.apps.cloud.rt.nyu.edu/v1/"`. If you miss this parameter you would be connecting to the vendor's SaaS platform and NYU provisioned virtual keys will not work.
174
:::
185

19-
## Onboarding
20-
Send an email to `genai-support@nyu.edu` to start the onboarding process.
21-
226
## Getting started with Portkey
237
As part of the onboarding process, you would have received an invite which gives you access to a workspace. We will also add virtual keys for LLMs to your workspace as part of the onboarding process. Once you've accepted it, head over to `https://app.portkey.ai/` and select the sign-in with Single Sign-On option and proceed with your NYU email address.
248

‎docs/genai/02_onboarding/_category_.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
{
2-
"label": "On-boarding"
2+
"label": "On-boarding & Setup"
33
}

docs/genai/03_external_llms/02_catalogue.md renamed to docs/genai/03_external_llms/01_catalogue.md

File renamed without changes.

docs/genai/03_external_llms/03_openwebui.md renamed to docs/genai/03_external_llms/02_openwebui.md

File renamed without changes.

0 commit comments

Comments
 (0)