Skip to content

Commit ac6f5dd

Browse files
committed
sync with v0.20.0
2 parents 34461b2 + 7d6d735 commit ac6f5dd

594 files changed

Lines changed: 41210 additions & 18118 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/pull_request_template.md

Lines changed: 5 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,14 @@
11
## Description
2-
[Provide a brief description of the changes in this PR]
32

3+
[Provide a brief description of the changes in this PR]
44

55
## How Has This Been Tested?
6-
[Describe the tests you ran to verify your changes]
7-
8-
9-
## Accepted Risk (provide if relevant)
10-
N/A
11-
126

13-
## Related Issue(s) (provide if relevant)
14-
N/A
15-
16-
17-
## Mental Checklist:
18-
- All of the automated tests pass
19-
- All PR comments are addressed and marked resolved
20-
- If there are migrations, they have been rebased to latest main
21-
- If there are new dependencies, they are added to the requirements
22-
- If there are new environment variables, they are added to all of the deployment methods
23-
- If there are new APIs that don't require auth, they are added to PUBLIC_ENDPOINT_SPECS
24-
- Docker images build and basic functionalities work
25-
- Author has done a final read through of the PR right before merge
7+
[Describe the tests you ran to verify your changes]
268

279
## Backporting (check the box to trigger backport action)
10+
2811
Note: You have to check that the action passes, otherwise resolve the conflicts manually and tag the patches.
12+
2913
- [ ] This PR should be backported (make sure to check that the backport attempt succeeds)
14+
- [ ] [Optional] Override Linear Check

.github/workflows/docker-build-push-cloud-web-container-on-tag.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,7 @@ jobs:
6666
NEXT_PUBLIC_POSTHOG_HOST=${{ secrets.POSTHOG_HOST }}
6767
NEXT_PUBLIC_SENTRY_DSN=${{ secrets.SENTRY_DSN }}
6868
NEXT_PUBLIC_GTM_ENABLED=true
69+
NEXT_PUBLIC_FORGOT_PASSWORD_ENABLED=true
6970
# needed due to weird interactions with the builds for different platforms
7071
no-cache: true
7172
labels: ${{ steps.meta.outputs.labels }}

.github/workflows/docker-build-push-model-server-container-on-tag.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -118,6 +118,6 @@ jobs:
118118
TRIVY_DB_REPOSITORY: "public.ecr.aws/aquasecurity/trivy-db:2"
119119
TRIVY_JAVA_DB_REPOSITORY: "public.ecr.aws/aquasecurity/trivy-java-db:1"
120120
with:
121-
image-ref: docker.io/onyxdotapp/onyx-model-server:${{ github.ref_name }}
121+
image-ref: docker.io/${{ env.REGISTRY_IMAGE }}:${{ github.ref_name }}
122122
severity: "CRITICAL,HIGH"
123123
timeout: "10m"
Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
name: Ensure PR references Linear
2+
3+
on:
4+
pull_request:
5+
types: [opened, edited, reopened, synchronize]
6+
7+
jobs:
8+
linear-check:
9+
runs-on: ubuntu-latest
10+
steps:
11+
- name: Check PR body for Linear link or override
12+
run: |
13+
PR_BODY="${{ github.event.pull_request.body }}"
14+
15+
# Looking for "https://linear.app" in the body
16+
if echo "$PR_BODY" | grep -qE "https://linear\.app"; then
17+
echo "Found a Linear link. Check passed."
18+
exit 0
19+
fi
20+
21+
# Looking for a checked override: "[x] Override Linear Check"
22+
if echo "$PR_BODY" | grep -q "\[x\].*Override Linear Check"; then
23+
echo "Override box is checked. Check passed."
24+
exit 0
25+
fi
26+
27+
# Otherwise, fail the run
28+
echo "No Linear link or override found in the PR description."
29+
exit 1

.github/workflows/pr-python-connector-tests.yml

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,19 @@ env:
2626
GOOGLE_GMAIL_OAUTH_CREDENTIALS_JSON_STR: ${{ secrets.GOOGLE_GMAIL_OAUTH_CREDENTIALS_JSON_STR }}
2727
# Slab
2828
SLAB_BOT_TOKEN: ${{ secrets.SLAB_BOT_TOKEN }}
29-
29+
# Zendesk
30+
ZENDESK_SUBDOMAIN: ${{ secrets.ZENDESK_SUBDOMAIN }}
31+
ZENDESK_EMAIL: ${{ secrets.ZENDESK_EMAIL }}
32+
ZENDESK_TOKEN: ${{ secrets.ZENDESK_TOKEN }}
33+
# Salesforce
34+
SF_USERNAME: ${{ secrets.SF_USERNAME }}
35+
SF_PASSWORD: ${{ secrets.SF_PASSWORD }}
36+
SF_SECURITY_TOKEN: ${{ secrets.SF_SECURITY_TOKEN }}
37+
# Airtable
38+
AIRTABLE_TEST_BASE_ID: ${{ secrets.AIRTABLE_TEST_BASE_ID }}
39+
AIRTABLE_TEST_TABLE_ID: ${{ secrets.AIRTABLE_TEST_TABLE_ID }}
40+
AIRTABLE_TEST_TABLE_NAME: ${{ secrets.AIRTABLE_TEST_TABLE_NAME }}
41+
AIRTABLE_ACCESS_TOKEN: ${{ secrets.AIRTABLE_ACCESS_TOKEN }}
3042
jobs:
3143
connectors-check:
3244
# See https://runs-on.com/runners/linux/

.vscode/env_template.txt

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,8 @@
55
# For local dev, often user Authentication is not needed
66
AUTH_TYPE=disabled
77

8+
# Skip warm up for dev
9+
SKIP_WARM_UP=True
810

911
# Always keep these on for Dev
1012
# Logs all model prompts to stdout
@@ -27,6 +29,7 @@ REQUIRE_EMAIL_VERIFICATION=False
2729

2830
# Set these so if you wipe the DB, you don't end up having to go through the UI every time
2931
GEN_AI_API_KEY=<REPLACE THIS>
32+
OPENAI_API_KEY=<REPLACE THIS>
3033
# If answer quality isn't important for dev, use gpt-4o-mini since it's cheaper
3134
GEN_AI_MODEL_VERSION=gpt-4o
3235
FAST_GEN_AI_MODEL_VERSION=gpt-4o

.vscode/launch.template.jsonc

Lines changed: 43 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@
2828
"Celery heavy",
2929
"Celery indexing",
3030
"Celery beat",
31+
"Celery monitoring",
3132
],
3233
"presentation": {
3334
"group": "1",
@@ -51,7 +52,8 @@
5152
"Celery light",
5253
"Celery heavy",
5354
"Celery indexing",
54-
"Celery beat"
55+
"Celery beat",
56+
"Celery monitoring",
5557
],
5658
"presentation": {
5759
"group": "1",
@@ -269,6 +271,31 @@
269271
},
270272
"consoleTitle": "Celery indexing Console"
271273
},
274+
{
275+
"name": "Celery monitoring",
276+
"type": "debugpy",
277+
"request": "launch",
278+
"module": "celery",
279+
"cwd": "${workspaceFolder}/backend",
280+
"envFile": "${workspaceFolder}/.vscode/.env",
281+
"env": {},
282+
"args": [
283+
"-A",
284+
"onyx.background.celery.versioned_apps.monitoring",
285+
"worker",
286+
"--pool=solo",
287+
"--concurrency=1",
288+
"--prefetch-multiplier=1",
289+
"--loglevel=INFO",
290+
"--hostname=monitoring@%n",
291+
"-Q",
292+
"monitoring",
293+
],
294+
"presentation": {
295+
"group": "2",
296+
},
297+
"consoleTitle": "Celery monitoring Console"
298+
},
272299
{
273300
"name": "Celery beat",
274301
"type": "debugpy",
@@ -355,5 +382,20 @@
355382
"PYTHONPATH": "."
356383
},
357384
},
385+
{
386+
"name": "Install Python Requirements",
387+
"type": "node",
388+
"request": "launch",
389+
"runtimeExecutable": "bash",
390+
"runtimeArgs": [
391+
"-c",
392+
"pip install -r backend/requirements/default.txt && pip install -r backend/requirements/dev.txt && pip install -r backend/requirements/ee.txt && pip install -r backend/requirements/model_server.txt"
393+
],
394+
"cwd": "${workspaceFolder}",
395+
"console": "integratedTerminal",
396+
"presentation": {
397+
"group": "3"
398+
}
399+
},
358400
]
359401
}

CONTRIBUTING.md

Lines changed: 49 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,10 @@ As an open source project in a rapidly changing space, we welcome all contributi
1212

1313
The [GitHub Issues](https://github.com/onyx-dot-app/onyx/issues) page is a great place to start for contribution ideas.
1414

15+
To ensure that your contribution is aligned with the project's direction, please reach out to Hagen (or any other maintainer) on the Onyx team
16+
via [Slack](https://join.slack.com/t/onyx-dot-app/shared_invite/zt-2twesxdr6-5iQitKZQpgq~hYIZ~dv3KA) /
17+
[Discord](https://discord.gg/TDJ59cGV2X) or [email](mailto:founders@onyx.app).
18+
1519
Issues that have been explicitly approved by the maintainers (aligned with the direction of the project)
1620
will be marked with the `approved by maintainers` label.
1721
Issues marked `good first issue` are an especially great place to start.
@@ -23,8 +27,8 @@ If you have a new/different contribution in mind, we'd love to hear about it!
2327
Your input is vital to making sure that Onyx moves in the right direction.
2428
Before starting on implementation, please raise a GitHub issue.
2529

26-
And always feel free to message us (Chris Weaver / Yuhong Sun) on
27-
[Slack](https://join.slack.com/t/danswer/shared_invite/zt-1w76msxmd-HJHLe3KNFIAIzk_0dSOKaQ) /
30+
Also, always feel free to message the founders (Chris Weaver / Yuhong Sun) on
31+
[Slack](https://join.slack.com/t/onyx-dot-app/shared_invite/zt-2twesxdr6-5iQitKZQpgq~hYIZ~dv3KA) /
2832
[Discord](https://discord.gg/TDJ59cGV2X) directly about anything at all.
2933

3034
### Contributing Code
@@ -42,7 +46,7 @@ Our goal is to make contributing as easy as possible. If you run into any issues
4246
That way we can help future contributors and users can avoid the same issue.
4347

4448
We also have support channels and generally interesting discussions on our
45-
[Slack](https://join.slack.com/t/danswer/shared_invite/zt-1w76msxmd-HJHLe3KNFIAIzk_0dSOKaQ)
49+
[Slack](https://join.slack.com/t/onyx-dot-app/shared_invite/zt-2twesxdr6-5iQitKZQpgq~hYIZ~dv3KA)
4650
and
4751
[Discord](https://discord.gg/TDJ59cGV2X).
4852

@@ -123,7 +127,47 @@ Once the above is done, navigate to `onyx/web` run:
123127
npm i
124128
```
125129

126-
#### Docker containers for external software
130+
## Formatting and Linting
131+
132+
### Backend
133+
134+
For the backend, you'll need to setup pre-commit hooks (black / reorder-python-imports).
135+
First, install pre-commit (if you don't have it already) following the instructions
136+
[here](https://pre-commit.com/#installation).
137+
138+
With the virtual environment active, install the pre-commit library with:
139+
140+
```bash
141+
pip install pre-commit
142+
```
143+
144+
Then, from the `onyx/backend` directory, run:
145+
146+
```bash
147+
pre-commit install
148+
```
149+
150+
Additionally, we use `mypy` for static type checking.
151+
Onyx is fully type-annotated, and we want to keep it that way!
152+
To run the mypy checks manually, run `python -m mypy .` from the `onyx/backend` directory.
153+
154+
### Web
155+
156+
We use `prettier` for formatting. The desired version (2.8.8) will be installed via a `npm i` from the `onyx/web` directory.
157+
To run the formatter, use `npx prettier --write .` from the `onyx/web` directory.
158+
Please double check that prettier passes before creating a pull request.
159+
160+
# Running the application for development
161+
162+
## Developing using VSCode Debugger (recommended)
163+
164+
We highly recommend using VSCode debugger for development.
165+
See [CONTRIBUTING_VSCODE.md](./CONTRIBUTING_VSCODE.md) for more details.
166+
167+
Otherwise, you can follow the instructions below to run the application for development.
168+
169+
## Manually running the application for development
170+
### Docker containers for external software
127171

128172
You will need Docker installed to run these containers.
129173

@@ -135,7 +179,7 @@ docker compose -f docker-compose.dev.yml -p onyx-stack up -d index relational_db
135179

136180
(index refers to Vespa, relational_db refers to Postgres, and cache refers to Redis)
137181

138-
#### Running Onyx locally
182+
### Running Onyx locally
139183

140184
To start the frontend, navigate to `onyx/web` and run:
141185

@@ -223,35 +267,6 @@ If you want to make changes to Onyx and run those changes in Docker, you can als
223267
docker compose -f docker-compose.dev.yml -p onyx-stack up -d --build
224268
```
225269

226-
### Formatting and Linting
227-
228-
#### Backend
229-
230-
For the backend, you'll need to setup pre-commit hooks (black / reorder-python-imports).
231-
First, install pre-commit (if you don't have it already) following the instructions
232-
[here](https://pre-commit.com/#installation).
233-
234-
With the virtual environment active, install the pre-commit library with:
235-
236-
```bash
237-
pip install pre-commit
238-
```
239-
240-
Then, from the `onyx/backend` directory, run:
241-
242-
```bash
243-
pre-commit install
244-
```
245-
246-
Additionally, we use `mypy` for static type checking.
247-
Onyx is fully type-annotated, and we want to keep it that way!
248-
To run the mypy checks manually, run `python -m mypy .` from the `onyx/backend` directory.
249-
250-
#### Web
251-
252-
We use `prettier` for formatting. The desired version (2.8.8) will be installed via a `npm i` from the `onyx/web` directory.
253-
To run the formatter, use `npx prettier --write .` from the `onyx/web` directory.
254-
Please double check that prettier passes before creating a pull request.
255270

256271
### Release Process
257272

CONTRIBUTING_VSCODE.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# VSCode Debugging Setup
2+
3+
This guide explains how to set up and use VSCode's debugging capabilities with this project.
4+
5+
## Initial Setup
6+
7+
1. **Environment Setup**:
8+
- Copy `.vscode/.env.template` to `.vscode/.env`
9+
- Fill in the necessary environment variables in `.vscode/.env`
10+
2. **launch.json**:
11+
- Copy `.vscode/launch.template.jsonc` to `.vscode/launch.json`
12+
13+
## Using the Debugger
14+
15+
Before starting, make sure the Docker Daemon is running.
16+
17+
1. Open the Debug view in VSCode (Cmd+Shift+D on macOS)
18+
2. From the dropdown at the top, select "Clear and Restart External Volumes and Containers" and press the green play button
19+
3. From the dropdown at the top, select "Run All Onyx Services" and press the green play button
20+
4. CD into web, run "npm i" followed by npm run dev.
21+
5. Now, you can navigate to onyx in your browser (default is http://localhost:3000) and start using the app
22+
6. You can set breakpoints by clicking to the left of line numbers to help debug while the app is running
23+
7. Use the debug toolbar to step through code, inspect variables, etc.
24+
25+
## Features
26+
27+
- Hot reload is enabled for the web server and API servers
28+
- Python debugging is configured with debugpy
29+
- Environment variables are loaded from `.vscode/.env`
30+
- Console output is organized in the integrated terminal with labeled tabs

README.md

Lines changed: 4 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
<a name="readme-top"></a>
44

55
<h2 align="center">
6-
<a href="https://www.onyx.app/"> <img width="50%" src="https://github.com/onyx-dot-app/onyx/blob/logo/LogoOnyx.png?raw=true)" /></a>
6+
<a href="https://www.onyx.app/"> <img width="50%" src="https://github.com/onyx-dot-app/onyx/blob/logo/OnyxLogoCropped.jpg?raw=true)" /></a>
77
</h2>
88

99
<p align="center">
@@ -13,7 +13,7 @@
1313
<a href="https://docs.onyx.app/" target="_blank">
1414
<img src="https://img.shields.io/badge/docs-view-blue" alt="Documentation">
1515
</a>
16-
<a href="https://join.slack.com/t/danswer/shared_invite/zt-1w76msxmd-HJHLe3KNFIAIzk_0dSOKaQ" target="_blank">
16+
<a href="https://join.slack.com/t/onyx-dot-app/shared_invite/zt-2twesxdr6-5iQitKZQpgq~hYIZ~dv3KA" target="_blank">
1717
<img src="https://img.shields.io/badge/slack-join-blue.svg?logo=slack" alt="Slack">
1818
</a>
1919
<a href="https://discord.gg/TDJ59cGV2X" target="_blank">
@@ -24,7 +24,7 @@
2424
</a>
2525
</p>
2626

27-
<strong>[Onyx](https://www.onyx.app/)</strong> (Formerly Danswer) is the AI Assistant connected to your company's docs, apps, and people.
27+
<strong>[Onyx](https://www.onyx.app/)</strong> (formerly Danswer) is the AI Assistant connected to your company's docs, apps, and people.
2828
Onyx provides a Chat interface and plugs into any LLM of your choice. Onyx can be deployed anywhere and for any
2929
scale - on a laptop, on-premise, or to cloud. Since you own the deployment, your user data and chats are fully in your
3030
own control. Onyx is dual Licensed with most of it under MIT license and designed to be modular and easily extensible. The system also comes fully ready
@@ -119,7 +119,7 @@ There are two editions of Onyx:
119119
- Whitelabeling
120120
- API key authentication
121121
- Encryption of secrets
122-
- Any many more! Checkout [our website](https://www.onyx.app/) for the latest.
122+
- And many more! Checkout [our website](https://www.onyx.app/) for the latest.
123123

124124
To try the Onyx Enterprise Edition:
125125

@@ -133,15 +133,3 @@ Looking to contribute? Please check out the [Contribution Guide](CONTRIBUTING.md
133133
## ⭐Star History
134134

135135
[![Star History Chart](https://api.star-history.com/svg?repos=onyx-dot-app/onyx&type=Date)](https://star-history.com/#onyx-dot-app/onyx&Date)
136-
137-
## ✨Contributors
138-
139-
<a href="https://github.com/onyx-dot-app/onyx/graphs/contributors">
140-
<img alt="contributors" src="https://contrib.rocks/image?repo=onyx-dot-app/onyx"/>
141-
</a>
142-
143-
<p align="right" style="font-size: 14px; color: #555; margin-top: 20px;">
144-
<a href="#readme-top" style="text-decoration: none; color: #007bff; font-weight: bold;">
145-
↑ Back to Top ↑
146-
</a>
147-
</p>

0 commit comments

Comments
 (0)