Skip to content

Commit c742fd8

Browse files
authored
examples: Add Qdrant CVE semantic search POC (#323)
Adds a local vulnerability search example using Encoderfile for embeddings and Qdrant for vector storage.
1 parent 391be40 commit c742fd8

6 files changed

Lines changed: 1704 additions & 0 deletions

File tree

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
# CVE Semantic Search — Encoderfile + Qdrant POC
2+
3+
A fully local, privacy-first vulnerability search system demonstrating
4+
how [Encoderfile](https://github.com/mozilla-ai/encoderfile) and
5+
[Qdrant](https://qdrant.tech/) work together.
6+
7+
**Encoderfile** generates embeddings locally.
8+
**Qdrant** stores and searches vectors — self-hosted, no data leaves the network.
9+
10+
## The Use Case
11+
12+
Internal security teams need to search across their vulnerability reports, pen test findings, and bug bounty submissions using natural language. Queries like "authentication bypass in our VPN infrastructure" should return semantically relevant results — not just keyword matches.
13+
14+
The data is too sensitive for cloud embedding APIs. Encoderfile + Qdrant
15+
keeps everything local.
16+
17+
### Key Characteristics
18+
19+
- No Python ML Runtime: Encoderfile runs as a single, compiled binary.
20+
21+
- Air-gapped by Default: Zero external network calls required at inference time.
22+
23+
- Auditable: The embedding binary is reproducible and hashable for strict compliance environments.
24+
25+
## Quick Start
26+
27+
### 1. Install dependencies
28+
29+
```bash
30+
uv sync
31+
```
32+
33+
To include the FastEmbed fallback:
34+
35+
```bash
36+
uv sync --extra fastembed
37+
```
38+
39+
### 2. Download and Start Encoderfile
40+
41+
Download the `all-MiniLM-L6-v2` binary for your specific OS and architecture (e.g., macOS ARM, Linux x86) from the [Mozilla AI Hugging Face repository](https://huggingface.co/mozilla-ai/encoderfile/tree/main/sentence-transformers/all-MiniLM-L6-v2).
42+
43+
Once downloaded, make it executable and start the server:
44+
45+
```bash
46+
chmod +x all-MiniLM-L6-v2.encoderfile
47+
./all-MiniLM-L6-v2.encoderfile serve --http-port 8080
48+
```
49+
50+
### 3. Run the demo
51+
52+
```bash
53+
# Run with default example query
54+
uv run main.py
55+
56+
# Run a specific query
57+
uv run main.py -q "SQL injection in file transfer software"
58+
59+
# Interactive search mode
60+
uv run main.py --interactive
61+
```
62+
63+
The demo auto-detects whether Encoderfile is running. If not, it falls back
64+
to FastEmbed (Qdrant's own embedding library, same model) so you can see
65+
the integration working regardless.
66+
67+
## What It Does
68+
69+
1. Embeds 30 real CVE descriptions (from NVD's public database)
70+
2. Stores the vectors + metadata in Qdrant
71+
3. Runs a query and shows ranked results
72+
73+
Try queries like:
74+
- `authentication bypass in VPN products`
75+
- `SQL injection in file transfer software`
76+
- `supply chain compromise through software updates`
77+
- `zero-day exploited by nation-state actors`
78+
79+
## File Structure
80+
81+
```
82+
cve-search-poc/
83+
├── main.py # Ingest → search, with optional --interactive mode
84+
├── embedders.py # Embedding providers (Encoderfile primary, FastEmbed fallback)
85+
├── cve_data.json # Sample CVE data (30 real CVEs from NVD)
86+
├── pyproject.toml # uv project definition and dependencies
87+
└── README.md
88+
```
89+
90+
## Extending This
91+
92+
**More data**: Replace `cve_data.json` with a script that pulls from
93+
[NVD's API](https://nvd.nist.gov/developers/vulnerabilities) or
94+
ingests your internal vulnerability reports.
95+
96+
**Full RAG**: Add [Llamafile](https://github.com/Mozilla-Ocho/llamafile)
97+
as the generation layer. Retrieve relevant CVEs from Qdrant, pass them
98+
as context to a local LLM, get natural language summaries.
99+
100+
**Production Qdrant**: Swap `QdrantClient(":memory:")` for
101+
`QdrantClient("localhost", port=6333)` pointed at a Docker or
102+
bare-metal Qdrant instance with persistence.
103+
Lines changed: 212 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,212 @@
1+
[
2+
{
3+
"id": "CVE-2024-3094",
4+
"severity": "CRITICAL",
5+
"description": "Malicious code was discovered in the upstream tarballs of xz-utils beginning with version 5.6.0. The backdoor manipulates the build process to inject code into the resulting liblzma library, which is used by sshd. This results in unauthorized remote access to affected systems via SSH.",
6+
"affected": "xz-utils 5.6.0, 5.6.1",
7+
"category": "supply-chain"
8+
},
9+
{
10+
"id": "CVE-2024-21762",
11+
"severity": "CRITICAL",
12+
"description": "A out-of-bounds write vulnerability in Fortinet FortiOS allows a remote unauthenticated attacker to execute arbitrary code or command via specially crafted HTTP requests. This affects the SSL VPN component and can be exploited without authentication.",
13+
"affected": "FortiOS 7.4.0 through 7.4.2, 7.2.0 through 7.2.6",
14+
"category": "remote-code-execution"
15+
},
16+
{
17+
"id": "CVE-2023-44487",
18+
"severity": "HIGH",
19+
"description": "The HTTP/2 protocol allows a denial of service via rapid reset. An attacker can send a large number of HTTP/2 requests and immediately cancel them, creating asymmetric resource consumption on the server. This was exploited in the wild as a zero-day to launch the largest DDoS attacks ever recorded.",
20+
"affected": "Multiple HTTP/2 implementations including nginx, Apache, and cloud providers",
21+
"category": "denial-of-service"
22+
},
23+
{
24+
"id": "CVE-2023-4966",
25+
"severity": "CRITICAL",
26+
"description": "Citrix NetScaler ADC and NetScaler Gateway contain a buffer overflow vulnerability that allows sensitive information disclosure when configured as a Gateway or AAA virtual server. Attackers can steal session tokens and hijack authenticated sessions without credentials.",
27+
"affected": "NetScaler ADC and Gateway 14.1 before 14.1-8.50, 13.1 before 13.1-49.15",
28+
"category": "information-disclosure"
29+
},
30+
{
31+
"id": "CVE-2024-1709",
32+
"severity": "CRITICAL",
33+
"description": "ConnectWise ScreenConnect contains an authentication bypass vulnerability that allows an attacker to create administrative accounts, upload malicious extensions, and gain remote code execution on the ScreenConnect server. This was widely exploited in the wild.",
34+
"affected": "ScreenConnect versions prior to 23.9.8",
35+
"category": "authentication-bypass"
36+
},
37+
{
38+
"id": "CVE-2023-36884",
39+
"severity": "HIGH",
40+
"description": "Microsoft Office and Windows contain a remote code execution vulnerability related to HTML processing. An attacker can craft a Microsoft Office document that, when opened, allows remote code execution in the context of the user. This was exploited by a Russian threat actor group targeting NATO summit attendees.",
41+
"affected": "Microsoft Office 2013-2021, Windows 10, Windows 11, Windows Server 2022",
42+
"category": "remote-code-execution"
43+
},
44+
{
45+
"id": "CVE-2023-34362",
46+
"severity": "CRITICAL",
47+
"description": "MOVEit Transfer contains a SQL injection vulnerability that could allow an unauthenticated attacker to gain access to the MOVEit Transfer database. An attacker could submit crafted payloads to escalate privileges and gain unauthorized access to the file transfer environment.",
48+
"affected": "MOVEit Transfer versions before 2021.0.6, 2021.1.4, 2022.0.4, 2022.1.5, 2023.0.1",
49+
"category": "sql-injection"
50+
},
51+
{
52+
"id": "CVE-2024-27198",
53+
"severity": "CRITICAL",
54+
"description": "JetBrains TeamCity contains an authentication bypass vulnerability that allows an unauthenticated attacker with HTTP access to perform administrative actions. This includes creating admin accounts, modifying build configurations, and executing arbitrary code on the build server.",
55+
"affected": "TeamCity before 2023.11.4",
56+
"category": "authentication-bypass"
57+
},
58+
{
59+
"id": "CVE-2023-42793",
60+
"severity": "CRITICAL",
61+
"description": "JetBrains TeamCity contains an authentication bypass vulnerability allowing unauthenticated attackers to achieve remote code execution on the TeamCity server via a crafted request to the REST API. This affects on-premises instances and was exploited by nation-state threat actors.",
62+
"affected": "TeamCity on-premises before 2023.05.4",
63+
"category": "authentication-bypass"
64+
},
65+
{
66+
"id": "CVE-2023-46805",
67+
"severity": "HIGH",
68+
"description": "Ivanti Connect Secure and Ivanti Policy Secure contain an authentication bypass vulnerability in the web component. When chained with CVE-2024-21887, this allows an attacker to craft malicious requests to execute arbitrary commands on the appliance. Widely exploited by Chinese threat actors.",
69+
"affected": "Ivanti Connect Secure 9.x, 22.x, Ivanti Policy Secure 9.x, 22.x",
70+
"category": "authentication-bypass"
71+
},
72+
{
73+
"id": "CVE-2024-23897",
74+
"severity": "CRITICAL",
75+
"description": "Jenkins has a vulnerability in its built-in CLI that allows unauthenticated attackers to read arbitrary files from the Jenkins controller file system. The args4j library used by Jenkins expands file paths in CLI arguments, enabling attackers to read SSH keys, credentials, and build secrets.",
76+
"affected": "Jenkins weekly up to 2.441, LTS up to 2.426.2",
77+
"category": "information-disclosure"
78+
},
79+
{
80+
"id": "CVE-2024-0204",
81+
"severity": "CRITICAL",
82+
"description": "Fortra GoAnywhere MFT contains an authentication bypass vulnerability that allows an unauthorized user to create an admin user via the administration portal. This can lead to complete takeover of the managed file transfer server and access to all transferred files.",
83+
"affected": "GoAnywhere MFT before 7.4.1",
84+
"category": "authentication-bypass"
85+
},
86+
{
87+
"id": "CVE-2023-20198",
88+
"severity": "CRITICAL",
89+
"description": "Cisco IOS XE contains a privilege escalation vulnerability in the web UI feature. An unauthenticated remote attacker can create an account on the device with privilege level 15 access, granting full control of the compromised system. Tens of thousands of devices were exploited in the wild.",
90+
"affected": "Cisco IOS XE with web UI enabled",
91+
"category": "privilege-escalation"
92+
},
93+
{
94+
"id": "CVE-2024-6387",
95+
"severity": "HIGH",
96+
"description": "A signal handler race condition was found in OpenSSH server sshd. If a client does not authenticate within LoginGraceTime seconds, the sshd SIGALRM handler is called asynchronously and calls various functions that are not async-signal-safe. A remote unauthenticated attacker may exploit this to achieve code execution as root.",
97+
"affected": "OpenSSH 8.5p1 through 9.7p1",
98+
"category": "remote-code-execution"
99+
},
100+
{
101+
"id": "CVE-2023-38545",
102+
"severity": "HIGH",
103+
"description": "A heap-based buffer overflow vulnerability exists in curl's SOCKS5 proxy handshake. When curl is asked to use a SOCKS5 proxy and the hostname is too long, curl switches to local name resolution and copies the excessively long hostname to a heap buffer, causing a buffer overflow that could lead to code execution.",
104+
"affected": "curl 7.69.0 through 8.3.0, libcurl 7.69.0 through 8.3.0",
105+
"category": "buffer-overflow"
106+
},
107+
{
108+
"id": "CVE-2024-4577",
109+
"severity": "CRITICAL",
110+
"description": "PHP CGI contains an argument injection vulnerability on Windows. Due to the way PHP handles character encoding conversions, an unauthenticated attacker can bypass previous protections for CVE-2012-1823 and inject arbitrary arguments to PHP binary, leading to remote code execution on Windows servers running PHP in CGI mode.",
111+
"affected": "PHP 8.3 before 8.3.8, 8.2 before 8.2.20, 8.1 before 8.1.29 on Windows",
112+
"category": "remote-code-execution"
113+
},
114+
{
115+
"id": "CVE-2024-3400",
116+
"severity": "CRITICAL",
117+
"description": "A command injection vulnerability in the GlobalProtect feature of Palo Alto Networks PAN-OS allows an unauthenticated attacker to execute arbitrary code with root privileges on the firewall. This was exploited as a zero-day in targeted attacks against organizations.",
118+
"affected": "PAN-OS 10.2, 11.0, 11.1 with GlobalProtect gateway configured",
119+
"category": "command-injection"
120+
},
121+
{
122+
"id": "CVE-2023-22515",
123+
"severity": "CRITICAL",
124+
"description": "Atlassian Confluence Data Center and Server contain a broken access control vulnerability that allows an external attacker to create unauthorized administrator accounts and access Confluence instances. This was exploited as a zero-day by a nation-state threat actor.",
125+
"affected": "Confluence Data Center and Server 8.0.0 through 8.5.1",
126+
"category": "authentication-bypass"
127+
},
128+
{
129+
"id": "CVE-2024-21887",
130+
"severity": "CRITICAL",
131+
"description": "A command injection vulnerability in the web components of Ivanti Connect Secure and Ivanti Policy Secure allows an authenticated administrator to send specially crafted requests to execute arbitrary commands on the appliance. When chained with CVE-2023-46805, no authentication is required.",
132+
"affected": "Ivanti Connect Secure 9.x, 22.x, Ivanti Policy Secure 9.x, 22.x",
133+
"category": "command-injection"
134+
},
135+
{
136+
"id": "CVE-2023-23397",
137+
"severity": "CRITICAL",
138+
"description": "Microsoft Outlook contains a privilege escalation vulnerability where a specially crafted email can leak the victim's Net-NTLMv2 hash to an attacker-controlled server. The vulnerability is triggered automatically when the email is retrieved and processed by the Outlook client, requiring no user interaction.",
139+
"affected": "Microsoft Outlook for Windows, all versions",
140+
"category": "privilege-escalation"
141+
},
142+
{
143+
"id": "CVE-2024-2389",
144+
"severity": "CRITICAL",
145+
"description": "Progress Flowmon contains a command injection vulnerability that allows an unauthenticated attacker to gain access to the Flowmon web interface via a specially crafted API request. Successful exploitation gives full system access to the network monitoring platform.",
146+
"affected": "Flowmon before 12.3.5, 11.1.14",
147+
"category": "command-injection"
148+
},
149+
{
150+
"id": "CVE-2023-48788",
151+
"severity": "CRITICAL",
152+
"description": "Fortinet FortiClient EMS contains a SQL injection vulnerability that allows an unauthenticated attacker to execute unauthorized code or commands via specially crafted requests to the database. This affects the endpoint management server and can lead to complete compromise of managed endpoints.",
153+
"affected": "FortiClient EMS 7.2.0 through 7.2.2, 7.0.1 through 7.0.10",
154+
"category": "sql-injection"
155+
},
156+
{
157+
"id": "CVE-2024-20353",
158+
"severity": "HIGH",
159+
"description": "A vulnerability in the management and VPN web servers for Cisco Adaptive Security Appliance and Firepower Threat Defense allows an unauthenticated remote attacker to cause a denial of service. The attacker can force the device to reload unexpectedly, disrupting VPN and firewall services.",
160+
"affected": "Cisco ASA Software, Cisco FTD Software",
161+
"category": "denial-of-service"
162+
},
163+
{
164+
"id": "CVE-2023-29357",
165+
"severity": "CRITICAL",
166+
"description": "Microsoft SharePoint Server contains a privilege escalation vulnerability that allows an attacker to gain administrator privileges. An attacker who has obtained spoofed JWT authentication tokens can use them to execute a network attack which bypasses authentication and gives them access to administrator privileges.",
167+
"affected": "Microsoft SharePoint Server 2019",
168+
"category": "privilege-escalation"
169+
},
170+
{
171+
"id": "CVE-2024-28986",
172+
"severity": "CRITICAL",
173+
"description": "SolarWinds Web Help Desk contains a Java deserialization vulnerability that allows a remote attacker to execute commands on the host machine. The vulnerability exists in the way the application handles serialized Java objects and can be exploited without authentication.",
174+
"affected": "SolarWinds Web Help Desk 12.8.3 HF1 and prior",
175+
"category": "remote-code-execution"
176+
},
177+
{
178+
"id": "CVE-2023-29059",
179+
"severity": "HIGH",
180+
"description": "3CX Desktop App versions shipped with a compromised DLL that was part of a supply chain attack. The trojanized application was signed with a valid 3CX certificate and distributed through the official update mechanism, allowing attackers to deploy information-stealing malware to thousands of organizations.",
181+
"affected": "3CX Desktop App 18.12.407 and 18.12.416 for Windows, 18.11.1213 for macOS",
182+
"category": "supply-chain"
183+
},
184+
{
185+
"id": "CVE-2023-27997",
186+
"severity": "CRITICAL",
187+
"description": "A heap-based buffer overflow vulnerability exists in the SSL VPN component of Fortinet FortiOS. A remote attacker can execute arbitrary code or commands via specifically crafted requests. This pre-authentication vulnerability affects the SSL VPN portal and was exploited by threat actors targeting government organizations.",
188+
"affected": "FortiOS 7.2.0 through 7.2.4, 7.0.0 through 7.0.11, 6.4.0 through 6.4.12",
189+
"category": "buffer-overflow"
190+
},
191+
{
192+
"id": "CVE-2024-5806",
193+
"severity": "CRITICAL",
194+
"description": "Progress MOVEit Transfer contains an authentication bypass vulnerability in the SFTP module. An attacker can bypass authentication by manipulating SSH public key paths, allowing unauthorized access to the file transfer system and all stored files without valid credentials.",
195+
"affected": "MOVEit Transfer 2023.0 before 2023.0.11, 2023.1 before 2023.1.6, 2024.0 before 2024.0.2",
196+
"category": "authentication-bypass"
197+
},
198+
{
199+
"id": "CVE-2023-20269",
200+
"severity": "MEDIUM",
201+
"description": "Cisco Adaptive Security Appliance and Firepower Threat Defense contain a vulnerability in the remote access VPN feature that allows an unauthenticated remote attacker to conduct brute force credential attacks against existing VPN profiles. Successful exploitation reveals valid VPN credentials.",
202+
"affected": "Cisco ASA Software, Cisco FTD Software with remote access VPN enabled",
203+
"category": "brute-force"
204+
},
205+
{
206+
"id": "CVE-2024-21893",
207+
"severity": "HIGH",
208+
"description": "A server-side request forgery vulnerability in the SAML component of Ivanti Connect Secure, Ivanti Policy Secure, and Ivanti Neurons for ZTA allows an attacker to access certain restricted resources without authentication. This was chained with other vulnerabilities for full system compromise.",
209+
"affected": "Ivanti Connect Secure 9.x, 22.x, Ivanti Policy Secure 9.x, 22.x",
210+
"category": "ssrf"
211+
}
212+
]

0 commit comments

Comments
 (0)