Skip to content

Commit 7fd3e68

Browse files
committed
Fix Zenoh NIF integration and convert scripts to Livebook
- Update all Zenoh API calls to use the correct v0.7.2 API - Convert all .exs scripts to .livemd Livebook notebooks - Force local NIF builds for all services using rustler_precompiled - Re-enable Zenoh integration in livebook_executor - Update service configurations for proper Zenoh session management - Remove unused shared_utils.exs file - Test Zenoh pub/sub functionality with ra_mailbox service
1 parent 4bda72e commit 7fd3e68

27 files changed

Lines changed: 3133 additions & 5063 deletions

.gitignore

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -106,4 +106,5 @@ rel/*/releases/*/cookie
106106
# Release archives
107107
*.tar.gz
108108
pretrained_weights/
109-
thirdparty/vsk-rush-registry/
109+
thirdparty/vsk-rush-registry/
110+
apps/ra_mailbox/*@*/
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
import Config
2+
3+
config :rustler_precompiled, :force_build, zenohex: true

apps/forge-client/lib/dashboard.ex

Lines changed: 20 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -3,36 +3,40 @@ defmodule ForgeClient.Dashboard do
33
Service Dashboard for monitoring active Zenoh liveliness tokens in the Forge VR platform.
44
"""
55

6+
use GenServer
7+
68
def start do
7-
run_dashboard()
9+
GenServer.start_link(__MODULE__, [], name: __MODULE__)
810
end
911

10-
defp run_dashboard do
12+
def init(_) do
1113
IO.puts("Forge VR Service Dashboard")
1214
IO.puts("==========================")
1315
IO.puts("Active VR Services:")
1416
IO.puts("")
1517

1618
# Start the session and monitoring
17-
{:ok, session} = Zenohex.open()
19+
{:ok, session_id} = Zenohex.Session.open()
1820

1921
# Subscribe to liveliness queries under "forge/services/**"
20-
liveliness_subscriber =
21-
Zenohex.Session.declare_subscriber(session, "forge/services/**", liveliness: true)
22+
{:ok, subscriber_id} = Zenohex.Session.declare_subscriber(session_id, "forge/services/**", self())
2223

23-
# Listen for changes
24-
loop(liveliness_subscriber)
24+
{:ok, %{session_id: session_id, subscriber_id: subscriber_id}}
2525
end
2626

27-
defp loop(subscriber) do
28-
Zenohex.Subscriber.loop(subscriber, fn sample ->
29-
case sample.kind do
30-
:put ->
31-
IO.puts("[+] #{sample.key_expr}")
27+
def handle_info(%Zenohex.Sample{} = sample, state) do
28+
case sample.kind do
29+
:put ->
30+
IO.puts("[+] #{sample.key_expr}")
31+
32+
:delete ->
33+
IO.puts("[-] #{sample.key_expr}")
34+
end
35+
{:noreply, state}
36+
end
3237

33-
:delete ->
34-
IO.puts("[-] #{sample.key_expr}")
35-
end
36-
end)
38+
def terminate(_reason, %{session_id: session_id, subscriber_id: subscriber_id}) do
39+
Zenohex.Subscriber.undeclare(subscriber_id)
40+
Zenohex.Session.close(session_id)
3741
end
3842
end

apps/forge-client/mix.exs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ defmodule ForgeClient.MixProject do
2929
defp deps do
3030
[
3131
{:zenohex, "~> 0.7.2"},
32+
{:rustler, ">= 0.0.0", optional: true},
3233
{:jason, "~> 1.4.4"},
3334
{:credo, "~> 1.7", only: [:dev], runtime: false}
3435
]

apps/livebook_executor/README.md

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
# Livebook Executor Service
2+
3+
An Elixir service that can execute Livebook (`.livemd`) notebooks programmatically and perform Google searches.
4+
5+
## Features
6+
7+
- **Notebook Execution**: Execute Livebook notebooks with custom parameters
8+
- **Google Search**: Perform web searches using Google Custom Search API
9+
- **Zenoh Integration**: Service communication via Zenoh pub/sub
10+
- **CLI Interface**: Command-line tools for direct usage
11+
- **REST-like API**: JSON-based request/response protocol
12+
13+
## Installation
14+
15+
1. Add to your supervision tree or run as an escript
16+
2. Set environment variables for Google Search:
17+
```bash
18+
export GOOGLE_API_KEY="your_google_api_key"
19+
export GOOGLE_SEARCH_ENGINE_ID="your_search_engine_id"
20+
```
21+
22+
## Usage
23+
24+
### CLI Commands
25+
26+
```bash
27+
# Start the server
28+
./livebook_executor
29+
30+
# Execute a notebook
31+
./livebook_executor execute elixir/qwen3vl_inference.livemd --param model=qwen2-vl-7b
32+
33+
# Search Google
34+
./livebook_executor search "elixir programming" --num-results 5
35+
36+
# List available notebooks
37+
./livebook_executor list
38+
39+
# Show help
40+
./livebook_executor help
41+
```
42+
43+
### Zenoh API
44+
45+
Send JSON requests to the `livebook_executor/requests` topic:
46+
47+
```json
48+
{
49+
"action": "execute_notebook",
50+
"notebook_path": "elixir/qwen3vl_inference.livemd",
51+
"params": {"model": "qwen2-vl-7b"}
52+
}
53+
```
54+
55+
```json
56+
{
57+
"action": "search_google",
58+
"query": "elixir programming",
59+
"options": {"num_results": 5}
60+
}
61+
```
62+
63+
```json
64+
{
65+
"action": "list_notebooks"
66+
}
67+
```
68+
69+
Receive responses on the `livebook_executor/results` topic:
70+
71+
```json
72+
{
73+
"action": "search_google",
74+
"result": {
75+
"query": "elixir programming",
76+
"total_results": "12300000",
77+
"search_time": 0.45,
78+
"results": [
79+
{
80+
"title": "Elixir Programming Language",
81+
"link": "https://elixir-lang.org/",
82+
"snippet": "Elixir is a dynamic, functional language designed for building scalable and maintainable applications...",
83+
"display_link": "elixir-lang.org"
84+
}
85+
]
86+
},
87+
"timestamp": "2024-01-16T20:40:35Z"
88+
}
89+
```
90+
91+
### Programmatic API
92+
93+
```elixir
94+
# Start the service
95+
{:ok, _} = Application.ensure_all_started(:livebook_executor)
96+
97+
# Execute a notebook
98+
{:ok, result} = LivebookExecutor.Server.execute_notebook("elixir/qwen3vl_inference.livemd", %{"model" => "qwen2-vl-7b"})
99+
100+
# Search Google
101+
{:ok, results} = LivebookExecutor.Server.search_google("elixir programming", %{"num_results" => 5})
102+
103+
# List notebooks
104+
{:ok, notebooks} = LivebookExecutor.Server.list_notebooks()
105+
```
106+
107+
## Dependencies
108+
109+
- `zenohex` - Zenoh communication
110+
- `jason` - JSON encoding/decoding
111+
- `req` - HTTP client for Google Search API
112+
- `livebook` - Livebook runtime (for future notebook execution)
113+
114+
## Google Custom Search Setup
115+
116+
1. Go to [Google Cloud Console](https://console.cloud.google.com/)
117+
2. Create a new project or select existing one
118+
3. Enable the Custom Search JSON API
119+
4. Create credentials (API Key)
120+
5. Go to [Custom Search Engine](https://cse.google.com/)
121+
6. Create a new search engine
122+
7. Get the Search Engine ID
123+
8. Set environment variables:
124+
```bash
125+
export GOOGLE_API_KEY="your_api_key_here"
126+
export GOOGLE_SEARCH_ENGINE_ID="your_search_engine_id_here"
127+
```
128+
129+
## Architecture
130+
131+
The service consists of:
132+
133+
- **CLI Module**: Command-line interface for direct usage
134+
- **Server Module**: GenServer handling requests and Zenoh communication
135+
- **Application Module**: OTP application setup
136+
137+
The service integrates with the existing Zenoh-based service architecture used by other forge components.
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
import Config
2+
3+
config :rustler_precompiled, :force_build, zenohex: true
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
defmodule LivebookExecutor.Application do
2+
@moduledoc false
3+
4+
use Application
5+
6+
@impl true
7+
def start(_type, _args) do
8+
children = [
9+
{LivebookExecutor.Server, []}
10+
]
11+
12+
opts = [strategy: :one_for_one, name: LivebookExecutor.Supervisor]
13+
Supervisor.start_link(children, opts)
14+
end
15+
end

0 commit comments

Comments
 (0)