Skip to content

Commit 90bdea3

Browse files
authored
Update examples (#84)
* update readme Signed-off-by: kerthcet <kerthcet@gmail.com> * Chagne the release version Signed-off-by: kerthcet <kerthcet@gmail.com> --------- Signed-off-by: kerthcet <kerthcet@gmail.com>
1 parent 7286c61 commit 90bdea3

4 files changed

Lines changed: 31 additions & 14 deletions

File tree

Cargo.lock

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[package]
22
name = "arms"
3-
version = "0.1.0"
3+
version = "0.0.1"
44
edition = "2024"
55

66
[dependencies]

README.md

Lines changed: 26 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -2,30 +2,35 @@
22

33
[![Latest Release](https://img.shields.io/github/v/release/inftyai/amrs?include_prereleases)](https://github.com/inftyai/amrs/releases/latest)
44

5-
The Adaptive Model Routing System (AMRS) is a framework designed to select the best-fit model for exploration and exploitation. (still under development)
5+
The Adaptive Model Routing System (AMRS) is a framework designed to select the best-fit model for exploration and exploitation. Rust core with python bindings. Still under active development 🚧.
66

7-
Thanks to [async-openai](https://github.com/64bit/async-openai), AMRS builds on top of it to provide adaptive model routing capabilities.
7+
AMRS builds on top of [async-openai](https://github.com/64bit/async-openai) to provide API services for quick setup. Thanks to open source 💙.
88

99
## Features
1010

11-
- Flexible routing strategies, including:
12-
- **Random**: Randomly selects a model from the available models.
11+
- **Endpoints Support** (only basic ones because of limited resources):
12+
- Chat Completions
13+
- Responses
14+
- More on the way
15+
16+
- **Flexible Routing Strategies**:
17+
- **Random(default)**: Randomly selects a model from the available models.
1318
- **WRR**: Weighted Round Robin selects models based on predefined weights.
14-
- **UCB1**: Upper Confidence Bound based model selection (coming soon).
19+
- **UCB1**: Upper Confidence Bound for balancing exploration and exploitation (coming soon).
1520
- **Adaptive**: Dynamically selects models based on performance metrics (coming soon).
1621

17-
- Broad provider support:
18-
- OpenAI compatible providers (DeepInfra, OpenRouter, etc.)
22+
- **Various Providers Support**:
23+
- OpenAI compatible providers (OpenAI, DeepInfra, etc.)
1924
- More on the way
2025

2126
## How to use
2227

23-
Here's a simple example with the Weighted Round Robin (WRR) routing mode:
28+
Here's a simple example with the Weighted Round Robin (WRR) routing mode. Before running the code, make sure to set your provider API key in the environment variable by running `export <PROVIDER>_API_KEY="your_openai_api_key"`.
29+
Here we use OpenAI as an example.
2430

2531

2632
```rust
27-
// Before running the code, make sure to set your OpenAI API key in the environment variable:
28-
// export OPENAI_API_KEY="your_openai_api_key"
33+
# Make sure OPENAI_API_KEY is set in your environment variables before running this code.
2934

3035
use tokio::runtime::Runtime;
3136
use arms::client;
@@ -55,12 +60,22 @@ let mut client = client::Client::new(config);
5560
let request = chat::CreateChatCompletionRequestArgs::default()
5661
.messages([
5762
chat::ChatCompletionRequestSystemMessage::from("You are a helpful assistant.").into(),
58-
chat::ChatCompletionRequestUserMessage::from("How is the weather today?").into(),
63+
chat::ChatCompletionRequestUserMessage::from("Who won the FIFA World Cup in 2025?").into(),
5964
])
6065
.build()
6166
.unwrap();
6267

6368
let result = Runtime::new().unwrap().block_on(client.create_completion(request));
69+
match result {
70+
Ok(response) => {
71+
for choice in response.choices {
72+
println!("Response: {:?}", choice.message.content);
73+
}
74+
}
75+
Err(e) => {
76+
eprintln!("Error: {}", e);
77+
}
78+
}
6479
```
6580

6681
## Contributing

src/main.rs

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,9 @@ fn main() {
4747

4848
match result {
4949
Ok(response) => {
50-
println!("Response: {:?}", response);
50+
for choice in response.choices {
51+
println!("Response: {:?}", choice.message.content);
52+
}
5153
}
5254
Err(e) => {
5355
eprintln!("Error: {}", e);

0 commit comments

Comments
 (0)