Skip to content

Commit a2158f1

Browse files
LawhyYuanHeyitianlian
authored
feat(examples/strands_sglang): update to strands-sglang 0.4.2 (#2106)
Co-authored-by: YuanHe <lawhy@amazon.com> Co-authored-by: Chengxing Xie <xiechengxing34@gmail.com>
1 parent 6a0ee15 commit a2158f1

5 files changed

Lines changed: 492 additions & 17 deletions

File tree

examples/strands_sglang/README.md

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ This example connects `slime` with [`strands-sglang`](https://github.com/horizon
1313
`strands-sglang` bridges the gap by extending `strands` with SGLang's native `/generate` endpoint:
1414

1515
- Captures exact token IDs during generation (no retokenization drift)
16-
- Automatically tracks `loss_mask` via `token_manager`
16+
- Automatically tracks `loss_mask` via the `Rollout` tracker (`model.rollout`)
1717
- Provides `ToolLimiter` for clean trajectory truncation
1818

1919
## Install Dependencies
@@ -22,11 +22,9 @@ This example connects `slime` with [`strands-sglang`](https://github.com/horizon
2222
2. Go to slime folder: `cd /root/slime`
2323
3. Install slime: `pip install -e . --no-deps`
2424
4. Go to the example folder: `cd /root/slime/examples/strands_sglang`
25-
5. Install other dependencies: `pip install -r requirements.txt`
25+
5. Install `strands-sglang`: `pip install strands-sglang==0.4.2`
2626

27-
> NOTE: `strands-sglang` is under rapid development, so we recommend using the GitHub repo version: `strands-sglang @ git+https://github.com/horizon-rl/strands-sglang.git`
28-
29-
> NOTE: We use camel-ai's subprocess code interpreter for python code execution, which is NOT a good practice; it's just for convenience of this example.
27+
> NOTE: The `execute_python_code` tool runs code via `subprocess_interpreter.py`, a self-contained interpreter vendored from camel-ai so this example does not depend on the full `camel-ai` package. It runs model-generated code in a local subprocess with **no isolation**, which is NOT a good practice; it is here only for the convenience of this example. Use a sandboxed interpreter (Docker, e2b, microsandbox, ...) for anything beyond local experimentation.
3028
3129
## Prepare Model
3230

examples/strands_sglang/__init__.py

Whitespace-only changes.

examples/strands_sglang/generate_with_strands.py

Lines changed: 9 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,5 @@
1-
# Updated with strands-sglang 0.3.2
21
import logging
32

4-
from camel.interpreters import SubprocessInterpreter
53
from strands import Agent, tool
64
from strands_sglang import SGLangModel, ToolLimiter, get_client_from_slime_args
75
from strands_sglang.tool_parsers import HermesToolParser
@@ -10,6 +8,8 @@
108
from slime.rollout.sglang_rollout import GenerateState
119
from slime.utils.types import Sample
1210

11+
from .subprocess_interpreter import SubprocessInterpreter
12+
1313
logger = logging.getLogger(__name__)
1414

1515
SYSTEM_PROMPT = """
@@ -22,7 +22,6 @@
2222
""".strip()
2323

2424
MAX_TOOL_ITERS = 5
25-
MAX_TOOL_CALLS = None # No limit
2625

2726

2827
@tool
@@ -51,7 +50,7 @@ async def generate(args, sample: Sample, sampling_params) -> Sample:
5150
sampling_params=sampling_params,
5251
)
5352

54-
tool_limiter = ToolLimiter(max_tool_iters=MAX_TOOL_ITERS, max_tool_calls=MAX_TOOL_CALLS)
53+
tool_limiter = ToolLimiter(max_tool_iters=MAX_TOOL_ITERS)
5554
agent = Agent(
5655
model=model,
5756
tools=[execute_python_code],
@@ -71,12 +70,12 @@ async def generate(args, sample: Sample, sampling_params) -> Sample:
7170
sample.status = Sample.Status.TRUNCATED
7271
logger.warning(f"TRUNCATED: {type(e).__name__}: {e}")
7372

74-
# Extract token trajectory from token_manager
75-
tm = model.token_manager
76-
prompt_len = len(tm.segments[0]) # system + user are first segment
77-
sample.tokens = tm.token_ids
78-
sample.loss_mask = tm.loss_mask[prompt_len:]
79-
sample.rollout_log_probs = tm.logprobs[prompt_len:]
73+
# Extract token trajectory from the rollout tracker
74+
rollout = model.rollout
75+
prompt_len = rollout.initial_prompt_length # system + user are the first segment
76+
sample.tokens = rollout.token_ids
77+
sample.loss_mask = rollout.loss_mask[prompt_len:]
78+
sample.rollout_log_probs = rollout.logprobs[prompt_len:]
8079
sample.response_length = len(sample.tokens) - prompt_len
8180
sample.response = model.tokenizer.decode(sample.tokens[prompt_len:], skip_special_tokens=False)
8281
# Tool iteration and tool call count are different because multiple parallel tool calls count as 1 iteration

examples/strands_sglang/requirements.txt

Lines changed: 0 additions & 2 deletions
This file was deleted.

0 commit comments

Comments
 (0)