Skip to content

Commit 3727e01

Browse files
cxl-git-hubchenxilongawni
authored
Fix empty /v1/models response for locally loaded models (ml-explore#713)
* server: expose local --model in /v1/models. use full local path as model id for /v1/models 在加载本地模型时,/v1/models返回空列表,导致 open webui获取不到模型,没办法使用。所以这里把本地模型的全路径作为 model_id返回。 * nits * fix test --------- Co-authored-by: chenxilong <cxl750529174@163.com> Co-authored-by: Awni Hannun <awni.hannun@gmail.com>
1 parent 0957964 commit 3727e01

2 files changed

Lines changed: 13 additions & 0 deletions

File tree

mlx_lm/server.py

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1444,6 +1444,18 @@ def probably_mlx_lm(repo):
14441444
for repo in downloaded_models
14451445
]
14461446

1447+
if self.response_generator.cli_args.model:
1448+
model_path = Path(self.response_generator.cli_args.model)
1449+
if model_path.exists():
1450+
model_id = str(model_path.resolve())
1451+
models.append(
1452+
{
1453+
"id": model_id,
1454+
"object": "model",
1455+
"created": self.created,
1456+
}
1457+
)
1458+
14471459
response = {"object": "list", "data": models}
14481460

14491461
response_json = json.dumps(response).encode()

tests/test_server.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@ def __init__(self, with_draft=False):
4040
"min_p": 0.0,
4141
"max_tokens": 512,
4242
"chat_template_args": {},
43+
"model": None,
4344
},
4445
)
4546

0 commit comments

Comments
 (0)