Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@ Features: SVE, NEON
This project demonstrates running large language models on CPU using llama.cpp compiled with Arm baseline optimizations and accelerated using NEON SIMD and SVE (when supported and enabled).

The stack includes:
- llama.cpp server with Arm NEON optimizations (SVE optional)
- Quantized SmolLM2-135M-Instruct model bundled in the image
- Simple web-based chat interface
- Prebuilt llama.cpp server runtime
Comment thread
yejseo01 marked this conversation as resolved.
- Quantized SmolLM2 135M model bundled in the image
- Built-in web chat interface
- No GPU required - pure CPU inference

## Prerequisites
Expand Down
18 changes: 3 additions & 15 deletions compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,18 +15,6 @@ services:
retries: 3
start_period: 60s

chat-ui:
platform: linux/arm64
build:
context: ./simple-chat
args:
ENABLE_SVE: OFF
depends_on:
llama-server:
condition: service_healthy
ports:
- "3000:3000"

x-topo:
name: "Topo CPU AI Chat"
description: |
Expand All @@ -37,9 +25,9 @@ x-topo:
accelerated using NEON SIMD and SVE (when supported and enabled).

The stack includes:
- llama.cpp server with Arm NEON optimizations (SVE optional)
- Quantized SmolLM2-135M-Instruct model bundled in the image
- Simple web-based chat interface
- Prebuilt llama.cpp server runtime
- Quantized SmolLM2 135M model bundled in the image
- Built-in web chat interface
- No GPU required - pure CPU inference

Perfect for demos and testing! The bundled SmolLM2-135M-Instruct model
Expand Down
2 changes: 0 additions & 2 deletions simple-chat/requirements.txt

This file was deleted.

219 changes: 0 additions & 219 deletions simple-chat/static/app.js

This file was deleted.

Loading