Skip to content

Commit f49535d

Browse files
Kabuki94Kabuki94
committed
feat(ai): bake default model set into the image (qwen2.5-coder:7b + nomic-embed-text)
Embed the global default models at build time and define the override contract in the unified mios.toml dotfile. Tuned for the 12 GB system- RAM baseline: CPU-only inference with ~8 GB available to the model. Default model set (researched against the 8 GB resident envelope): - qwen2.5-coder:7b chat / code primary Q4_K_M ~4.7 GB on disk, ~5-6 GB resident Apache 2.0; 128K-token context HumanEval ~88%; trained heavily on bash, PowerShell, Containerfiles, systemd units, TOML Strong filesystem-path reasoning - nomic-embed-text embeddings (v1.5) ~270 MB Q4 GGUF; 768-dim; 8192-token context OpenAI /v1/embeddings-shaped via LocalAI automation/37-ollama-prep.sh -- rewritten as a real build-time baker (was a no-op stub from the prior commit). Pulls the model set into /usr/share/ollama/models on the immutable composefs surface (FHS- correct for "architecture-independent immutable data files"). The final /var cleanup at the end of the Containerfile RUN does NOT touch /usr/share, so the seed survives into the deployed image. - MIOS_OLLAMA_BAKE_MODELS=<csv> at build time overrides the default set; empty disables baking entirely (CI builds that just validate the pipeline can opt out). automation/build.sh -- 37-ollama-prep.sh removed from CONTAINERFILE_SCRIPTS so it runs as a regular pipeline step instead of being orphaned. usr/libexec/mios/ollama-firstboot.sh -- replaces the network-pull- only first-boot path with a two-layer flow: 1. Hardlink-copy the build-baked seed (/usr/share/ollama/models) into /var/lib/ollama/models (the writable runtime store ollama uses via OLLAMA_MODELS). Hardlinks keep on-disk usage to a single copy until ollama mutates a manifest. cp -al falls back to plain cp -a on cross-FS boundaries (composefs /usr -> ext4 /var). 2. Network pull only as a fallback for any model the seed did not include (e.g. operator overrides via /etc/mios/install.env). usr/share/containers/systemd/ollama.container -- two volumes: /usr/share/ollama:/usr/share/ollama:ro,Z immutable seed (RO) /var/lib/ollama:/var/lib/ollama:Z runtime store (RW) OLLAMA_MODELS=/var/lib/ollama/models so ollama itself reads/writes the runtime path. The seed is reachable to firstboot via the RO mount. usr/lib/tmpfiles.d/mios-ollama.conf (new) -- declares /var/lib/ollama and /var/lib/ollama/models with mios-ollama: mios-ollama ownership. Architectural Law 2 -- /var paths are never imperatively mkdir'd at build time. Unified user-overrides dotfile (usr/share/mios/mios.toml + env.defaults): the [ai] section grows four new keys with explicit semantics: bake_models -- comma-separated build-time bake list (-> MIOS_OLLAMA_BAKE_MODELS) ram_floor_gb -- recommended floor (informational; logged in postcheck so operators see what they signed up for) seed_dir -- /usr/share/ollama/models (immutable, build-baked) runtime_dir -- /var/lib/ollama/models (writable, first-boot-seeded) The [ai] section comment block enumerates the alternates considered (qwen2.5-coder:14b for 24 GB+ systems, llama3.2:3b for low-RAM / fast-response profiles) and their resource cost, so operators can swap deliberately rather than guessing. INDEX.md -- new section 2a documents the default set with disk / resident sizing, license, and the build-baked / first-boot-seeded flow so the model surface is discoverable next to the OpenAI API endpoint table.
1 parent 97dc135 commit f49535d

8 files changed

Lines changed: 288 additions & 51 deletions

File tree

‎INDEX.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,28 @@ the manifest at `/v1/mcp` is what MiOS agents read to populate that
3535
`tools` array. The `x-mios:` prefix is a documentation marker only --
3636
the served URL is `/v1/mcp`.
3737

38+
### 2a. Default model set
39+
40+
| Slot | Default | Quant | Disk | Resident | Notes |
41+
|---|---|---|---|---|---|
42+
| chat / code | `qwen2.5-coder:7b` | Q4_K_M | ~4.7 GB | ~5-6 GB | Apache 2.0; 128K context; HumanEval ~88%; multi-language including bash, PowerShell, Containerfiles, systemd units, TOML |
43+
| embeddings | `nomic-embed-text` (v1.5) | Q4 | ~270 MB | <1 GB | Apache 2.0; 768-dim; 8192-token context; OpenAI `/v1/embeddings`-shape via LocalAI |
44+
45+
Sized for the 12 GB system-RAM baseline (CPU-only inference, ~8 GB
46+
available to the model). Build-baked into
47+
`/usr/share/ollama/models` (immutable composefs surface) by
48+
`automation/37-ollama-prep.sh`; first-boot service
49+
(`mios-ollama-firstboot.service`) hardlink-copies the seed into
50+
`/var/lib/ollama/models` (the writable `OLLAMA_MODELS` path).
51+
52+
Override the build-time set with `MIOS_OLLAMA_BAKE_MODELS=<csv>` (e.g.
53+
`"qwen2.5-coder:14b,nomic-embed-text"` for 24 GB+ profiles). Override
54+
the runtime selection by editing `[ai].model` / `[ai].embed_model` in
55+
`/etc/mios/mios.toml` (host) or `~/.config/mios/mios.toml` (per-user)
56+
and restarting `mios-ollama.service`. The `[ai]` section in
57+
`usr/share/mios/mios.toml` documents the alternates considered and
58+
their resource cost.
59+
3860
## 3. Architectural laws (enforced; non-negotiable)
3961

4062
| # | Law | Enforced by |

‎automation/37-ollama-prep.sh‎

Lines changed: 96 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,103 @@
11
#!/bin/bash
2-
# 37-ollama-prep: DEPRECATED -- replaced by mios-ollama-firstboot.service.
2+
# 37-ollama-prep -- bake the default 'MiOS' model set into the image.
33
#
4-
# This script tried to embed the default LLM model set into
5-
# /var/lib/ollama at build time, but bootc's image-commit step runs a
6-
# /var cleanup that removes everything except /var/{tmp,cache} (per
7-
# Architectural Law 2 -- NO-MKDIR-IN-VAR; /var is a mutable host
8-
# surface, not part of the immutable composefs layer). The pulled
9-
# models therefore never survive into the deployed image.
4+
# Models land in /usr/share/ollama/models (immutable composefs surface,
5+
# FHS-correct for "architecture-independent immutable data"). The
6+
# build-time /var cleanup at the end of the Containerfile RUN does NOT
7+
# touch /usr/share, so the seed survives into the deployed image.
108
#
11-
# The work has moved to a runtime first-boot service that pulls the
12-
# default models AFTER the deploy, into the persistent /var/lib/ollama:
9+
# At first boot, mios-ollama-firstboot.service hardlink-copies the seed
10+
# into /var/lib/ollama/models (the writable runtime location ollama.
11+
# container reads/writes via OLLAMA_MODELS). Hardlinks keep the on-disk
12+
# footprint single-copy until ollama mutates a model file.
1313
#
14-
# usr/lib/systemd/system/mios-ollama-firstboot.service
15-
# usr/libexec/mios/ollama-firstboot.sh
14+
# Default model set (researched for the 12 GB RAM workstation baseline):
15+
# qwen2.5-coder:7b -- chat / code primary (~4.7 GB Q4_K_M)
16+
# best open-source coder in the 7B class as of
17+
# 2026; 128K context; Apache 2.0
18+
# nomic-embed-text -- embedding (~270 MB Q4_K_M)
19+
# 768-dim, 8192-token context, MTEB-competitive
1620
#
17-
# That service is sentinel-guarded (/var/lib/mios/.ollama-firstboot-
18-
# done) so it runs once per deploy, and the models survive
19-
# 'bootc upgrade' / 'bootc rollback' as ordinary /var state.
20-
#
21-
# The script is kept as an explicit no-op stub (rather than deleted)
22-
# so existing CONTAINERFILE_SCRIPTS references in automation/build.sh
23-
# don't break. Future cleanups can drop both this file and that
24-
# reference together.
21+
# Override the set with MIOS_OLLAMA_BAKE_MODELS=<csv> at build time --
22+
# e.g. MIOS_OLLAMA_BAKE_MODELS="qwen2.5-coder:14b,nomic-embed-text" for
23+
# the 24 GB+ profile. Empty value disables baking entirely (useful for
24+
# CI builds that only validate the pipeline).
2525
set -euo pipefail
26-
echo "[37-ollama-prep] deprecated -- model pull happens at first boot via mios-ollama-firstboot.service"
26+
27+
# shellcheck source=lib/common.sh
28+
source "$(dirname "$0")/lib/common.sh"
29+
30+
DEFAULT_MODELS="qwen2.5-coder:7b,nomic-embed-text"
31+
BAKE_MODELS="${MIOS_OLLAMA_BAKE_MODELS:-${DEFAULT_MODELS}}"
32+
BAKE_DIR="/usr/share/ollama/models"
33+
34+
if [[ -z "$BAKE_MODELS" ]]; then
35+
log "MIOS_OLLAMA_BAKE_MODELS empty -- skipping build-time model bake"
36+
exit 0
37+
fi
38+
39+
# Idempotency guard: if the seed dir already has manifests, assume a
40+
# previous bake step in this build already populated it.
41+
if [[ -d "${BAKE_DIR}/manifests" ]] && [[ -n "$(ls -A "${BAKE_DIR}/manifests" 2>/dev/null)" ]]; then
42+
log "Seed models already present at ${BAKE_DIR}; skipping re-bake"
43+
exit 0
44+
fi
45+
46+
# ollama is installed as part of packages-ai (see usr/share/mios/PACKAGES.md).
47+
# If it's not on PATH the build context didn't include the AI pack -- skip
48+
# rather than fail so unrelated CI builds still pass.
49+
if ! command -v ollama >/dev/null 2>&1; then
50+
log "ollama binary not found -- skipping bake (AI pack missing from this build)"
51+
exit 0
52+
fi
53+
54+
# Bake destination; OLLAMA_MODELS is the canonical override env var
55+
# the upstream binary respects.
56+
install -d -m 0755 "${BAKE_DIR}"
57+
export OLLAMA_MODELS="${BAKE_DIR}"
58+
59+
# Start a temporary ollama serve instance just for the duration of the
60+
# pull. Detach it ourselves so we keep stable control over its lifetime.
61+
log "Starting transient ollama serve (host=127.0.0.1:11434, store=${BAKE_DIR})"
62+
OLLAMA_HOST="127.0.0.1:11434" ollama serve >/tmp/ollama-bake.log 2>&1 &
63+
OLLAMA_PID=$!
64+
65+
# Wait up to 30 s for the API. If it never comes up, log + skip rather
66+
# than fail the build.
67+
for attempt in $(seq 1 15); do
68+
if scurl -sf "http://127.0.0.1:11434/api/tags" >/dev/null 2>&1; then
69+
break
70+
fi
71+
sleep 2
72+
done
73+
if ! scurl -sf "http://127.0.0.1:11434/api/tags" >/dev/null 2>&1; then
74+
log "WARN: ollama serve never became reachable -- skipping bake"
75+
kill "$OLLAMA_PID" 2>/dev/null || true
76+
wait "$OLLAMA_PID" 2>/dev/null || true
77+
exit 0
78+
fi
79+
80+
# Pull each requested model. A failure on one model is non-fatal (build
81+
# continues; first-boot service catches up).
82+
failures=0
83+
IFS=',' read -ra MODELS <<< "$BAKE_MODELS"
84+
for raw_model in "${MODELS[@]}"; do
85+
model="${raw_model// /}"
86+
[[ -z "$model" ]] && continue
87+
log "Pulling ${model} into ${BAKE_DIR}"
88+
if OLLAMA_HOST="127.0.0.1:11434" ollama pull "$model"; then
89+
log " [ok] ${model}"
90+
else
91+
log " [WARN] ${model} pull failed -- first-boot service will retry"
92+
failures=$((failures + 1))
93+
fi
94+
done
95+
96+
# Stop the transient serve cleanly.
97+
kill "$OLLAMA_PID" 2>/dev/null || true
98+
wait "$OLLAMA_PID" 2>/dev/null || true
99+
100+
# Report the on-disk size so build logs make the +N GB cost obvious.
101+
seed_size="$(du -sh "${BAKE_DIR}" 2>/dev/null | awk '{print $1}')"
102+
log "Bake complete -- ${BAKE_DIR} = ${seed_size:-?} (failures: ${failures})"
27103
exit 0

‎automation/build.sh‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -176,7 +176,14 @@ if [[ ! -f "$PACKAGES_MD" ]]; then
176176
fi
177177

178178
# ── Script classification ────────────────────────────────────────────────────
179-
CONTAINERFILE_SCRIPTS="08-system-files-overlay.sh 37-ollama-prep.sh 99-postcheck.sh"
179+
# CONTAINERFILE_SCRIPTS are explicitly skipped from the main loop because
180+
# they're invoked elsewhere in the build flow:
181+
# 08-system-files-overlay.sh -- called explicitly by the OCI overlay step
182+
# 99-postcheck.sh -- called explicitly below after the loop
183+
# 37-ollama-prep.sh used to live here but was orphaned (never called).
184+
# It is now a regular pipeline step that bakes the default model set
185+
# into /usr/share/ollama/models, so it runs in the main loop.
186+
CONTAINERFILE_SCRIPTS="08-system-files-overlay.sh 99-postcheck.sh"
180187

181188
NON_FATAL_SCRIPTS="
182189
05-enable-external-repos.sh

‎usr/lib/tmpfiles.d/mios-ollama.conf‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# /usr/lib/tmpfiles.d/mios-ollama.conf
2+
# Runtime state for the mios-ollama Quadlet.
3+
#
4+
# /var/lib/ollama is the writable model store ollama reads/writes via
5+
# OLLAMA_MODELS=/var/lib/ollama/models. mios-ollama-firstboot.service
6+
# seeds it at first boot from /usr/share/ollama/models (the build-time
7+
# bake on the immutable composefs surface).
8+
#
9+
# Architectural Law 2 (NO-MKDIR-IN-VAR): every /var path is declared
10+
# here, never imperatively mkdir'd at build time.
11+
12+
d /var/lib/ollama 0755 mios-ollama mios-ollama - -
13+
d /var/lib/ollama/models 0755 mios-ollama mios-ollama - -

‎usr/libexec/mios/ollama-firstboot.sh‎

Lines changed: 77 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,56 +1,104 @@
11
#!/usr/bin/env bash
22
# usr/libexec/mios/ollama-firstboot.sh
33
#
4-
# Pull the default 'MiOS' inference models on first boot. Run by
5-
# mios-ollama-firstboot.service after ollama.service is active. Models
6-
# land in /var/lib/ollama (mutable, persistent across bootc upgrades),
7-
# which is why this can't be done at build time -- the OCI build's
8-
# final /var cleanup would delete them.
4+
# First-boot model bootstrap for the mios-ollama Quadlet.
95
#
10-
# Sentinel-guarded: writes /var/lib/mios/.ollama-firstboot-done so a
11-
# re-fire (after a bootc rollback or service restart) is a no-op.
12-
# Re-pulling specific models on demand is still possible via
13-
# 'podman exec mios-ollama ollama pull <model>'.
6+
# Build-time path (preferred): automation/37-ollama-prep.sh has already
7+
# baked the default model set into /usr/share/ollama/models on the
8+
# immutable composefs surface. This script hardlink-copies that seed
9+
# into /var/lib/ollama/models on first boot so ollama can both read
10+
# AND write models there (the writable runtime store), without paying
11+
# a second on-disk copy until ollama itself mutates a manifest.
1412
#
15-
# Default model set tracks the OpenAI-API-shaped surface advertised in
16-
# usr/share/mios/ai/v1/models.json; keep them in sync.
13+
# Network fallback: if the seed dir is empty (e.g. CI built the image
14+
# with MIOS_OLLAMA_BAKE_MODELS= to keep image size down), pull the
15+
# user-configured model set via 'podman exec mios-ollama ollama pull'.
16+
# The model list is read from /etc/mios/install.env (MIOS_AI_MODEL,
17+
# MIOS_AI_EMBED_MODEL) so the build's globally-defaulted set carries
18+
# through to runtime, with operator overrides taking precedence.
19+
#
20+
# Sentinel: /var/lib/mios/.ollama-firstboot-done -- delete to force
21+
# a re-run.
1722
set -euo pipefail
1823
# shellcheck source=/usr/lib/mios/paths.sh
1924
source /usr/lib/mios/paths.sh 2>/dev/null || true
2025
: "${MIOS_VAR_DIR:=/var/lib/mios}"
2126

2227
SENTINEL="${MIOS_VAR_DIR}/.ollama-firstboot-done"
28+
SEED_DIR="/usr/share/ollama/models"
29+
RUNTIME_DIR="/var/lib/ollama/models"
2330
CONTAINER="mios-ollama"
24-
DEFAULT_MODELS=(
25-
"qwen2.5-coder:7b"
26-
"nomic-embed-text"
27-
)
2831

2932
_log() { logger -t mios-ollama-firstboot "$*" 2>/dev/null || true; echo "[ollama-firstboot] $*" >&2; }
3033

3134
if [[ -f "$SENTINEL" ]]; then
32-
_log "sentinel exists ($SENTINEL); models already pulled -- nothing to do"
35+
_log "sentinel exists ($SENTINEL); first-boot already done"
3336
exit 0
3437
fi
3538

36-
# Wait for the ollama container to be running. The service unit has
37-
# After=ollama.service so it should be up, but the container may need
38-
# a few seconds to finish initializing the model store on first start.
39+
# ── Layer 1: hardlink-copy the build-baked seed into the runtime dir ──
40+
# 'cp -al' tries hardlinks first and falls back to a regular copy when
41+
# the source/destination cross filesystems. Models then occupy a single
42+
# inode-level copy until ollama mutates one of the manifest files.
43+
if [[ -d "$SEED_DIR" ]] && [[ -n "$(ls -A "$SEED_DIR" 2>/dev/null)" ]]; then
44+
if [[ ! -d "$RUNTIME_DIR" ]] || [[ -z "$(ls -A "$RUNTIME_DIR" 2>/dev/null)" ]]; then
45+
_log "Seeding $RUNTIME_DIR from $SEED_DIR (hardlink-copy)"
46+
install -d -m 0755 "$RUNTIME_DIR"
47+
# Two-pass: try hardlink first, fall back to plain copy if it
48+
# crosses a filesystem boundary (likely between /usr composefs
49+
# and /var ext4 on bootc deployments).
50+
if cp -al "${SEED_DIR}/." "${RUNTIME_DIR}/" 2>/dev/null; then
51+
_log " [ok] hardlink-copy succeeded"
52+
else
53+
_log " [info] hardlink crossed FS boundary; falling back to cp -a"
54+
cp -a "${SEED_DIR}/." "${RUNTIME_DIR}/"
55+
fi
56+
chown -R mios-ollama:mios-ollama "$RUNTIME_DIR" 2>/dev/null || true
57+
else
58+
_log "Runtime dir $RUNTIME_DIR already populated; skipping seed copy"
59+
fi
60+
else
61+
_log "No build-baked seed at $SEED_DIR -- network pull fallback"
62+
fi
63+
64+
# ── Layer 2: ensure the configured models are present, pulling any gaps ──
65+
# Read the runtime model selection from install.env so operators can
66+
# override the build-baked default without rebuilding the image.
67+
DEFAULT_CHAT="qwen2.5-coder:7b"
68+
DEFAULT_EMBED="nomic-embed-text"
69+
MIOS_AI_MODEL="$DEFAULT_CHAT"
70+
MIOS_AI_EMBED_MODEL="$DEFAULT_EMBED"
71+
if [[ -f /etc/mios/install.env ]]; then
72+
# shellcheck disable=SC1091
73+
source /etc/mios/install.env 2>/dev/null || true
74+
fi
75+
76+
# Wait for the ollama container to be up. After=ollama.service in the
77+
# unit, but the model store may need a few seconds to settle on first
78+
# start.
3979
for attempt in 1 2 3 4 5 6 7 8 9 10; do
4080
if podman exec "$CONTAINER" ollama list >/dev/null 2>&1; then
4181
break
4282
fi
4383
_log "waiting for $CONTAINER (attempt $attempt/10)"
4484
sleep 5
4585
done
46-
4786
if ! podman exec "$CONTAINER" ollama list >/dev/null 2>&1; then
48-
_log "ERROR: $CONTAINER not reachable after 50s; aborting first-boot pull"
49-
exit 1
87+
_log "WARN: $CONTAINER not reachable after 50 s; skipping pull check"
88+
install -d -m 0755 "$(dirname "$SENTINEL")"
89+
touch "$SENTINEL"
90+
exit 0
5091
fi
5192

5293
failures=0
53-
for model in "${DEFAULT_MODELS[@]}"; do
94+
for model in "$MIOS_AI_MODEL" "$MIOS_AI_EMBED_MODEL"; do
95+
[[ -z "$model" ]] && continue
96+
# If 'ollama list' already shows it, skip the pull -- saves a long
97+
# network round-trip when the build-baked seed already supplied it.
98+
if podman exec "$CONTAINER" ollama list 2>/dev/null | awk 'NR>1{print $1}' | grep -qx "$model"; then
99+
_log " [present] $model"
100+
continue
101+
fi
54102
_log "pulling $model"
55103
if podman exec "$CONTAINER" ollama pull "$model" 2>&1 | logger -t mios-ollama-firstboot; then
56104
_log " [ok] $model"
@@ -60,16 +108,15 @@ for model in "${DEFAULT_MODELS[@]}"; do
60108
fi
61109
done
62110

63-
# Drop the sentinel even when some pulls failed -- we don't want every
64-
# boot to retry indefinitely. Operators can re-run by deleting the
65-
# sentinel: rm /var/lib/mios/.ollama-firstboot-done && systemctl start
66-
# mios-ollama-firstboot
111+
# Drop the sentinel even when some pulls fail -- avoid an indefinite
112+
# retry loop on every boot. Manual recovery: delete the sentinel and
113+
# restart mios-ollama-firstboot.service.
67114
install -d -m 0755 "$(dirname "$SENTINEL")"
68115
touch "$SENTINEL"
69116

70117
if (( failures > 0 )); then
71-
_log "first-boot pull complete with $failures failure(s)"
72-
exit 0 # non-fatal so the boot doesn't degrade
118+
_log "first-boot complete with $failures pull failure(s)"
119+
exit 0 # non-fatal; don't degrade the boot
73120
fi
74-
_log "first-boot pull complete -- all models present"
121+
_log "first-boot complete -- model set ready at $RUNTIME_DIR"
75122
exit 0

‎usr/share/containers/systemd/ollama.container‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,20 @@ Group=mios-ollama
1717
# 0.0.0.0 on the host side makes the surface contract explicit.
1818
Environment=OLLAMA_HOST=0.0.0.0:11434
1919
Environment=OLLAMA_ORIGINS=*
20+
# Point ollama at the writable runtime store. mios-ollama-firstboot.
21+
# service hardlink-copies the build-baked seed (/usr/share/ollama/
22+
# models -- immutable composefs surface) into this path on first boot.
23+
# Subsequent 'ollama pull' commands write here too, so this is the
24+
# single source of truth ollama reads.
25+
Environment=OLLAMA_MODELS=/var/lib/ollama/models
26+
27+
# Volumes (Architectural Law 2 -- /var paths declared via tmpfiles.d):
28+
# /var/lib/ollama writable runtime model store
29+
# /usr/share/ollama immutable seed mounted RO so the firstboot service
30+
# and ollama itself can both reach it
31+
Volume=/var/lib/ollama:/var/lib/ollama:Z
32+
Volume=/usr/share/ollama:/usr/share/ollama:ro,Z
33+
2034
PublishPort=0.0.0.0:11434:11434
2135

2236
# OCI labels for Podman Desktop discovery. ollama is an alternate local

‎usr/share/mios/env.defaults‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,11 +48,23 @@ MIOS_FORGE_HTTP_PORT="3000"
4848
MIOS_FORGE_SSH_PORT="2222"
4949

5050
# ── AI inference (LAW 5: localhost OpenAI-compatible only) ────────────────────
51+
# Default model set researched for the 12 GB system-RAM workstation
52+
# baseline (CPU-only inference, ~8 GB available to the model):
53+
# qwen2.5-coder:7b primary chat/code (Q4_K_M ~4.7 GB on disk)
54+
# nomic-embed-text embeddings (~270 MB)
55+
# Both are baked into /usr/share/ollama/models at build time by
56+
# automation/37-ollama-prep.sh (override with MIOS_OLLAMA_BAKE_MODELS).
57+
# See [ai] in /usr/share/mios/mios.toml for the canonical reference and
58+
# the override semantics.
5159
MIOS_AI_ENDPOINT="http://localhost:8080/v1"
5260
MIOS_AI_MODEL="qwen2.5-coder:7b"
5361
MIOS_AI_EMBED_MODEL="nomic-embed-text"
5462
MIOS_AI_KEY=""
5563
MIOS_AI_PORT="8080"
64+
MIOS_AI_BAKE_MODELS="qwen2.5-coder:7b,nomic-embed-text"
65+
MIOS_AI_RAM_FLOOR_GB="12"
66+
MIOS_AI_SEED_DIR="/usr/share/ollama/models"
67+
MIOS_AI_RUNTIME_DIR="/var/lib/ollama/models"
5668

5769
# ── Network ───────────────────────────────────────────────────────────────────
5870
MIOS_QUADLET_NETWORK="mios.network"

0 commit comments

Comments
 (0)