Skip to content

Commit d95234e

Browse files
committed
docs: add LibreChat on ZimaCube 2 blog post
1 parent 1359163 commit d95234e

9 files changed

Lines changed: 128 additions & 5 deletions

app/blog/page.tsx

Lines changed: 26 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
import Image from 'next/image'
22
import Link from 'next/link'
3+
import { Pin } from 'lucide-react'
34
import { blog } from '@/lib/source'
5+
import { cn } from '@/lib/utils'
46
import type { Metadata } from 'next'
57

68
export const metadata: Metadata = {
@@ -17,6 +19,7 @@ interface FeedEntry {
1719
author?: string
1820
ogImage?: string
1921
ogImagePosition?: string
22+
featured?: boolean
2023
}
2124

2225
const dateFormatter = new Intl.DateTimeFormat('en-US', {
@@ -45,10 +48,14 @@ export default function BlogPage() {
4548
author: (post as any).author,
4649
ogImage: post.ogImage,
4750
ogImagePosition: (post as any).ogImagePosition,
51+
featured: (post as any).featured,
4852
})
4953
}
5054

51-
entries.sort((a, b) => new Date(b.date).getTime() - new Date(a.date).getTime())
55+
entries.sort((a, b) => {
56+
if (!!b.featured !== !!a.featured) return b.featured ? 1 : -1
57+
return new Date(b.date).getTime() - new Date(a.date).getTime()
58+
})
5259

5360
return (
5461
<div>
@@ -61,18 +68,32 @@ export default function BlogPage() {
6168

6269
<div className="grid grid-cols-1 gap-6 md:grid-cols-2">
6370
{entries.map((entry) => (
64-
<article key={entry.url}>
71+
<article key={entry.url} className={cn(entry.featured && 'md:col-span-2')}>
6572
<Link
6673
href={entry.url}
67-
className="group flex h-full flex-col rounded-lg border border-border overflow-hidden transition-colors hover:bg-muted"
74+
className={cn(
75+
'group flex h-full flex-col rounded-lg border border-border overflow-hidden transition-colors hover:bg-muted',
76+
entry.featured && 'border-primary/40 ring-1 ring-primary/20',
77+
)}
6878
>
69-
<div className="relative h-48 w-full overflow-hidden bg-muted">
79+
<div
80+
className={cn(
81+
'relative w-full overflow-hidden bg-muted',
82+
entry.featured ? 'h-56 md:h-72' : 'h-48',
83+
)}
84+
>
85+
{entry.featured && (
86+
<span className="absolute left-3 top-3 z-10 inline-flex items-center gap-1.5 rounded-full bg-primary px-2.5 py-1 text-xs font-medium text-primary-foreground shadow-sm">
87+
<Pin className="size-3" aria-hidden="true" />
88+
Pinned
89+
</span>
90+
)}
7091
{entry.ogImage ? (
7192
<Image
7293
src={entry.ogImage}
7394
alt={entry.title}
7495
fill
75-
sizes="(max-width: 768px) 100vw, 50vw"
96+
sizes={entry.featured ? '100vw' : '(max-width: 768px) 100vw, 50vw'}
7697
className="object-cover transition-transform group-hover:scale-105"
7798
style={
7899
entry.ogImagePosition ? { objectPosition: entry.ogImagePosition } : undefined

content/blog/2026-02-18_2026_roadmap.mdx

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ ogImage: /images/blog/2026-02-18_2026_roadmap.png
1010
ogMetaImage: /images/blog/2026-02-18_2026_roadmap_og.png
1111
ogImagePosition: center 10%
1212
author: danny
13+
featured: true
1314
---
1415

1516
import Discord from '@/components/icons/discord'
Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
---
2+
title: 'Running LibreChat on the ZimaCube 2'
3+
date: 2026-06-03
4+
description: "The first ZimaCube ran the LibreChat demo's stateful backend for three years. When the ZimaCube 2 arrived, I tested the hardware that had given the old one trouble, then rebuilt the stack as a home AI hub."
5+
tags:
6+
- zimacube
7+
- zimaos
8+
- self-hosted
9+
- docker
10+
- guide
11+
author: berry
12+
ogImage: /images/blog/2026-06-03_librechat_zimacube.png
13+
category: guide
14+
---
15+
16+
I run the infrastructure behind LibreChat's public demo at [chat.librechat.ai](https://chat.librechat.ai). For roughly three years, a large part of that backend ran on a ZimaCube, the first-generation unit. When IceWhale offered to send the ZimaCube 2, I had two questions. The first was whether they had fixed the single hardware problem the original gave me. The second was whether the things I had come to rely on, the operating system in particular, were still there.
17+
18+
## Three years on the first ZimaCube
19+
20+
The demo is a production deployment. It has served more than 100,000 users to date, and the original ZimaCube held its stateful tier the entire time: MongoDB, Meilisearch, the RAG API, and Redis.
21+
22+
Those four services make a useful stress profile, because each one loads a different subsystem. MongoDB drove the NVMe, where its working set and journal writes dominated disk I/O. Meilisearch saturated CPU cores whenever it rebuilt an index. The RAG API ran embeddings on the GPU. Redis stayed resident in RAM. Together they kept disk, CPU, GPU, and memory under more or less constant pressure.
23+
24+
It was never a dedicated machine, either. That first Cube was a considerably more powerful configuration than the 2, and alongside the demo backend it ran around 130 other containers for my own projects. It carried that workload for three years with one exception.
25+
26+
The exception was thermal. Under sustained high traffic the NVMe drives ran warm, climbing toward their throttle threshold without ever crossing it. I never fully isolated the cause. My working theory is that the seventh bay sits on PCIe lanes that were not specified for three or four NVMe drives under sustained write load, though I cannot rule out something in how I had the drives arranged.
27+
28+
## What changed on the ZimaCube 2
29+
30+
Before installing anything, I moved most of those drives into the 2 and reran the write-heavy load that used to heat them.
31+
32+
This time they stayed well within their normal range. I cannot attribute that to a single cause from the outside, whether a board revision, a different lane allocation, or better drive placement. What is visible is physical: the 2 has additional vents along the lower side panels that the first generation did not have. More airflow across the drive bays is consistent with the temperatures I recorded.
33+
34+
![ZimaOS storage view, showing the NVMe drives I moved over from the first Cube](/images/blog/2026-06-03_librechat_zimacube_storage.png)
35+
36+
That result was enough to commit to the rest of the build.
37+
38+
## The operating system
39+
40+
With the thermal question settled, ZimaOS was the part that stood out.
41+
42+
It is built as an appliance, and the design follows from that. The root filesystem is a read-only A/B image with atomic updates, so a failed update rolls back to the previous image rather than leaving the system half-written, and user data is never part of that image. Everything persistent lives on `/DATA`, on a separate volume. Administration runs through the web dashboard: installing, starting, updating, and configuring apps. On a machine the rest of the household depends on, not needing shell access for routine work is a meaningful operational property.
43+
44+
![The ZimaOS desktop: app launcher and live system stats, with Frigate and LibreChat running side by side](/images/blog/2026-06-03_librechat_zimacube_zimaos.png)
45+
46+
The detail that mattered most to me is how little the OS consumes on its own. Its idle footprint is small enough to be negligible against the workloads, which on an 8 GB system is the margin between running comfortably and swapping.
47+
48+
## Deploying the stack
49+
50+
If you run LibreChat somewhere other than ZimaOS, only the next paragraph is platform-specific. Everything after it applies to any deployment, whether a VPS, a NUC, or a spare machine.
51+
52+
On ZimaOS, I imported a pinned Compose file through the dashboard: LibreChat v0.8.6, MongoDB 8.0, and Meilisearch 1.35. The install runs asynchronously, and the dashboard pulls and verifies each image before the app transitions to `running`. That took about four minutes here with no visible progress in between, which is expected rather than a hang.
53+
54+
![Configuring the LibreChat app in the ZimaOS dashboard: Docker image, ports, and /DATA volume mounts](/images/blog/2026-06-03_librechat_zimacube_install.png)
55+
56+
From there it is standard LibreChat. The first launch presents a registration page, and each member of the household creates an account with separate history, presets, and model selection.
57+
58+
![LibreChat registration page served from the Cube](/images/blog/2026-06-03_librechat_zimacube_register.png)
59+
60+
Provider keys go either into the stack's environment or per user under Settings. The model picker then exposes whatever those keys grant access to: OpenAI, Anthropic, Google, and the rest.
61+
62+
![Model picker showing OpenAI, Anthropic and Google endpoints](/images/blog/2026-06-03_librechat_zimacube_picker.png)
63+
64+
That registration-to-picker sequence is the same regardless of where the stack runs. The Compose import is the only ZimaOS-specific step.
65+
66+
I left the RAG stack out of this build, meaning the vector database and retrieval API that drove the GPU on the old unit. On an 8 GB box already running an NVR, that is a deliberate trade-off. On a 16 GB system I would include it.
67+
68+
## Memory footprint
69+
70+
Idle resident memory per container, read from each container's cgroup memory accounting rather than from `docker stats`:
71+
72+
| Container | Idle RSS |
73+
| -------------------- | ----------- |
74+
| LibreChat API (Node) | ~364 MB |
75+
| MongoDB | ~295 MB |
76+
| Meilisearch | ~97 MB |
77+
| **Stack total** | **~756 MB** |
78+
79+
That is under a gigabyte for the chat UI, the database, and full-text search across the entire conversation history. The same unit also runs Frigate as an NVR, which holds 3 to 4 GB for real-time object detection across two Coral TPUs, so the two workloads share the machine directly. I set per-service memory limits in Compose so that neither can starve the other. Cold start from stopped to healthy is roughly four seconds with images cached.
80+
81+
## Set your own secrets
82+
83+
LibreChat encrypts stored user credentials with keys taken from its environment, and those keys have to be your own, not the template defaults.
84+
85+
<Callout type="warning" title="Before first login">
86+
Set your own `JWT_SECRET`, `JWT_REFRESH_SECRET`, `CREDS_KEY`, and `CREDS_IV` before any real
87+
account is created. The [credentials generator](/docs/toolkit/credentials-generator) produces all
88+
four at the correct lengths. The template values exist only to get a first boot running.
89+
</Callout>
90+
91+
If you later expose the instance outside your network, put it behind a reverse proxy or a tunnel. With the secrets already replaced, that is purely a networking change.
92+
93+
## Verdict
94+
95+
A home AI hub puts limited demands on the hardware and the OS underneath it. The box needs to run reliably under continuous load and recover from a bad update without losing data, and it needs to leave the services enough memory to work in. The ZimaCube 2 did that. On the one issue I had with the first unit, the drive temperatures under sustained load, it measured better.
96+
97+
IceWhale provided the unit. The measurements in this post are my own, taken on the device.
98+
99+
---
100+
101+
_ZimaCube 2 · ZimaOS v1.6.1 · i3-1215U (6 cores, 8 threads) · 8 GB RAM · LibreChat v0.8.6. Screenshots are from a throwaway account._
17.4 KB
Loading
362 KB
Loading
51.2 KB
Loading
24.2 KB
Loading
293 KB
Loading
418 KB
Loading

0 commit comments

Comments
 (0)