Skip to content

Commit f15b917

Browse files
localai-botmudler
andauthored
feat(usage): track and visualise usage per API key (#9920)
* feat(usage): add Source, APIKeyID, APIKeyName columns to UsageRecord Adds three additive columns plus UsageSource* constants. The columns are auto-migrated by InitDB. APIKeyID is a nullable foreign reference to UserAPIKey.ID; APIKeyName is snapshotted on each row so revoked keys keep showing their name in history. Refs: #9862 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(usage): backfill Source on pre-feature usage rows InitDB now classifies any pre-existing usage_record with an empty source: 'legacy-api-key' user -> legacy, everything else -> web. The backfill is idempotent (only touches NULL/empty rows). Refs: #9862 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(usage): add GetUserUsageBySource aggregator Groups by (bucket, source, api_key_id, api_key_name). Filters out legacy by default. Returns both per-bucket detail and roll-ups (by_source, by_key sorted desc and capped at 200, grand_total). The MAX(created_at) projection is iterated via Rows().Scan into a string column and parsed manually because the SQLite driver surfaces the aggregated timestamp as a string, which database/sql refuses to scan directly into time.Time. Postgres returns a real timestamp; the same string path handles its RFC3339 form too. Refs: #9862 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(usage): log Rows() errors and assert LastUsed in tests Adds rows.Err() and Rows() open-failure logging in computeSourceTotals so silent data drops surface in logs. Logs on parseLastUsedString format misses for the same reason. Strengthens the snapshot-survival test to assert LastUsed is a recent timestamp, locking the SQLite time-string parser behaviour. Refs: #9862 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(usage): add admin GetAllUsageBySource with filters and truncation Optional user_id and api_key_id filters (composed with AND). Legacy bucket is included for admin callers. truncated=true when more than 200 distinct keys would be in the by_key roll-up. Refs: #9862 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(auth): plumb auth_source and auth_apikey through Echo context tryAuthenticate now sets auth_source on every successful branch (web for session/Bearer-session, apikey for Bearer-key/x-api-key/ token-cookie, legacy for legacy env key match). For named-key branches it also stores the resolved *UserAPIKey under auth_apikey so downstream middlewares can snapshot id+name without re-validating. Refs: #9862 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(auth): expand tryAuthenticate godoc and cover Bearer-session branch Documents all three context-keys side effects (auth_source, auth_apikey, _auth_session) plus the split of responsibilities with the parent Middleware. Adds a test for the Bearer-as-session-token classification so future regressions there fail loudly. Refs: #9862 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(usage): UsageMiddleware records source + snapshots key name Reads auth_source and auth_apikey from the Echo context (set by auth.Middleware in the previous task). Snapshots UserAPIKey.ID and Name onto each row so revoked keys remain readable in history. Falls back to source=web when no auth_source is set (auth disabled or unrecognised path). Refs: #9862 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(usage): add /api/auth/usage/sources and admin variant Self endpoint filters legacy server-side; admin endpoint includes legacy and accepts user_id + api_key_id filters. Response includes buckets, totals.{by_source, by_key, grand_total}, and a truncated flag set when the per-key roll-up was capped at 200. Refs: #9862 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(routes): mark test mirror handlers as keep-in-sync with production The newTestAuthApp helper duplicates production route handlers inline because it cannot use RegisterAuthRoutes (which requires a *application.Application). Naming the source path on each mirror makes the drift contract explicit for future maintainers. Refs: #9862 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): add usageApi.getMySources/getAdminSources + i18n strings Refs: #9862 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): add Sources tab skeleton with data fetch Adds Usage page tab that fetches /api/auth/usage/sources (or the admin variant). Renders raw totals plus a placeholder key list; real visualisations land in subsequent commits. Restructures the existing tab button block so Models and Sources are visible to non-admins (Users remains admin-only). Refs: #9862 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): source mix ribbon + searchable/sortable sources table Replaces the SourcesTab placeholder rendering with two reusable components: SourceMixRibbon (one segmented bar per source class) and SourcesTable (search + sort + revoked-key dim). Pulls the current API key list to detect revoked keys. Refs: #9862 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): skip revoked-key detection until the key list is known existingKeyIds defaulted to an empty Set, which made every live api_key row render as (revoked) during the brief window before apiKeysApi.list() resolved, and permanently after a fetch failure. Use null as the unknown state and suppress the revoked badge until the parent provides a real Set. Refs: #9862 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): top-N stacked time chart and drill-in chip for Sources tab Top 7 sources by total tokens get distinct colours; the rest roll up into 'Other'. Clicking a row in the SourcesTable dims everything except that series in the chart; the chip is the canonical clear. Refs: #9862 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(usage): document per-API-key Sources tab and endpoints Extends features/authentication.md Usage Tracking section with: - A 'Sources' tab description and source-class taxonomy - Endpoint documentation for /api/auth/usage/sources and the admin variant - Response shape example with by_source / by_key / grand_total - Migration note about pre-feature row backfill Refs: #9862 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(usage): silence errcheck on deferred rows.Close CI errcheck flagged the bare 'defer rows.Close()' in computeSourceTotals. Wrap in a closure that discards the close error explicitly; an error here is non-actionable since we have already drained the rows and logged any iteration failure. Refs: #9862 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(usage): bound batcher intake and add Shutdown/FlushNow hooks The pre-existing usage batcher had no cap on its add() path; the usageMaxPending=5000 constant only guarded the re-queue path after a failed write, leaving memory growth unbounded if the DB fell behind. This commit: - Adds the cap to add() so saturation drops new records (rate-limited warn at 1/1024) instead of growing unbounded. - Raises usageMaxPending to 50000 to absorb realistic inference bursts. - Replaces the package-level batcher global with a mutex-guarded pair plus a currentBatcher() accessor so Init / Shutdown cycles are race-free. - Adds ShutdownUsageRecorder() for graceful drain on process exit (not yet wired into app shutdown, just published). - Adds FlushNow() for deterministic tests; the middleware suite no longer needs 6s sleeps per spec and now runs in ~50ms instead of 18s. - Re-queue on failed flush is now cap-aware: prepends as much of the failed batch as fits alongside concurrent arrivals, instead of dropping the whole batch when full. Refs: #9862 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(usage): drain usage batcher on graceful shutdown Registers ShutdownUsageRecorder with the existing signals.RegisterGracefulTerminationHandler so SIGINT/SIGTERM synchronously flushes any in-memory usage records before the process exits. Without this, up to one flush interval (5s) of recorded usage was lost when LocalAI restarted. Refs: #9862 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
1 parent 959de86 commit f15b917

18 files changed

Lines changed: 1822 additions & 56 deletions

File tree

core/http/app.go

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ import (
2828
"github.com/mudler/LocalAI/core/services/monitoring"
2929
"github.com/mudler/LocalAI/core/services/nodes"
3030
"github.com/mudler/LocalAI/core/services/quantization"
31+
"github.com/mudler/LocalAI/pkg/signals"
3132

3233
"github.com/mudler/xlog"
3334
)
@@ -267,9 +268,12 @@ func API(application *application.Application) (*echo.Echo, error) {
267268
e.Static("/generated-videos", videoPath)
268269
}
269270

270-
// Initialize usage recording when auth DB is available
271+
// Initialize usage recording when auth DB is available, and ensure the
272+
// batcher drains its in-memory queue on graceful shutdown so the last
273+
// few seconds of usage don't disappear when the process exits.
271274
if application.AuthDB() != nil {
272275
httpMiddleware.InitUsageRecorder(application.AuthDB())
276+
signals.RegisterGracefulTerminationHandler(httpMiddleware.ShutdownUsageRecorder)
273277
}
274278

275279
// Auth is applied to _all_ endpoints. Filtering out endpoints to bypass is

core/http/auth/db.go

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,9 +38,15 @@ func InitDB(databaseURL string) (*gorm.DB, error) {
3838
}
3939

4040
// Backfill: users created before the provider column existed have an empty
41-
// provider treat them as local accounts so the UI can identify them.
41+
// provider - treat them as local accounts so the UI can identify them.
4242
db.Exec("UPDATE users SET provider = ? WHERE provider = '' OR provider IS NULL", ProviderLocal)
4343

44+
// Backfill: pre-feature usage_records have no source column. Classify them so the
45+
// new per-source aggregators include them.
46+
if err := BackfillUsageSource(db); err != nil {
47+
return nil, fmt.Errorf("failed to backfill usage source: %w", err)
48+
}
49+
4450
// Create composite index on users(provider, subject) for fast OAuth lookups
4551
if err := db.Exec("CREATE INDEX IF NOT EXISTS idx_users_provider_subject ON users(provider, subject)").Error; err != nil {
4652
// Ignore error on postgres if index already exists

core/http/auth/middleware.go

Lines changed: 48 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,10 @@ import (
1616
)
1717

1818
const (
19-
contextKeyUser = "auth_user"
20-
contextKeyRole = "auth_role"
19+
contextKeyUser = "auth_user"
20+
contextKeyRole = "auth_role"
21+
contextKeyAPIKey = "auth_apikey"
22+
contextKeySource = "auth_source"
2123
)
2224

2325
// Middleware returns an Echo middleware that handles authentication.
@@ -75,6 +77,7 @@ func Middleware(db *gorm.DB, appConfig *config.ApplicationConfig) echo.Middlewar
7577
}
7678
c.Set(contextKeyUser, syntheticUser)
7779
c.Set(contextKeyRole, RoleAdmin)
80+
c.Set(contextKeySource, UsageSourceLegacy)
7881
authenticated = true
7982
}
8083
}
@@ -213,6 +216,20 @@ func GetUserRole(c echo.Context) string {
213216
return role
214217
}
215218

219+
// GetAPIKey returns the resolved API key from the echo context, or nil.
220+
// Nil for session-cookie and legacy-env-key authentication.
221+
func GetAPIKey(c echo.Context) *UserAPIKey {
222+
k, _ := c.Get(contextKeyAPIKey).(*UserAPIKey)
223+
return k
224+
}
225+
226+
// GetSource returns the request's authentication source: UsageSourceAPIKey,
227+
// UsageSourceWeb, UsageSourceLegacy, or empty if no authentication was performed.
228+
func GetSource(c echo.Context) string {
229+
s, _ := c.Get(contextKeySource).(string)
230+
return s
231+
}
232+
216233
// RequireRouteFeature returns a global middleware that checks the user has access
217234
// to the feature required by the matched route. It uses the RouteFeatureRegistry
218235
// to look up the required feature for each route pattern + HTTP method.
@@ -421,47 +438,67 @@ func RequireQuota(db *gorm.DB) echo.MiddlewareFunc {
421438
}
422439

423440
// tryAuthenticate attempts to authenticate the request using the database.
441+
//
442+
// On success it returns the user and, as a side effect, sets the following
443+
// values on the Echo context:
444+
// - contextKeySource ("auth_source"): always set, one of UsageSourceWeb /
445+
// UsageSourceAPIKey. UsageSourceLegacy is set elsewhere by the parent
446+
// Middleware when a legacy env key matches.
447+
// - contextKeyAPIKey ("auth_apikey"): set to the resolved *UserAPIKey for
448+
// named-key branches (Bearer, x-api-key, xi-api-key, token cookie).
449+
// - "_auth_session": session record, used by Middleware to drive cookie
450+
// rotation. Only set on the session-cookie branch.
451+
//
452+
// contextKeyUser and contextKeyRole are populated by the parent Middleware
453+
// after this function returns.
424454
func tryAuthenticate(c echo.Context, db *gorm.DB, appConfig *config.ApplicationConfig) *User {
425455
hmacSecret := appConfig.Auth.APIKeyHMACSecret
426456

427-
// a. Session cookie
457+
// a. Session cookie -> web UI
428458
if cookie, err := c.Cookie(sessionCookie); err == nil && cookie.Value != "" {
429459
if user, session := ValidateSession(db, cookie.Value, hmacSecret); user != nil {
430460
// Store session for rotation check in middleware
431461
c.Set("_auth_session", session)
462+
c.Set(contextKeySource, UsageSourceWeb)
432463
return user
433464
}
434465
}
435466

436-
// b. Authorization: Bearer token
467+
// b. Authorization: Bearer
437468
authHeader := c.Request().Header.Get("Authorization")
438469
if strings.HasPrefix(authHeader, "Bearer ") {
439470
token := strings.TrimPrefix(authHeader, "Bearer ")
440471

441-
// Try as session ID first
472+
// b1. Session token via Bearer -> still web UI
442473
if user, _ := ValidateSession(db, token, hmacSecret); user != nil {
474+
c.Set(contextKeySource, UsageSourceWeb)
443475
return user
444476
}
445477

446-
// Try as user API key
478+
// b2. Named API key
447479
if key, err := ValidateAPIKey(db, token, hmacSecret); err == nil {
480+
c.Set(contextKeySource, UsageSourceAPIKey)
481+
c.Set(contextKeyAPIKey, key)
448482
return &key.User
449483
}
450484
}
451485

452-
// c. x-api-key / xi-api-key headers
486+
// c. x-api-key / xi-api-key -> named API key
453487
for _, header := range []string{"x-api-key", "xi-api-key"} {
454-
if key := c.Request().Header.Get(header); key != "" {
455-
if apiKey, err := ValidateAPIKey(db, key, hmacSecret); err == nil {
488+
if k := c.Request().Header.Get(header); k != "" {
489+
if apiKey, err := ValidateAPIKey(db, k, hmacSecret); err == nil {
490+
c.Set(contextKeySource, UsageSourceAPIKey)
491+
c.Set(contextKeyAPIKey, apiKey)
456492
return &apiKey.User
457493
}
458494
}
459495
}
460496

461-
// d. token cookie (legacy)
497+
// d. token cookie -> named API key
462498
if cookie, err := c.Cookie("token"); err == nil && cookie.Value != "" {
463-
// Try as user API key
464499
if key, err := ValidateAPIKey(db, cookie.Value, hmacSecret); err == nil {
500+
c.Set(contextKeySource, UsageSourceAPIKey)
501+
c.Set(contextKeyAPIKey, key)
465502
return &key.User
466503
}
467504
}

core/http/auth/middleware_test.go

Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -303,4 +303,122 @@ var _ = Describe("Auth Middleware", func() {
303303
}
304304
})
305305
})
306+
307+
Describe("auth context plumbing for usage source", func() {
308+
// probeApp builds a minimal echo app with the auth middleware and a single
309+
// "/probe" route that captures the user, source, and apikey from context.
310+
type probe struct {
311+
user *auth.User
312+
source string
313+
key *auth.UserAPIKey
314+
}
315+
probeApp := func(db *gorm.DB, appConfig *config.ApplicationConfig, p *probe) *echo.Echo {
316+
e := echo.New()
317+
e.Use(auth.Middleware(db, appConfig))
318+
e.GET("/probe", func(c echo.Context) error {
319+
p.user = auth.GetUser(c)
320+
p.source = auth.GetSource(c)
321+
p.key = auth.GetAPIKey(c)
322+
return c.NoContent(http.StatusOK)
323+
})
324+
return e
325+
}
326+
327+
It("session cookie sets source=web, apikey=nil", func() {
328+
db := testDB()
329+
appConfig := config.NewApplicationConfig()
330+
user := createTestUser(db, "alice@example.com", auth.RoleUser, auth.ProviderLocal)
331+
token := createTestSession(db, user.ID)
332+
333+
var p probe
334+
app := probeApp(db, appConfig, &p)
335+
rec := doRequest(app, http.MethodGet, "/probe", withSessionCookie(token))
336+
337+
Expect(rec.Code).To(Equal(http.StatusOK))
338+
Expect(p.user).ToNot(BeNil())
339+
Expect(p.user.ID).To(Equal(user.ID))
340+
Expect(p.source).To(Equal(auth.UsageSourceWeb))
341+
Expect(p.key).To(BeNil())
342+
})
343+
344+
It("Bearer session token sets source=web, apikey=nil", func() {
345+
db := testDB()
346+
appConfig := config.NewApplicationConfig()
347+
user := createTestUser(db, "alice@example.com", auth.RoleUser, auth.ProviderLocal)
348+
token := createTestSession(db, user.ID)
349+
350+
var p probe
351+
app := probeApp(db, appConfig, &p)
352+
rec := doRequest(app, http.MethodGet, "/probe", withBearerToken(token))
353+
354+
Expect(rec.Code).To(Equal(http.StatusOK))
355+
Expect(p.user).ToNot(BeNil())
356+
Expect(p.user.ID).To(Equal(user.ID))
357+
Expect(p.source).To(Equal(auth.UsageSourceWeb))
358+
Expect(p.key).To(BeNil())
359+
})
360+
361+
It("Bearer API key sets source=apikey and exposes the resolved *UserAPIKey", func() {
362+
db := testDB()
363+
appConfig := config.NewApplicationConfig()
364+
user := createTestUser(db, "alice@example.com", auth.RoleUser, auth.ProviderLocal)
365+
plaintext, key, err := auth.CreateAPIKey(db, user.ID, "ci", auth.RoleUser, appConfig.Auth.APIKeyHMACSecret, nil)
366+
Expect(err).ToNot(HaveOccurred())
367+
368+
var p probe
369+
app := probeApp(db, appConfig, &p)
370+
rec := doRequest(app, http.MethodGet, "/probe", withBearerToken(plaintext))
371+
372+
Expect(rec.Code).To(Equal(http.StatusOK))
373+
Expect(p.source).To(Equal(auth.UsageSourceAPIKey))
374+
Expect(p.key).ToNot(BeNil())
375+
Expect(p.key.ID).To(Equal(key.ID))
376+
})
377+
378+
It("x-api-key header sets source=apikey", func() {
379+
db := testDB()
380+
appConfig := config.NewApplicationConfig()
381+
user := createTestUser(db, "alice@example.com", auth.RoleUser, auth.ProviderLocal)
382+
plaintext, _, err := auth.CreateAPIKey(db, user.ID, "ci", auth.RoleUser, appConfig.Auth.APIKeyHMACSecret, nil)
383+
Expect(err).ToNot(HaveOccurred())
384+
385+
var p probe
386+
app := probeApp(db, appConfig, &p)
387+
rec := doRequest(app, http.MethodGet, "/probe", withXApiKey(plaintext))
388+
389+
Expect(rec.Code).To(Equal(http.StatusOK))
390+
Expect(p.source).To(Equal(auth.UsageSourceAPIKey))
391+
Expect(p.key).ToNot(BeNil())
392+
})
393+
394+
It("token cookie sets source=apikey", func() {
395+
db := testDB()
396+
appConfig := config.NewApplicationConfig()
397+
user := createTestUser(db, "alice@example.com", auth.RoleUser, auth.ProviderLocal)
398+
plaintext, _, err := auth.CreateAPIKey(db, user.ID, "ci", auth.RoleUser, appConfig.Auth.APIKeyHMACSecret, nil)
399+
Expect(err).ToNot(HaveOccurred())
400+
401+
var p probe
402+
app := probeApp(db, appConfig, &p)
403+
rec := doRequest(app, http.MethodGet, "/probe", withTokenCookie(plaintext))
404+
405+
Expect(rec.Code).To(Equal(http.StatusOK))
406+
Expect(p.source).To(Equal(auth.UsageSourceAPIKey))
407+
Expect(p.key).ToNot(BeNil())
408+
})
409+
410+
It("legacy env key sets source=legacy, apikey=nil", func() {
411+
db := testDB()
412+
appConfig := config.NewApplicationConfig()
413+
appConfig.ApiKeys = []string{"legacy-secret"}
414+
415+
var p probe
416+
app := probeApp(db, appConfig, &p)
417+
rec := doRequest(app, http.MethodGet, "/probe", withBearerToken("legacy-secret"))
418+
419+
Expect(rec.Code).To(Equal(http.StatusOK))
420+
Expect(p.source).To(Equal(auth.UsageSourceLegacy))
421+
Expect(p.key).To(BeNil())
422+
})
423+
})
306424
})

0 commit comments

Comments
 (0)