Gen UI — Generative User Interfaces with Multi-Agent AI

AI doesn't just answer — it builds the UI.

Gen UI is an experimental system where LLMs generate fully interactive Angular component trees in real-time, instead of returning plain text or markdown. Describe any interface — a dashboard, portfolio, survey, pricing page — and watch it materialise in seconds.

Inspired by Google Research's Generative UI paper (Nov 2025), which demonstrated that LLMs can create immersive visual experiences and interactive tools on the fly for any prompt.

Live Demo

How It Works

User prompt
    │
    ▼
┌──────────────┐     ┌──────────────┐     ┌─────────────┐
│ Ethics Gate   │ ──▶ │ Router Agent │ ──▶ │ Web Search  │ (conditional)
│  (safety)     │     │ (planning)   │     │ (enrichment)│
└──────────────┘     └──────────────┘     └─────────────┘
                            │
                            ▼
                     ┌──────────────┐     ┌─────────────┐
                     │  Summarizer  │ ──▶ │ UX Designer │ (conditional)
                     │(compression) │     │ (structure)  │
                     └──────────────┘     └─────────────┘
                            │
                            ▼
                     ┌──────────────┐
                     │ AI Provider  │  ← streams UI schema chunks
                     │ (generation) │
                     └──────┬───────┘
                            │
                     ┌──────▼──────────────────┐
                     │  Validate + Repair Loop  │
                     │  Validator ──valid──▶ ✅  │
                     │     │ invalid             │
                     │  Repair (up to 2 rounds)  │
                     └─────────────────────────┘
                            │
                            ▼
                   Angular component tree
                   rendered in the browser

The backend orchestrates 7 specialised agents in a pipeline. The output is a JSON UI schema that the frontend's dynamic renderer turns into real Angular components — cards, tables, charts, forms, wizards — with zero client-side code generation.

Agent Architecture

#	Agent	Role
0	Ethics Gate	Safety pre-filter — blocks illegal, unsafe, and prompt-injection requests using pattern matching + optional LLM review
1	Router Agent	Decides generation mode (patch vs replace), model tier (fast / balanced / quality), and whether to activate UX planning or web search
2	Web Search	Enriches prompts with real-time information via OpenAI web search (conditional, keyword-triggered)
3	Summarizer	Compresses conversation history — keeps last 8 messages verbatim, truncates/summarises older context, builds a UI state digest
4	UX Designer	Structural planner — recommends layout, sections, interaction model, density, icon suggestions. Does not produce the final schema
5	AI Provider	Streams the UI schema using the resolved provider + model (see supported providers below)
6	Validator	Multi-layer validation: manifest compliance, prop type checks, density heuristics, interaction policy enforcement, icon/emoji rules
7	Repair Agent	Two-pass repair: (1) deterministic sanitiser via manifest (free), (2) LLM-based repair with tier escalation. Up to 2 rounds
–	Copy Agent	Optional microcopy polisher — compact labels (≤ 3 words) and helper text (≤ 10 words)

Every agent follows a deterministic-first design: fast pattern matching runs before any LLM call, minimising latency and cost.

AI Providers

Provider	Notes
Google Gemini	Default provider; OpenAI-compatible endpoint
OpenRouter	Meta-provider — access to Gemini, Claude, GPT, Llama, etc.
OpenAI	GPT-4o, GPT-4.1
Anthropic	Claude 3.5 / 4 via native SDK
Groq	Llama models; OpenAI-compatible

Fallback chain: gemini → openrouter → openai → anthropic → groq

The LayerLLM system maps each agent layer (router, summariser, schema, repair, etc.) to a separate provider + model + tier, all configurable via environment variables.

Design System — 36 Components

The frontend ships a manifest-driven design system. The AI generates a JSON schema tree and the renderer instantiates real Angular components:

Category	Components
Form	`input` · `select` · `checkbox` · `radio` · `textarea` · `button`
Layout	`container` · `grid` · `card` · `tabs` · `accordion` · `flexbox` · `split-layout`
Data Display	`table` · `list` · `listbox` · `basic-chart` · `timeline` · `carousel` · `audio-player` · `video-player` · `stats-card` · `progress-ring` · `flow-diagram` · `chart-bar`
Typography	`heading` · `paragraph` · `divider`
Navigation	`wizard-stepper` · `menu` · `toolbar` · `stepper`
Feedback	`badge` · `alert` · `progress-bar`

All components are responsive (single-column on mobile), theme-aware (dark / light), and support the client-side data engine.

Client-Side Data Engine

Filtering, sorting, and pagination happen entirely in the browser — zero backend round-trips after the initial schema arrives.

Data components (table, list) register as sources with an id
Form controls (input, select, checkbox) target a source via filterTarget + filterField + filterOperator
As the user types or selects, the engine filters data instantly using Angular signals
Supports operators: contains, equals, gt, lt, gte, lte, in

Current Challenges & Open Problems

Generated UI Event Handling

The biggest open question in generative UI is: what happens when a user clicks a button?

In a traditional app, buttons trigger API calls, navigation, or state mutations that the developer has explicitly coded. In a generated UI, buttons exist because the LLM put them there — but the runtime has no way to know what backend action they should trigger.

Where we are today:

Wired interactions work — filter clearing, pagination (nextPage_, prevPage_), tab switching, accordion toggling, wizard step navigation, column sorting, copy-to-clipboard. These are all client-side and the engine handles them automatically.
Dead-end buttons are explicitly forbidden — the AI prompt instructs the model to never create a button that has no real client-side action (e.g. "View Details", "Learn More", "Get Started"). If a CTA can't be wired to a real interaction, it should be omitted or replaced with a badge/paragraph.
Cross-component communication — form controls can target data components for filtering, but there's no general-purpose event bus for arbitrary component-to-component messaging yet.

What's hard:

Stateful multi-view navigation — a portfolio site might have a "Projects" tab that navigates to a projects detail view. The AI can generate both views, but wiring per-item navigation (click project card → show project detail) requires a state machine the LLM would need to define.
Write operations — a contact form's "Submit" button implies a backend API call that doesn't exist. The system must either (a) prevent such buttons, (b) generate mock/preview behaviour, or (c) dynamically spin up API endpoints (which opens a massive security surface).
Event semantics — when the LLM generates {"type": "button", "props": {"label": "Delete"}}, what should delete? The LLM knows the intent, but that intent isn't captured in the schema in a machine-executable way.

Possible directions:

Declarative action schemas — extend the UI schema with an actions array that describes intents ({"action": "navigate", "target": "#section-projects"}, {"action": "copy", "value": "..."})
LLM-in-the-loop interactions — on button click, send the event back to the LLM and let it respond with a schema patch (already partially supported via the patch mode)
Sandboxed function generation — let the LLM emit small JS functions that run in a sandboxed iframe (high complexity, high flexibility)

This is an active area of exploration. Contributions and ideas are very welcome.

Tech Stack

Layer	Technology
Frontend	Angular 21 (Standalone Components, Signals, CDK Virtual Scroll)
Backend	NestJS, TypeORM, Bull queues
Database	PostgreSQL 14+
Cache	Redis 6+
Monorepo	Nx
Auth	JWT + OAuth (GitHub, Google)
WebSocket	Socket.IO (real-time streaming)
Containerisation	Docker & Docker Compose

Resilience Patterns

Pattern	Purpose
Circuit Breaker	Prevents cascading failures on AI provider outages
Connection Pool	Manages concurrent provider connections
Message Batcher	Batches database writes for interaction events
Provider Fallback	Streams primary provider; retries with fallback chain on 429/5xx
Tier Escalation	Repair starts at `fast` tier, escalates to `quality` if first attempt fails
Write-Behind Cache	Hot state in Redis, async write queue to PostgreSQL
Manifest-Driven Validation	Single source of truth for schema validation, sanitisation, and prompt generation

Quick Start

Prerequisites

Node.js 20+
Docker and Docker Compose

1. Clone and Install

git clone https://github.com/devashishSarmah/gen-ui.git
cd gen-ui
npm install

2. Configure Environment

cp .env.example .env

Required variables:

Variable	Description
`POSTGRES_PASSWORD`	Secure password for PostgreSQL
`REDIS_PASSWORD`	Secure password for Redis
`JWT_SECRET`	Signing secret (min 32 chars)
`FRONTEND_URL`	Frontend origin (default with proxy: `http://localhost`)
`NGINX_HTTP_PORT` / `NGINX_HTTPS_PORT`	Host ports exposed by reverse proxy (defaults: `80` / `443`)
At least one AI provider key	`GEMINI_API_KEY`, `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, or `OPENROUTER_API_KEY`

Optional OAuth:

Variable	Description
`GITHUB_CLIENT_ID` / `GITHUB_CLIENT_SECRET`	GitHub OAuth
`GOOGLE_CLIENT_ID` / `GOOGLE_CLIENT_SECRET`	Google OAuth (redirect URI: `<FRONTEND_URL>/auth/google/callback`)

3. Start with Docker

npm run docker:up        # Start all services
npm run docker:logs      # View logs
npm run docker:down      # Stop

Service	URL
Frontend (via Nginx)	http://localhost (or https://localhost)
Backend API (via Nginx)	http://localhost/api
Health check (via Nginx)	http://localhost/api/health

Note: Docker setup uses an internal reverse proxy (nginx) as the only host-exposed service. backend and frontend are not directly exposed.

4. Development Mode

# Infrastructure only
docker-compose up postgres redis -d

# In separate terminals
npm run start:backend
npm run start:frontend

Project Structure

gen-ui/
├── apps/
│   ├── backend/
│   │   └── src/
│   │       ├── ai/
│   │       │   ├── agents/          # 7 specialised agents
│   │       │   ├── providers/       # AI provider adapters
│   │       │   ├── tools/           # Web search, etc.
│   │       │   ├── prompts/         # System prompt + renderer schema
│   │       │   ├── agent-orchestrator.service.ts
│   │       │   ├── layer-llm.service.ts
│   │       │   └── model-resolver.service.ts
│   │       ├── auth/                # JWT + OAuth (GitHub, Google)
│   │       ├── gateway/             # WebSocket gateway (Socket.IO)
│   │       ├── state/               # State manager + replay
│   │       ├── common/              # Circuit breaker, pooling, batching
│   │       └── entities/            # TypeORM entities (6 tables)
│   │
│   └── frontend/
│       └── src/app/
│           ├── conversations/       # Chat view, sidebar, welcome screen
│           ├── shared/components/
│           │   └── ui-schema-renderer/  # Dynamic component renderer
│           ├── core/
│           │   ├── services/        # WebSocket, client data engine, interaction
│           │   └── stores/          # Signal-based state (conversations, UI)
│           └── auth/                # Login, register, OAuth callback
│
├── libs/
│   ├── design-system/               # 34 UI components + showcase
│   └── shared/                      # DTOs and interfaces
│
└── docker-compose.yml

Development Commands

# Dev servers
npm run start:backend           # NestJS (port 3000)
npm run start:frontend          # Angular (port 4200)

# Build
npm run build:backend
npm run build:frontend

# Docker
npm run docker:up               # All services
npm run docker:dev              # Dev mode with hot reload
npm run docker:down

# Database
npm run db:migrate              # Run migrations
npm run db:migrate:generate     # Generate migration from entity changes
npm run db:migrate:revert       # Revert last migration

CI/CD (GitHub Actions -> EC2)

This repo includes a full pipeline at .github/workflows/cicd.yml.

What it does:

CI on PR/push:
- npm ci --legacy-peer-deps
- npm run generate:schema
- production builds for frontend and backend
On main push (or manual run):
- builds and pushes Docker images to GHCR:
  - ghcr.io/<owner>/gen-ui-backend
  - ghcr.io/<owner>/gen-ui-frontend
  - ghcr.io/<owner>/gen-ui-nginx
- tags each image with:
  - commit SHA
  - latest
Deploy job:
- uploads deploy files to EC2:
  - infra/deploy/docker-compose.ec2.yml
  - infra/scripts/ec2-cleanup.sh
- pulls new images and restarts via Docker Compose on EC2
- runs cleanup script after deploy

Required GitHub Secrets

Set these in Settings -> Secrets and variables -> Actions:

EC2_HOST: public IP or DNS of EC2
EC2_USER: SSH user (for example ec2-user or ubuntu)
EC2_SSH_KEY: private key content (PEM)
EC2_PORT: SSH port (usually 22)
EC2_APP_DIR: absolute deploy directory on EC2 (for example /opt/genui)
GHCR_USERNAME: GitHub username/org account used for GHCR auth
GHCR_TOKEN: PAT with at least read:packages

Optional GitHub Actions variable:

DOCKER_PLATFORMS: defaults to linux/amd64.
- Use linux/arm64 for Graviton EC2.
- Use linux/amd64,linux/arm64 for multi-arch images.

One-time EC2 Setup

# 1) Create deploy directory
sudo mkdir -p /opt/genui
sudo chown -R "$USER":"$USER" /opt/genui

# 2) Place your runtime env file (same keys as .env.example)
cp /path/to/your/.env /opt/genui/.env

# 3) Ensure Docker + Compose are installed and running
docker --version
docker compose version || docker-compose --version
sudo systemctl enable docker
sudo systemctl start docker

The deploy workflow expects .env at ${EC2_APP_DIR}/.env.

Manual Deploy Commands on EC2 (Fallback)

cd /opt/genui
export BACKEND_IMAGE=ghcr.io/<owner>/gen-ui-backend
export FRONTEND_IMAGE=ghcr.io/<owner>/gen-ui-frontend
export NGINX_IMAGE=ghcr.io/<owner>/gen-ui-nginx
export IMAGE_TAG=<commit_sha_or_latest>

docker compose -f infra/deploy/docker-compose.ec2.yml --env-file .env pull
docker compose -f infra/deploy/docker-compose.ec2.yml --env-file .env up -d --remove-orphans

If your server only has legacy Compose binary, replace docker compose with docker-compose.

Cleanup Steps on EC2

Automatic cleanup runs after each deploy via infra/scripts/ec2-cleanup.sh.

Manual run:

cd /opt/genui
chmod +x infra/scripts/ec2-cleanup.sh
IMAGE_RETENTION_HOURS=168 PRUNE_VOLUMES=false ./infra/scripts/ec2-cleanup.sh

What it prunes:

old unused images
old build cache
old stopped containers

It does not prune volumes unless you explicitly set PRUNE_VOLUMES=true.

Inspiration & References

Generative UI: LLMs are Effective UI Generators — Google Research (2025)
Generative UI Paper (PDF)
Generative UI Project Page

License

MIT

An experimental project by @devashishSarmah

Name		Name	Last commit message	Last commit date
Latest commit History 70 Commits
.github		.github
apps		apps
infra		infra
libs		libs
postman		postman
.dockerignore		.dockerignore
.env.example		.env.example
.eslintrc.json		.eslintrc.json
.gitignore		.gitignore
.nvmrc		.nvmrc
.postcssrc.json		.postcssrc.json
README.md		README.md
docker-compose.dev.yml		docker-compose.dev.yml
docker-compose.yml		docker-compose.yml
jest.preset.js		jest.preset.js
nx.json		nx.json
package-lock.json		package-lock.json
package.json		package.json
postcss.config.js		postcss.config.js
tsconfig.base.json		tsconfig.base.json
tsconfig.typeorm.json		tsconfig.typeorm.json

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Gen UI — Generative User Interfaces with Multi-Agent AI

How It Works

Agent Architecture

AI Providers

Design System — 36 Components

Client-Side Data Engine

Current Challenges & Open Problems

Generated UI Event Handling

Tech Stack

Resilience Patterns

Quick Start

Prerequisites

1. Clone and Install

2. Configure Environment

3. Start with Docker

4. Development Mode

Project Structure

Development Commands

CI/CD (GitHub Actions -> EC2)

Required GitHub Secrets

One-time EC2 Setup

Manual Deploy Commands on EC2 (Fallback)

Cleanup Steps on EC2

Inspiration & References

License

About

Uh oh!

Releases

Packages

Uh oh!

Contributors

Uh oh!

Languages

Folders and files

Latest commit

History

Repository files navigation

Gen UI — Generative User Interfaces with Multi-Agent AI

How It Works

Agent Architecture

AI Providers

Design System — 36 Components

Client-Side Data Engine

Current Challenges & Open Problems

Generated UI Event Handling

Tech Stack

Resilience Patterns

Quick Start

Prerequisites

1. Clone and Install

2. Configure Environment

3. Start with Docker

4. Development Mode

Project Structure

Development Commands

CI/CD (GitHub Actions -> EC2)

Required GitHub Secrets

One-time EC2 Setup

Manual Deploy Commands on EC2 (Fallback)

Cleanup Steps on EC2

Inspiration & References

License

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages