Skip to content

Commit 4c3de58

Browse files
committed
docs(research): add plain-language summaries and guide
1 parent d84a316 commit 4c3de58

12 files changed

Lines changed: 138 additions & 1 deletion

README.academic.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ The repository combines a working Next.js CMS platform with a documented archite
88

99
Quick links:
1010

11+
- Research guide: [docs/research/README.md](./docs/research/README.md)
1112
- Executive summary: [docs/research/executive-summary.md](./docs/research/executive-summary.md)
1213
- Vibe-Guard protocol: [docs/vibe-guard/protocol-overview.md](./docs/vibe-guard/protocol-overview.md)
1314
- Commercial version: [README.commercial.md](./README.commercial.md)

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ It combines three goals in one repository:
1515
- Commercial overview: [README.commercial.md](./README.commercial.md)
1616
- Academic overview: [README.academic.md](./README.academic.md)
1717
- Developer guide: [docs/dev/README.md](./docs/dev/README.md)
18+
- Research guide: [docs/research/README.md](./docs/research/README.md)
1819
- Docs index: [docs/README.md](./docs/README.md)
1920
- Security audit: [docs/security/security-audit.md](./docs/security/security-audit.md)
2021
- Vibe-Guard protocol: [docs/vibe-guard/protocol-overview.md](./docs/vibe-guard/protocol-overview.md)

docs/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ If you’re reading the repository as a research artifact, jump to the Research
2626

2727
## Research
2828

29+
- [Research Guide (Start Here)](./research/README.md)
2930
- [Executive Summary](./research/executive-summary.md)
3031
- [Core Concepts](./research/core-concepts.md)
3132
- [Methodology](./research/methodology.md)

docs/dev/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
This guide is for engineers using `CMS Nova` as a reusable Next.js CMS template (not the research angle).
44

5-
If you’re here for the case study / Vibe-Guard research docs, start at `docs/research/executive-summary.md`.
5+
If you’re here for the case study / Vibe-Guard research docs, start at `docs/research/README.md`.
66

77
## Quick Start (Local)
88

docs/research/README.md

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# Research Guide (Start Here)
2+
3+
This folder documents a case study about security and architectural integrity in **AI-assisted software development**, using `CMS Nova` as the main research artifact.
4+
5+
## TL;DR (1 minute)
6+
7+
- AI coding can produce changes that look correct **inside one file**, but violate **system-wide security rules** (auth boundaries, publication rules, route visibility, secret handling).
8+
- The case study identifies recurring reasoning failures (e.g. `Naming Bias`, `Lack of Global Context`) and frames the human role as `MVU` (Minimum Viable Understanding).
9+
- The proposed mitigation is `Vibe-Guard`: treat security as **policy verification** (central rules + automated checks), not as “remember everything everywhere”.
10+
- This repo includes a small concrete example of that idea: a default-private policy for `/api/*` routes plus a static checker (`npm run security:check-api`).
11+
12+
## Plain-language (ELI5)
13+
14+
Imagine a helper who is very good at building **one LEGO house at a time**.
15+
Each house looks fine. But a city is not just houses — it also has **city rules**:
16+
17+
- “Only teachers can enter the teacher room.” (authorization)
18+
- “Only finished posters go on the public wall.” (published-only content)
19+
- “Don’t put secret keys on the street.” (secret handling)
20+
21+
AI tools often follow the “house logic” but forget the “city rules” unless those rules are written down and checked.
22+
`Vibe-Guard` is the idea of writing the city rules once, centrally, and having an automated guard verify that every new house still follows them.
23+
24+
## What’s in This Repo (Research Artifacts)
25+
26+
- Concepts and framing:
27+
- [Core Concepts](./core-concepts.md)
28+
- Evidence and narrative:
29+
- [Case Study: CMS Nova](./case-study-cms-nova.md)
30+
- [Findings Matrix](./findings-matrix.md)
31+
- Method and reproducibility:
32+
- [Methodology](./methodology.md)
33+
- [API Visibility Policy](../../src/server/policy/api-visibility.js)
34+
- [API Policy Checker](../../scripts/security/check-api-policy.js)
35+
- Architecture intervention record:
36+
- [Architecture Evolution](./architecture-evolution.md)
37+
38+
## Suggested Reading Order
39+
40+
1. [Executive Summary](./executive-summary.md)
41+
2. [Core Concepts](./core-concepts.md)
42+
3. [Case Study: CMS Nova](./case-study-cms-nova.md)
43+
4. [Methodology](./methodology.md)
44+
5. [Findings Matrix](./findings-matrix.md)
45+
6. [Architecture Evolution](./architecture-evolution.md)

docs/research/architecture-evolution.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,17 @@
11
# Architecture Evolution
22

3+
## TL;DR (1 minute)
4+
5+
- The initial repo worked, but boundaries were blurry: server/client concerns mixed, public/admin rules scattered, and business-specific features leaked into “core”.
6+
- The intervention encodes boundaries in folders (`server/`, `modules/`, `shared/`, `verticals/`) and makes policy-sensitive behavior harder to change accidentally.
7+
- This supports the research thesis: AI can keep local code correct while eroding global integrity unless architecture + policy are explicit.
8+
9+
## Plain-language (ELI5)
10+
11+
Think of the repo like a big house.
12+
If everything is thrown into one big room, it’s easy to accidentally grab the wrong thing and break a rule.
13+
By putting things into labeled rooms (server-only, shared, business-specific), we make it easier for humans and tools to follow “house rules” consistently.
14+
315
## Purpose
416

517
This document explains the structural evolution of `CMS Nova` from an organically grown application into a more explicit platform architecture. It is written as both a technical record and a research artifact for the Vibe-Guard case study.

docs/research/case-study-cms-nova.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,18 @@
11
# Case Study: CMS Nova
22

3+
## TL;DR (1 minute)
4+
5+
- AI-assisted changes can improve one visible security issue while silently weakening another cross-cutting rule.
6+
- The most common failures were not “syntax bugs”, but **policy bugs**: what should be public, who can mutate state, what counts as “published”, and where secrets/logs can go.
7+
- This repo includes a small “default private” guardrail for `/api/*` routes as an example of policy externalization.
8+
9+
## Plain-language (ELI5)
10+
11+
`CMS Nova` is like a school with many rooms.
12+
Some rooms are public (hallway), some are private (teacher office).
13+
AI helpers often build features fast, but sometimes they accidentally leave a door unlocked because the door is in a different file than the feature they were working on.
14+
The case study is about finding those “unlocked doors” and then writing down rules so they are harder to reintroduce.
15+
316
## Why CMS Nova Matters
417

518
`CMS Nova` is a strong research artifact because it is neither a toy example nor a narrowly scoped demo. It contains dynamic content modeling, authentication, admin workflows, plugin configuration, public routing, file handling, and productization efforts through a CLI. That combination makes it large enough for structural failures to emerge.

docs/research/core-concepts.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,12 @@
11
# Core Concepts
22

3+
## Plain-language (ELI5)
4+
5+
- **Naming Bias:** trusting the *label* on a box instead of checking what’s inside.
6+
- **Lack of Global Context:** fixing one room in a house without understanding how the whole house works.
7+
- **MVU:** the minimum “map” a human needs to notice when something is locally fine but globally unsafe.
8+
- **Vibe-Guard:** a rulebook + a referee (central rules that can be checked automatically).
9+
310
## Naming Bias
411

512
`Naming Bias` describes the tendency of AI coding systems to use names as security signals. If a file, route, variable, or module sounds harmless, the model may infer that it is safe to expose, relax, or bypass. This inference is often made without validating the actual sensitivity of the underlying data or checking project-wide policies.

docs/research/executive-summary.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,19 @@
11
# Executive Summary
22

3+
## TL;DR (1 minute)
4+
5+
- AI-assisted coding can generate changes that look correct **inside one file**, but break **system-wide security rules** (auth boundaries, publication rules, route visibility, secret handling).
6+
- `CMS Nova` is used as a realistic case study; audits of AI-assisted paths surfaced **24 vulnerabilities** across multiple systems.
7+
- The proposed mitigation is `Vibe-Guard`: move critical expectations into **central policy** and check changes against them automatically.
8+
9+
## Plain-language (ELI5)
10+
11+
Think of software like a big LEGO city.
12+
An AI helper can build one LEGO house very fast, and that house can look “correct”.
13+
But cities also have rules: “only grown-ups can open this door” and “don’t put secrets outside”.
14+
AI systems often follow the house-level logic but forget the city-level rules unless those rules are written down and checked.
15+
`Vibe-Guard` is the idea of writing the city rules once and having an automated guard verify every change.
16+
317
This research investigates the security behavior of AI-assisted software development through a real-world case study: `CMS Nova`, a Next.js-based headless CMS that evolved through iterative human-AI collaboration.
418

519
The central claim is that modern coding models often preserve local correctness while weakening global system integrity. In practice, they can produce working code that appears reasonable inside a single file, yet violates cross-cutting security invariants such as authorization boundaries, publication rules, route exposure constraints, and secret-handling policies.

docs/research/findings-matrix.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,16 @@
11
# Findings Matrix
22

3+
## How to Use This (30 seconds)
4+
5+
- Each row is one “what went wrong” story: **Issue → Root cause → AI failure mode → Intervention → Outcome**.
6+
- Read it as a map from *symptom* to *structural fix*, not as a list of random bugs.
7+
- If you only read one thing: look for repeated patterns in the **AI Failure Mode** column.
8+
9+
## Plain-language (ELI5)
10+
11+
This table is like a “mistake notebook”.
12+
It lists the same kind of mistakes happening again and again (e.g., leaving a door unlocked), and then shows the fix (e.g., adding a lock that is checked every time).
13+
314
## Purpose
415

516
This matrix summarizes the main findings from the `CMS Nova` case study in a format suitable for research writing, portfolio presentation, and discussion of the Vibe-Guard proposal.

0 commit comments

Comments
 (0)