Skip to content

Commit 79a5797

Browse files
RitaChen609RitaChen609
committed
ghcp(crawl): ex0 bootstrap scaffolding
1 parent 7ea79ef commit 79a5797

File tree

4 files changed

+281
-0
lines changed

4 files changed

+281
-0
lines changed

‎.copilot-track/crawl/README.md‎

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
# Copilot Crawl Track — README
2+
3+
This directory (`.copilot-track/crawl/`) holds AI-assisted crawl-and-modernisation artefacts for the **MarkLogic Java Client API** repository. It is not part of the production library; contents are for developer guidance and AI context only.
4+
5+
---
6+
7+
## What Is the Crawl Track?
8+
9+
The crawl track is an incremental, evidence-driven workflow for making large-scale changes to this codebase using AI assistance (GitHub Copilot / agent mode). Changes are broken into small, reviewable PRs that form a **chain** — each PR builds on the previous one.
10+
11+
---
12+
13+
## Chain-PR Pattern
14+
15+
A chain-PR is a sequence of pull requests where each PR:
16+
17+
1. Targets the **previous PR's branch** (not `main`) as its base, creating a linear dependency chain.
18+
2. Carries a **single, focused concern** (e.g., "migrate HTTP client from HttpClient to OkHttp", "update Jackson version", "replace deprecated API calls in DocMgr").
19+
3. Is reviewed and merged in order — do **not** merge PR N+1 before PR N is merged and its branch updated.
20+
21+
```
22+
main ← PR-1 (foundation) ← PR-2 (layer A) ← PR-3 (layer B) ← ...
23+
```
24+
25+
When the base PR merges, rebase subsequent PRs down the chain to keep them conflict-free:
26+
27+
```bash
28+
git fetch origin
29+
git checkout feature/crawl-layer-B
30+
git rebase origin/feature/crawl-layer-A
31+
git push --force-with-lease
32+
```
33+
34+
---
35+
36+
## Evidence in PRs
37+
38+
Every crawl PR must include evidence that the change is safe. Accepted evidence types:
39+
40+
| Evidence | Where to add it |
41+
| ------------------------------------------------ | -------------------------------------------------------------------------- |
42+
| Passing CI green-check (unit + functional tests) | Shown automatically on the PR by Jenkins |
43+
| Before/after compile output | Paste in PR description under `## Build evidence` |
44+
| Test-coverage delta | Add `## Test delta` section; attach Gradle test report if coverage dropped |
45+
| Copilot prompt used | Add `## Prompt used` section (see below) |
46+
| Manual verification steps | Add `## Manual verification` with exact commands run |
47+
48+
PRs that lack evidence will be marked **needs-evidence** and not merged.
49+
50+
---
51+
52+
## Prompt Usage
53+
54+
AI prompts that drove a crawl change belong in the PR description under `## Prompt used`. This creates an audit trail and lets reviewers reproduce or adjust the change.
55+
56+
**Template:**
57+
58+
```markdown
59+
## Prompt used
60+
61+
> Agent mode, model: claude-sonnet-4-6
62+
>
63+
> "Migrate all usages of `com.marklogic.client.impl.OkHttpServices` constructor that
64+
> pass a plain `String` password to instead use `char[]` and call `Arrays.fill` after use.
65+
> Do not modify test files. Only change files under marklogic-client-api/src/main/."
66+
67+
Files changed by prompt: <!-- list them -->
68+
Files reviewed manually: <!-- list them -->
69+
```
70+
71+
Storing prompts in PRs helps future crawl passes understand _why_ a change was made, not just _what_ changed.
72+
73+
---
74+
75+
## Adding New Crawl Artefacts
76+
77+
Place any generated files, diff summaries, or migration notes inside this directory as flat Markdown or JSON files. Suggested naming:
78+
79+
```
80+
.copilot-track/crawl/
81+
├── README.md ← this file
82+
├── 001-<topic>.md ← plan / notes for crawl step 1
83+
├── 002-<topic>.md ← plan / notes for crawl step 2
84+
└── ...
85+
```
86+
87+
Keep each step file small (< 200 lines). Reference `ai-track-docs/` for system-level context.
88+
89+
---
90+
91+
## Related Docs
92+
93+
- [ai-track-docs/SYSTEM-OVERVIEW.md](../../ai-track-docs/SYSTEM-OVERVIEW.md) — what this project does and how it is structured
94+
- [ai-track-docs/build-test.md](../../ai-track-docs/build-test.md) — how to build and run tests locally
95+
- [ai-track-docs/architecture.mmd](../../ai-track-docs/architecture.mmd) — Mermaid architecture diagram

‎ai-track-docs/SYSTEM-OVERVIEW.md‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# System Overview — MarkLogic Java Client API
2+
3+
## Purpose
4+
5+
The **MarkLogic Java Client API** (`com.marklogic:marklogic-client-api`) is a Java library that exposes MarkLogic Server's REST API as a type-safe, fluent Java interface. It supports reading, writing, deleting, and querying JSON, XML, binary, and text documents, as well as ACID multi-statement transactions, semantic (SPARQL/RDF), Full-text search, alerting, Data Services, and Row Manager (Optic API).
6+
7+
## Repository Root
8+
9+
`marklogic-client-api-parent` (Gradle multi-project).
10+
11+
## Modules
12+
13+
| Module | Description |
14+
| -------------------------------------- | ---------------------------------------------------------------------------- |
15+
| `marklogic-client-api` | Core library — all production source code |
16+
| `marklogic-client-api-functionaltests` | Functional / integration tests requiring a live MarkLogic instance |
17+
| `ml-development-tools` | Kotlin-based developer tooling (code generation helpers) |
18+
| `test-app` | MarkLogic application deployed to the test server (modules, schemas, config) |
19+
| `examples` | Standalone usage examples |
20+
21+
## Runtime Requirements
22+
23+
- **Java 17** (minimum; Java 21 also supported and tested in CI)
24+
- **MarkLogic Server** (for integration/functional tests) — started via `docker-compose.yaml`
25+
26+
## Technology Stack
27+
28+
- Build: **Gradle** (wrapper at `./gradlew`)
29+
- Test framework: **JUnit 5** (unit) + MarkLogic functional test harness
30+
- CI: **Jenkins** (`Jenkinsfile`) — Docker-based MarkLogic, parallel Java 17/21 builds
31+
- Primary language: **Java**; developer tooling in **Kotlin**
32+
33+
## Key External Dependencies
34+
35+
- OkHttp (HTTP client transport)
36+
- Jackson (JSON serialization)
37+
- SLF4J / Logback (logging)
38+
39+
## Relationship to MarkLogic Server
40+
41+
All network communication travels over the **MarkLogic REST Management and Client APIs** (typically port 8000/8002). The library never connects directly to MarkLogic's internal ports; authentication is via HTTP Digest or certificate.

‎ai-track-docs/architecture.mmd‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
%%{init: {"theme": "neutral"}}%%
2+
graph TD
3+
subgraph callers["Calling Code"]
4+
APP["Java Application / Examples"]
5+
end
6+
7+
subgraph core["marklogic-client-api (core)"]
8+
DatabaseClient["DatabaseClient\n(entry-point factory)"]
9+
DocMgr["Document Managers\n(JSON / XML / Text / Binary / Generic)"]
10+
QueryMgr["QueryManager\n(search, cts, SPARQL)"]
11+
TxMgr["TransactionManager\n(ACID multi-statement)"]
12+
RowMgr["RowManager\n(Optic / SQL)"]
13+
DataSvc["Data Services\n(generated proxies)"]
14+
RESTServices["RESTServices\n(OkHttp transport layer)"]
15+
end
16+
17+
subgraph devtools["ml-development-tools"]
18+
CodeGen["Proxy Code Generator\n(Kotlin)"]
19+
end
20+
21+
subgraph testapp["test-app"]
22+
MLConfig["ml-config\n(DB, forests, REST server)"]
23+
MLModules["ml-modules\n(XQuery / SJS modules)"]
24+
MLSchemas["ml-schemas\n(TDE templates)"]
25+
end
26+
27+
subgraph server["MarkLogic Server (Docker / remote)"]
28+
REST["REST Client API\n(port 8000 / 8002)"]
29+
XDBC["e-node internal"]
30+
end
31+
32+
APP --> DatabaseClient
33+
DatabaseClient --> DocMgr
34+
DatabaseClient --> QueryMgr
35+
DatabaseClient --> TxMgr
36+
DatabaseClient --> RowMgr
37+
DatabaseClient --> DataSvc
38+
DocMgr --> RESTServices
39+
QueryMgr --> RESTServices
40+
TxMgr --> RESTServices
41+
RowMgr --> RESTServices
42+
DataSvc --> RESTServices
43+
RESTServices -->|"HTTP Digest / Cert auth"| REST
44+
REST --> XDBC
45+
46+
CodeGen -->|"generates Java proxy classes"| DataSvc
47+
48+
MLConfig -->|"mlDeploy"| REST
49+
MLModules -->|"mlDeploy"| REST
50+
MLSchemas -->|"mlReloadSchemas"| REST

‎ai-track-docs/build-test.md‎

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
# Build & Test Guide
2+
3+
## Prerequisites
4+
5+
| Requirement | Notes |
6+
| -------------- | -------------------------------------------------------------------------------------------- |
7+
| JDK 17+ | JDK 21 also works; set `JAVA_HOME` or rely on Gradle toolchain auto-provisioning |
8+
| Docker | Required for functional tests (MarkLogic container) |
9+
| Gradle wrapper | Use `./gradlew` (Linux/macOS) or `gradlew.bat` (Windows); do **not** install Gradle globally |
10+
11+
---
12+
13+
## Quick Build (no tests)
14+
15+
```bash
16+
./gradlew clean build -x test
17+
```
18+
19+
---
20+
21+
## Unit Tests — core library only
22+
23+
```bash
24+
./gradlew :marklogic-client-api:test
25+
```
26+
27+
Unit tests have **no external dependencies**; they run without MarkLogic.
28+
29+
---
30+
31+
## Developer-Tools Tests
32+
33+
```bash
34+
./gradlew :ml-development-tools:test
35+
```
36+
37+
---
38+
39+
## Functional / Integration Tests
40+
41+
Functional tests require a running MarkLogic instance. Start it with Docker Compose first:
42+
43+
```bash
44+
docker compose up -d
45+
```
46+
47+
Then deploy the test application and run the functional tests:
48+
49+
```bash
50+
./gradlew :test-app:mlDeploy :test-app:mlReloadSchemas
51+
./gradlew :marklogic-client-api-functionaltests:test
52+
```
53+
54+
> **Tip:** The `Jenkinsfile` contains the authoritative CI test sequence if the local steps diverge.
55+
56+
---
57+
58+
## Running a Specific Test Class
59+
60+
```bash
61+
./gradlew :marklogic-client-api:test --tests "com.marklogic.client.test.SomeTest"
62+
```
63+
64+
---
65+
66+
## Gradle Properties
67+
68+
Key properties live in `gradle.properties` and `marklogic-client-api/gradle.properties`. Override on the command line with `-P<key>=<value>`:
69+
70+
```bash
71+
./gradlew :marklogic-client-api:test -PmlHost=localhost -PmlPort=8000
72+
```
73+
74+
---
75+
76+
## Build Artifacts
77+
78+
After a successful build the JAR is at:
79+
80+
```
81+
marklogic-client-api/build/libs/marklogic-client-api-<version>.jar
82+
```
83+
84+
---
85+
86+
## Linting / Static Analysis
87+
88+
No dedicated lint step is configured in the current Gradle build.
89+
90+
---
91+
92+
## Common Pitfalls
93+
94+
- Functional tests **will hang or fail** if Docker is not running or MarkLogic has not finished starting. Wait ~30 s after `docker compose up -d` before deploying.
95+
- Java toolchain auto-provisioning requires internet access on first run. On air-gapped machines set `org.gradle.java.installations.paths` in `~/.gradle/gradle.properties`.

0 commit comments

Comments
 (0)