Skip to content

Commit 943f136

Browse files
committed
feat(docs): add details on custom model fine-tuning roles and permissions
1 parent 3d18784 commit 943f136

4 files changed

Lines changed: 18 additions & 1 deletion

File tree

CHANGELOG.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -204,7 +204,7 @@ SPDX-License-Identifier: MIT-0
204204

205205
- **Discovery accessible from CLI and SDK** — Discovery can now be run programmatically via the IDP SDK (`client.discovery.run()`) and CLI (`idp-cli discover`), enabling users with many document classes to automate schema generation without the Web UI. Supports both modes: without ground truth (exploratory) and with ground truth (optimized). ([#228](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/issues/228))
206206

207-
- **Custom Model Fine-tuning**Fine-tune Amazon Nova models using processed IDP documents directly from the Web UI. Create a fine-tuning job from a Test Set (which provides documents with ground truth data), generate training data in JSONL format, submit fine-tuning jobs to Bedrock, and deploy the resulting custom models for use in extraction workflows. See [Custom Model Fine-tuning](./docs/custom-model-finetuning.md) for details.
207+
- **Custom Model Fine-tuning**Improve extraction and classification accuracy for your specific document types by fine-tuning Amazon Nova models on your own labeled data — no ML expertise required. Select a Test Set with ground truth, choose a base model, and the system handles training data generation, Bedrock fine-tuning, and on-demand model deployment automatically. Custom models are billed pay-per-token with no idle costs. Available to Admin and Author roles. See [Custom Model Fine-tuning](./docs/custom-model-finetuning.md) for details.
208208
- **Web UI**: New "Custom Models" page with job creation form (test set selector, base model selector, train/validation split), jobs table with status tracking, and detailed job view with deployment status and configuration version creation
209209
- **CLI / SDK**: `idp-cli finetuning create`, `idp-cli finetuning status`, `idp-cli finetuning list`, `idp-cli finetuning delete` commands for programmatic job management
210210
- **GraphQL API**: New `createFinetuningJob`, `getFinetuningJob`, `listFinetuningJobs`, `deleteFinetuningJob` mutations/queries with `FinetuningJob` type and real-time status fields

docs/custom-model-finetuning.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,13 +28,20 @@ flowchart LR
2828
5. **Deploy Endpoint**: The trained model is deployed as an on-demand Custom Model Deployment
2929
6. **Use in Configuration**: Select the custom model when creating or editing configurations
3030

31+
## Roles & Permissions
32+
33+
Custom Model Fine-tuning is available to **Admin** and **Author** roles. The GraphQL mutations (`createFinetuningJob`, `deleteFinetuningJob`) are enforced server-side via AppSync `@aws_auth` directives, so Reviewer and Viewer roles cannot create or delete jobs even via direct API calls. The query APIs (`listFinetuningJobs`, `getFinetuningJob`, `listAvailableModels`) are accessible to all authenticated users at the schema level, but the UI only exposes the Custom Models page to Admin and Author.
34+
35+
> **Note on config-version scoping**: Fine-tuning jobs are currently global — they are not filtered by `allowedConfigVersions`. A scoped Author can see all fine-tuning jobs and create jobs from any test set. However, when applying a custom model to a configuration version (via "Create Config Version"), the config-version scope is enforced — the Author can only target versions within their scope. See [RBAC](./rbac.md) for the full permission matrix.
36+
3137
## Prerequisites
3238

3339
- An existing Test Set with:
3440
- At least 10 documents
3541
- At least 2 document classes
3642
- Baseline classifications for all documents
3743
- Appropriate IAM permissions for Bedrock model customization
44+
- **Admin** or **Author** role in the IDP Accelerator
3845

3946
## Using the UI
4047

docs/rbac.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,12 @@ TEST STUDIO
6767
View/run test sets ✅ ✅ ❌ ❌
6868
Create/delete test sets ✅ ✅ ❌ ❌
6969
70+
CUSTOM MODEL FINE-TUNING
71+
List/view fine-tuning jobs ✅ ✅ ❌ ❌
72+
Create fine-tuning jobs ✅ ✅ ❌ ❌
73+
Delete fine-tuning jobs ✅ ✅ ❌ ❌
74+
List available models ✅ ✅ ❌ ❌
75+
7076
CAPACITY PLANNING
7177
Calculate capacity ✅ ✅ ❌ ✅
7278
@@ -178,6 +184,7 @@ Every GraphQL **mutation** and many **queries** have `@aws_auth(cognito_groups:
178184
| `startTestRun`, `addTestSet`, `addTestSetFromUpload`, `deleteTests`, `deleteTestSets` | Admin, Author |
179185
| `syncBdaIdp`, `uploadDiscoveryDocument`, `deleteDiscoveryJob`, `autoDetectSections` | Admin, Author |
180186
| `copyToBaseline` | Admin, Author |
187+
| `createFinetuningJob`, `deleteFinetuningJob` | Admin, Author |
181188
| `processChanges`, `completeSectionReview`, `claimReview`, `releaseReview`, `skipAllSectionsReview` | Admin, Reviewer |
182189
| `sendAgentChatMessage`, `deleteChatSession`, `updateChatSessionTitle`, `deleteAgentJob` | All authenticated users (see note below) |
183190
| `updateAgentChatMessage` | All authenticated users (also IAM for backend) |
@@ -196,6 +203,7 @@ Every GraphQL **mutation** and many **queries** have `@aws_auth(cognito_groups:
196203
| `listConfigurationLibrary`, `getConfigurationLibraryFile` | Admin, Author, Viewer |
197204
| `listDiscoveryJobs` | Admin, Author |
198205
| `getTestRun`, `getTestRuns`, `getTestRunStatus`, `compareTestRuns`, `getTestSets`, `listBucketFiles`, `validateTestFileName` | Admin, Author |
206+
| `listFinetuningJobs`, `getFinetuningJob`, `validateTestSetForFinetuning`, `listAvailableModels` | All authenticated (UI limited to Admin, Author) |
199207
| `queryKnowledgeBase`, `chatWithDocument` | All authenticated |
200208
| `listUsers` | All authenticated (non-admin sees only self in resolver) |
201209
| `getMyProfile` | All authenticated |
@@ -295,4 +303,5 @@ To add a new role:
295303
- **Knowledge Base queries** do not currently enforce config-version scope. KB results may include documents from out-of-scope config versions.
296304
- **Agent Companion Chat** analytics queries (Athena) do not filter by config-version scope.
297305
- **GetDocument API** (direct document access by URL) does not enforce config-version scope at the resolver level. UI navigation hides out-of-scope documents, but direct API access is not blocked.
306+
- **Custom Model Fine-tuning** jobs are global — not scoped by `allowedConfigVersions`. A scoped Author can see all fine-tuning jobs and create jobs from any test set. However, when applying a custom model to a configuration version (via the "Create Config Version" modal), the config-version scope IS enforced — the Author can only target versions within their scope.
298307
- These limitations are tracked for Phase 3 implementation.

src/ui/src/components/genaiidp-layout/navigation.tsx

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,7 @@ export const authorNavItems = [
8080
items: [
8181
{ type: 'link', text: 'View/Edit Configuration', href: `#${CONFIGURATION_PATH}` },
8282
{ type: 'link', text: 'Discovery', href: `#${DISCOVERY_PATH}` },
83+
{ type: 'link', text: 'Custom Models', href: `#${CUSTOM_MODELS_PATH}` },
8384
{ type: 'link', text: 'Capacity Planning', href: `#${CAPACITY_PLANNING_PATH}` },
8485
{ type: 'link', text: 'View Pricing', href: `#${PRICING_PATH}` },
8586
],

0 commit comments

Comments
 (0)