Skip to content

Commit 6190010

Browse files
authored
docs: add architecture diagram for intrinsics (#998)
Signed-off-by: Jake LoRocco <jake.lorocco@ibm.com>
1 parent 8a835c5 commit 6190010

5 files changed

Lines changed: 42 additions & 84 deletions

File tree

docs/docs/advanced/intrinsics.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ reliable than prompting a general-purpose model for these specialized micro-task
1919
> - **OpenAIBackend** — uses a Granite Switch model served via vLLM with
2020
> `load_embedded_adapters=True`. Only intrinsics embedded in the model are
2121
> available — check the model's `adapter_index.json` for the list.
22+
> See `docs/docs/examples/granite-switch/README.md`
2223
>
2324
> Intrinsics do not work with Ollama or other remote backends.
2425
127 KB
Loading

docs/examples/granite-switch/README.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -26,11 +26,7 @@ python -m vllm.entrypoints.openai.api_server \
2626

2727
## Available adapters
2828

29-
Not all intrinsics are embedded in every Granite Switch model. Check the model's
30-
`adapter_index.json` for the list of available adapters. The current model
31-
includes: `answerability`, `citations`, `context_relevance`, `guardian-core`,
32-
`hallucination_detection`, `query_clarification`, `query_rewrite`, and
33-
`requirement-check`.
29+
Not all intrinsics are embedded in every Granite Switch model. You should check the model's `adapter_index.json` file for a definitive list. For granite switch models pre-built by IBM, we include a list of models in the Mellea `model_id`.
3430

3531
## Files
3632

@@ -52,7 +48,11 @@ Shows how to manually load embedded adapters using
5248
you only need a subset of adapters or want more control over adapter
5349
registration.
5450

51+
## Architecture
52+
![Granite Libraries Software Stack Architecture in Mellea](../../docs/images/granite-libraries-mellea-architecture.png)
53+
5554
## Related
5655

5756
- [`../intrinsics/`](../intrinsics/) — the same intrinsics using `LocalHFBackend`
58-
- [Intrinsics documentation](../../docs/docs/advanced/intrinsics.md)
57+
- [Intrinsics Documentation](../../docs/docs/advanced/intrinsics.md)
58+
- [Official Granite Switch Documentation](https://github.com/generative-computing/granite-switch)

docs/examples/intrinsics/README.md

Lines changed: 34 additions & 77 deletions
Original file line numberDiff line numberDiff line change
@@ -2,54 +2,6 @@
22

33
This directory contains examples for using Mellea's intrinsic functions - specialized model capabilities accessed through adapters.
44

5-
## Files
6-
7-
### intrinsics.py
8-
Core example showing how to directly use intrinsics with adapters.
9-
10-
**Key Features:**
11-
- Creating and adding adapters to backends
12-
- Using `Intrinsic` component for specialized tasks
13-
- Working with Granite Common adapters (aLoRA-based)
14-
- Understanding adapter output formats
15-
16-
### answerability.py
17-
Checks if a question can be answered given the context.
18-
19-
### citations.py
20-
Validates and extracts citations from generated text.
21-
22-
### context_relevance.py
23-
Assesses if retrieved context is relevant to a query.
24-
25-
### hallucination_detection.py
26-
Detects when model outputs contain hallucinated information.
27-
28-
### query_rewrite.py
29-
Rewrites queries for better retrieval or understanding.
30-
31-
### uncertainty.py
32-
Estimates the model's certainty about answering a question.
33-
34-
### requirement_check.py
35-
Detect if text adheres to provided requirements.
36-
37-
### policy_guardrails.py
38-
Checks if a scenario is compliant/non-compliant/ambiguous with respect to a given policy,
39-
40-
### guardian_core.py
41-
Uses the guardian-core LoRA adapter for safety risk detection, including prompt-level harm, response-level social bias, RAG groundedness, and custom criteria.
42-
43-
### factuality_detection.py
44-
Detects if the the model's output is factually incorrect relative to context.
45-
46-
### factuality_correction.py
47-
Corrects a factually incorrect response relative to context.
48-
49-
### context_attribution.py
50-
Identifies sentences in conversation history and documents that most influenced the response.
51-
52-
535
## Concepts Demonstrated
546

557
- **Intrinsic Functions**: Specialized model capabilities beyond text generation
@@ -61,25 +13,20 @@ Identifies sentences in conversation history and documents that most influenced
6113
## Basic Usage
6214

6315
```python
64-
from mellea.backends.huggingface import LocalHFBackend
65-
from mellea.backends.adapters.adapter import IntrinsicAdapter
66-
from mellea.stdlib.components import Intrinsic
67-
import mellea.stdlib.functional as mfuncs
16+
from mellea import model_ids, start_backend
17+
from mellea.stdlib import functional as mfuncs
18+
from mellea.stdlib.components.intrinsic import core
6819

69-
# Create backend and adapter
70-
backend = LocalHFBackend(model_id="ibm-granite/granite-4.0-micro")
71-
adapter = IntrinsicAdapter("requirement_check",
72-
base_model_name=backend.base_model_name)
73-
backend.add_adapter(adapter)
74-
75-
# Use intrinsic
76-
out, new_ctx = mfuncs.act(
77-
Intrinsic(
78-
"requirement_check",
79-
intrinsic_kwargs={"requirement": "The assistant is helpful."}),
80-
ctx,
81-
backend
20+
ctx, backend = start_backend(
21+
"hf", model_id=model_ids.IBM_GRANITE_4_1_3B, context_type="chat"
8222
)
23+
24+
response, ctx = mfuncs.chat("What is 2 + 2?", ctx, backend)
25+
print(f"Response: {response.content}")
26+
27+
# NOTE: There are additional functions for other intrinsics as well.
28+
result = core.check_certainty(ctx, backend)
29+
print(f"Certainty score: {result}")
8330
```
8431

8532
OpenAIBackends also support a type of embedded adapter for Granite Switch models:
@@ -89,8 +36,15 @@ backend = OpenAIBackend(
8936
load_embedded_adapters=True, # Auto-loads adapters from huggingface repo.
9037
...
9138
)
39+
```
40+
41+
The underlying intrinsics can also be utilized directly when generating:
42+
```python
43+
from mellea.stdlib.components import Intrinsic
44+
import mellea.stdlib.functional as mfuncs
45+
46+
...
9247

93-
# Assumes the model has the `requirement_check` adapter embedded.
9448
out, new_ctx = mfuncs.act(
9549
Intrinsic(
9650
"requirement_check",
@@ -103,29 +57,32 @@ out, new_ctx = mfuncs.act(
10357
For complete runnable examples using the OpenAI backend with Granite Switch,
10458
see [`../granite-switch/`](../granite-switch/).
10559

106-
> **Note:** Not all intrinsics are embedded in every Granite Switch model. The
107-
> current model includes: `answerability`, `citations`, `context_relevance`,
108-
> `guardian-core`, `hallucination_detection`, `query_clarification`,
109-
> `query_rewrite`, and `requirement-check`. Check the model's
110-
> `adapter_index.json` for the full list.
60+
> **Note:** Not all intrinsics are embedded in every Granite Switch model. You should check
61+
> the model's `adapter_index.json` file for a definitive list. For granite switch models
62+
> pre-built by IBM, we include a list of models in the Mellea `model_id`.
11163
11264
## Available Intrinsics
11365

114-
- **requirement_check**: Validate requirements (used by ALoraRequirement)
11566
- **answerability**: Determine if question is answerable
11667
- **citations**: Extract and validate citations
68+
- **context-attribution**: Identify context sentences that most influenced response
11769
- **context_relevance**: Assess context-query relevance
70+
- **factuality_correction**: Correct factually incorrect responses
71+
- **factuality_detection**: Detect factually incorrect responses
72+
- **guardian-core**: Safety risk detection (harm, bias, groundedness, custom criteria)
11873
- **hallucination_detection**: Detect hallucinated content
74+
- **policy_guardrails**: Determine if scenario complies with policy
75+
- **query_clarification**: Generate a clarification request if needed, otherwise "CLEAR".
11976
- **query_rewrite**: Improve query formulation
77+
- **requirement_check**: Validate requirements (used by ALoraRequirement)
12078
- **uncertainty**: Estimate certainty about answering a question
121-
- **policy_guardrails**: Determine if scenario complies with policy
122-
- **guardian-core**: Safety risk detection (harm, bias, groundedness, custom criteria)
123-
- **factuality_detection**: Detect factually incorrect responses
124-
- **factuality_correction**: Correct factually incorrect responses
125-
- **context-attribution**: Identify context sentences that most influenced response
79+
80+
## Architecture
81+
![Granite Libraries Software Stack Architecture in Mellea](../../docs/images/granite-libraries-mellea-architecture.png)
12682

12783
## Related Documentation
12884

12985
- See `mellea/stdlib/components/intrinsic/` for intrinsic implementations
13086
- See `mellea/backends/adapters/` for adapter system
13187
- See `docs/dev/intrinsics_and_adapters.md` for architecture details
88+
- See `docs/docs/examples/granite-switch/README.md` for more about granite-switch

docs/examples/intrinsics/context_relevance.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
ctx, backend = start_backend(
1515
"hf", model_id=model_ids.IBM_GRANITE_4_MICRO_3B, context_type="chat"
1616
)
17-
# NOTE: this example uses Granite 4.0 micro because there is no context_relevance intrinsic for Graniet 4.1
17+
# NOTE: this example uses Granite 4.0 micro because there is no context_relevance intrinsic for Granite 4.1
1818

1919
question = "Who is the CEO of Microsoft?"
2020
document = (

0 commit comments

Comments
 (0)