Skip to content

Commit 08ef19c

Browse files
neagualexaneagualexa
committed
refactoring for mued with simlified structure and better testing
1 parent 441984c commit 08ef19c

21 files changed

+636
-991
lines changed

‎AGENTS.md‎

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# AGENTS.md
2+
3+
This file provides guidance to AI agents when working with code in this repository.
4+
5+
## Project Overview
6+
7+
This is a boilerplate for creating AI educational chatbots that integrate with the **Lambda-Feedback** educational platform. It deploys as an AWS Lambda function (containerized via Docker) that receives student chat messages with educational context and returns LLM-powered chatbot responses.
8+
9+
## Commands
10+
11+
**Testing:**
12+
```bash
13+
pytest # Run all unit tests
14+
python tests/manual_agent_run.py # Test agent locally with example inputs
15+
python tests/manual_agent_requests.py # Test running Docker container
16+
```
17+
18+
**Docker:**
19+
```bash
20+
docker build -t llm_chat .
21+
docker run --env-file .env -p 8080:8080 llm_chat
22+
```
23+
24+
**Manual API test (while Docker is running):**
25+
```bash
26+
curl -X POST http://localhost:8080/2015-03-31/functions/function/invocations \
27+
-H 'Content-Type: application/json' \
28+
-d '{"body":"{\"conversationId\": \"12345Test\", \"messages\": [{\"role\": \"USER\", \"content\": \"hi\"}], \"user\": {\"type\": \"LEARNER\"}}"}'
29+
```
30+
31+
**Run a single test:**
32+
```bash
33+
pytest tests/test_module.py # Run specific test file
34+
pytest tests/test_index.py::test_function_name # Run specific test
35+
```
36+
37+
## Architecture
38+
39+
### Request Flow
40+
41+
```
42+
Lambda event → index.py (handler)
43+
→ validates via lf_toolkit ChatRequest schema
44+
→ src/module.py (chat_module)
45+
→ extracts muEd API context (messages, conversationId, question context, user type)
46+
→ parses educational context to prompt text via src/agent/context.py
47+
→ src/agent/agent.py (BaseAgent / LangGraph)
48+
→ routes to call_llm or summarize_conversation node
49+
→ calls LLM provider (OpenAI / Google / Azure / Ollama)
50+
→ returns ChatResponse (output, summary, conversationalStyle, processingTime)
51+
```
52+
53+
### Key Files
54+
55+
| File | Role |
56+
|------|------|
57+
| `index.py` | AWS Lambda entry point; parses event body, validates schema |
58+
| `src/module.py` | Transforms muEd API request → invokes agent → builds ChatResponse |
59+
| `src/agent/agent.py` | LangGraph stateful graph; manages message history and summarization |
60+
| `src/agent/prompts.py` | System prompts for tutor behavior, summarization, style detection |
61+
| `src/agent/llm_factory.py` | Factory classes for each LLM provider (OpenAI, Google, Azure, Ollama) |
62+
| `src/agent/context.py` | Converts muEd question/submission context dicts to LLM prompt text |
63+
| `tests/utils.py` | Shared test helpers: `assert_valid_chat_request`, `assert_valid_chat_response` |
64+
| `tests/example_inputs/` | Real muEd payloads used for end-to-end tests |
65+
66+
### Agent Logic (LangGraph)
67+
68+
`BaseAgent` maintains a state graph with two nodes:
69+
- **`call_llm`**: Invokes the LLM with system prompt + conversation summary + conversational style preference
70+
- **`summarize_conversation`**: Triggered when message count exceeds ~11; summarizes history and also extracts the student's preferred conversational style
71+
72+
Messages are trimmed after summarization to keep context window manageable. The `summary` and `conversationalStyle` fields persist across calls via the `ChatRequest` metadata.
73+
74+
### muEd API Format
75+
76+
`src/module.py` handles the muEd request format (https://mued.org/). The `context` field in `ChatRequest` contains nested educational data (question parts, student submissions, task info) that gets parsed into a tutoring prompt via `src/agent/context.py`.
77+
78+
### LLM Configuration
79+
80+
LLM provider and model are set via environment variables (see `.env.example`). The `llm_factory.py` selects the provider at runtime. The Lambda function name/identity is set in `config.json`.
81+
82+
The agent uses **two separate LLM instances** — `self.llm` for chat responses and `self.summarisation_llm` for conversation summarisation and style analysis. By default both use the same provider, but you can point them at different models (e.g. a cheaper model for summarisation) by changing the class in `agent.py`.
83+
84+
## Deployment
85+
86+
- Pushing to `dev` branch triggers the dev deployment GitHub Actions workflow
87+
- Pushing to `main` triggers staging deployment, with manual approval required for production
88+
- All environment variables (API keys, model names) are injected via GitHub Actions secrets/variables — do not hardcode them

‎CLAUDE.md‎

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Project Overview
6+
7+
This is a boilerplate for creating AI educational chatbots that integrate with the **Lambda-Feedback** educational platform. It deploys as an AWS Lambda function (containerized via Docker) that receives student chat messages with educational context and returns LLM-powered chatbot responses.
8+
9+
## Commands
10+
11+
**Testing:**
12+
```bash
13+
pytest # Run all unit tests
14+
python tests/manual_agent_run.py # Test agent locally with example inputs
15+
python tests/manual_agent_requests.py # Test running Docker container
16+
```
17+
18+
**Docker:**
19+
```bash
20+
docker build -t llm_chat .
21+
docker run --env-file .env -p 8080:8080 llm_chat
22+
```
23+
24+
**Manual API test (while Docker is running):**
25+
```bash
26+
curl -X POST http://localhost:8080/2015-03-31/functions/function/invocations \
27+
-H 'Content-Type: application/json' \
28+
-d '{"body":"{\"conversationId\": \"12345Test\", \"messages\": [{\"role\": \"USER\", \"content\": \"hi\"}], \"user\": {\"type\": \"LEARNER\"}}"}'
29+
```
30+
31+
**Run a single test:**
32+
```bash
33+
pytest tests/test_module.py # Run specific test file
34+
pytest tests/test_index.py::test_function_name # Run specific test
35+
```
36+
37+
## Architecture
38+
39+
### Request Flow
40+
41+
```
42+
Lambda event → index.py (handler)
43+
→ validates via lf_toolkit ChatRequest schema
44+
→ src/module.py (chat_module)
45+
→ extracts muEd API context (messages, conversationId, question context, user type)
46+
→ parses educational context to prompt text via src/agent/context.py
47+
→ src/agent/agent.py (BaseAgent / LangGraph)
48+
→ routes to call_llm or summarize_conversation node
49+
→ calls LLM provider (OpenAI / Google / Azure / Ollama)
50+
→ returns ChatResponse (output, summary, conversationalStyle, processingTime)
51+
```
52+
53+
### Key Files
54+
55+
| File | Role |
56+
|------|------|
57+
| `index.py` | AWS Lambda entry point; parses event body, validates schema |
58+
| `src/module.py` | Transforms muEd API request → invokes agent → builds ChatResponse |
59+
| `src/agent/agent.py` | LangGraph stateful graph; manages message history and summarization |
60+
| `src/agent/prompts.py` | System prompts for tutor behavior, summarization, style detection |
61+
| `src/agent/llm_factory.py` | Factory classes for each LLM provider (OpenAI, Google, Azure, Ollama) |
62+
| `src/agent/context.py` | Converts muEd question/submission context dicts to LLM prompt text |
63+
| `tests/utils.py` | Shared test helpers: `assert_valid_chat_request`, `assert_valid_chat_response` |
64+
| `tests/example_inputs/` | Real muEd payloads used for end-to-end tests |
65+
66+
### Agent Logic (LangGraph)
67+
68+
`BaseAgent` maintains a state graph with two nodes:
69+
- **`call_llm`**: Invokes the LLM with system prompt + conversation summary + conversational style preference
70+
- **`summarize_conversation`**: Triggered when message count exceeds ~11; summarizes history and also extracts the student's preferred conversational style
71+
72+
Messages are trimmed after summarization to keep context window manageable. The `summary` and `conversationalStyle` fields persist across calls via the `ChatRequest` metadata.
73+
74+
### muEd API Format
75+
76+
`src/module.py` handles the muEd request format (https://mued.org/). The `context` field in `ChatRequest` contains nested educational data (question parts, student submissions, task info) and the `user` field contains user-specific information (e.g., user type, preferences, task progress) that gets parsed into a tutoring prompt via `src/agent/context.py`.
77+
78+
### LLM Configuration
79+
80+
LLM provider and model are set via environment variables (see `.env.example`). The `llm_factory.py` selects the provider at runtime. The Lambda function name/identity is set in `config.json`.
81+
82+
The agent uses **two separate LLM instances** — `self.llm` for chat responses and `self.summarisation_llm` for conversation summarisation and style analysis. By default both use the same provider, but you can point them at different models (e.g. a cheaper model for summarisation) by changing the class in `agent.py`.
83+
84+
## Deployment
85+
86+
- Pushing to `dev` branch triggers the dev deployment GitHub Actions workflow
87+
- Pushing to `main` triggers staging deployment, with manual approval required for production
88+
- All environment variables (API keys, model names) are injected via GitHub Actions secrets/variables — do not hardcode them

‎README.md‎

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,9 @@ Also, don't forget to update or delete the Quickstart chapter from the `README.m
8080

8181
You can create your own invocation to your own agents hosted anywhere. Copy or update the `agent.py` from `src/agent/` and edit it to match your LLM agent requirements. Import the new invocation in the `module.py` file.
8282

83-
You agent can be based on an LLM hosted anywhere, you have available currently OpenAI, AzureOpenAI, and Ollama models but you can introduce your own API call in the `src/agent/utils/llm_factory.py`.
83+
Your agent can be based on an LLM hosted anywhere. OpenAI, Google AI, Azure OpenAI, and Ollama are available out of the box via `src/agent/llm_factory.py`, and you can add your own provider there too.
84+
85+
The agent uses **two separate LLM instances** — `self.llm` for chat responses and `self.summarisation_llm` for conversation summarisation and style analysis. By default both use the same provider, but you can point them at different models (e.g. a cheaper or faster model for summarisation) by changing the class in `agent.py`.
8486

8587
### Prerequisites
8688

@@ -98,13 +100,16 @@ You agent can be based on an LLM hosted anywhere, you have available currently O
98100
├── docs/ # docs for devs and users
99101
├── src/
100102
│ ├── agent/
101-
│ │ ├── utils/ # utils for the agent, including the llm_factory
102-
│ │ ├── agent.py # the agent logic
103-
│ │ └── prompts.py # the system prompts defining the behaviour of the chatbot
104-
│ └── module.py
103+
│ │ ├── agent.py # LangGraph stateful agent logic
104+
│ │ ├── context.py # converts muEd context dicts to LLM prompt text
105+
│ │ ├── llm_factory.py # factory classes for each LLM provider
106+
│ │ └── prompts.py # system prompts defining the behaviour of the chatbot
107+
│ └── module.py
105108
└── tests/ # contains all tests for the chat function
109+
├── example_inputs/ # muEd example payloads for end-to-end tests
106110
├── manual_agent_requests.py # allows testing of the docker container through API requests
107111
├── manual_agent_run.py # allows testing of any LLM agent on a couple of example inputs
112+
├── utils.py # shared test helpers
108113
├── test_example_inputs.py # pytests for the example input files
109114
├── test_index.py # pytests
110115
└── test_module.py # pytests

‎docs/dev.md‎

Lines changed: 89 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
*Brief description of what this chat function does, from the developer perspective*
33

44
## Inputs
5-
*Specific input parameters which can be supplied when the calling this chat function.*
5+
*Specific input parameters which can be supplied when calling this chat function.*
66

77
## Outputs
88
*Output schema/values for this function*
@@ -24,12 +24,6 @@ pytest
2424

2525
### Run the Chat Script
2626

27-
You can run the Python function itself. Make sure to have a main function in either `src/module.py` or `index.py`.
28-
29-
```bash
30-
python src/module.py
31-
```
32-
3327
You can also use the `manual_agent_run.py` script to test the agents with example inputs from Lambda Feedback questions and synthetic conversations.
3428
```bash
3529
python tests/manual_agent_run.py
@@ -64,7 +58,7 @@ This will start the chat function and expose it on port `8080` and it will be op
6458
```bash
6559
curl --location 'http://localhost:8080/2015-03-31/functions/function/invocations' \
6660
--header 'Content-Type: application/json' \
67-
--data '{"body":"{\"message\": \"hi\", \"params\": {\"conversation_id\": \"12345Test\", \"conversation_history\": [{\"type\": \"user\",
61+
--data '{"body":"{\"conversationId\": \"12345Test\", \"messages\": [{\"role\": \"USER\", \"content\": \"hi\"}], \"user\": {\"type\": \"LEARNER\"}}"}'
6862
```
6963

7064
#### Call Docker Container
@@ -83,21 +77,97 @@ http://localhost:8080/2015-03-31/functions/function/invocations
8377
Body (stringified within body for API request):
8478

8579
```JSON
86-
{"body":"{\"message\": \"hi\", \"params\": {\"conversation_id\": \"12345Test\", \"conversation_history\": [{\"type\": \"user\", \"content\": \"hi\"}]}}"}
80+
{"body":"{\"conversationId\": \"12345Test\", \"messages\": [{\"role\": \"USER\", \"content\": \"hi\"}], \"user\": {\"type\": \"LEARNER\"}}"}
8781
```
8882

89-
Body with optional Params:
90-
```JSON
83+
Body with optional fields:
84+
```json
9185
{
92-
"message":"hi",
93-
"params":{
94-
"conversation_id":"12345Test",
95-
"conversation_history":[{"type":"user","content":"hi"}],
96-
"summary":" ",
97-
"conversational_style":" ",
98-
"question_response_details": "",
99-
"include_test_data": true,
86+
"conversationId": "<uuid>",
87+
"messages": [
88+
{ "role": "USER", "content": "<previous user message>" },
89+
{ "role": "ASSISTANT", "content": "<previous assistant reply>" },
90+
{ "role": "USER", "content": "<current message>" }
91+
],
92+
"user": {
93+
"type": "LEARNER",
94+
"preference": {
95+
"conversationalStyle": "<stored style string>"
96+
},
97+
"taskProgress": {
98+
"timeSpentOnQuestion": "30 minutes",
99+
"accessStatus": "a good amount of time spent on this question today.",
100+
"markedDone": "This question is still being worked on.",
101+
"currentPart": {
102+
"position": 0,
103+
"timeSpentOnPart": "10 minutes",
104+
"markedDone": "This part is not marked done.",
105+
"responseAreas": [
106+
{
107+
"responseType": "EXPRESSION",
108+
"totalSubmissions": 3,
109+
"wrongSubmissions": 2,
110+
"latestSubmission": {
111+
"submission": "<student's last answer>",
112+
"feedback": "<feedback text from evaluator>",
113+
"answer": "<reference answer used for evaluation>"
114+
}
115+
}
116+
]
117+
}
100118
}
119+
},
120+
"context": {
121+
"summary": "<compressed conversation history>",
122+
"set": {
123+
"title": "Fundamentals",
124+
"number": 2,
125+
"description": "<set description>"
126+
},
127+
"question": {
128+
"title": "Understanding Polymorphism",
129+
"number": 3,
130+
"guidance": "<teacher guidance>",
131+
"content": "<master question content>",
132+
"estimatedTime": "15-25 minutes",
133+
"parts": [
134+
{
135+
"position": 0,
136+
"content": "<part prompt>",
137+
"answerContent": "<part answer>",
138+
"workedSolutionSections": [
139+
{ "position": 0, "title": "Step 1", "content": "..." }
140+
],
141+
"structuredTutorialSections": [
142+
{ "position": 0, "title": "Hint", "content": "..." }
143+
],
144+
"responseAreas": [
145+
{
146+
"position": 0,
147+
"responseType": "EXPRESSION",
148+
"answer": "<reference answer>",
149+
"preResponseText": "<label shown before input>"
150+
}
151+
]
152+
}
153+
]
154+
}
155+
}
101156
}
102157
```
103158

159+
Response:
160+
161+
```json
162+
{
163+
"output": {
164+
"role": "ASSISTANT",
165+
"content": "<assistant reply text>"
166+
},
167+
"metadata": {
168+
"summary": "<updated conversation summary>",
169+
"conversationalStyle": "<updated style string>",
170+
"processingTimeMs": 1234
171+
}
172+
}
173+
```

0 commit comments

Comments
 (0)