Skip to content

Commit 6437438

Browse files
neagualexaneagualexa
authored
Chatbots info for teachers and students (#33)
* descriptions of the chatbots for teacher and students (todo: example chats) * example responses + chats term * minor changes * update button * instructions teachers to turn on/off chatbots * restructure to avoid repetition
1 parent b6d7e83 commit 6437438

11 files changed

Lines changed: 338 additions & 67 deletions

File tree

‎docs/advanced/chat_functions/info.md‎

Lines changed: 40 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,38 +1,57 @@
11
# Chat Functions - More information
22

3-
Chat functions are the microservices that Lambda Feedback calls to provide the underlying functionality of a chatbot. Students can chat with the chatbots and ask for help or further explanations regarding the Question that they are working on. Each chatbot has its own personality and approach to assisting the students.
3+
Chat functions are the microservices that Lambda Feedback calls to provide the underlying functionality of a chatbot. Each chat function connects to a [Large Language Model (LLM)](https://uit.stanford.edu/service/techtraining/ai-demystified/llm) configured with its own role prompt, giving each chatbot its own personality and approach to assisting students.
44

5-
The chatbots have at their basis a [Large Language Model (LLM)](https://en.wikipedia.org/wiki/Large_language_model) which received information regarding:
5+
For a higher-level description of the chatbots currently available and the context each one is given about the student and question, see the [Chatbots guide for teachers](../../teacher/guides/chatbots.md) (or the [student-facing version](../../student/chatbots.md)).
66

7-
- The raw markdown content of the question the student is on currently, including:
8-
- The question name, number and content
9-
- The final answer, structured tutorial, and worked solutions of the question
10-
- The guidance (blurb and time estimate) from the teacher for the question
11-
- The set name, number and description
12-
- All parts with their number, content and done status (current part emphasised)
13-
- All response areas and their respective expected answers
14-
- The progress of the student on all parts of the Question, including:
15-
- The total number of responses and the number of wrong responses the student has made for each response area
16-
- The last responses the student has made for each response area and the received feedback
17-
- The time duration the student has spent on the respective question and current part on that day
7+
## Deployed chatbots
8+
<!-- TODO: This information needs to be autoloaded in a similar manner to evaluation function documentation -->
189

19-
---
10+
The role prompts and key behaviours of the chatbots currently deployed on Lambda Feedback are described below.
2011

21-
## Available Chat functions
12+
### Informational Chatbot
2213

23-
Currently the students have access to the following chat functions that host their own specific chatbot. Many others are in development.
14+
**Role:** A patient AI tutor focused on student-centred learning — aims to foster critical thinking, active engagement, and confidence-building.
2415

25-
Click on the links below for information on each chatbot:
16+
**Key behaviours from the role prompt**
2617

27-
[1. Informational Chatbot](https://github.com/lambda-feedback/informationalChatFunction/blob/main/docs/user.md)
18+
- **Step-by-step guidance:** breaks problems into smaller steps and offers hints or intermediate steps before giving the final answer. Will share the complete answer if the student explicitly asks, but only after first encouraging exploration.
19+
- **Error reflection:** treats mistakes as opportunities — helps the student work out *why* something went wrong rather than silently correcting it.
20+
- **Awareness of materials:** grounds responses in the question's content, answer, worked solution, and the teacher's guidance — paraphrasing rather than quoting verbatim.
21+
- **Adaptive support:** if the student keeps struggling, evaluates their progress and time spent on the question and gradually offers more detailed and specific guidance.
22+
- **Engagement:** ends interactions with a question to keep the dialogue going and gauge comprehension.
23+
- **Praise** is reserved for genuine effort or breakthroughs to avoid sounding insincere.
24+
- **Stays on topic:** politely redirects students who ask about unrelated material.
2825

26+
[More details and source →](https://github.com/lambda-feedback/informationalChatFunction)
2927

30-
[2. Concise Chatbot](https://github.com/lambda-feedback/conciseChatFunction/blob/main/docs/user.md)
28+
### Concise Chatbot
3129

30+
**Role:** A tutor that gives short, direct answers.
3231

33-
[3. Reflective Chatbot](https://github.com/lambda-feedback/reflectiveChatFunction/blob/main/docs/user.md)
32+
**Key behaviours from the role prompt**
3433

34+
- **Direct and minimal:** answers the question and stops — no extra details, explanations, or examples unless the student asks.
35+
- **Aware of struggle:** if the student seems stuck or frustrated, references their progress so far and how long they have spent on the question relative to the teacher's guidance time.
36+
- **Stays on topic:** redirects unrelated questions back to the current material with a short refusal.
37+
- **No filler:** does not end messages with concluding statements or summaries.
38+
39+
[More details and source →](https://github.com/lambda-feedback/conciseChatFunction)
40+
41+
### Reflective Chatbot
42+
43+
**Role:** A Socratic tutor that guides students to discover knowledge through questioning rather than direct instruction.
44+
45+
**Key behaviours from the role prompt**
46+
47+
- **Always ends with a question:** every response finishes with a follow-up question that pushes the student's thinking forward.
48+
- **Counter-questions over answers:** when a student asks a direct question, responds with a question that guides them toward the answer rather than handing it over. If it does share a fact, it immediately follows with a question that asks the student to apply or extend it.
49+
- **Uses a varied question toolkit:** clarifying ("What do you mean by…?"), assumption-probing ("What are you assuming here?"), evidence-based, perspective, implication, and meta-questions about why a question matters.
50+
- **Diagnoses where the student is stuck:** if a student is frustrated, asks about their thought process to locate the gap, drawing on their progress and time spent.
51+
- **Never provides complete answers:** always leaves room for the student to think and respond.
52+
53+
[More details and source →](https://github.com/lambda-feedback/reflectiveChatFunction)
3554

3655
## Chat Function Development
3756

38-
Are you interested in developing your own chatbot? Then check out the [Quickstart guide](quickstart.md) to develop and deploy your own AI chat function for Lambda Feedback.
57+
Are you interested in developing your own chatbot? Check out the [Quickstart guide](quickstart.md) to develop and deploy your own AI chat function for Lambda Feedback.

‎docs/advanced/chat_functions/local.md‎

Lines changed: 110 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,19 @@
11
# Running and Testing Chat function Locally
22

3-
You can run the Python function for your chat function itself by writing a `main()` function, or you can call the [`testbench_prompts.py`](https://github.com/lambda-feedback/lambda-chat/blob/main/src/agents/utils/testbench_prompts.py) script that runs a similar pipeline to the `module.py`.
3+
## Run Unit Tests
4+
5+
You can run the unit tests using `pytest`:
46

57
```bash
6-
python src/agents/utils/testbench_prompts.py
8+
pytest
79
```
810

9-
You can also use the `test_prompts.py` script to test the chat function with example inputs from Lambda Feedback questions and synthetic conversations.
11+
## Run the Chat Script
12+
13+
You can use the `manual_agent_run.py` script to test the agents with example inputs from Lambda Feedback questions and synthetic chats:
14+
1015
```bash
11-
python src/agents/utils/test_prompts.py
16+
python tests/manual_agent_run.py
1217
```
1318

1419
## Testing using the Docker Image [:material-docker:](https://www.docker.com/)
@@ -44,35 +49,121 @@ This will start the evaluation function and expose it on port `8080` and it will
4449
```bash
4550
curl --location 'http://localhost:8080/2015-03-31/functions/function/invocations' \
4651
--header 'Content-Type: application/json' \
47-
--data '{"body":"{\"message\": \"hi\", \"params\": {\"conversation_id\": \"12345Test\", \"conversation_history\": [{\"type\": \"user\", \"content\": \"hi\"}]}}"}'
52+
--data '{"body":"{\"conversationId\": \"12345Test\", \"messages\": [{\"role\": \"USER\", \"content\": \"hi\"}], \"user\": {\"type\": \"LEARNER\"}}"}'
53+
```
54+
55+
### Call Docker Container
56+
57+
#### A. Call Docker with Python Requests
58+
59+
In the `tests/` folder you can find the `manual_agent_requests.py` script that calls the POST URL of the running Docker container. It reads input files matching the expected schema, so you can use it to validate your chatbot end-to-end.
60+
61+
```bash
62+
python tests/manual_agent_requests.py
4863
```
4964

50-
### Call Docker Container From Postman
65+
#### B. Call Docker Container From Postman
5166

5267
POST URL:
5368

5469
```bash
5570
http://localhost:8080/2015-03-31/functions/function/invocations
5671
```
5772

58-
Body:
73+
Body (stringified within `body` for the API request):
5974

6075
```JSON
61-
{"body":"{\"message\": \"hi\", \"params\": {\"conversation_id\": \"12345Test\", \"conversation_history\": [{\"type\": \"user\", \"content\": \"hi\"}]}}"}
76+
{"body":"{\"conversationId\": \"12345Test\", \"messages\": [{\"role\": \"USER\", \"content\": \"hi\"}], \"user\": {\"type\": \"LEARNER\"}}"}
6277
```
6378

64-
Body with optional Params:
65-
```JSON
79+
Input Body with optional fields:
80+
```json
6681
{
67-
"message":"hi",
68-
"params":{
69-
"conversation_id":"12345Test",
70-
"conversation_history":[{"type":"user","content":"hi"}],
71-
"summary":" ",
72-
"conversational_style":" ",
73-
"question_response_details": "",
74-
"include_test_data": true,
75-
"agent_type": {agent_name}
82+
"conversationId": "<uuid>",
83+
"messages": [
84+
{ "role": "USER", "content": "<previous user message>" },
85+
{ "role": "ASSISTANT", "content": "<previous assistant reply>" },
86+
{ "role": "USER", "content": "<current message>" }
87+
],
88+
"user": {
89+
"type": "LEARNER",
90+
"preference": {
91+
"conversationalStyle": "<stored style string>"
92+
},
93+
"taskProgress": {
94+
"timeSpentOnQuestion": "30 minutes",
95+
"accessStatus": "a good amount of time spent on this question today.",
96+
"markedDone": "This question is still being worked on.",
97+
"currentPart": {
98+
"position": 0,
99+
"timeSpentOnPart": "10 minutes",
100+
"markedDone": "This part is not marked done.",
101+
"responseAreas": [
102+
{
103+
"responseType": "EXPRESSION",
104+
"totalSubmissions": 3,
105+
"wrongSubmissions": 2,
106+
"latestSubmission": {
107+
"submission": "<student's last answer>",
108+
"feedback": "<feedback text from evaluator>",
109+
"answer": "<reference answer used for evaluation>"
110+
}
111+
}
112+
]
113+
}
76114
}
115+
},
116+
"context": {
117+
"summary": "<compressed chat history>",
118+
"set": {
119+
"title": "Fundamentals",
120+
"number": 2,
121+
"description": "<set description>"
122+
},
123+
"question": {
124+
"title": "Understanding Polymorphism",
125+
"number": 3,
126+
"guidance": "<teacher guidance>",
127+
"content": "<master question content>",
128+
"estimatedTime": "15-25 minutes",
129+
"parts": [
130+
{
131+
"position": 0,
132+
"content": "<part prompt>",
133+
"answerContent": "<part answer>",
134+
"workedSolutionSections": [
135+
{ "position": 0, "title": "Step 1", "content": "..." }
136+
],
137+
"structuredTutorialSections": [
138+
{ "position": 0, "title": "Hint", "content": "..." }
139+
],
140+
"responseAreas": [
141+
{
142+
"position": 0,
143+
"responseType": "EXPRESSION",
144+
"answer": "<reference answer>",
145+
"preResponseText": "<label shown before input>"
146+
}
147+
]
148+
}
149+
]
150+
}
151+
}
152+
}
153+
```
154+
155+
Output Response:
156+
157+
```json
158+
{
159+
"output": {
160+
"role": "ASSISTANT",
161+
"content": "<assistant reply text>"
162+
},
163+
"metadata": {
164+
"summary": "<updated chat summary>",
165+
"conversationalStyle": "<updated style string>",
166+
"processingTimeMs": 1234
167+
}
77168
}
78169
```

‎docs/advanced/chat_functions/quickstart.md‎

Lines changed: 89 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -23,31 +23,96 @@ Chat functions host a chatbot. Chatbots capture and automate the process of assi
2323

2424
- the chat function expects the following arguments when it being called:
2525

26-
Body with necessary Params:
26+
Body with necessary fields:
2727

2828
```JSON
2929
{
30-
"message":"hi",
31-
"params":{
32-
"conversation_id":"12345Test",
33-
"conversation_history": [{"type":"user","content":"hi"}]
30+
"conversationId": "12345Test",
31+
"messages": [{ "role": "USER", "content": "hi" }],
32+
"user": { "type": "LEARNER" }
33+
}
34+
```
35+
36+
Body with optional fields:
37+
38+
```JSON
39+
{
40+
"conversationId": "12345Test",
41+
"messages": [
42+
{ "role": "USER", "content": "<previous user message>" },
43+
{ "role": "ASSISTANT", "content": "<previous assistant reply>" },
44+
{ "role": "USER", "content": "hi" }
45+
],
46+
"user": {
47+
"type": "LEARNER",
48+
"preference": { "conversationalStyle": "<stored style string>" },
49+
"taskProgress": {
50+
"timeSpentOnQuestion": "30 minutes",
51+
"accessStatus": "a good amount of time spent on this question today.",
52+
"markedDone": "This question is still being worked on.",
53+
"currentPart": {
54+
"position": 0,
55+
"timeSpentOnPart": "10 minutes",
56+
"markedDone": "This part is not marked done.",
57+
"responseAreas": [
58+
{
59+
"responseType": "EXPRESSION",
60+
"totalSubmissions": 3,
61+
"wrongSubmissions": 2,
62+
"latestSubmission": {
63+
"submission": "<student's last answer>",
64+
"feedback": "<feedback text from evaluator>",
65+
"answer": "<reference answer used for evaluation>"
66+
}
67+
}
68+
]
69+
}
70+
}
71+
},
72+
"context": {
73+
"summary": "<compressed chat history>",
74+
"set": { "title": "Fundamentals", "number": 2, "description": "<set description>" },
75+
"question": {
76+
"title": "Understanding Polymorphism",
77+
"number": 3,
78+
"guidance": "<teacher guidance>",
79+
"content": "<master question content>",
80+
"estimatedTime": "15-25 minutes",
81+
"parts": [
82+
{
83+
"position": 0,
84+
"content": "<part prompt>",
85+
"answerContent": "<part answer>",
86+
"workedSolutionSections": [
87+
{ "position": 0, "title": "Step 1", "content": "..." }
88+
],
89+
"structuredTutorialSections": [
90+
{ "position": 0, "title": "Hint", "content": "..." }
91+
],
92+
"responseAreas": [
93+
{
94+
"position": 0,
95+
"responseType": "EXPRESSION",
96+
"answer": "<reference answer>",
97+
"preResponseText": "<label shown before input>"
98+
}
99+
]
100+
}
101+
]
102+
}
34103
}
35104
}
36105
```
37106

38-
Body with optional Params:
107+
Expected response:
39108

40109
```JSON
41110
{
42-
"message":"hi",
43-
"params":{
44-
"conversation_id":"12345Test",
45-
"conversation_history":[{"type":"user","content":"hi"}],
46-
"summary":" ",
47-
"conversational_style":" ",
48-
"question_response_details": "",
49-
"include_test_data": true,
50-
"agent_type": {agent_name}
111+
"output": { "role": "ASSISTANT", "content": "<assistant reply text>" },
112+
"metadata": {
113+
"summary": "<updated chat summary>",
114+
"conversationalStyle": "<updated style string>",
115+
"processingTimeMs": 1234
51116
}
52117
}
53118
```
@@ -62,7 +127,7 @@ Chat functions host a chatbot. Chatbots capture and automate the process of assi
62127

63128
4. Changes can be tested locally by running the pipeline tests using:
64129
```bash
65-
pytest src/module_test.py
130+
pytest
66131
```
67132
[Running and Testing Chat Functions Locally](local.md){ .md-button }
68133

@@ -78,16 +143,14 @@ Chat functions host a chatbot. Chatbots capture and automate the process of assi
78143
curl --location 'https://<***>.execute-api.eu-west-2.amazonaws.com/default/chat/chatFunctionBoilerplate-dev' \
79144
--header 'Content-Type: application/json' \
80145
--data '{
81-
"message": "hi",
82-
"params": {
83-
"conversation_id": "12345Test",
84-
"conversation_history": [
85-
{
86-
"type": "user",
87-
"content": "hi"
88-
}
89-
]
90-
}
146+
"conversationId": "12345Test",
147+
"messages": [
148+
{
149+
"role": "USER",
150+
"content": "hi"
151+
}
152+
],
153+
"user": { "type": "LEARNER" }
91154
}'
92155

93156
7. Once the `dev` chat function is fully tested, you can merge the code to the default branch (`main`). This will trigger the `main.yml` workflow, which will deploy the `staging` and `prod` versions of your chat function. Please contact the ADMIN to provide you the URLS for the `staging` and `prod` versions of your chat function.

0 commit comments

Comments
 (0)