Skip to content

Commit 610d43a

Browse files
m-messerm-messer
committed
Updated README
1 parent 2ed3e6a commit 610d43a

1 file changed

Lines changed: 102 additions & 46 deletions

File tree

‎readme.md‎

Lines changed: 102 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -1,96 +1,152 @@
1-
# Evaluation Function Template Repository
1+
# isExactEqual
22

3-
This template repository contains the boilerplate code needed in order to create an AWS Lambda function that can be written by any tutor to grade a response area in any way they like.
3+
[![Request Production Deploy](https://img.shields.io/badge/Request-Production_Deploy-2ea44f?style=for-the-badge)](https://github.com/lambda-feedback/IsExactEqual/issues/new?template=release-request.yml)
44

5-
This version is specifically for python, however the ultimate goal is to make similar boilerplate repositories in any language, allowing tutors the freedom to code in what they feel most comfortable with.
5+
An evaluation function that checks exact equality between a student response and the correct answer. Both inputs are cast to a specified type before comparison using Python's `==` operator. The function is deployed on the [lambda-feedback](https://github.com/lambda-feedback) platform.
66

77
## Table of Contents
8-
- [Evaluation Function Template Repository](#evaluation-function-template-repository)
8+
- [isExactEqual](#isexactequal)
99
- [Table of Contents](#table-of-contents)
1010
- [Repository Structure](#repository-structure)
1111
- [Usage](#usage)
12-
- [Getting Started](#getting-started)
12+
- [Parameters](#parameters)
13+
- [Examples](#examples)
1314
- [How it works](#how-it-works)
1415
- [Docker & Amazon Web Services (AWS)](#docker--amazon-web-services-aws)
1516
- [Middleware Functions](#middleware-functions)
1617
- [GitHub Actions](#github-actions)
18+
- [Documentation](#documentation)
1719
- [Pre-requisites](#pre-requisites)
1820
- [Contact](#contact)
1921

2022
## Repository Structure
2123

22-
```bash
24+
```
2325
app/
24-
__init__.py
25-
evaluation.py # Script containing the main evaluation_function
26-
docs.md # Documentation page for this function (required)
27-
evaluation_tests.py # Unittests for the main evaluation_function
28-
requirements.txt # list of packages needed for algorithm.py
29-
Dockerfile # for building whole image to deploy to AWS
26+
evaluation.py # Main evaluation_function
27+
evaluation_tests.py # Unit tests
28+
schema.json # JSON schema for input validation
29+
requirements.txt # Python dependencies
30+
Dockerfile # Container image for AWS Lambda
31+
docs/
32+
user.md # End-user documentation
33+
dev.md # Developer reference
3034
3135
.github/
3236
workflows/
33-
test-and-deploy.yml # Testing and deployment pipeline
37+
test-lint.yml # Run tests and linting on pull requests
38+
staging-deploy.yml # Deploy to staging on push to main
39+
production-deploy.yml # Deploy to production
3440
35-
config.json # Specify the name of the evaluation function in this file
41+
config.json # Evaluation function name
3642
.gitignore
3743
```
3844

3945
## Usage
4046

41-
### Getting Started
42-
43-
1. Clone this repository
44-
2. Change the name of the evaluation function in `config.json`
45-
3. The name must be unique. To view existing grading functions, go to:
46-
47-
- [Staging API Gateway Integrations](https://eu-west-2.console.aws.amazon.com/apigateway/main/develop/integrations/attach?api=c1o0u8se7b&region=eu-west-2&routes=0xsoy4q)
48-
- [Production API Gateway Integrations](https://eu-west-2.console.aws.amazon.com/apigateway/main/develop/integrations/attach?api=cttolq2oph&integration=qpbgva8&region=eu-west-2&routes=0xsoy4q)
47+
Send a POST request to the deployed function endpoint with the following JSON body.
48+
49+
### Parameters
50+
51+
| Field | Type | Required | Description |
52+
|---|---|---|---|
53+
| `response` | any | Yes | The student's submitted response |
54+
| `answer` | any | Yes | The correct answer |
55+
| `params.type` | string | Yes | Cast type for both inputs before comparison. One of: `"int"`, `"float"`, `"str"`, `"dict"` |
56+
| `params.display_submission_count` | boolean | No | If `true`, includes a submission count message in the feedback |
57+
| `params.submission_context.submissions_per_student_per_response_area` | integer | No | Number of previously processed submissions for this student and response area |
58+
59+
### Examples
60+
61+
**Simple string comparison:**
62+
```json
63+
{
64+
"response": "hydrophobic",
65+
"answer": "hydrophobic",
66+
"params": {
67+
"type": "str"
68+
}
69+
}
70+
```
71+
```json
72+
{
73+
"is_correct": true
74+
}
75+
```
4976

50-
4. Merge commits into the default branch
51-
- This will trigger the `test-and-deploy.yml` workflow, which will build the docker image, push it to a shared ECR repository, then call the backend `grading-function/ensure` route to build the necessary infrastructure to make the function available from the client app.
77+
**Integer comparison:**
78+
```json
79+
{
80+
"response": "45",
81+
"answer": "45",
82+
"params": {
83+
"type": "int"
84+
}
85+
}
86+
```
87+
```json
88+
{
89+
"is_correct": true
90+
}
91+
```
5292

53-
5. You are now ready to start developing your function:
54-
55-
- Edit the `app/evaluation.py` file, which ultimately gets called when the function is given the `eval` command
56-
- Edit the `app/evaluation_tests.py` file to add tests which get run:
57-
- Every time you commit to this repo, before the image is built and deployed
58-
- Whenever the `healthcheck` command is supplied to the deployed function
59-
- Edit the `app/docs.md` file to reflect your changes. This file is baked into the function's image, and is made available using the `docs` command. This feature is used to display this function's documentation on our [Documentation](https://lambda-feedback.github.io/Documentation/) website once it's been hooked up!
93+
**With submission count feedback:**
94+
```json
95+
{
96+
"response": "1",
97+
"answer": "1",
98+
"params": {
99+
"type": "int",
100+
"display_submission_count": true,
101+
"submission_context": {
102+
"submissions_per_student_per_response_area": 2
103+
}
104+
}
105+
}
106+
```
107+
```json
108+
{
109+
"is_correct": true,
110+
"feedback": "You have submitted 3 responses."
111+
}
112+
```
60113

61114
---
62115

63116
## How it works
64117

65-
The function is built on top of a custom base layer, [BaseEvaluationFunctionLayer](https://github.com/lambda-feedback/BaseEvalutionFunctionLayer), which tools, tests and schema checking relevant to all evaluation functions.
118+
The function is built on top of a custom base layer, [BaseEvaluationFunctionLayer](https://github.com/lambda-feedback/BaseEvalutionFunctionLayer), which provides tooling, testing helpers, and schema checking common to all evaluation functions.
66119

67120
### Docker & Amazon Web Services (AWS)
68121

69-
The grading scripts are hosted AWS Lambda, using containers to run a docker image of the app. Docker is a popular tool in software development that allows programs to be hosted on any machine by bundling all its requirements and dependencies into a single file called an **image**.
70-
71-
Images are run within **containers** on AWS, which give us a lot of flexibility over what programming language and packages/libraries can be used. For more information on Docker, read this [introduction to containerisation](https://www.freecodecamp.org/news/a-beginner-friendly-introduction-to-containers-vms-and-docker-79a9e3e119b/). To learn more about AWS Lambda, click [here](https://geekflare.com/aws-lambda-for-beginners/).
122+
The evaluation function is hosted on AWS Lambda and packaged as a Docker container. Docker bundles the function and all its dependencies into a single image, which AWS runs on demand inside a container. For background reading, see this [introduction to containerisation](https://www.freecodecamp.org/news/a-beginner-friendly-introduction-to-containers-vms-and-docker-79a9e3e119b/) and the [AWS Lambda documentation](https://docs.aws.amazon.com/lambda/latest/dg/welcome.html).
72123

73124
### Middleware Functions
74-
In order to run the algorithm and schema on AWS Lambda, some middleware functions have been provided to handle, validate and return the data so all you need to worry about is the evaluation script and testing.
75125

76-
The code needed to build the image using all the middleware functions are available in the [BaseEvaluationFunctionLayer](https://github.com/lambda-feedback/BaseEvalutionFunctionLayer) repository.
126+
Middleware provided by [BaseEvaluationFunctionLayer](https://github.com/lambda-feedback/BaseEvalutionFunctionLayer) handles request validation, error formatting, and response serialisation. The `evaluation.py` file only needs to implement the core comparison logic.
77127

78128
### GitHub Actions
79-
Whenever a commit is made to the GitHub repository, the new code will go through a pipeline, where it will be tested for syntax errors and code coverage. The pipeline used is called **GitHub Actions** and the scripts for these can be found in `.github/workflows/`.
80129

81-
On top of that, when starting a new evaluation function, you will have to complete a set of unit test scripts, which not only make sure your code is reliable, but also helps you to build a _specification_ for how the code should function before you start programming.
130+
Three pipelines are configured in `.github/workflows/`:
82131

83-
Once the code passes all these tests, it will then be uploaded to AWS and will be deployed and ready to go in only a few minutes.
132+
- **`test-lint.yml`** — runs on every pull request: executes the unit test suite and linting via `flake8`.
133+
- **`staging-deploy.yml`** — runs on every push to `main`: re-runs tests, then builds the Docker image, pushes it to the shared ECR repository, and deploys to the staging environment.
134+
- **`production-deploy.yml`** — promotes a tested build to the production environment.
84135

85-
## Pre-requisites
86-
Although all programming can be done through the GitHub interface, it is recommended you do this locally on your machine. To do this, you must have installed:
136+
## Documentation
87137

88-
- Python 3.8 or higher.
138+
- [`app/docs/user.md`](app/docs/user.md) — end-user guide explaining inputs and parameters
139+
- [`app/docs/dev.md`](app/docs/dev.md) — developer reference with detailed input/output specs and examples
89140

90-
- GitHub Desktop or the `git` CLI.
141+
## Pre-requisites
91142

92-
- A code editor such as Atom, VS Code, or Sublime.
143+
To develop and test locally:
93144

94-
Copy this template over by clicking **Use this template** button found in the repository on GitHub. Save it to the `lambda-feedback` Organisation.
145+
- Python 3.8 or higher
146+
- Docker
147+
- `git` CLI or GitHub Desktop
148+
- A code editor (VS Code, PyCharm, etc.)
95149

96150
## Contact
151+
152+
For questions or issues, open a GitHub issue or visit the [lambda-feedback organisation](https://github.com/lambda-feedback).

0 commit comments

Comments
 (0)