Skip to content

Commit e526f05

Browse files
ArthurCRodriguesArthurCRodrigues
authored
Merge pull request #327 from webtech-network/refactor-github-action-docs
docs: refactor GitHub Action documentation
2 parents ce0bdb3 + c6b5d8b commit e526f05

9 files changed

Lines changed: 924 additions & 312 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -356,7 +356,7 @@ jobs:
356356
- **Module overview and internals:** [docs/github_action/README.md](docs/github_action/README.md)
357357
- **Configuration reference and troubleshooting:** [docs/github_action/configuration.md](docs/github_action/configuration.md)
358358
- **Reference implementation:** [webtech-network/demo-autograder](https://github.com/webtech-network/demo-autograder)
359-
- **How to adapt the demo to your course:** [docs/github_action/demo-autograder.md](docs/github_action/demo-autograder.md)
359+
- **How to adapt the demo to your course:** [docs/github_action/quick-start.md](docs/github_action/quick-start.md)
360360
361361
---
362362

‎docs/github_action/README.md‎

Lines changed: 138 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -1,80 +1,166 @@
1-
# GitHub Action module
1+
# GitHub Action — Overview
22

3-
This module packages the Autograder as a Docker-based GitHub Action so repositories can grade submissions automatically in CI.
3+
The Prisma Autograder ships as a Docker-based GitHub Action that grades student submissions automatically inside CI workflows. It is designed for **GitHub Classroom** but works with any repository that follows the expected directory layout.
44

5-
It is primarily designed for GitHub Classroom style workflows, but can be used by any repository that follows the expected directory contract.
5+
---
66

7-
## Execution modes
7+
## How it works
88

9-
| Mode | How config is loaded | When to use |
10-
|------|---------------------|-------------|
11-
| `repo` (default) | From config files checked out inside `submission/.github/autograder/` | Repository-based setups where grading config lives in the student repo |
12-
| `external` | Fetched from the Autograder Cloud API at runtime | Centralised setups where a single Autograder Cloud instance manages all assignments |
9+
```
10+
Student pushes code → GitHub Action triggers → Autograder grades → Results exported
11+
```
1312

14-
## What it does (repo mode)
13+
The Action:
1514

16-
When a workflow runs in `repo` mode:
15+
1. Reads student files from the `submission/` directory.
16+
2. Loads grading configuration (from repo files **or** from the Autograder Cloud).
17+
3. Builds and executes the full grading pipeline (criteria tree → sandbox → grading → feedback).
18+
4. Exports the result (to GitHub Classroom check run **or** to the Autograder Cloud API).
1719

18-
1. Reads student files from `submission/`.
19-
2. Reads grading config from `submission/.github/autograder/`.
20-
3. Builds and executes the Autograder pipeline.
21-
4. Updates the GitHub check run with the final score.
22-
5. Optionally commits `relatorio.md` with feedback.
20+
---
2321

24-
## What it does (external mode)
22+
## Execution modes
2523

26-
When a workflow runs in `external` mode:
24+
The Action supports two mutually exclusive modes:
2725

28-
1. Fetches the grading configuration from the Autograder Cloud (`GET /api/v1/configs/id/{id}`).
29-
2. Reads student files from `submission/`.
30-
3. Builds and executes the Autograder pipeline.
31-
4. Posts the grading result back to the Autograder Cloud (`POST /api/v1/submissions/external-results`).
32-
5. If grading fails for any reason, posts a `failed` status payload before exiting non-zero.
26+
| Mode | Config source | Result destination | Use case |
27+
|------|--------------|-------------------|----------|
28+
| **`repo`** (default) | JSON files in `submission/.github/autograder/` | GitHub check run + `relatorio.md` | Config lives in the student repo |
29+
| **`external`** | Autograder Cloud API | Autograder Cloud API | Centralised config managed by the instructor |
3330

34-
## Internal architecture
31+
### Repo mode flow
3532

36-
| Component | Responsibility |
37-
|---|---|
38-
| [`action.yml`](https://github.com/webtech-network/autograder/blob/main/action.yml) | Declares Action inputs/outputs and maps inputs to container env vars. |
39-
| [`Dockerfile.actions`](https://github.com/webtech-network/autograder/blob/main/Dockerfile.actions) | Builds runtime image used by the Action. |
40-
| [`github_action/entrypoint.sh`](https://github.com/webtech-network/autograder/blob/main/github_action/entrypoint.sh) | Validates required env vars and executes Python entrypoint with flags. |
41-
| [`github_action/main.py`](https://github.com/webtech-network/autograder/blob/main/github_action/main.py) | Parses arguments, validates runtime options, reads submission files, and starts grading. |
42-
| [`github_action/github_action_service.py`](https://github.com/webtech-network/autograder/blob/main/github_action/github_action_service.py) | Builds pipeline from JSON configs and handles GitHub export operations. |
43-
| [`github_action/github_classroom_exporter.py`](https://github.com/webtech-network/autograder/blob/main/github_action/github_classroom_exporter.py) | Export adapter that reports score and feedback through `GithubActionService`. |
44-
| [`github_action/cloud_client.py`](https://github.com/webtech-network/autograder/blob/main/github_action/cloud_client.py) | HTTP client for the Autograder Cloud API (config fetch + result submission). Uses exponential back-off on 5xx/network errors; raises `CloudClientError` on 4xx. |
45-
| [`github_action/cloud_exporter.py`](https://github.com/webtech-network/autograder/blob/main/github_action/cloud_exporter.py) | Export adapter for external mode. Builds the `ExternalResultCreate` payload and calls `CloudClient`. Provides `submit_failure()` for the failure path. |
33+
```
34+
submission/.github/autograder/criteria.json ──┐
35+
submission/.github/autograder/feedback.json ──┤──► build_pipeline() ──► run ──► GitHub Check Run
36+
submission/.github/autograder/setup.json ──┘ └► relatorio.md
37+
```
38+
39+
### External mode flow
40+
41+
```
42+
Autograder Cloud ──GET /api/v1/configs/id/{id}──► build_pipeline() ──► run ──► POST /api/v1/submissions/external-results
43+
```
4644

47-
## Repository contract required by the Action
45+
If grading fails in external mode, a `status: "failed"` payload is submitted before the Action exits non-zero.
4846

49-
Your workflow should checkout repository content into a folder named `submission`:
47+
---
48+
49+
## Repository contract
50+
51+
Your workflow **must** checkout the repository into a directory named `submission`:
5052

5153
```yaml
5254
- uses: actions/checkout@v4
5355
with:
5456
path: submission
5557
```
5658
57-
Inside that checkout, the Action expects:
59+
### Repo mode file requirements
60+
61+
```
62+
submission/
63+
├── .github/
64+
│ └── autograder/
65+
│ ├── criteria.json # Required — grading rubric
66+
│ ├── feedback.json # Optional (can be {})
67+
│ └── setup.json # Optional (can be {})
68+
├── index.html # Student files (read recursively)
69+
├── styles.css
70+
└── app.js
71+
```
72+
73+
### External mode file requirements
5874

59-
- `submission/.github/autograder/criteria.json` (required)
60-
- `submission/.github/autograder/feedback.json` (optional, empty object allowed)
61-
- `submission/.github/autograder/setup.json` (optional, empty object allowed)
75+
Only the student source files are needed. No `.github/autograder/` directory required — config comes from the cloud.
6276

63-
## Important runtime notes
77+
```
78+
submission/
79+
├── main.py # Student files (read recursively)
80+
└── utils.py
81+
```
82+
83+
!!! note
84+
The Action skips `.git/` and `.github/` directories when collecting student files.
6485

65-
- `template-preset: custom` is currently rejected by the Action entrypoint.
66-
- `feedback-type: ai` requires `openai-key`.
67-
- The current notification logic looks for a check run named `grading`.
68-
- `include-feedback: "true"` enables feedback generation and may update `relatorio.md`.
69-
- In `external` mode, `include_feedback` is taken from the cloud config, **not** from the `include-feedback` action input.
70-
- In `external` mode, `submission-language` is validated against the `languages` list in the cloud config. Omitting it defaults to the first language in that list.
86+
---
7187

72-
## Reference repository
88+
## Internal architecture
89+
90+
| Component | Responsibility |
91+
|-----------|---------------|
92+
| `action.yml` | Declares Action inputs/outputs; maps inputs to container environment variables |
93+
| `Dockerfile.actions` | Builds the runtime Docker image (Python 3.10 + Node.js 20) |
94+
| `github_action/entrypoint.sh` | Validates required env vars; executes Python entrypoint with CLI flags |
95+
| `github_action/main.py` | Parses arguments, validates options, reads submission files, starts grading |
96+
| `github_action/github_action_service.py` | Builds pipeline from config; handles GitHub/Cloud export |
97+
| `github_action/cloud_client.py` | HTTP client for Cloud API with retry + exponential backoff |
98+
| `github_action/cloud_exporter.py` | Builds result payload and submits to Cloud API |
99+
| `github_action/github_classroom_exporter.py` | Reports score to GitHub Classroom check run |
100+
101+
### Docker image
102+
103+
The Action runs inside a Docker container built from `Dockerfile.actions`:
104+
105+
- **Base:** `python:3.10-slim`
106+
- **Includes:** Node.js 20 (for JavaScript template grading), `tree`, `curl`
107+
- **Entry:** `/app/github_action/entrypoint.sh`
108+
109+
---
110+
111+
## Supported template presets
112+
113+
| Preset | Language | Description |
114+
|--------|----------|-------------|
115+
| `webdev` | HTML/CSS/JS | Web development assignments (DOM, CSS, structure) |
116+
| `input_output` | Python, Java, C++ | Standard I/O programs with test cases |
117+
| `api` | Any | REST API endpoint testing |
118+
119+
!!! warning
120+
`template-preset: custom` is currently **not supported** by the Action and will cause an immediate exit.
121+
122+
---
123+
124+
## Feedback modes
125+
126+
| Mode | Requires | Description |
127+
|------|----------|-------------|
128+
| `default` | Nothing extra | Rule-based feedback from the grading pipeline |
129+
| `ai` | `openai-key` | AI-generated feedback using OpenAI API |
130+
131+
When `include-feedback: "true"` (repo mode), the Action commits feedback as `relatorio.md` to the repository.
132+
133+
---
134+
135+
## Quick reference
136+
137+
Minimal repo-mode workflow:
138+
139+
```yaml
140+
- uses: webtech-network/autograder@main
141+
with:
142+
template-preset: "webdev"
143+
feedback-type: "default"
144+
include-feedback: "true"
145+
```
146+
147+
Minimal external-mode workflow:
148+
149+
```yaml
150+
- uses: webtech-network/autograder@main
151+
with:
152+
execution-mode: "external"
153+
grading-config-id: "42"
154+
autograder-cloud-url: ${{ secrets.AUTOGRADER_CLOUD_URL }}
155+
autograder-cloud-token: ${{ secrets.AUTOGRADER_CLOUD_TOKEN }}
156+
feedback-type: "default"
157+
template-preset: "input_output"
158+
```
73159
74-
Use **[`webtech-network/demo-autograder`](https://github.com/webtech-network/demo-autograder)** as the baseline implementation. It demonstrates a working workflow, config files, and expected repository layout.
160+
---
75161
76-
## Next
162+
## Next steps
77163
78-
- Configuration details: [configuration.md](configuration.md)
79-
- External mode deep-dive: [external-mode.md](external-mode.md)
80-
- Demo walkthrough: [demo-autograder.md](demo-autograder.md)
164+
- [Configuration Reference](configuration.md) — all inputs, outputs, secrets, and permissions
165+
- [Quick Start Guide](quick-start.md) — step-by-step setup for both modes
166+
- [External Mode](external-mode.md) — deep dive into cloud-based grading

0 commit comments

Comments
 (0)