Skip to content

Commit 9e77d12

Browse files
authored
Splitagain (#429)
Please don't review yet!!!! I'm sure CodeRabbit will have a lot to say, and I'll probably not be finished until Monday. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Added full LLM Rubric Autograde docs and clarified The LLM Rubric Grader as a separate instructor‑approval workflow; streamlined rubric creation, grading steps, UI guidance, images, examples, and rubric requirement behavior. Reworked grading and assessment authoring content for clearer navigation, simplified workflows, and updated sample visuals and captions. * **Changelog** * Added August 2025 entry: Join a Course token is now four words. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
1 parent 23af792 commit 9e77d12

13 files changed

Lines changed: 137 additions & 91 deletions

File tree

source/img/No-overrindengrade.png

-150 KB
Binary file not shown.

source/img/Overridengarde.png

-154 KB
Binary file not shown.
173 KB
Loading

source/img/filter.png

-70.4 KB
Binary file not shown.

source/img/grading-assign.png

221 KB
Loading

source/img/grading-unit.png

303 KB
Loading

source/img/gradingqueue.png

-6.45 KB
Binary file not shown.

source/img/latepenalty.png

189 KB
Loading

source/instructors/authoring/assessments/add-assessment.rst

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ Auto-Graded Assessments
2424
free-text
2525
grade-book
2626
llm-based-rubric
27+
llm-based-auto-rubric
2728
math-assessments
2829
multiple-choice
2930
parsons-puzzle
@@ -69,4 +70,5 @@ Refer to the specific topics for the type of assessment:
6970
- :ref:`Grade Book <grade-book>`
7071
- :ref:`Parsons Puzzle <parsons-puzzle>`
7172
- :ref:`Random Assessment <random>`
72-
- :ref:`LLM Rubric and LLM Rubric Autograde <llm-based-rubric>`
73+
- :ref:`LLM Rubric Grader <llm-based-rubric>`
74+
- :ref:`LLM Rubric Autograde <llm-based-auto-rubric>`
Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
.. meta::
2+
:description: Use an LLM to generate a rubric and grade based on that rubric.
3+
4+
.. _llm-based-auto-rubric:
5+
6+
7+
The LLM Rubric Auto-grader
8+
==========================
9+
10+
The **LLM Rubric Autograde** assessment automatically grades student submissions using an LLM-generated rubric and displays feedback directly to students without requiring instructor approval.
11+
12+
It can be found in the **Auto-Graded** section of the assessments menu. If you wish to approve the LLM grading before it is displayed to the student, see the :ref:`LLM Rubric <llm-based-rubric>` assessment in the **Manually Graded** section.
13+
14+
More information about adding assessments can be found in our :ref:`assessment documentation <add-assessment>`.
15+
16+
There are two steps to the **LLM Rubric Autograde** process:
17+
18+
1. Rubric generation
19+
2. LLM-based grading using the generated rubric
20+
21+
22+
Add an **LLM Rubric Autograde** assessment to your guide page and follow the steps below.
23+
24+
LLM-Based Rubric Creation (Step 1)
25+
----------------------------------
26+
27+
Complete the sections below to set up your rubric grader. For more information on **General**, **Metadata** (optional) and **Files** (optional) see :ref:`Assessments <assessments>`.
28+
29+
1. Complete **General**.
30+
31+
2. Click **Grading** in the navigation pane and complete the following information:
32+
33+
.. image:: /img/guides/llmbasedrubric.png
34+
:height: 600
35+
:alt: Generate a rubric
36+
37+
- Add a solution file (1) if you wish the rubric creation process to consider your solution.
38+
39+
- Click the **Generate Rubrics** (2) button to initiate the process.
40+
41+
The **Rubric Creation Agent** uses the following items to generate the rubric items:
42+
43+
- The assessment name
44+
- Instructions provided in the **General** tab of the assessment
45+
- Content of the Guide Page where the assessment is being added
46+
- Contents of the provided solution file
47+
- The Course, Module, and Assignment name
48+
- Requirements specified in the Rubric creation tab
49+
50+
.. Note:: If you do not specify rubric requirements, the system will generate rubric items using general code grading norms.
51+
52+
Add your requirements in the **Rubric Requirements** dialog (optional):
53+
54+
.. image:: /img/guides/llmrubricreqs.png
55+
:height: 600
56+
:alt: Area to add your rubric requirements
57+
58+
59+
- Once you are done, click **Generate Using AI**.
60+
- You can provide additional rubric items by clicking **Add Rubric** and entering information.
61+
- Once you have reviewed the rubric items and other settings, click **Save** to save the assessment.
62+
63+
64+
LLM Grading Based on the Created Rubric (Step 2)
65+
------------------------------------------------
66+
67+
The grading occurs when the student clicks the **Check It** button. The LLM Grading agent uses the following to grade the student's work:
68+
69+
- Instructions provided in the **General** tab of the assessment
70+
- Contents of the Guide page where the assessment is located
71+
- Contents of the specified solution file
72+
- The student file
73+
- The rubric generated in the previous step to identify the grading criteria
74+
75+
Sample feedback for the Auto-Graded Version:
76+
77+
.. image:: /img/guides/rubricfinal.png
78+
:height: 600
79+
:alt: Final grading information displayed to the student
80+
81+
Rubric Requirements Example
82+
---------------------------
83+
(You can view another example on the :ref:`LLM Rubric Grader <llm-based-rubric>` page.)
84+
85+
Use only the following criteria for evaluating the student code:
86+
87+
- The code correctly implements the requested task and outputs the correct values.
88+
- Variable and function names are descriptive and clearly indicate their purpose in the program.
89+
- The code includes at least two meaningful comments.

0 commit comments

Comments
 (0)