|
| 1 | +.. meta:: |
| 2 | + :description: Use an LLM to generate a rubric and grade based on that rubric. |
| 3 | + |
| 4 | +.. _llm-based-auto-rubric: |
| 5 | + |
| 6 | + |
| 7 | +The LLM Rubric Auto-grader |
| 8 | +========================== |
| 9 | + |
| 10 | +The **LLM Rubric Autograde** assessment automatically grades student submissions using an LLM-generated rubric and displays feedback directly to students without requiring instructor approval. |
| 11 | + |
| 12 | +It can be found in the **Auto-Graded** section of the assessments menu. If you wish to approve the LLM grading before it is displayed to the student, see the :ref:`LLM Rubric <llm-based-rubric>` assessment in the **Manually Graded** section. |
| 13 | + |
| 14 | +More information about adding assessments can be found in our :ref:`assessment documentation <add-assessment>`. |
| 15 | + |
| 16 | +There are two steps to the **LLM Rubric Autograde** process: |
| 17 | + |
| 18 | +1. Rubric generation |
| 19 | +2. LLM-based grading using the generated rubric |
| 20 | + |
| 21 | + |
| 22 | +Add an **LLM Rubric Autograde** assessment to your guide page and follow the steps below. |
| 23 | + |
| 24 | +LLM-Based Rubric Creation (Step 1) |
| 25 | +---------------------------------- |
| 26 | + |
| 27 | +Complete the sections below to set up your rubric grader. For more information on **General**, **Metadata** (optional) and **Files** (optional) see :ref:`Assessments <assessments>`. |
| 28 | + |
| 29 | +1. Complete **General**. |
| 30 | + |
| 31 | +2. Click **Grading** in the navigation pane and complete the following information: |
| 32 | + |
| 33 | +.. image:: /img/guides/llmbasedrubric.png |
| 34 | + :height: 600 |
| 35 | + :alt: Generate a rubric |
| 36 | + |
| 37 | +- Add a solution file (1) if you wish the rubric creation process to consider your solution. |
| 38 | + |
| 39 | +- Click the **Generate Rubrics** (2) button to initiate the process. |
| 40 | + |
| 41 | +The **Rubric Creation Agent** uses the following items to generate the rubric items: |
| 42 | + |
| 43 | +- The assessment name |
| 44 | +- Instructions provided in the **General** tab of the assessment |
| 45 | +- Content of the Guide Page where the assessment is being added |
| 46 | +- Contents of the provided solution file |
| 47 | +- The Course, Module, and Assignment name |
| 48 | +- Requirements specified in the Rubric creation tab |
| 49 | + |
| 50 | +.. Note:: If you do not specify rubric requirements, the system will generate rubric items using general code grading norms. |
| 51 | + |
| 52 | +Add your requirements in the **Rubric Requirements** dialog (optional): |
| 53 | + |
| 54 | +.. image:: /img/guides/llmrubricreqs.png |
| 55 | + :height: 600 |
| 56 | + :alt: Area to add your rubric requirements |
| 57 | + |
| 58 | + |
| 59 | +- Once you are done, click **Generate Using AI**. |
| 60 | +- You can provide additional rubric items by clicking **Add Rubric** and entering information. |
| 61 | +- Once you have reviewed the rubric items and other settings, click **Save** to save the assessment. |
| 62 | + |
| 63 | + |
| 64 | +LLM Grading Based on the Created Rubric (Step 2) |
| 65 | +------------------------------------------------ |
| 66 | + |
| 67 | +The grading occurs when the student clicks the **Check It** button. The LLM Grading agent uses the following to grade the student's work: |
| 68 | + |
| 69 | +- Instructions provided in the **General** tab of the assessment |
| 70 | +- Contents of the Guide page where the assessment is located |
| 71 | +- Contents of the specified solution file |
| 72 | +- The student file |
| 73 | +- The rubric generated in the previous step to identify the grading criteria |
| 74 | + |
| 75 | +Sample feedback for the Auto-Graded Version: |
| 76 | + |
| 77 | +.. image:: /img/guides/rubricfinal.png |
| 78 | + :height: 600 |
| 79 | + :alt: Final grading information displayed to the student |
| 80 | + |
| 81 | +Rubric Requirements Example |
| 82 | +--------------------------- |
| 83 | +(You can view another example on the :ref:`LLM Rubric Grader <llm-based-rubric>` page.) |
| 84 | + |
| 85 | +Use only the following criteria for evaluating the student code: |
| 86 | + |
| 87 | +- The code correctly implements the requested task and outputs the correct values. |
| 88 | +- Variable and function names are descriptive and clearly indicate their purpose in the program. |
| 89 | +- The code includes at least two meaningful comments. |
0 commit comments