LLM-based rubric grading updates (#379)

Perhaps Mohit can first review for accuracy and to be sure I haven't
missed anything and then Jairo for general doc stuff. Also, Jairo, I
added images because I think this doc really needs them but I am having
trouble getting sizes right. I would appreciate your advice!

Author: jairovelasquez

source/img/guides/llmbasedrubric.png

@@ -11,27 +11,28 @@ Auto-graded assessments
    :hidden:
 
    assessments
-   ai-assessment-generation
-   student-submission
+   advanced-code-test
+   ai-assessment-generation  
    assessment-security
-   auto-grade-scripts
-   parameterized
+   autograde-free-text
+   auto-grade-scripts  
    delete-assessment
-   edit-assessment
    duplicate-assessment
+   edit-assessment
    edit-assessment-points
-   standard-code-test
-   advanced-code-test
-   partial-points
-   multiple-choice
    fill-in-blanks
    free-text
-   autograde-free-text
    grade-book
+   llm-based-rubric
+   math-assessments
+   multiple-choice
+   parameterized
    parsons-puzzle
+   partial-points
    random
-   math-assessments
    splice
+   standard-code-test
+   student-submission
    ungraded-assessments
 
 

source/img/guides/llmrubricreqs.png

@@ -5,7 +5,7 @@
 
 Assessments
 ===========
-Assessments can be used to determine how well students understand course material, and can be automatically or manually graded. Codio offers a wide range of assessment types, including autom-graded code tests, multiple choice tests, fill in the blanks, drop-down selection, free text responses and assignment grading. Assessments can be interspersed throughout tutorial materials or stand alone using an :ref:`assignment level script <auto-grade-scripts>`. 
+Assessments can be used to determine how well students understand course material, and can be automatically or manually graded. Codio offers a wide range of assessment types, including auto-graded code tests, multiple choice tests, fill in the blanks, drop-down selection, free text responses and assignment grading. Assessments can be interspersed throughout tutorial materials or stand alone using an :ref:`assignment level script <auto-grade-scripts>`. 
 
 You can view the results of assessments in a course from the teacher dashboard.
 

source/img/guides/rubricapproval.png

@@ -0,0 +1,102 @@
+.. meta::
+   :description: Use an LLM to generate a rubric and grade based on that rubric
+   
+.. _llm-based-rubric:
+
+
+LLM-Based Rubric Grader
+=======================
+You can find the **LLM Based Rubric** assessment in the **Manually Graded** section of the Assessments tab. More information about adding assessments can be found in our :ref:`assessment documentation <add-assessment>`.
+
+There are three steps to this process. 
+
+1. Rubric generation
+2. LLM-based grading using the generated rubric
+3. Manual approval of LLM-generated grades by the instructor
+
+.. Note:: Step 3 is required - LLM feedback will not be released to the student until the instructor reviews and approves it.
+
+LLM-Based Rubric Creation (Step 1)
+----------------------------------
+
+Add an **LLM Based Rubric** assessment to your guide page and follow the steps below.
+
+
+1. On the **General** page, enter the following information:
+
+   - **Name** - Enter a short name that describes the test. This name is displayed in the teacher dashboard so the name should reflect the challenge and thereby be clear when reviewing.
+
+     Toggle the **Show Name** setting to hide the name in the challenge text the student sees.
+     
+   - **Instructions** - Enter text that is shown to the student using optional Markdown formatting.
+
+2. Click **Grading** in the navigation pane and complete the following information:
+
+   - Add a solution file (1) if you wish the rubric creation process to consider your solution.
+
+      .. image:: /img/guides/llmbasedrubric.png
+         :height: 600
+         :alt: Generate a rubric
+
+   - Click the **Generate Rubrics** (2) button to initiate the process.
+
+   The **Rubric Creation Agent** uses the following items to generate the rubric items:
+
+   - The assessment name
+   - Instructions provided in the **General** tab of the assessment
+   - Content of the Guide Page where the assessment is being added
+   - Contents of the provided solution file
+   - The Course, Module, and Assignment name
+   - Requirements specified in the Rubric creation tab
+
+   .. Note:: If you do not add rubric requirements; the process will use general code grading norms to supply rubric items.
+
+   (optional) Add your requirements in the **Rubrics Requirements** dialog: 
+
+      .. image:: /img/guides/llmrubricreqs.png
+         :height: 600
+         :alt: Area to add your rubric requirements
+
+
+   - Once you are done, click **Generate Using AI**. 
+
+
+   - You can provide additional rubric items by clicking **Add Rubric** and entering information.
+   - Once you have reviewed the rubric items and other settings, click **Save** to save the assessment.
+
+
+LLM Grading Based on the Created Rubric (Step 2)
+------------------------------------------------
+
+The LLM Grading agent uses the following to grade the student work:
+
+- Instructions provided in the **General** tab of the assessment
+- Contents of the Guide page where the assessment is located
+- Contents of the specified solution file
+- The student file
+- The rubric generated in the previous step to identify the grading criteria
+
+.. Note:: The grading occurs when the student clicks the **Check It** button. The student receives information about whether they have passed or failed each rubric item, but does not see the rest of the LLM-generated feedback until after the instructor conducts their review.
+
+.. image:: /img/guides/rubricprelim.png
+    :height: 600
+    :alt: Preliminary grading information displayed to the student
+
+
+Manual Approval by Instructor (Step 3)
+--------------------------------------
+The final step involves the instructor opening the student assignment and selecting the passing test cases as part of the :ref:`grading process <grading>`. The instructor can also edit the comments generated by the LLM. Once this process is complete, click on **Apply Grade**. Once the feedback is released to students, it cannot be modified.
+
+   .. image:: /img/guides/rubricapproval.png
+      :height: 600
+      :alt: The teacher approves each comment generated by the LLM grader
+
+
+Rubric Requirements Example
+---------------------------
+
+.. code-block:: text
+
+      Use the following criteria, assigning equal weight to each one.
+      - Program correctness
+      - Proper and efficient usage of a dictionary data structure

source/img/guides/rubricprelim.png

@@ -19,6 +19,7 @@ We do not report small bug fixes or issues that affect individual users, who we
 
 **May 2025**
 
+- `[F] [23rd]`: Add Ability to :ref:`Grade using an LLM-based rubric grader <llm-based-rubric>`
 - `[F] [23rd]`: Add Ability to :ref:`Require all assessments be answered before marking as complete <complete-requires-all-answered>`
 
 **Apr 2025**