Skip to content

Commit 0051a14

Browse files
authored
assessments update part 2 (#430)
<!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Restructured instructor assessment docs for clearer, consistent headings and capitalization. * Renumbered and reorganized step flows and navigation instructions (including adjusted step numbers in advanced code test). * Converted many detailed UI walkthroughs into condensed narratives and cross‑references; replaced inline steps with higher‑level guidance. * Introduced tables for answer types and grading parameters; standardized examples, test/debug guidance, and launcher notes. * Clarified grading workflows, partial‑points guidance, and free‑text/autograde references. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
1 parent 9e77d12 commit 0051a14

12 files changed

Lines changed: 177 additions & 197 deletions
49.3 KB
Loading
22.9 KB
Loading
58.6 KB
Loading

source/img/guides/freetexticon.png

127 KB
Loading

source/img/guides/notpartial.png

-4.42 KB
Loading

source/img/guides/partial.png

16.3 KB
Loading

source/instructors/authoring/assessments/advanced-code-test.rst

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -46,11 +46,11 @@ Complete each section to set up your advanced code test. For more information on
4646
- **Rationale** - Enter guidance for the assessment. This is always visible to the teacher when the project is opened in the course or when opening the student's project.
4747
- **Use maximum score** - Toggle to enable assessment final score to be the highest score attained of all runs.
4848

49-
5. **(Optional)** Complete **Metadata**.
49+
4. **(Optional)** Complete **Metadata**.
5050

51-
6. **(Optional)** Complete **Files**.
51+
5. **(Optional)** Complete **Files**.
5252

53-
7. Click **Create** to complete the process.
53+
6. Click **Create** to complete the process.
5454

5555

5656
See a Working Example

source/instructors/authoring/assessments/auto-grade-scripts.rst

Lines changed: 60 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -5,18 +5,19 @@
55

66
Assignment Level Scripts
77
========================
8+
89
You can use assignment level scripts to evaluate student code, normalize points, and mark for participation grading. Assignment level scripts are added in the **Script Grading** field on the :ref:`Script Grading <grade-weights>` settings page. These scripts can then transfer the grading value into the grading field. Assignment level scripts are run when an assignment is **Marked as Complete**.
910

1011
.. Note:: The script must execute within 3 minutes or a timeout error occurs. There is a maximum size for the feedback that can be returned of 1Mb. If this limit is exceeded, the message **Payload content length greater than maximum allowed: 1048576** will be returned.
1112

1213
If you are using an LMS platform with Codio, be sure to enter a percentage value in the **Grade Weight** field to maintain compatibility with LMS gradebooks. This value is then transferred into your LMS gradebook once you :ref:`release the grades <release-grades>`.
1314

14-
Secure scripts
15+
Secure Scripts
1516
--------------
1617
If you store grading scripts in the **.guides/secure** folder, they run securely and students cannot see the script or the files in the folder. Only instructors can access this folder.
17-
You can find more information about assessment security :ref:`here <assessment-security>`.
18+
You can find more information about assessment security in the :ref:`Assessment Security <assessment-security>` page.
1819

19-
Access assessment results
20+
Access Assessment Results
2021
-------------------------
2122
You can access student scores for the auto-graded assessments in an assignment. All this information is in JSON format and can be accessed in the ``CODIO_AUTOGRADE_ENV`` environment variable. The following tabs show the format of this data. The first tab shows just the assessment portion of the data and the second depicts all the available values.
2223

@@ -258,42 +259,45 @@ The student grade is calculated based on whether they answered the question, not
258259
- In the course assignment settings :ref:`Grade Weights <grade-weights>` section, enable **Script Grading** set **Set custom script path** to that file and disable **Assessments Grading**.
259260

260261

261-
Regrade an individual student's assignment
262-
------------------------------------------
263-
If students have clicked **Mark as Complete** and the custom script is triggered, you can regrade their work by resetting the `complete` switch, and then set it to *complete* again, which triggers the custom script to run again.
262+
Regrade an Individual Student's Assignment
263+
-------------------------------------------
264+
If students have clicked **Mark as Complete** and the custom script has been triggered, you can regrade their work by clicking the three vertical dots next to the student's name to access additional actions. Select **Mark as Incomplete**, then click the three vertical dots again and select **Mark as Complete** to retrigger the custom script.
264265

265-
Regrade all student's assignments
266+
Regrade All Students' Assignments
266267
---------------------------------
267-
You can regrade all student's assignments that have already been auto-graded from the **Actions** button on the assignment page.
268+
You can regrade all students' assignments that have already been auto-graded by clicking the **Regrade Completed** button at the bottom of the assignment page.
269+
270+
1. Navigate to the assignment and click it.
271+
2. Then click **Regrade Completed** at the bottom of the page. This is useful if you have found a bug in your assignment level grading script. **Regrade Completed** does not run individual code test assessments.
268272

269-
1. Navigate to the assignment and open it.
270-
2. Click the **Actions** button and then click **Regrade Completed**. This is useful if you have found a bug in your assignment level grading script. **Regrade Completed** does not run individual code test assessments.
273+
Test and Debug Your Grading Scripts
274+
------------------------------------
271275

272-
Test and debug your grading scripts
273-
-----------------------------------
274-
.. Note:: Codio provides the ability to test your auto-grading scripts when creating your project, this should be done before publishing your project to a course. Once an assignment has been published to the course, any changes made to files in the student workspace (/home/codio/workspace) are not reflected in the published assignment. Grading scripts should be stored in the **.guides/secure** folder. Files in the .guides and guides/secure folders can be published even if students have already started.
276+
Codio provides the ability to test your auto-grading scripts when creating your project, this should be done before publishing your project to a course. Once an assignment has been published to the course, any changes made to files in the student workspace (/home/codio/workspace) are not reflected in the published assignment. Grading scripts should be stored in the **.guides/secure** folder. Files in the **.guides** and **.guides/secure** folders can be published even if students have already started.
275277

276278

277-
Test your script in the IDE
278-
...........................
279+
Test Your Script in the IDE
280+
----------------------------
281+
279282
You can test your auto-grading script in the Codio IDE from the **Education > Test Autograde Script** on the menu bar. This option allows you to specify the location of your auto-grading script and run it against the current project content. It also allows you simulate scores attained by any auto-graded assessments located in the Codio Guide and select which autograded assessments to test.
280283

281284
.. image:: /img/autograde-test.png
282285
:alt: Autograde Test
283286

284287
Be sure to take the following into account when using this feature:
285288

286-
- When you click **Test Script**:
289+
1. When you click **Test Script**:
290+
291+
- All output to ``stdout`` and ``stderr`` are displayed in the dialog.
292+
- The grade returned by your test script is at the bottom of the output section.
287293

288-
- All output to ``stdout`` and ``stderr`` are displayed in the dialog.
289-
- The grade returned by your test script is at the bottom of the output section.
294+
2. ``stdout`` and ``stderr`` output is not available when running the actual auto-grading script (not in test mode) because it runs invisibly when the assignment is marked as complete. Because of this, you should only generate output for testing and debugging.
295+
3. If you want your script to provide feedback to the student, you should output it to a file that can be accessed by the student when opening the project at a later date. In this case, you should allow read-only access to the project from the assignment settings after being marked as complete.
290296

291-
- ``stdout`` and ``stderr`` output is not available when running the actual auto-grading script (not in test mode) because it runs invisibly when the assignment is marked as complete. Because of this, you should only generate output for testing and debugging.
292-
- If you want your script to provide feedback to the student, you should output it to a file that can be accessed by the student when opening the project at a later date. In this case, you should allow read-only access to the project from the assignment settings after being marked as complete.
297+
Test Your Script Using Bootstrap Launcher
298+
------------------------------------------
293299

294-
Test your script using bootstrap launcher
295-
.........................................
296-
You can also use a simple bootstrap launcher that loads and executes the script from a remote location so that you can edit and debug independently of the Codio box. The following example bash script shows a Python script that is located as a Gist on GitHub. This script might be called **.guides/secure/launcher.sh**.
300+
You can also use a simple bootstrap launcher that loads and executes the script from a remote location so that you can edit and debug independently of the Codio box. The following example is a bash launcher script that downloads and runs a Python script from a GitHub Gist. This script would be saved as **.guides/secure/launcher.sh**.
297301

298302
.. code:: bash
299303
@@ -312,7 +316,7 @@ Sending Points to Codio
312316

313317
Codio provides a Python library to facilitate reporting points from your custom scripts. There are four functions in this library: `send_grade`, `send_grade_v2`, `send_partial` and `send_partial_v2`.
314318

315-
.. Note:: Partial points are not used in assignment level scripts, see :ref:`Allow Partial Points <partial-points>` for more information about setting up partial points.
319+
.. Note:: Partial points are not used in assignment level scripts. See :ref:`Allow Partial Points <partial-points>` for more information about setting up partial points.
316320

317321
In order to use this library you need to add the following code to the top of your grading script:
318322

@@ -336,23 +340,31 @@ The calls to use these functions are as follows:
336340
337341
send_grade(grade)
338342
339-
`grade` - Should be the percent correct for the assessment.
343+
or:
340344

341345
.. code:: python
342346
343347
send_grade_v2(grade, feedback, format=FORMAT_V2_TXT, extra_credit=None)
344348
345-
`grade` - Should be the percent correct for the assessment.
346-
347-
`feedback` - The buffer containing the feedback for your student - maximum size is 1 Mb.
348349
349-
`format` - The format can be Markdown, HTML or text and the default is text.
350+
.. list-table::
351+
:widths: 20 80
352+
:header-rows: 1
350353

351-
`extra_credit` - Extra points beyond the value for doing this correctly. These do not get passed to an LMS system automatically, just the percentage correct.
354+
* - Field
355+
- Description
356+
* - ``grade``
357+
- Should be the percent correct for the assessment.
358+
* - ``feedback``
359+
- The buffer containing the feedback for your student - maximum size is 1 Mb.
360+
* - ``format``
361+
- The format can be Markdown, HTML or text and the default is text.
362+
* - ``extra_credit``
363+
- Extra points beyond the value for doing this correctly. These do not get passed to an LMS system automatically, just the percentage correct.
352364

353365
.. _autograde-enhance:
354366

355-
Auto-grading enhancements
367+
Auto-Grading Enhancements
356368
-------------------------
357369

358370
The V2 versions of the grading functions allow you to:
@@ -362,14 +374,25 @@ The V2 versions of the grading functions allow you to:
362374
- Notify (instructors and students) and reopen assignments for a student on grade script failure.
363375

364376

365-
If you don't use the send_grade_v2 functions, this URL (passed as an environment variable) can be used:```CODIO_AUTOGRADE_V2_URL```
377+
If you don't use the send_grade_v2 functions, this URL (passed as an environment variable) can be used:``CODIO_AUTOGRADE_V2_URL``
366378

367379
These variables allow POST and GET requests with the following parameters:
368380

369-
- **Grade** (```CODIO_AUTOGRADE_V2_URL```) - return 0-100 percent. This is the percent correct out of total possible points.
370-
- **Feedback** - text
371-
- **Format** - html, md, txt - txt is default
372-
- **Penalty** - Penalty is number between 0-100,
381+
.. list-table::
382+
:widths: 20 80
383+
:header-rows: 1
384+
385+
* - Field
386+
- Description
387+
* - **Grade** (``CODIO_AUTOGRADE_V2_URL``)
388+
- Return 0-100 percent. This is the percent correct out of total possible points.
389+
* - **Feedback**
390+
- Text
391+
* - **Format**
392+
- html, md, txt - txt is default
393+
* - **Penalty**
394+
- Penalty is number between 0-100
395+
373396

374397
If you want to calculate penalties in the grading script you can use the **completedDate** (in UTC format) in ``CODIO_AUTOGRADE_ENV`` to calculate penalties. See Python example below.
375398

@@ -430,7 +453,7 @@ These Python and Bash files that can be loaded by a bootstrap script or as expla
430453

431454
main()
432455

433-
Example grading scripts
456+
Example Grading Scripts
434457
-----------------------
435458
This section provides example assignment level scripts using the older methods to send grades.
436459

source/instructors/authoring/assessments/edit-assessment-points.rst

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,11 @@ Edit Assessment Points
77
======================
88
To edit assessment points, follow these steps:
99

10-
1. In the Guide Editor, click the **Assessments** button to view the list of all assessments.
10+
1. In the Guide Editor, click the **Assessments** button.
1111

12-
2. Click the assessment to open it and modify the points.
12+
2. Then click **"View Existing Assessments"** in the bottom right corner to view the list of all assessments.
13+
14+
3. Modify the point value in the box to the left of the assessment you want to update. Once done, click the **Close** button on the bottom left.
1315

1416
.. image:: /img/assessmentpoints.png
1517
:alt: Edit Assessment Points

source/instructors/authoring/assessments/edit-assessment.rst

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,12 +8,16 @@ Edit an Assessment
88
To edit an assessment, either:
99

1010

11-
1. In the Guide Editor, click the **Edit: <Assessment Name>** button to the right of the assessment.
11+
- In the Guide Editor, click the **Edit: <Assessment Name>** button to the right of the assessment.
1212

1313
.. image:: /img/guides/editassessmentbutton.png
1414
:alt: Edit Assessment
1515

16-
2. Click the **Assessment** button to view the list of all assessments and click the assessment to open it and make your changes.
16+
Or, in the Guide Editor, use the following steps:
1717

18+
1. Click the **Assessment** button.
19+
2. Click **"View Existing Assessments"** in the bottom right corner.
20+
3. Click the assessment to open it and make your changes.
21+
1822
.. image:: /img/guides/editassessmentlist.png
1923
:alt: Edit Assessment List

0 commit comments

Comments
 (0)