You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Restructured instructor assessment docs for clearer, consistent
headings and capitalization.
* Renumbered and reorganized step flows and navigation instructions
(including adjusted step numbers in advanced code test).
* Converted many detailed UI walkthroughs into condensed narratives and
cross‑references; replaced inline steps with higher‑level guidance.
* Introduced tables for answer types and grading parameters;
standardized examples, test/debug guidance, and launcher notes.
* Clarified grading workflows, partial‑points guidance, and
free‑text/autograde references.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Copy file name to clipboardExpand all lines: source/instructors/authoring/assessments/advanced-code-test.rst
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -46,11 +46,11 @@ Complete each section to set up your advanced code test. For more information on
46
46
- **Rationale** - Enter guidance for the assessment. This is always visible to the teacher when the project is opened in the course or when opening the student's project.
47
47
- **Use maximum score** - Toggle to enable assessment final score to be the highest score attained of all runs.
Copy file name to clipboardExpand all lines: source/instructors/authoring/assessments/auto-grade-scripts.rst
+60-37Lines changed: 60 additions & 37 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,18 +5,19 @@
5
5
6
6
Assignment Level Scripts
7
7
========================
8
+
8
9
You can use assignment level scripts to evaluate student code, normalize points, and mark for participation grading. Assignment level scripts are added in the **Script Grading** field on the :ref:`Script Grading <grade-weights>` settings page. These scripts can then transfer the grading value into the grading field. Assignment level scripts are run when an assignment is **Marked as Complete**.
9
10
10
11
.. Note:: The script must execute within 3 minutes or a timeout error occurs. There is a maximum size for the feedback that can be returned of 1Mb. If this limit is exceeded, the message **Payload content length greater than maximum allowed: 1048576** will be returned.
11
12
12
13
If you are using an LMS platform with Codio, be sure to enter a percentage value in the **Grade Weight** field to maintain compatibility with LMS gradebooks. This value is then transferred into your LMS gradebook once you :ref:`release the grades <release-grades>`.
13
14
14
-
Secure scripts
15
+
Secure Scripts
15
16
--------------
16
17
If you store grading scripts in the **.guides/secure** folder, they run securely and students cannot see the script or the files in the folder. Only instructors can access this folder.
17
-
You can find more information about assessment security :ref:`here <assessment-security>`.
18
+
You can find more information about assessment security in the :ref:`Assessment Security <assessment-security>` page.
18
19
19
-
Access assessment results
20
+
Access Assessment Results
20
21
-------------------------
21
22
You can access student scores for the auto-graded assessments in an assignment. All this information is in JSON format and can be accessed in the ``CODIO_AUTOGRADE_ENV`` environment variable. The following tabs show the format of this data. The first tab shows just the assessment portion of the data and the second depicts all the available values.
22
23
@@ -258,42 +259,45 @@ The student grade is calculated based on whether they answered the question, not
258
259
- In the course assignment settings :ref:`Grade Weights <grade-weights>` section, enable **Script Grading** set **Set custom script path** to that file and disable **Assessments Grading**.
259
260
260
261
261
-
Regrade an individual student's assignment
262
-
------------------------------------------
263
-
If students have clicked **Mark as Complete** and the custom script is triggered, you can regrade their work by resetting the `complete` switch, and then set it to *complete* again, which triggers the custom script to run again.
262
+
Regrade an Individual Student's Assignment
263
+
-------------------------------------------
264
+
If students have clicked **Mark as Complete** and the custom script has been triggered, you can regrade their work by clicking the three vertical dots next to the student's name to access additional actions. Select **Mark as Incomplete**, then click the three vertical dots again and select **Mark as Complete** to retrigger the custom script.
264
265
265
-
Regrade all student's assignments
266
+
Regrade All Students' Assignments
266
267
---------------------------------
267
-
You can regrade all student's assignments that have already been auto-graded from the **Actions** button on the assignment page.
268
+
You can regrade all students' assignments that have already been auto-graded by clicking the **Regrade Completed** button at the bottom of the assignment page.
269
+
270
+
1. Navigate to the assignment and click it.
271
+
2. Then click **Regrade Completed** at the bottom of the page. This is useful if you have found a bug in your assignment level grading script. **Regrade Completed** does not run individual code test assessments.
268
272
269
-
1. Navigate to the assignment and open it.
270
-
2. Click the **Actions** button and then click **Regrade Completed**. This is useful if you have found a bug in your assignment level grading script. **Regrade Completed** does not run individual code test assessments.
273
+
Test and Debug Your Grading Scripts
274
+
------------------------------------
271
275
272
-
Test and debug your grading scripts
273
-
-----------------------------------
274
-
.. Note:: Codio provides the ability to test your auto-grading scripts when creating your project, this should be done before publishing your project to a course. Once an assignment has been published to the course, any changes made to files in the student workspace (/home/codio/workspace) are not reflected in the published assignment. Grading scripts should be stored in the **.guides/secure** folder. Files in the .guides and guides/secure folders can be published even if students have already started.
276
+
Codio provides the ability to test your auto-grading scripts when creating your project, this should be done before publishing your project to a course. Once an assignment has been published to the course, any changes made to files in the student workspace (/home/codio/workspace) are not reflected in the published assignment. Grading scripts should be stored in the **.guides/secure** folder. Files in the **.guides** and **.guides/secure** folders can be published even if students have already started.
275
277
276
278
277
-
Test your script in the IDE
278
-
...........................
279
+
Test Your Script in the IDE
280
+
----------------------------
281
+
279
282
You can test your auto-grading script in the Codio IDE from the **Education > Test Autograde Script** on the menu bar. This option allows you to specify the location of your auto-grading script and run it against the current project content. It also allows you simulate scores attained by any auto-graded assessments located in the Codio Guide and select which autograded assessments to test.
280
283
281
284
.. image:: /img/autograde-test.png
282
285
:alt:Autograde Test
283
286
284
287
Be sure to take the following into account when using this feature:
285
288
286
-
- When you click **Test Script**:
289
+
1. When you click **Test Script**:
290
+
291
+
- All output to ``stdout`` and ``stderr`` are displayed in the dialog.
292
+
- The grade returned by your test script is at the bottom of the output section.
287
293
288
-
- All output to ``stdout`` and ``stderr`` are displayed in the dialog.
289
-
- The grade returned by your test script is at the bottom of the output section.
294
+
2. ``stdout`` and ``stderr`` output is not available when running the actual auto-grading script (not in test mode) because it runs invisibly when the assignment is marked as complete. Because of this, you should only generate output for testing and debugging.
295
+
3. If you want your script to provide feedback to the student, you should output it to a file that can be accessed by the student when opening the project at a later date. In this case, you should allow read-only access to the project from the assignment settings after being marked as complete.
290
296
291
-
- ``stdout`` and ``stderr`` output is not available when running the actual auto-grading script (not in test mode) because it runs invisibly when the assignment is marked as complete. Because of this, you should only generate output for testing and debugging.
292
-
- If you want your script to provide feedback to the student, you should output it to a file that can be accessed by the student when opening the project at a later date. In this case, you should allow read-only access to the project from the assignment settings after being marked as complete.
297
+
Test Your Script Using Bootstrap Launcher
298
+
------------------------------------------
293
299
294
-
Test your script using bootstrap launcher
295
-
.........................................
296
-
You can also use a simple bootstrap launcher that loads and executes the script from a remote location so that you can edit and debug independently of the Codio box. The following example bash script shows a Python script that is located as a Gist on GitHub. This script might be called **.guides/secure/launcher.sh**.
300
+
You can also use a simple bootstrap launcher that loads and executes the script from a remote location so that you can edit and debug independently of the Codio box. The following example is a bash launcher script that downloads and runs a Python script from a GitHub Gist. This script would be saved as **.guides/secure/launcher.sh**.
297
301
298
302
.. code:: bash
299
303
@@ -312,7 +316,7 @@ Sending Points to Codio
312
316
313
317
Codio provides a Python library to facilitate reporting points from your custom scripts. There are four functions in this library: `send_grade`, `send_grade_v2`, `send_partial` and `send_partial_v2`.
314
318
315
-
.. Note:: Partial points are not used in assignment level scripts, see :ref:`Allow Partial Points <partial-points>` for more information about setting up partial points.
319
+
.. Note:: Partial points are not used in assignment level scripts. See :ref:`Allow Partial Points <partial-points>` for more information about setting up partial points.
316
320
317
321
In order to use this library you need to add the following code to the top of your grading script:
318
322
@@ -336,23 +340,31 @@ The calls to use these functions are as follows:
336
340
337
341
send_grade(grade)
338
342
339
-
`grade` - Should be the percent correct for the assessment.
`grade` - Should be the percent correct for the assessment.
346
-
347
-
`feedback` - The buffer containing the feedback for your student - maximum size is 1 Mb.
348
349
349
-
`format` - The format can be Markdown, HTML or text and the default is text.
350
+
.. list-table::
351
+
:widths: 20 80
352
+
:header-rows: 1
350
353
351
-
`extra_credit` - Extra points beyond the value for doing this correctly. These do not get passed to an LMS system automatically, just the percentage correct.
354
+
* - Field
355
+
- Description
356
+
* - ``grade``
357
+
- Should be the percent correct for the assessment.
358
+
* - ``feedback``
359
+
- The buffer containing the feedback for your student - maximum size is 1 Mb.
360
+
* - ``format``
361
+
- The format can be Markdown, HTML or text and the default is text.
362
+
* - ``extra_credit``
363
+
- Extra points beyond the value for doing this correctly. These do not get passed to an LMS system automatically, just the percentage correct.
352
364
353
365
.. _autograde-enhance:
354
366
355
-
Auto-grading enhancements
367
+
Auto-Grading Enhancements
356
368
-------------------------
357
369
358
370
The V2 versions of the grading functions allow you to:
@@ -362,14 +374,25 @@ The V2 versions of the grading functions allow you to:
362
374
- Notify (instructors and students) and reopen assignments for a student on grade script failure.
363
375
364
376
365
-
If you don't use the send_grade_v2 functions, this URL (passed as an environment variable) can be used:```CODIO_AUTOGRADE_V2_URL```
377
+
If you don't use the send_grade_v2 functions, this URL (passed as an environment variable) can be used:``CODIO_AUTOGRADE_V2_URL``
366
378
367
379
These variables allow POST and GET requests with the following parameters:
368
380
369
-
- **Grade** (```CODIO_AUTOGRADE_V2_URL```) - return 0-100 percent. This is the percent correct out of total possible points.
370
-
- **Feedback** - text
371
-
- **Format** - html, md, txt - txt is default
372
-
- **Penalty** - Penalty is number between 0-100,
381
+
.. list-table::
382
+
:widths: 20 80
383
+
:header-rows: 1
384
+
385
+
* - Field
386
+
- Description
387
+
* - **Grade** (``CODIO_AUTOGRADE_V2_URL``)
388
+
- Return 0-100 percent. This is the percent correct out of total possible points.
389
+
* - **Feedback**
390
+
- Text
391
+
* - **Format**
392
+
- html, md, txt - txt is default
393
+
* - **Penalty**
394
+
- Penalty is number between 0-100
395
+
373
396
374
397
If you want to calculate penalties in the grading script you can use the **completedDate** (in UTC format) in ``CODIO_AUTOGRADE_ENV`` to calculate penalties. See Python example below.
375
398
@@ -430,7 +453,7 @@ These Python and Bash files that can be loaded by a bootstrap script or as expla
430
453
431
454
main()
432
455
433
-
Example grading scripts
456
+
Example Grading Scripts
434
457
-----------------------
435
458
This section provides example assignment level scripts using the older methods to send grades.
0 commit comments