Skip to content

Commit 23af792

Browse files
authored
assessment updates part 1 (#428)
<!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Reorganized and clarified assessment authoring with a new common-configuration section and step-by-step "How to Add a New Assessment." * Added AI-generated assessment guidance, example prompts, and a "How to Generate Assessment with AI" workflow (with note that layout must be set manually). * Introduced "Auto-Graded Assessment Types" and standardized "Test Types" guidance. * Expanded secure-folder/file-organization, student workspace limits, and updated examples/visuals for consistency. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
1 parent 7888f23 commit 23af792

12 files changed

Lines changed: 304 additions & 164 deletions
99.9 KB
Loading
43.8 KB
Loading
32.3 KB
Loading
40.4 KB
Loading
18.1 KB
Loading

source/instructors/authoring/assessments/add-assessment.rst

Lines changed: 12 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,11 @@
33

44
.. _add-assessment:
55

6-
Auto-graded assessments
7-
=======================
6+
Auto-Graded Assessments
7+
========================
88

99
.. toctree::
10-
:caption: Auto-graded assessments
10+
:caption: Auto-Graded Assessments
1111
:hidden:
1212

1313
assessments
@@ -26,7 +26,6 @@ Auto-graded assessments
2626
llm-based-rubric
2727
math-assessments
2828
multiple-choice
29-
parameterized
3029
parsons-puzzle
3130
partial-points
3231
random
@@ -36,10 +35,14 @@ Auto-graded assessments
3635
ungraded-assessments
3736

3837

38+
Codio offers a variety of auto-graded assessments to streamline your grading process. Below, you'll find instructions on how to add assessments to your assignment, along with links to pages detailing specific assessment types and their features. Some auto-graded assessments can also be generated automatically.
3939

40-
Add a New Assessment
41-
--------------------
42-
To add a new assessment for a course, follow these steps:
40+
Assessments can be auto-generated based on the content found on a guides page. Currently, the following assessment types support auto-generation: :ref:`Multiple Choice <multiple-choice>`, :ref:`Fill in the Blanks <fill-in-blanks>`, :ref:`Free Text <free-text>`, :ref:`Standard Code Test <standard-code-test>`, and :ref:`Parsons Puzzle <parsons-puzzle>`. For more information, please check out :ref:`Generating Assessments With AI <ai-assessment-generation>`.
41+
42+
43+
How to Add a New Assessment
44+
----------------------------
45+
To add a new assessment for an assignment, follow these steps:
4346

4447
1. Open the assignment in the course, and click the **Edit** button to open the Guide Editor. You can also click the **Tools** menu and choose **Guides > Edit**.
4548
2. Click the **Assessments** button and choose the type of assessment you want to add to the assignment.
@@ -51,11 +54,9 @@ To add a new assessment for a course, follow these steps:
5154

5255
While creating/editing the assessment, you can still see/edit the guide page.
5356

54-
Assessment Auto-Generation
55-
++++++++++++++++++++++++++
56-
57-
Assessments can be auto-generated based on the text found on a guides page. Currently only :ref:`Multiple Choice <multiple-choice>`, :ref:`Fill in the Blanks <fill-in-blanks>`, :ref:`Free Text <free-text>`, :ref:`Standard Code Test <standard-code-test>`, and :ref:`Parsons Puzzle <parsons-puzzle>` assessments can be auto-generated. For more information, please check out :ref:`Generating Assessments With AI <ai-assessment-generation>`
5857

58+
Auto-Graded Assessment Types
59+
-----------------------------
5960

6061
Refer to the specific topics for the type of assessment:
6162

source/instructors/authoring/assessments/advanced-code-test.rst

Lines changed: 86 additions & 88 deletions
Large diffs are not rendered by default.

source/instructors/authoring/assessments/ai-assessment-generation.rst

Lines changed: 30 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,8 +21,37 @@ Important points to consider when auto-generating assessments:
2121

2222
This is a new feature, and if it does not work properly for you, please let us know via the support chat or by emailing help@codio.com. If you would like us to take a look, send us your course name, the module, the assignment, and the page.
2323

24+
.. note:: Not all assessments can be AI-generated. If you don't see a "Generate" button in the bottom right corner when you click on an assessment, that assessment type does not support AI generation.
25+
26+
27+
How to Generate an Assessment with AI
28+
-------------------------------------
29+
30+
Assessments can be auto-generated using the text on the current guides page as context. Follow the steps below to auto-generate an assessment:
31+
32+
1. Select an assessment from the Assessments list.
33+
34+
2. Press the **Generate** button at bottom right corner.
35+
36+
.. image:: /img/guides/generate-assessment-button.png
37+
:alt: Generate assessment button
38+
39+
3. The Generation Prompt will open, press **Generate Using AI** to preview the generated assessment.
40+
41+
.. image:: /img/guides/assessment-generation-prompt.png
42+
:alt: Assessment Generation Prompt
43+
44+
45+
4. When you are satisfied with the result, press **Apply** and then **Create**.
46+
47+
48+
.. important:: The generate assessment feature does not configure the page :ref:`layout; <layout>` you should specify the layout depending on how you want to present the information to the students.
49+
50+
51+
52+
2453
Example Prompts
25-
---------------
54+
----------------
2655
**Multiple choice**
2756

2857
- Create a question about how to refactor the code below with multiple correct answers: (include code)

source/instructors/authoring/assessments/assessment-security.rst

Lines changed: 100 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -4,33 +4,115 @@
44
.. _assessment-security:
55

66
Assessment Security
7-
===================
7+
====================
88

9-
Data for assessments is stored in individual files in the ``.guides/assessments`` folder.
10-
Other folders automatically created in the ``.guides`` directory are: ``.guides/img`` and ``.guides/content``.
11-
Student assignments only receive the ``.guides`` folder and the ``.guides/img`` folder. If you have not provided student access to the file tree in the :ref:`page layout <page>`, the ``.guides`` folder will not be visible.
9+
File Organization
10+
-----------------
11+
12+
Assessment data is stored in individual files within the ``.guides/assessments`` folder. The ``.guides`` directory automatically contains the following subdirectories:
13+
14+
- ``.guides/assessments`` - Assessment configuration files
15+
- ``.guides/img`` - Image assets
16+
- ``.guides/content`` - Content files
17+
18+
Student Workspace Limitations
19+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
20+
21+
When students receive an assignment, they only get access to:
22+
23+
- The ``.guides`` folder
24+
- The ``.guides/img`` folder
25+
26+
If you have not enabled student access to the file tree in the :ref:`page layout <page>`, the ``.guides`` folder will not be visible to them.
27+
28+
29+
30+
Using the .guides Folder
31+
------------------------
1232

1333

14-
The ``.guides`` folder is a good place to store bash files or data files because they can't be easily deleted by the student via the file tree. Keep in mind though, if students have access to the terminal they can use it to view any files in their Linux container, including the ``.guides`` folder.
34+
Benefits and Considerations
35+
~~~~~~~~~~~~~~~~~~~~~~~~~~~
1536

16-
Files that are stored in the ``.guides`` folder can be updated even if students have already started an assignment.
17-
When you publish an assignment that students have already started, pre-existing files in the student workspace ``\home\codio\workspace`` will not be updated in order to make sure student work is not overwritten.
18-
More information on this topic may be found on the :ref:`modify assignments <modify-assignments>` page.
37+
The ``.guides`` folder provides a secure location for storing bash scripts and data files, as students cannot easily delete them through the file tree interface. However, be aware that students with terminal access can view any files in their Linux container, including contents of the ``.guides`` folder.
1938

20-
Blocking student access to the terminal
39+
Updating Published Assignments
40+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
41+
42+
Files stored in the ``.guides`` folder can be updated even after students have started an assignment. When you republish an assignment that students have already begun:
43+
44+
- Files in ``.guides`` **will be updated**.
45+
- Pre-existing files in the student workspace (``/home/codio/workspace``) **will not be updated** to prevent overwriting student work.
46+
47+
For more information, see the :ref:`modify assignments <modify-assignments>` page.
48+
49+
50+
51+
Blocking Student Access to the Terminal
2152
---------------------------------------
2253

23-
The :ref:`terminal <terminal>` can be accessed from the file tree, the menu or by using the key combination (Shift+Alt+T on a PC, Shift+Option+T on a Mac). If you want to block all student access to the terminal you need to hide the file tree in the page layout, :ref:`customize the IDE menu<custom-ide>` and block the keyboard short cut in :ref:`project preferences<project-prefs>`.
2454

25-
The Secure folder
55+
The :ref:`terminal <terminal>` can be accessed through multiple methods:
56+
57+
- From the file tree
58+
- From the menu
59+
- Via keyboard shortcuts (Shift+Alt+T on PC, Shift+Option+T on Mac)
60+
61+
To completely block student access to the terminal, you must:
62+
63+
1. Hide the file tree in the page layout
64+
2. :ref:`Customize the IDE menu <custom-ide>` to remove terminal access
65+
3. Block the keyboard shortcut in :ref:`project preferences <project-prefs>`
66+
67+
68+
69+
The Secure Folder
2670
-----------------
2771

28-
If you are creating your own testing scripts or unit tests you can create a folder named ``.guides\secure`` to safely store test files. The secure directory is not copied over to the student version of the assignment.
29-
When a student runs a code test that references a file in the ``.guides\secure`` folder, an **ephemeral** container is created that is a combination of the student workspace and the ``.guides\secure`` folder the instructor created.
30-
This container is referred to as **ephemeral** because it only exists during the execution of the assessment. If you open a student project to view their work, you will not have access to the secure folder because it does not exist in the student environment.
72+
Overview
73+
~~~~~~~~
74+
75+
The ``.guides/secure`` folder provides a protected location for storing testing scripts and unit tests. This folder offers enhanced security because it is **not copied** to the student version of the assignment.
76+
77+
How It Works
78+
~~~~~~~~~~~~
79+
80+
When a student runs a code test that references files in ``.guides/secure``, an **ephemeral container** is created that combines:
81+
82+
- The student workspace
83+
- The instructor's ``.guides/secure`` folder
84+
85+
This container is called "ephemeral" because it only exists during assessment execution. If you open a student project to review their work, you will not see the secure folder—it does not exist in the student environment.
86+
87+
Important Limitations
88+
~~~~~~~~~~~~~~~~~~~~~
89+
90+
91+
**Do not write to the secure folder** during test execution, as changes will not persist. All output must be conveyed through:
92+
93+
- Standard output (stdout)
94+
- The feedback buffer (described in individual assessment documentation)
95+
96+
Assessment Types Supporting Secure Folder
97+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
98+
99+
The following assessment types can access the secure folder:
100+
101+
- :ref:`Advanced Code Test <advanced-code-test>`
102+
- :ref:`Free Text Autograde <free-text-autograde>`
103+
- :ref:`Assignment Level Scripts <auto-grade-scripts>`
104+
105+
Using Secure Folder with Standard Code Tests
106+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
107+
108+
:ref:`Standard Code Tests <standard-code-test>` do not have access to the secure folder by default. However, you can force a Standard Code Test to run in an **ephemeral container** by including a file path with ``.guides/secure`` in the **Command** field on the **Execution** tab.
109+
110+
Implementation Methods
111+
~~~~~~~~~~~~~~~~~~~~~~
112+
113+
You can trigger ephemeral container creation by:
31114

32-
Your test should not write any information to the ``.guides\secure`` folder because it does not persist.
33-
All output should conveyed using either stdout or the feedback buffer that is described in the individual assessesment types that can use the secure folder, :ref:`Advanced Code Test <advanced-code-test>`, :ref:`Free Text Autograde <free-text-autograde>` and :ref:`Assignment Level Scripts <auto-grade-scripts>`.
115+
- Passing a secure folder file name as a parameter
116+
- Including the file name on the command line (it will be ignored by your program, but signals Codio to mount the secure folder)
34117

35-
Standard code tests do not have access to the secure folder by default but you *can* force a :ref:`Standard Code Test <standard-code-test>` to run in an **ephemeral** container by including a file name with the ``.guides\secure`` path in the **Command** portion on the **Execution** tab.
36-
This could be done if you are passing a file name as a parameter or if you have no command line parameters, you can put the file name on the command line and it will be ignored by your program but Codio will run the test in a container with the Secure folder mounted. Then you can specify a file in ``.guides\secure`` as an input parameter.
118+
Once the ephemeral container is created, you can specify files in ``.guides/secure`` as input parameters for your tests.

source/instructors/authoring/assessments/assessments.rst

Lines changed: 41 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,17 +3,55 @@
33

44
.. _assessments:
55

6+
67
Assessments
78
===========
8-
Assessments can be used to determine how well students understand course material, and can be automatically or manually graded. Codio offers a wide range of assessment types, including auto-graded code tests, multiple choice tests, fill in the blanks, drop-down selection, free text responses and assignment grading. Assessments can be interspersed throughout tutorial materials or stand alone using an :ref:`assignment level script <auto-grade-scripts>`.
9+
10+
Assessments can be used to determine how well students understand course material, and can be automatically or manually graded. Codio offers a wide range of assessment types, including auto-graded code tests, multiple choice tests, fill in the blanks, drop-down selection, free text responses, and assignment grading. Assessments can be interspersed throughout tutorial materials or stand alone using an :ref:`assignment level script <auto-grade-scripts>`.
911

1012
You can view the results of assessments in a course from the teacher dashboard.
1113

12-
Review the following topics to gain an understanding of using assessments:
14+
**Assessment Topics**
1315

1416
- :ref:`Ungraded Assessments <ungraded-assessments>`
1517
- :ref:`Add a New Assessment <add-assessment>`
1618
- :ref:`Edit an Assessment <edit-assessment>`
1719
- :ref:`Edit Points for an Assessment <edit-assessment-points>`
1820
- :ref:`Delete an Assessment <delete-assessment>`
19-
- :ref:`Assessment Security <assessment-security>`
21+
- :ref:`Assessment Security <assessment-security>`
22+
23+
Common Assessment Configuration
24+
--------------------------------
25+
26+
The following configuration options are common to all assessment types:
27+
28+
General Settings
29+
~~~~~~~~~~~~~~~~
30+
31+
.. image:: /img/guides/assessment_general.png
32+
:alt: General
33+
34+
- **Name** - Enter a short name that describes the test. This name is displayed in the teacher dashboard, so it should clearly reflect the challenge. Toggle the **Show Name** setting to hide the name in the challenge text the student sees.
35+
36+
- **Instructions** - Enter text that is shown to the student using optional Markdown formatting.
37+
38+
Metadata (Optional)
39+
~~~~~~~~~~~~~~~~~~~
40+
41+
.. image:: /img/guides/assessment_metadata.png
42+
:alt: Metadata
43+
44+
- **Bloom's Level** - Select the appropriate level of `Bloom's Taxonomy <https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/>`_ for the current assessment.
45+
46+
- **Learning Objectives** - Specify the educational goals of the assessment. Objectives typically begin with "Students Will Be Able To (SWBAT)." For example: *SWBAT follow the flow of recursive execution*.
47+
48+
- **Tags** - The **Content** and **Programming Language** tags are provided and required. To add custom tags, click **Add Tag** and enter the name and values.
49+
50+
Files (Optional)
51+
~~~~~~~~~~~~~~~~
52+
53+
.. image:: /img/guides/assessment_files.png
54+
:alt: Files
55+
56+
Check the boxes for any additional external files to include with the assessment when adding it to an assessment library. Selected files appear in the **Additional content** list.
57+

0 commit comments

Comments
 (0)