Skip to content

Commit 6d0c430

Browse files
leoaulasneo98UsamaSadiq
leoaulasneo98
authored and
UsamaSadiq
committed
feat: implement XqueueViewSet with full xqueue-watcher compatibility
 - Add XqueueViewSet with complete xqueue-watcher service compatibility - Implement get_submission service for retrieving pending submissions - Add put_result service with row-level locking (select_for_update(nowait=True)) to prevent race conditions - Ensure concurrent xqueue-watcher instances process each submission exactly once, even under high load - Implement standardized response format for backward compatibility - Add session management and authentication handling for XWatcher clients - Add comprehensive test coverage for core interactions feat: Improve transaction handling to prevent concurrent processing and add timeout mechanism (Get Submission) - Implemented locking to ensure submissions are processed by a single xqueue watcher. - Added timeout mechanism for submissions stuck in 'pulled' state. - Updated tests to cover new error scenarios and timeout handling. feat: Optimize submission retrieval and processing flow - Renamed variables from 'submission_record' to 'external_grader' throughout the codebase for better consistency with model naming - Added new 'retry' status to integrate with submission processing retry services - Removed unused 'is_processable' method that wasn't providing any value - Enhanced test coverage - Add new status external grader detail migration feat: Add event emission for external grader scores This commit adds the EXTERNAL_GRADER_SCORE_SUBMITTED event emission to the put_result endpoint in the XQueueViewSet. When a grader submits a result and the score is successfully saved, the system now emits an event with all the necessary information for the LMS to render the graded XBlock. Key changes: - Add queue_key field to ExternalGraderDetail model - Include queue_key in create_external_grader_detail method - Emit EXTERNAL_GRADER_SCORE_SUBMITTED event after successful score update - Implement robust false to propagate error and put submission in pending queue again - Add migration for the new queue_key field - Add openedx-events dependency This enables the event-driven approach for updating XBlocks with scoring data from the edx-submissions service, supporting the gradual migration away from HTTP-based XQueue callbacks.
1 parent 81cc930 commit 6d0c430

18 files changed

Lines changed: 458 additions & 399 deletions

‎docs/source/decisions/0004-transaction-handling-in-get_submission‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
Improving Transaction Handling in Get Submission
2+
================================================
3+
4+
Status
5+
------
6+
7+
**Provisional** *2025-03-19*
8+
9+
Context
10+
-------
11+
12+
We identified a concurrency issue in the submission processing flow. Multiple instances of the xqueue watcher were processing the same transaction simultaneously, causing duplicate handling of submissions. Additionally, submissions remained stuck in the "pulled" state indefinitely when errors or unexpected interruptions occurred during processing.
13+
14+
Decision
15+
--------
16+
17+
We introduced a robust locking mechanism at the database level to prevent concurrent processing of the same submission. Specifically, we employed database row-level locking (`select_for_update(nowait=True)`) on submissions to ensure only one xqueue watcher instance can process a transaction at a time. Additionally, we implemented a timeout-based fallback, marking submissions in the "pulled" state as available again if they exceed a defined timeout period (5 minutes).
18+
19+
Consequences
20+
------------
21+
22+
Positive
23+
~~~~~~~~
24+
25+
- Ensures data consistency by preventing duplicate transaction processing.
26+
- Implements automatic recovery for submissions stuck in the "pulled" state, improving overall system reliability.
27+
- Minimal impact on system performance due to efficient database-level locking.
28+
29+
Negative
30+
~~~~~~~~
31+
32+
- Slightly increased complexity in transaction handling logic.
33+
- Potential minor latency increase when contention occurs due to database locks.

‎requirements/base.txt‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,9 @@
44
#
55
# make upgrade
66
#
7-
asgiref==3.9.1
7+
asgiref==3.9.2
88
# via django
9+
django==4.2.25
910
django==4.2.25
1011
# via
1112
# -c common_constraints.txt
@@ -23,8 +24,8 @@ edx-django-release-util==1.5.0
2324
jsonfield==3.2.0
2425
# via -r base.in
2526
pytz==2025.2
26-
# via -r base.in
27-
pyyaml==6.0.2
27+
# via -r requirements/base.in
28+
pyyaml==6.0.3
2829
# via edx-django-release-util
2930
six==1.17.0
3031
# via edx-django-release-util

‎requirements/ci.txt‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ colorama==0.4.6
2020
# via
2121
# -r requirements/tox.txt
2222
# tox
23-
coverage[toml]==7.10.5
23+
coverage[toml]==7.10.7
2424
# via coveralls
2525
coveralls==4.0.1
2626
# via -r requirements/ci.in
@@ -57,7 +57,7 @@ pyproject-api==1.9.1
5757
# tox
5858
requests==2.32.5
5959
# via coveralls
60-
tox==4.28.4
60+
tox==4.30.2
6161
# via -r requirements/tox.txt
6262
urllib3==2.5.0
6363
# via requests

‎requirements/dev.txt‎

Lines changed: 113 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ alabaster==1.0.0
1414
# -r docs.txt
1515
# -r test.txt
1616
# sphinx
17-
asgiref==3.9.1
17+
asgiref==3.9.2
1818
# via
1919
# -r base.txt
2020
# -r docs.txt
@@ -25,32 +25,41 @@ astroid==3.3.11
2525
# -r test.txt
2626
# pylint
2727
# pylint-celery
28+
attrs==25.3.0
29+
# via
30+
# -r requirements/test.txt
31+
# openedx-events
2832
babel==2.17.0
2933
# via
3034
# -r docs.txt
3135
# -r test.txt
3236
# pydata-sphinx-theme
3337
# sphinx
34-
beautifulsoup4==4.13.5
38+
beautifulsoup4==4.14.2
3539
# via
3640
# -r docs.txt
3741
# -r test.txt
3842
# pydata-sphinx-theme
3943
certifi==2025.8.3
4044
# via
41-
# -r docs.txt
42-
# -r test.txt
45+
# -r requirements/docs.txt
46+
# -r requirements/test.txt
4347
# requests
48+
cffi==2.0.0
49+
# via
50+
# -r requirements/test.txt
51+
# pynacl
4452
charset-normalizer==3.4.3
4553
# via
4654
# -r docs.txt
4755
# -r test.txt
4856
# requests
49-
click==8.2.1
57+
click==8.3.0
5058
# via
5159
# -r test.txt
5260
# click-log
5361
# code-annotations
62+
# edx-django-utils
5463
# edx-lint
5564
click-log==0.4.0
5665
# via
@@ -60,7 +69,7 @@ code-annotations==2.3.0
6069
# via
6170
# -r test.txt
6271
# edx-lint
63-
coverage[toml]==7.10.5
72+
coverage[toml]==7.10.7
6473
# via
6574
# -r test.txt
6675
# pytest-cov
@@ -70,45 +79,81 @@ dill==0.4.0
7079
# via
7180
# -r test.txt
7281
# pylint
82+
django==4.2.25
7383
django==4.2.25
7484
# via
75-
# -c common_constraints.txt
76-
# -r base.txt
77-
# -r docs.txt
85+
# -c requirements/common_constraints.txt
86+
# -r requirements/base.txt
87+
# -r requirements/docs.txt
88+
# -r requirements/test.txt
89+
# django-crum
7890
# django-model-utils
91+
# django-waffle
7992
# djangorestframework
8093
# edx-django-release-util
94+
# edx-django-utils
8195
# jsonfield
96+
# openedx-events
97+
django-crum==0.7.9
98+
# via
99+
# -r requirements/test.txt
100+
# edx-django-utils
82101
django-model-utils==5.0.0
83102
# via
84-
# -r base.txt
85-
# -r docs.txt
86-
# -r test.txt
103+
# -r requirements/base.txt
104+
# -r requirements/docs.txt
105+
# -r requirements/test.txt
106+
django-waffle==5.0.0
107+
# via
108+
# -r requirements/test.txt
109+
# edx-django-utils
87110
djangorestframework==3.16.1
88111
# via
89-
# -r base.txt
90-
# -r docs.txt
112+
# -r requirements/base.txt
113+
# -r requirements/docs.txt
114+
# -r requirements/test.txt
115+
dnspython==2.8.0
116+
# via
117+
# -r requirements/test.txt
118+
# pymongo
91119
docutils==0.21.2
92120
# via
93121
# -r docs.txt
94122
# -r test.txt
95123
# pydata-sphinx-theme
96124
# sphinx
125+
edx-ccx-keys==2.0.2
126+
# via
127+
# -r requirements/test.txt
128+
# openedx-events
97129
edx-django-release-util==1.5.0
98130
# via
99-
# -r base.txt
100-
# -r docs.txt
101-
# -r test.txt
131+
# -r requirements/base.txt
132+
# -r requirements/docs.txt
133+
# -r requirements/test.txt
134+
edx-django-utils==8.0.1
135+
# via
136+
# -r requirements/test.txt
137+
# openedx-events
102138
edx-lint==5.3.7
103139
# via
104-
# -c constraints.txt
105-
# -r test.txt
140+
# -c requirements/constraints.txt
141+
# -r requirements/test.txt
142+
edx-opaque-keys[django]==3.0.0
143+
# via
144+
# -r requirements/test.txt
145+
# edx-ccx-keys
146+
# openedx-events
106147
factory-boy==3.3.3
107-
# via -r test.txt
108-
faker==37.6.0
148+
# via -r requirements/test.txt
149+
faker==37.8.0
109150
# via
110-
# -r test.txt
151+
# -r requirements/test.txt
111152
# factory-boy
153+
fastavro==1.12.0
154+
# via
155+
# -r requirements/test.txt
156+
# openedx-events
112157
freezegun==1.5.5
113158
# via -r test.txt
114159
idna==3.10
@@ -125,7 +170,7 @@ iniconfig==2.1.0
125170
# via
126171
# -r test.txt
127172
# pytest
128-
isort==6.0.1
173+
isort==6.1.0
129174
# via
130175
# -r test.txt
131176
# pylint
@@ -137,10 +182,10 @@ jinja2==3.1.6
137182
# sphinx
138183
jsonfield==3.2.0
139184
# via
140-
# -r base.txt
141-
# -r docs.txt
142-
# -r test.txt
143-
markupsafe==3.0.2
185+
# -r requirements/base.txt
186+
# -r requirements/docs.txt
187+
# -r requirements/test.txt
188+
markupsafe==3.0.3
144189
# via
145190
# -r docs.txt
146191
# -r test.txt
@@ -150,7 +195,9 @@ mccabe==0.7.0
150195
# -r test.txt
151196
# pylint
152197
mock==5.2.0
153-
# via -r test.txt
198+
# via -r requirements/test.txt
199+
openedx-events==10.2.0
200+
# via -r requirements/test.txt
154201
packaging==25.0
155202
# via
156203
# -r docs.txt
@@ -169,11 +216,19 @@ pluggy==1.6.0
169216
# pytest-cov
170217
pockets==0.9.1
171218
# via
172-
# -r docs.txt
173-
# -r test.txt
219+
# -r requirements/docs.txt
220+
# -r requirements/test.txt
174221
# sphinxcontrib-napoleon
222+
psutil==7.1.0
223+
# via
224+
# -r requirements/test.txt
225+
# edx-django-utils
175226
pycodestyle==2.14.0
176-
# via -r test.txt
227+
# via -r requirements/test.txt
228+
pycparser==2.23
229+
# via
230+
# -r requirements/test.txt
231+
# cffi
177232
pydata-sphinx-theme==0.15.4
178233
# via
179234
# -r docs.txt
@@ -207,13 +262,21 @@ pylint-plugin-utils==0.9.0
207262
# -r test.txt
208263
# pylint-celery
209264
# pylint-django
210-
pytest==8.4.1
265+
pymongo==4.15.2
211266
# via
212-
# -r test.txt
267+
# -r requirements/test.txt
268+
# edx-opaque-keys
269+
pynacl==1.6.0
270+
# via
271+
# -r requirements/test.txt
272+
# edx-django-utils
273+
pytest==8.4.2
274+
# via
275+
# -r requirements/test.txt
213276
# pytest-cov
214277
# pytest-django
215-
pytest-cov==6.2.1
216-
# via -r test.txt
278+
pytest-cov==7.0.0
279+
# via -r requirements/test.txt
217280
pytest-django==4.11.1
218281
# via -r test.txt
219282
python-dateutil==2.9.0.post0
@@ -226,10 +289,10 @@ python-slugify==8.0.4
226289
# code-annotations
227290
pytz==2025.2
228291
# via
229-
# -r base.txt
230-
# -r docs.txt
231-
# -r test.txt
232-
pyyaml==6.0.2
292+
# -r requirements/base.txt
293+
# -r requirements/docs.txt
294+
# -r requirements/test.txt
295+
pyyaml==6.0.3
233296
# via
234297
# -r base.txt
235298
# -r docs.txt
@@ -248,9 +311,10 @@ roman-numerals-py==3.1.0
248311
# sphinx
249312
six==1.17.0
250313
# via
251-
# -r base.txt
252-
# -r docs.txt
253-
# -r test.txt
314+
# -r requirements/base.txt
315+
# -r requirements/docs.txt
316+
# -r requirements/test.txt
317+
# edx-ccx-keys
254318
# edx-django-release-util
255319
# edx-lint
256320
# pockets
@@ -261,7 +325,7 @@ snowballstemmer==3.0.1
261325
# -r docs.txt
262326
# -r test.txt
263327
# sphinx
264-
soupsieve==2.7
328+
soupsieve==2.8
265329
# via
266330
# -r docs.txt
267331
# -r test.txt
@@ -320,6 +384,8 @@ stevedore==5.5.0
320384
# via
321385
# -r test.txt
322386
# code-annotations
387+
# edx-django-utils
388+
# edx-opaque-keys
323389
text-unidecode==1.3
324390
# via
325391
# -r test.txt
@@ -333,6 +399,7 @@ typing-extensions==4.15.0
333399
# -r docs.txt
334400
# -r test.txt
335401
# beautifulsoup4
402+
# edx-opaque-keys
336403
# pydata-sphinx-theme
337404
tzdata==2025.2
338405
# via
@@ -343,3 +410,6 @@ urllib3==2.5.0
343410
# -r docs.txt
344411
# -r test.txt
345412
# requests
413+
414+
# The following packages are considered to be unsafe in a requirements file:
415+
# setuptools

0 commit comments

Comments
 (0)