Skip to content

replace % with mod() SQL function to avoid invalid double-percentage …#923

Open
AngryUbuntuNerd wants to merge 4 commits into
celery:mainfrom
AngryUbuntuNerd:fix-invalid-sql
Open

replace % with mod() SQL function to avoid invalid double-percentage …#923
AngryUbuntuNerd wants to merge 4 commits into
celery:mainfrom
AngryUbuntuNerd:fix-invalid-sql

Conversation

@AngryUbuntuNerd

Copy link
Copy Markdown

fixes #922

invalid SQL before:

SELECT
	`django_celery_beat_periodictask`.`id`,
	`django_celery_beat_periodictask`.`name`,
	`django_celery_beat_periodictask`.`task`,
	`django_celery_beat_periodictask`.`interval_id`,
	`django_celery_beat_periodictask`.`crontab_id`,
	`django_celery_beat_periodictask`.`solar_id`,
	`django_celery_beat_periodictask`.`clocked_id`,
	`django_celery_beat_periodictask`.`args`,
	`django_celery_beat_periodictask`.`kwargs`,
	`django_celery_beat_periodictask`.`queue`,
	`django_celery_beat_periodictask`.`exchange`,
	`django_celery_beat_periodictask`.`routing_key`,
	`django_celery_beat_periodictask`.`headers`,
	`django_celery_beat_periodictask`.`priority`,
	`django_celery_beat_periodictask`.`expires`,
	`django_celery_beat_periodictask`.`expire_seconds`,
	`django_celery_beat_periodictask`.`one_off`,
	`django_celery_beat_periodictask`.`start_time`,
	`django_celery_beat_periodictask`.`enabled`,
	`django_celery_beat_periodictask`.`last_run_at`,
	`django_celery_beat_periodictask`.`total_run_count`,
	`django_celery_beat_periodictask`.`date_changed`,
	`django_celery_beat_periodictask`.`description`
FROM
	`django_celery_beat_periodictask`
LEFT OUTER JOIN `django_celery_beat_clockedschedule` ON
	(`django_celery_beat_periodictask`.`clocked_id` = `django_celery_beat_clockedschedule`.`id`)
WHERE
	(`django_celery_beat_periodictask`.`enabled` = 1
		AND NOT (((`django_celery_beat_clockedschedule`.`clocked_time` > '2025-07-29 14:05:32.092804'
			AND `django_celery_beat_clockedschedule`.`clocked_time` IS NOT NULL
			AND `django_celery_beat_periodictask`.`clocked_id` IS NOT NULL)
		OR (`django_celery_beat_periodictask`.`crontab_id` IS NOT NULL
			AND `django_celery_beat_periodictask`.`crontab_id` IN (
			SELECT
				U0.`id` AS `id`
			FROM
				`django_celery_beat_crontabschedule` U0
			WHERE
				(U0.`hour` REGEXP BINARY '^\\d+$'
					AND NOT (CASE
						WHEN (U0.`timezone` = 'UTC') THEN (((CAST(U0.`hour` AS signed integer) + 0) + 24) %% 24)
						ELSE CAST(U0.`hour` AS signed integer)
					END IN (12, 13, 14, 15, 16, 4))))
				AND `django_celery_beat_periodictask`.`crontab_id` IS NOT NULL))))

new SQL:

SELECT
	`django_celery_beat_periodictask`.`id`,
	`django_celery_beat_periodictask`.`name`,
	`django_celery_beat_periodictask`.`task`,
	`django_celery_beat_periodictask`.`interval_id`,
	`django_celery_beat_periodictask`.`crontab_id`,
	`django_celery_beat_periodictask`.`solar_id`,
	`django_celery_beat_periodictask`.`clocked_id`,
	`django_celery_beat_periodictask`.`args`,
	`django_celery_beat_periodictask`.`kwargs`,
	`django_celery_beat_periodictask`.`queue`,
	`django_celery_beat_periodictask`.`exchange`,
	`django_celery_beat_periodictask`.`routing_key`,
	`django_celery_beat_periodictask`.`headers`,
	`django_celery_beat_periodictask`.`priority`,
	`django_celery_beat_periodictask`.`expires`,
	`django_celery_beat_periodictask`.`expire_seconds`,
	`django_celery_beat_periodictask`.`one_off`,
	`django_celery_beat_periodictask`.`start_time`,
	`django_celery_beat_periodictask`.`enabled`,
	`django_celery_beat_periodictask`.`last_run_at`,
	`django_celery_beat_periodictask`.`total_run_count`,
	`django_celery_beat_periodictask`.`date_changed`,
	`django_celery_beat_periodictask`.`description`
FROM
	`django_celery_beat_periodictask`
LEFT OUTER JOIN `django_celery_beat_clockedschedule` ON
	(`django_celery_beat_periodictask`.`clocked_id` = `django_celery_beat_clockedschedule`.`id`)
WHERE
	(`django_celery_beat_periodictask`.`enabled` = 1
		AND NOT (((`django_celery_beat_clockedschedule`.`clocked_time` > '2025-07-29 10:42:42.813538'
			AND `django_celery_beat_clockedschedule`.`clocked_time` IS NOT NULL
			AND `django_celery_beat_periodictask`.`clocked_id` IS NOT NULL)
		OR (`django_celery_beat_periodictask`.`crontab_id` IS NOT NULL
			AND `django_celery_beat_periodictask`.`crontab_id` IN (
			SELECT
				U0.`id` AS `id`
			FROM
				`django_celery_beat_crontabschedule` U0
			WHERE
				(U0.`hour` REGEXP BINARY '^\\d+$'
					AND NOT (CASE
						WHEN (U0.`timezone` = 'UTC') THEN MOD(((CAST(U0.`hour` AS signed integer) + 0) + 24), 24)
						ELSE CAST(U0.`hour` AS signed integer)
					END IN (12, 13, 14, 15, 16, 4))))
				AND `django_celery_beat_periodictask`.`crontab_id` IS NOT NULL))))

@auvipy auvipy requested review from auvipy and Copilot July 29, 2025 15:15

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes a SQL generation issue where Django was producing invalid double-percentage (%%) operators in MySQL queries. The fix replaces the Python modulo operator (%) with Django's Func class using the SQL MOD() function to ensure proper SQL syntax.

Key Changes

  • Replaces Python modulo operator with Django's Func class using MOD function
  • Wraps the expression in ExpressionWrapper with proper output field type
  • Adds necessary imports for ExpressionWrapper and Func

Comment thread django_celery_beat/schedulers.py
@codecov

codecov Bot commented Jul 29, 2025

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.18%. Comparing base (f1f239c) to head (e992587).
⚠️ Report is 70 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #923      +/-   ##
==========================================
- Coverage   88.19%   88.18%   -0.02%     
==========================================
  Files          32       32              
  Lines        1008     1007       -1     
  Branches      105      105              
==========================================
- Hits          889      888       -1     
  Misses        101      101              
  Partials       18       18              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@auvipy auvipy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is nit pissible to modify the relevant tests as well?

@AngryUbuntuNerd

Copy link
Copy Markdown
Author

can you give me some guidance there:

  1. which test needs modifying?
  2. how do I run the tests locally? the README does not seem to say

@auvipy

auvipy commented Aug 3, 2025

Copy link
Copy Markdown
Member

this should be the primary file https://github.com/celery/django-celery-beat/blob/main/t/unit/test_schedulers.py

and for tests there are docker and tox for running tests locally

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

@auvipy auvipy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you please revisit this to update or add relevant tests?

@auvipy auvipy requested a review from Copilot October 6, 2025 10:57

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: Invalid MySQL query using version 2.8.1

3 participants