Migrate from djcelery to django_celery_results

[spaceone]

Hello,

Is there a documentation about migrating from djcelery to django_celery_results?

It seems there were 6 tables:

 public | djcelery_crontabschedule   | table | 
 public | djcelery_intervalschedule  | table | 
 public | djcelery_periodictask      | table | 
 public | djcelery_periodictasks     | table | 
 public | djcelery_taskstate         | table | 
 public | djcelery_workerstate       | table | 

And now only one:

public | django_celery_results_taskresult | table |

[spaceone]

I need this especially / only for from djcelery.models import TaskMeta → from django_celery_results.models import TaskResult (don't know if djcerely provides more than that).

[spaceone]

Calling python3 manage.py migrate django_celery_results raises:

Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3/dist-packages/…/manage.py", line 24, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/lib/python3/dist-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/lib/python3/dist-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/usr/lib/python3/dist-packages/django/core/management/commands/migrate.py", line 86, in handle
    executor.loader.check_consistent_history(connection)
  File "/usr/lib/python3/dist-packages/django/db/migrations/loader.py", line 299, in check_consistent_history
    connection.alias,
django.db.migrations.exceptions.InconsistentMigrationHistory: Migration myapp.0001_initial is applied before its dependency django_celery_results.0001_initial on database 'default'.

[spaceone]

If I want to create a django migration with both djcelery and django_celery_results installed I have a dependency problem:
djcelery depends on python-celery < 4 while django_celery_results depends on python-celery >= 4:

So I get either this or that traceback:

Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/lib/python2.7/dist-packages/myapp/manage.py", line 24, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line 341, in execute
    django.setup()
  File "/usr/lib/python2.7/dist-packages/django/__init__.py", line 27, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/lib/python2.7/dist-packages/django/apps/registry.py", line 108, in populate
    app_config.import_models(all_models)
  File "/usr/lib/python2.7/dist-packages/django/apps/config.py", line 199, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/lib/python2.7/dist-packages/myapp/models.py", line 52, in <module>
    from django_celery_results.models import TaskResult  # noqa: F811
  File "/usr/lib/python2.7/dist-packages/django_celery_results/models.py", line 9, in <module>
    from celery.five import python_2_unicode_compatible
ImportError: cannot import name python_2_unicode_compatible

Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/lib/python2.7/dist-packages/myapp/manage.py", line 24, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line 341, in execute
    django.setup()
  File "/usr/lib/python2.7/dist-packages/django/__init__.py", line 27, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/lib/python2.7/dist-packages/django/apps/registry.py", line 108, in populate
    app_config.import_models(all_models)
  File "/usr/lib/python2.7/dist-packages/django/apps/config.py", line 199, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/lib/python2.7/dist-packages/djcelery/models.py", line 15, in <module>
    from celery.utils.timeutils import timedelta_seconds
ImportError: No module named timeutils

[spaceone]

When I rename celery to celery4 and kombu to kombu2 both libraries can be imported at the same time.
manage.py sqlmigrate … now helps a little bit to get some SQL commands.

Workaround:

su - postgres -c "psql -d '$POSTGRES_DB' -c \"delete from django_migrations where app='import_api';\""
python3 migrate.py django_celery_results
su - postgres -c "psql -d '$POSTGRES_DB' -c 'ALTER TABLE "foo_bar" DROP CONSTRAINT "foo_barj_result_id_7383b884_fk_celery_taskmeta_id";'"
su - postgres -c "psql -d '$POSTGRES_DB' -c 'ALTER TABLE "foo_bar" ADD CONSTRAINT "foo_bar_result_id_7383b884_fk_django_celery_results_taskresult_id" FOREIGN KEY ("result_id") REFERENCES "django_celery_results_taskresult" ("id") DEFERRABLE INITIALLY DEFERRED;'"
python3 migrate.py --fake-initial foo

[auvipy]

When I rename celery to celery4 and kombu to kombu2 both libraries can be imported at the same time. manage.py sqlmigrate … now helps a little bit to get some SQL commands.

Workaround:

su - postgres -c "psql -d '$POSTGRES_DB' -c \"delete from django_migrations where app='import_api';\""
python3 migrate.py django_celery_results
su - postgres -c "psql -d '$POSTGRES_DB' -c 'ALTER TABLE "foo_bar" DROP CONSTRAINT "foo_barj_result_id_7383b884_fk_celery_taskmeta_id";'"
su - postgres -c "psql -d '$POSTGRES_DB' -c 'ALTER TABLE "foo_bar" ADD CONSTRAINT "foo_bar_result_id_7383b884_fk_django_celery_results_taskresult_id" FOREIGN KEY ("result_id") REFERENCES "django_celery_results_taskresult" ("id") DEFERRABLE INITIALLY DEFERRED;'"
python3 migrate.py --fake-initial foo

this approach could be documented

[auvipy]

• Migrating from djcelery django-celery-beat#151 (comment)