[ENG-10538] Post-NR Project PR (#11623)

* Create global_file_updated and global_reviews subscriptions if missing
* Use USER_FILE_UPDATED for group global_file_updated
* Use REVIEWS_SUBMISSION_STATUS for group global_reviews
* Add missing `is_digest=True` for new OSF user subscriptions
* Extend otf subscription creation to apply to _node_file_updated group
* Fix typo for `_is_digest`
* add is_digest_type property
* Add is_digest_type property to NotificationType and log message in emit method
* Fix is_digest handling in NotificationType.emit method
* Log message update in NotificationType.emit method for is_digest handling
* fix unit tests
* Move set-deafult-subscriptions out of non-effective try-except
* Add fake_sent field to Notification model and update notification creation logic
* add unique_together constraint
* Add 'PARTIAL_SUCCESS' status to EmailTask model and update email task handling logic
* NR migration [ENG-10040, ENG-10025, ENG-9854]
* Remove subscription if notifications.tasks.send_moderator_email_task fails with permission error
* Apply suggestion from @Ostap-Zherebetskyi
* remove datetime
* Add 'no_login_email_last_sent' field to OSFUser and update email task logic
* Enforce and improve permission check for subscriptions
* Fix typo in annotated_obj_qs for NODE_FILE_UPDATED
* Add unit tests for testing node_file_updated subscription detail
* Implement notifications cleanup task and related settings; add tests for functionality
* Fix legacy subscription ID for NODE_FILE_UPDATED: "guid_files_updated"
* Fix duplicate and mismatched type NODE_FILE(S)_UPDATED
* removed email.py
* Fix annotated qs for global reviews and update unit tests
* Update tests for node_file(s)_updated subscription detail
* Rename fixtures for notification subscription detail tests
* Annotate with legacy_id for serializer to handle created subscriptions
* Add unit tests for creating missing subscriptions on the fly
* clear useless code
* Rename migration name for NR post-release
* Improve unit test: test_emit_frequency_none
* Remove `seen` from `Notification` and re-make migrations
* `mark_sent()` now handles `fake_sent=True`, and only `save()` once
* Update default settings
* remove useless import
* Enhance SubscriptionList queryset with additional content types and refactor notification type handling
* fix unit test
* Refactor SubscriptionList queryset to use a single provider content type and update related notification handling; add script to update notification subscriptions' content types.
* fix CR comments
* Update comments
* split into 3 files
* remove populate_notification_subscriptions
* Renamed files, refactor of populate notification subscriptions user global file updated
* added try/except, added timers
* converted populate_notification_subscriptions_user_global_reviews.py
* fix batch time execution
* converted populate_notification_subscriptions_node_file_updated
* convert to separate update and create scripts
* updated, added parameters
* move to remove_after_use
* add time track to last batch, fix proper time track for batch in node_file_updated
* convert to use review_nt
* Fix unit tests due to new constraints
* Move missing subscription creation to a helper function in utils
* Subscription list view now creates missing attributes on-the-fly
  * Note: only works for when filter on "legacy id"; logs message to sentry for tracking purpose when filter on "event names".
* Fix broken `.exists()` due to complex annotated QS with `.distinct()`
* fix names of tasks
* Add notification subscriptions de-duplication v2 command
* fix naming
* Add deduplication command to notification migration
* Fix notification handling by updating legacy ID suffix from '_files_updated' to '_file_updated' across utils and views; adjust related test cases accordingly.
* fix unit tests
* Update the name of the NotificationType NODE_FILE_UPDATED to be consistent
* Rename notification type from node_files_updated to node_file_updated for consistency
* clean keys
* fix naming in templates, event_context
* remove leftover comments
* Refactor notification type references to use NotificationTypeEnum
  - Updated all instances of NotificationType.Type to NotificationTypeEnum in test files and application code.
  - Ensured consistency in notification type usage across various tests including auth, claims, events, and more.
  - This change improves clarity and maintainability by standardizing the notification type references.
* Fix notification type reference for FILE_UPDATED
* Refactor notification type references to use NotificationTypeEnum across views and scripts
* fix unit tests
* remove imports
* fix USER_CROSSREF_DOI_PENDING event_context
* sync naming
* added explanation
* Add Celery task to disable removed beat tasks from PeriodicTask entries
* Apply suggestion from @Ostap-Zherebetskyi
* Apply suggestion from @cslzchen
* Add migrations to remove duplicate notification subscriptions and refactor notification model
* removed comment
* Refactor no-login email filtering to ensure last sent date is before last login date
* Add test to exclude users logged in before no_login_email_last_sent
* Refactor no-login email user filtering to exclude those with pending EmailTasks and recent no-login emails
* Fix CollectionSubmission URLs to use absolute_url method for consistency
* Refactor logo handling in notification methods to improve consistency and readability
* Add management command to sync notification templates
* Apply suggestion from @cslzchen
* Add management command for deleting withdrawn or failed registration files to CeleryConfig
* fix: update logo handling in send_moderator_email_task for favicon support

---------

Co-authored-by: Longze Chen <cslzchen@gmail.com>
Co-authored-by: Ostap Zherebetskyi <ozherebetskyi@exoft.net>
Co-authored-by: Bohdan Odintsov <bodintsov@exoft.net>

Author: 3 people

addons/base/views.py

@@ -51,7 +51,7 @@
     DraftRegistration,
     Guid,
     FileVersionUserMetadata,
-    FileVersion, NotificationType
+    FileVersion, NotificationTypeEnum
 )
 from osf.metrics import PreprintView, PreprintDownload
 from osf.utils import permissions
@@ -575,14 +575,13 @@ def create_waterbutler_log(payload, **kwargs):
 
             if payload.get('email') or payload.get('errors'):
                 if payload.get('email'):
-                    notification_type = NotificationType.Type.USER_FILE_OPERATION_SUCCESS.instance
+                    notification_type = NotificationTypeEnum.USER_FILE_OPERATION_SUCCESS.instance
                 if payload.get('errors'):
-                    notification_type = NotificationType.Type.USER_FILE_OPERATION_FAILED.instance
+                    notification_type = NotificationTypeEnum.USER_FILE_OPERATION_FAILED.instance
                 notification_type.emit(
                     user=user,
                     subscribed_object=node,
                     event_context={
-                        'user_fullname': user.fullname,
                         'action': payload['action'],
                         'source_node': source_node._id,
                         'source_node_title': source_node.title,

addons/boa/tasks.py

@@ -14,7 +14,7 @@
 from addons.boa.boa_error_code import BoaErrorCode
 from framework import sentry
 from framework.celery_tasks import app as celery_app
-from osf.models import OSFUser, NotificationType
+from osf.models import OSFUser, NotificationTypeEnum
 from osf.utils.fields import ensure_str, ensure_bytes
 from website import settings as osf_settings
 
@@ -183,18 +183,15 @@ async def submit_to_boa_async(host, username, password, user_guid, project_guid,
 
     logger.info('Successfully uploaded query output to OSF.')
     logger.debug('Task ends <<<<<<<<')
-    NotificationType.Type.ADDONS_BOA_JOB_COMPLETE.instance.emit(
+    NotificationTypeEnum.ADDONS_BOA_JOB_COMPLETE.instance.emit(
         user=user,
         event_context={
             'user_fullname': user.fullname,
-            'query_file_name': query_file_name,
             'query_file_full_path': file_full_path,
             'output_file_name': output_file_name,
             'job_id': boa_job.id,
             'project_url': project_url,
             'boa_job_list_url': boa_settings.BOA_JOB_LIST_URL,
-            'boa_support_email': boa_settings.BOA_SUPPORT_EMAIL,
-            'osf_support_email': osf_settings.OSF_SUPPORT_EMAIL,
         }
     )
     return BoaErrorCode.NO_ERROR
@@ -209,12 +206,11 @@ def handle_boa_error(message, code, username, fullname, project_url, query_file_
         sentry.log_message(message, skip_session=True)
     except Exception:
         pass
-    NotificationType.Type.ADDONS_BOA_JOB_FAILURE.instance.emit(
+    NotificationTypeEnum.ADDONS_BOA_JOB_FAILURE.instance.emit(
         destination_address=username,
         event_context={
             'user_fullname': fullname,
             'code': code,
-            'query_file_name': query_file_name,
             'file_size': file_size,
             'message': message,
             'max_file_size': boa_settings.MAX_SUBMISSION_SIZE,

addons/boa/tests/test_tasks.py

@@ -9,7 +9,7 @@
 from addons.boa import settings as boa_settings
 from addons.boa.boa_error_code import BoaErrorCode
 from addons.boa.tasks import submit_to_boa, submit_to_boa_async, handle_boa_error
-from osf.models import NotificationType
+from osf.models import NotificationTypeEnum
 from osf_tests.factories import AuthUserFactory, ProjectFactory
 from tests.base import OsfTestCase
 from tests.utils import capture_notifications
@@ -66,7 +66,7 @@ def test_handle_boa_error(self):
                         job_id=self.job_id
                     )
                 assert len(notifications['emits']) == 1
-                assert notifications['emits'][0]['type'] == NotificationType.Type.ADDONS_BOA_JOB_FAILURE
+                assert notifications['emits'][0]['type'] == NotificationTypeEnum.ADDONS_BOA_JOB_FAILURE
                 mock_sentry_log_message.assert_called_with(self.error_message, skip_session=True)
                 mock_logger_error.assert_called_with(self.error_message)
                 assert return_value == BoaErrorCode.UNKNOWN

addons/osfstorage/tests/test_models.py

@@ -9,7 +9,7 @@
 
 from framework.auth import Auth
 from addons.osfstorage.models import OsfStorageFile, OsfStorageFileNode, OsfStorageFolder
-from osf.models import BaseFileNode, NotificationType
+from osf.models import BaseFileNode, NotificationTypeEnum
 from osf.exceptions import ValidationError
 from osf.utils.permissions import WRITE, ADMIN
 
@@ -750,7 +750,7 @@ def test_after_fork_copies_versions(self, node, node_settings, auth_obj):
             fork = node.fork_node(auth_obj)
 
         assert len(notifications['emits']) == 1
-        assert notifications['emits'][0]['type'] == NotificationType.Type.NODE_CONTRIBUTOR_ADDED_DEFAULT
+        assert notifications['emits'][0]['type'] == NotificationTypeEnum.NODE_CONTRIBUTOR_ADDED_DEFAULT
         fork_node_settings = fork.get_addon('osfstorage')
         fork_node_settings.reload()
 

admin/management/urls.py

@@ -17,5 +17,7 @@
     re_path(r'^ingest_cedar_metadata_templates', views.IngestCedarMetadataTemplates.as_view(), name='ingest_cedar_metadata_templates'),
     re_path(r'^bulk_resync', views.BulkResync.as_view(), name='bulk-resync'),
     re_path(r'^empty_metadata_dataarchive_registration_bulk_resync', views.EmptyMetadataDataarchiveRegistrationBulkResync.as_view(),
-            name='empty-metadata-dataarchive-registration-bulk-resync')
+            name='empty-metadata-dataarchive-registration-bulk-resync'),
+    re_path(r'^sync_notification_templates', views.SyncNotificationTemplates.as_view(),
+            name='sync_notification_templates')
 ]

admin/management/views.py

@@ -11,6 +11,7 @@
 from osf.management.commands.monthly_reporters_go import monthly_reporters_go
 from osf.management.commands.fetch_cedar_metadata_templates import ingest_cedar_metadata_templates
 from osf.management.commands.sync_doi_metadata import sync_doi_metadata, sync_doi_empty_metadata_dataarchive_registrations
+from osf.management.commands.populate_notification_types import populate_notification_types
 from scripts.find_spammy_content import manage_spammy_content
 from django.urls import reverse
 from django.shortcuts import redirect
@@ -172,3 +173,11 @@ def post(self, request):
         })
         messages.success(request, 'Resyncing with DataCite! It will take some time.')
         return redirect(reverse('management:commands'))
+
+
+class SyncNotificationTemplates(ManagementCommandPermissionView):
+
+    def post(self, request):
+        populate_notification_types()
+        messages.success(request, 'Notification templates have been successfully synced.')
+        return redirect(reverse('management:commands'))

admin/providers/views.py

@@ -1,7 +1,7 @@
 from django.shortcuts import redirect
 from django.views.generic import TemplateView
 from django.contrib import messages
-from osf.models import RegistrationProvider, OSFUser, CollectionProvider, NotificationType
+from osf.models import RegistrationProvider, OSFUser, CollectionProvider, NotificationTypeEnum
 from website.settings import DOMAIN
 
 
@@ -63,7 +63,7 @@ def post(self, request, *args, **kwargs):
 
         context['provider_url'] = f'{provider.domain or DOMAIN}{provider_type_word}/{(provider._id if not provider.domain else '').strip('/')}'
         messages.success(request, f'The following {target_type} was successfully added: {target_user.fullname} ({target_user.username})')
-        notification_type = NotificationType.Type.PROVIDER_MODERATOR_ADDED
+        notification_type = NotificationTypeEnum.PROVIDER_MODERATOR_ADDED
         notification_type.instance.emit(
             user=target_user,
             event_context=context,

admin/templates/management/commands.html

@@ -152,6 +152,19 @@ <h4><u>Resync empty metadata dataarchive registrations with DataCite</u></h4>
                     </nav>
                 </form>
             </section>
+            <section>
+                <h4><u>Sync Notification Templates</u></h4>
+                <p>
+                    Use this management command to sync notification templates. Warning: existing templates modifications via django admin will be overridden if they haven't been updated in code. In addition, templates are cached for 2 hours so changes won't be effective immediately.
+                </p>
+                <form method="post"
+                      action="{% url 'management:sync_notification_templates'%}">
+                    {% csrf_token %}
+                    <nav>
+                        <input class="btn btn-success" type="submit" value="Run" />
+                    </nav>
+                </form>
+            </section>
         </div>
     </section>
 {% endblock %}

admin/users/views.py

@@ -20,7 +20,7 @@
 from osf.models.base import Guid
 from osf.models.user import OSFUser
 from osf.models.spam import SpamStatus
-from osf.models.notification_type import NotificationType
+from osf.models.notification_type import NotificationTypeEnum
 from framework.auth import get_user
 from framework.auth.core import generate_verification_key
 
@@ -184,12 +184,11 @@ def post(self, request, *args, **kwargs):
                 message=f'User account {user.pk} disabled',
                 action_flag=USER_REMOVED
             )
-            NotificationType.Type.USER_REQUEST_DEACTIVATION_COMPLETE.instance.emit(
+            NotificationTypeEnum.USER_REQUEST_DEACTIVATION_COMPLETE.instance.emit(
                 user=user,
                 event_context={
                     'user_fullname': user.fullname,
                     'contact_email': OSF_SUPPORT_EMAIL,
-                    'can_change_preferences': False,
                 }
             )
         else:

admin_tests/preprints/test_views.py

@@ -8,7 +8,7 @@
 from django.contrib.messages.storage.fallback import FallbackStorage
 
 from tests.base import AdminTestCase
-from osf.models import Preprint, PreprintLog, PreprintRequest, NotificationType
+from osf.models import Preprint, PreprintLog, PreprintRequest, NotificationTypeEnum
 from framework.auth import Auth
 from osf_tests.factories import (
     AuthUserFactory,
@@ -719,7 +719,7 @@ def test_can_unwithdraw_preprint_without_moderation_workflow(self, withdrawal_re
             machine_state=DefaultStates.INITIAL.value)
         withdrawal_request.run_submit(admin)
 
-        with assert_notification(type=NotificationType.Type.PREPRINT_REQUEST_WITHDRAWAL_APPROVED):
+        with assert_notification(type=NotificationTypeEnum.PREPRINT_REQUEST_WITHDRAWAL_APPROVED):
             withdrawal_request.run_accept(admin, withdrawal_request.comment)
 
         assert preprint.machine_state == 'withdrawn'