Add anonymizing of user applications feature by wes-otf · Pull Request #4702 · HyphaApp/hypha

wes-otf · 2026-02-17T16:34:09Z

Would close #3441. Adds a new AnonymizedSubmission model that only contains minimal, non-PII datapoints. Also adds UI features for staff to select whether a user's application(s) should be deleted or converted to skeletoned/anonymized application.

The todos:

Implement AnonymizedSubmission model & creation methods
Handle manual individual submission deletions
Handle manual bulk submission deletions/anonymization
Handle submissions with individual user deletions
Handle submissions with bulk user deletions
Include anonymized submissions in the results dashboard
Add unit tests for new functionality

Test Steps

…

wes-otf · 2026-03-18T19:59:18Z

@frjo I'm going to add a unit test for the results view, some docs & do a final look over but it should be ready for review if you wanted to take a look before that

wes-otf · 2026-03-18T20:03:18Z

+            SubmissionSkeletonFilter.declared_filters
+        )
+        if not set(self.request.GET) & set(non_skeleton_fields):
+            skeleton_qs = SubmissionSkeletonFilter(


somewhat of a hacky solution to managing 2 models with one FilterView but seems to work?

seems like the # of SQL queries shoots up though so might need some optimizing

frjo · 2026-04-13T13:52:25Z

@wes-otf I rebased this with main to solve a small merge conflict.

frjo · 2026-04-13T19:31:35Z

@wes-otf When I was testing this I realised we have some existing issues with deleted/archive. When I started with fixing that more things followed as you can see in the PR now 😄.

The backend is nearly untouched. This part seems to works really well!
The wagtail admin part is nearly untouched as well, only added some classes for margins etc.

My changes:

Renamed everything skeleton -> anonymize, makes it easier to read the code I believe.
Added some tests.
Added "Anonymise" button in "MORE ACTIONS" section on submission detail. Anonymise also has its own confirm modal now.
Moved "Delete" button to "MORE ACTIONS" section for staff as well. I think this makes more sense.
Reverted changes to SubmissionDeleteView since they are not used any more.

frjo · 2026-04-13T20:06:05Z

The batch functions on submission/all only have a simple js alert while the same functions for a single submission has a nice modal. That is not logical. Will fix that in a separate PR.

frjo

This works very well in my testing, the backend seems solid. The anonymised submission has very little information left since all form_data is deleted. The feature talk about removing PII info, some might be surprised that almost nothing besides the value remain.

Maybe this can be communicated better?

…th wagtail & submission views

…ail form if enabled

…n of unit tests

…e actions section.

frjo · 2026-04-29T10:55:44Z

+ * `round` - round (if any) the original submission was in
+ * `submit_time` - time the original submission was submitted
+ * `screening_status` - screening status of the original submission
+ * `category` - the categories of the original submission


Are categories always preserved? Will try to test this for bulk_anonymize_submissions, submission_cleanup and on user delete.

yes they should be as of the latest push, needed to add form_fields to the values(...) call for them to be parsed

frjo · 2026-04-29T11:05:13Z

    timestamp = models.DateTimeField()
    type = models.CharField(choices=ACTIVITY_TYPES.items(), max_length=30)
-    user = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.PROTECT)
+    user = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.CASCADE)


This needs to be documented I think. Deleting a user will now delete the audit trail.

Would it not be better to anonymise the user account instead of deleting it? Or do the activities contain sensitive info?

I'm thinking that would likely be another PR/feature if we wanted to go that route. As it stands I'm the only one at OTF that can delete a user when a GDPR request comes in for deletion because all staff member gets in wagtail is an error saying the user is used elsewhere but never pointed to which specific activity that is. This is definitely more of a nuclear option but definitely GDPR compliant as activities can contain a host of data especially when it's a comment - open to your thoughts on this though!

frjo · 2026-04-29T11:08:29Z

        """
        return reverse("wagtailusers_users:edit", args=[self.id])

+    def delete(


Is this needed, does not the wagtail_hook handle this?

I added it for any manual .delete()s that could come of the user model (as we continue w retention tooling I could see it being used in accounts_cleanup)

frjo · 2026-04-29T11:19:08Z

+                user=self.request.user,
+                request=self.request,
+                source=submission,
+            )


This records an event but no adapter (slack/mail etc.) is configured. Is the submission not deleted at this stage so "submission" is empty?

Fixed by both adding a slack adapter and pushing the super call down past the messaging call

…ation notifications for slack, pulled `form_fields` from values for categories

wes-otf · 2026-04-30T17:00:42Z

@frjo thanks for the feedback! I appreciate the attention to detail in the review, especially the stuff around messaging I'm not sure if I wouldn't quite caught that

frjo

Found two bugs, other than that it seems to work nicely now.

wes-otf · 2026-05-01T16:22:05Z

Fat fingering got me again - gonna deploy to test and make sure all looks good over there. Thanks again @frjo!

wes-otf · 2026-05-01T17:57:52Z

Seems to work good on test - I think theres some general issues with the category filtering (for applications both normal and anonymized) that I need to look at when in the results view but I think that might be out of scope here

wes-otf force-pushed the feature/clean_user_pii branch 3 times, most recently from b389175 to 0bc0caf Compare March 3, 2026 15:48

wes-otf force-pushed the feature/clean_user_pii branch 2 times, most recently from 9ae8d11 to 30b37bd Compare March 18, 2026 19:57

wes-otf requested a review from frjo March 18, 2026 19:58

wes-otf commented Mar 18, 2026

View reviewed changes

wes-otf marked this pull request as ready for review March 19, 2026 16:09

wes-otf changed the title ~~WIP: Add "skeletoning"/anonymizing of user applications~~ Add "skeletoning"/anonymizing of user applications Mar 19, 2026

frjo added Type: Feature This is something new (not an enhancement of an existing thing). Type: Minor Minor change, used in release drafter labels Apr 3, 2026

frjo force-pushed the feature/clean_user_pii branch from 30b37bd to a5e160a Compare April 13, 2026 13:51

frjo requested changes Apr 14, 2026

View reviewed changes

frjo changed the title ~~Add "skeletoning"/anonymizing of user applications~~ Add anonymizing of user applications feature Apr 14, 2026

wes-otf and others added 12 commits April 22, 2026 17:30

Implemented the option to anonymize or delete user applications in bo…

debf643

…th wagtail & submission views

Added management command for anonymizing submissions & only show wagt…

838909b

…ail form if enabled

Added handling for wagtail bulk actions

f9ca7c1

Added bulk anonymization, fixed some previous copy/paste typos

93a7cca

Added missing migration, fixed/added docstrings, inital implementatio…

f868859

…n of unit tests

Test fixes and rebasing onto main

d6ab39a

Added more tests, fixed settings

edaf68c

Added skeleton application stats into results view

7f4c7fb

Update migrations after rebase.

bc06495

Fix user delete bug and add comment for SUBMISSION_SKELETONING_ENABLED.

5e49560

Seperate Anonymize and Delete button on submissions, move both to Mor…

c7d6cb4

…e actions section.

Cleanup unused code.

a70e3c1

frjo added 8 commits April 22, 2026 17:30

Add back applicant delete button.

ba60118

Use longer names so meaning clear.

907e54c

Simplyfy get_user_submission_count().

9fd3b99

Revert chenges in SubmissionDeleteView.

273cfca

Use the term anonymize and not skeleton througout.

a54deb8

Added and updated tests.

dbc6fdf

Correcting how average_value is calulated.

a9b8b8f

Add notice about anonymized_excluded_count.

7b7d9a0

frjo force-pushed the feature/clean_user_pii branch from b858e27 to 7b7d9a0 Compare April 22, 2026 15:32

frjo and others added 2 commits April 22, 2026 17:44

Update migrations after rebase.

b6ed347

Removed management command placeholders and added docs

549c4f8

frjo approved these changes Apr 24, 2026

View reviewed changes

Added categories + filtering in results view

3c9b438

frjo requested changes Apr 29, 2026

View reviewed changes

Fixed untranslated strings, better messaging logic, configed anonymiz…

9f25745

…ation notifications for slack, pulled `form_fields` from values for categories

wes-otf requested a review from frjo April 30, 2026 17:32

cancel button class fix

ca187ca

frjo requested changes Apr 30, 2026

View reviewed changes

Comment thread hypha/apply/users/wagtail_hooks.py Outdated

Comment thread hypha/apply/funds/services.py

Typo fixes & forgetting to save files

df2b0e9

frjo self-requested a review May 1, 2026 16:52

frjo approved these changes May 1, 2026

View reviewed changes

wes-otf added the Status: Tested - approved for live ✅ label May 1, 2026

wes-otf merged commit f705a99 into main May 1, 2026
6 checks passed

Conversation

wes-otf commented Feb 17, 2026 • edited by frjo Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Steps

Uh oh!

wes-otf commented Mar 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

frjo commented Apr 13, 2026

Uh oh!

frjo commented Apr 13, 2026

Uh oh!

frjo commented Apr 13, 2026

Uh oh!

frjo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

wes-otf commented Apr 30, 2026

Uh oh!

frjo left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

wes-otf commented May 1, 2026

Uh oh!

wes-otf commented May 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

wes-otf commented Feb 17, 2026 •

edited by frjo

Loading

wes-otf commented Mar 18, 2026 •

edited

Loading

frjo left a comment •

edited

Loading