Skip to content
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
debf643
Implemented the option to anonymize or delete user applications in bo…
wes-otf Feb 17, 2026
838909b
Added management command for anonymizing submissions & only show wagt…
wes-otf Feb 18, 2026
f9ca7c1
Added handling for wagtail bulk actions
wes-otf Feb 25, 2026
93a7cca
Added bulk anonymization, fixed some previous copy/paste typos
wes-otf Feb 26, 2026
f868859
Added missing migration, fixed/added docstrings, inital implementatio…
wes-otf Mar 3, 2026
d6ab39a
Test fixes and rebasing onto main
wes-otf Mar 3, 2026
edaf68c
Added more tests, fixed settings
wes-otf Mar 4, 2026
7f4c7fb
Added skeleton application stats into results view
wes-otf Mar 18, 2026
bc06495
Update migrations after rebase.
frjo Apr 13, 2026
5e49560
Fix user delete bug and add comment for SUBMISSION_SKELETONING_ENABLED.
frjo Apr 13, 2026
c7d6cb4
Seperate Anonymize and Delete button on submissions, move both to M…
frjo Apr 13, 2026
a70e3c1
Cleanup unused code.
frjo Apr 13, 2026
ba60118
Add back applicant delete button.
frjo Apr 13, 2026
907e54c
Use longer names so meaning clear.
frjo Apr 13, 2026
9fd3b99
Simplyfy get_user_submission_count().
frjo Apr 13, 2026
273cfca
Revert chenges in SubmissionDeleteView.
frjo Apr 13, 2026
a54deb8
Use the term anonymize and not skeleton througout.
frjo Apr 13, 2026
dbc6fdf
Added and updated tests.
frjo Apr 13, 2026
a9b8b8f
Correcting how average_value is calulated.
frjo Apr 13, 2026
7b7d9a0
Add notice about anonymized_excluded_count.
frjo Apr 13, 2026
b6ed347
Update migrations after rebase.
frjo Apr 22, 2026
549c4f8
Removed management command placeholders and added docs
wes-otf Apr 24, 2026
3c9b438
Added categories + filtering in results view
wes-otf Apr 28, 2026
9f25745
Fixed untranslated strings, better messaging logic, configed anonymiz…
wes-otf Apr 30, 2026
ca187ca
cancel button class fix
wes-otf Apr 30, 2026
df2b0e9
Typo fixes & forgetting to save files
wes-otf May 1, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions docs/setup/administrators/anonymizing-submissions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Anonymizing Submissions

Hypha offers submission anonymization to allow for less retention of Personally Identifiable Information (PII).

When the configuration option `SUBMISSION_ANONYMIZATION_ENABLED=True`, submissions can be either:

* Individually anonymized (Submission Detail View → "More Actions" → "Anonymize" button)
* Anonymized manually in bulk (Submissions All View → select submissions → "Anonymize" button in actions bar)
* Anonymized in bulk via user deletion (Wagtail Admin → "Settings" → "Users" → Select a user → "…" in top action/nav bar → "Delete" → "Anonymize all user submissions")
* Anonymized through the management command `python3 manage.py submission_cleanup --submissions [days]` (more info on that [here](cron-jobs.md#submission-cleanup))

When a submission is anonymized, the original submission is deleted and a derivative is created with the following attributes:

* `value` - applied for value (if it exists on the original submission)
* `status` - status of the original submission
* `page` - either the fund/lab that the original submission existed on
* `round` - round (if any) the original submission was in
* `submit_time` - time the original submission was submitted
* `screening_status` - screening status of the original submission
* `category` - the categories of the original submission
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are categories always preserved? Will try to test this for bulk_anonymize_submissions, submission_cleanup and on user delete.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes they should be as of the latest push, needed to add form_fields to the values(...) call for them to be parsed


and if the user that "owns" the submission still exists and isn't being actively deleted:

* `user` - stores the user that "owned" the original submission. When the user account is deleted, this field is set to null.
6 changes: 6 additions & 0 deletions docs/setup/administrators/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -299,6 +299,12 @@ See [here](machine-translations.md) for more information on setting up machine t

----

Should submission anonymization be enabled. Allows for both manual anonymization and utilization of cleanup commands to anonymize by time thresholds.

SUBMISSION_ANONYMIZATION_ENABLED = env.bool("SUBMISSION_ANONYMIZATION_ENABLED", False)

----

## Slack settings

SLACK_TOKEN = env.str('SLACK_TOKEN', None)
Expand Down
13 changes: 8 additions & 5 deletions docs/setup/administrators/cron-jobs.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,20 +10,23 @@ Accounts that haven't been logged into in 5 months can be marked as inactive wit
python3 manage.py accounts_cleanup
```

## Drafts cleanup
## Submission cleanup

Drafts that haven't been modified in a specified time (in days) can be deleted with the following command:
There are two options for the cleaning up of user submissions
* Drafts: **deletes** all drafts outside the specified timeframe. Drafts are never anonymized.
* Submissions: by default, anonymizes all submissions that are not drafts outside the specified timeline.
* If `--delete` is specified, the submissions will be deleted instead of anonymized.

```shell
python3 manage.py drafts_cleanup [days]
python3 manage.py submission_cleanup --drafts [days] --submissions [days]

# Or, to run without a confirmation prompt

python3 manage.py drafts_cleanup [days] --noinput
python3 manage.py submission_cleanup --drafts [days] --submissions [days] --noinput
```

Example: to delete all drafts that haven't been modified in a year without a confirmation prompt:

```shell
python3 manage.py drafts_cleanup 365 --noinput
python3 manage.py submission_cleanup --drafts [days] 365 --noinput
Comment thread
wes-otf marked this conversation as resolved.
Outdated
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
# Generated by Django 5.2.13 on 2026-04-13 18:50

import django.db.models.deletion
from django.conf import settings
from django.db import migrations, models


class Migration(migrations.Migration):
dependencies = [
("activity", "0091_alter_activity_options_and_more"),
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
]

operations = [
migrations.AlterField(
model_name="activity",
name="user",
field=models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL
),
),
migrations.AlterField(
model_name="event",
name="by",
field=models.ForeignKey(
null=True,
on_delete=django.db.models.deletion.CASCADE,
to=settings.AUTH_USER_MODEL,
),
),
migrations.AlterField(
model_name="event",
name="type",
field=models.CharField(
choices=[
("UPDATE_LEAD", "updated lead"),
("BATCH_UPDATE_LEAD", "batch updated lead"),
("EDIT_SUBMISSION", "edited submission"),
("APPLICANT_EDIT", "edited applicant"),
("NEW_SUBMISSION", "submitted new submission"),
("DRAFT_SUBMISSION", "submitted new draft submission"),
("SCREENING", "screened"),
("TRANSITION", "transitioned"),
("BATCH_TRANSITION", "batch transitioned"),
("DETERMINATION_OUTCOME", "sent determination outcome"),
("BATCH_DETERMINATION_OUTCOME", "sent batch determination outcome"),
("INVITED_TO_PROPOSAL", "invited to proposal"),
("REVIEWERS_UPDATED", "updated reviewers"),
("BATCH_REVIEWERS_UPDATED", "batch updated reviewers"),
("READY_FOR_REVIEW", "marked ready for review"),
("BATCH_READY_FOR_REVIEW", "marked batch ready for review"),
("NEW_REVIEW", "added new review"),
("COMMENT", "added comment"),
("PROPOSAL_SUBMITTED", "submitted proposal"),
("OPENED_SEALED", "opened sealed submission"),
("REVIEW_OPINION", "reviewed opinion"),
("DELETE_SUBMISSION", "deleted submission"),
("ANONYMIZE_SUBMISSION", "anonymized submission"),
("DELETE_REVIEW", "deleted review"),
("DELETE_REVIEW_OPINION", "deleted review opinion"),
("CREATED_PROJECT", "created project"),
("UPDATE_PROJECT_LEAD", "updated project lead"),
("UPDATE_PROJECT_TITLE", "updated project title"),
("EDIT_REVIEW", "edited review"),
("SEND_FOR_APPROVAL", "sent for approval"),
("APPROVE_PROJECT", "approved project"),
("ASSIGN_PAF_APPROVER", "assign project form approver"),
("APPROVE_PAF", "approved project form"),
("PROJECT_TRANSITION", "transitioned project"),
("REQUEST_PROJECT_CHANGE", "requested project change"),
("SUBMIT_CONTRACT_DOCUMENTS", "submitted contract documents"),
("UPLOAD_DOCUMENT", "uploaded document to project"),
("UPLOAD_CONTRACT", "uploaded contract to project"),
("APPROVE_CONTRACT", "approved contract"),
("CREATE_INVOICE", "created invoice for project"),
("UPDATE_INVOICE_STATUS", "updated invoice status"),
("APPROVE_INVOICE", "approve invoice"),
("DELETE_INVOICE", "deleted invoice"),
("SENT_TO_COMPLIANCE", "sent project to compliance"),
("UPDATE_INVOICE", "updated invoice"),
("SUBMIT_REPORT", "submitted report"),
("SKIPPED_REPORT", "skipped report"),
("REPORT_FREQUENCY_CHANGED", "changed report frequency"),
("DISABLED_REPORTING", "disabled reporting"),
("REPORT_NOTIFY", "notified report"),
("REVIEW_REMINDER", "reminder to review"),
("BATCH_DELETE_SUBMISSION", "batch deleted submissions"),
("BATCH_ANONYMIZE_SUBMISSION", "batch anonymized submissions"),
("BATCH_ARCHIVE_SUBMISSION", "batch archive submissions"),
("BATCH_INVOICE_STATUS_UPDATE", "batch update invoice status"),
("STAFF_ACCOUNT_CREATED", "created new account"),
("STAFF_ACCOUNT_EDITED", "edited account"),
("ARCHIVE_SUBMISSION", "archived submission"),
("UNARCHIVE_SUBMISSION", "unarchived submission"),
("REMOVE_TASK", "remove task"),
("INVITE_COAPPLICANT", "invite co-applicant"),
],
max_length=50,
verbose_name="verb",
),
),
]
4 changes: 2 additions & 2 deletions hypha/apply/activity/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ def get_absolute_url(self):
class Activity(models.Model):
timestamp = models.DateTimeField()
type = models.CharField(choices=ACTIVITY_TYPES.items(), max_length=30)
user = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.PROTECT)
user = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.CASCADE)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be documented I think. Deleting a user will now delete the audit trail.

Would it not be better to anonymise the user account instead of deleting it? Or do the activities contain sensitive info?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking that would likely be another PR/feature if we wanted to go that route. As it stands I'm the only one at OTF that can delete a user when a GDPR request comes in for deletion because all staff member gets in wagtail is an error saying the user is used elsewhere but never pointed to which specific activity that is. This is definitely more of a nuclear option but definitely GDPR compliant as activities can contain a host of data especially when it's a comment - open to your thoughts on this though!


source_content_type = models.ForeignKey(
ContentType,
Expand Down Expand Up @@ -328,7 +328,7 @@ class Event(models.Model):
when = models.DateTimeField(auto_now_add=True)
type = models.CharField(_("verb"), choices=MESSAGES.choices, max_length=50)
by = models.ForeignKey(
settings.AUTH_USER_MODEL, on_delete=models.PROTECT, null=True
settings.AUTH_USER_MODEL, on_delete=models.CASCADE, null=True
)
content_type = models.ForeignKey(
ContentType, blank=True, null=True, on_delete=models.CASCADE
Expand Down
5 changes: 5 additions & 0 deletions hypha/apply/activity/options.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ class MESSAGES(TextChoices):
OPENED_SEALED = "OPENED_SEALED", _("opened sealed submission")
REVIEW_OPINION = "REVIEW_OPINION", _("reviewed opinion")
DELETE_SUBMISSION = "DELETE_SUBMISSION", _("deleted submission")
ANONYMIZE_SUBMISSION = "ANONYMIZE_SUBMISSION", _("anonymized submission")
DELETE_REVIEW = "DELETE_REVIEW", _("deleted review")
DELETE_REVIEW_OPINION = "DELETE_REVIEW_OPINION", _("deleted review opinion")
CREATED_PROJECT = "CREATED_PROJECT", _("created project")
Expand Down Expand Up @@ -64,6 +65,10 @@ class MESSAGES(TextChoices):
REPORT_NOTIFY = "REPORT_NOTIFY", _("notified report")
REVIEW_REMINDER = "REVIEW_REMINDER", _("reminder to review")
BATCH_DELETE_SUBMISSION = "BATCH_DELETE_SUBMISSION", _("batch deleted submissions")
BATCH_ANONYMIZE_SUBMISSION = (
"BATCH_ANONYMIZE_SUBMISSION",
_("batch anonymized submissions"),
)
BATCH_ARCHIVE_SUBMISSION = (
"BATCH_ARCHIVE_SUBMISSION",
_("batch archive submissions"),
Expand Down
89 changes: 0 additions & 89 deletions hypha/apply/funds/management/commands/drafts_cleanup.py

This file was deleted.

Loading
Loading