Fix profile photo upload (#2242)

[gregjkal]

It appears that boto3's s3transfer library uses threads by default for uploads, which deadlock under gevent's monkey-patched threading. The gunicorn arbiter kills the hung worker after 30 seconds, surfacing as "upstream request timeout." This may have been an issue since the project started using S3 storage alongside gunicorn's gevent workers.

Reproduced locally by switching from runserver to gunicorn with gevent, confirmed the fix by setting AWS_S3_USE_THREADS = False.

• AWS_S3_USE_THREADS = False in settings, disabling boto3's threaded transfers that deadlock under gevent
• Use isinstance(data, UploadedFile) to detect new uploads instead of the unreliable != comparison between FieldFile and InMemoryUploadedFile
• Delete old files via storage.delete() instead of FieldFile.delete(), which closes the pending upload's file handle
• Invalidate ImageKit's cached thumbnail after upload so it regenerates from the new source image
• Fix user.image -> user.profile_image in 5 templates (missed during field rename in e0fe6d6)
• Use get_thumbnail_url in news detail template instead of accessing image_thumbnail.url directly
• Broaden error suppression in get_thumbnail_url and get_hq_image_url for S3 storage errors
• Clean up delete_account to properly invalidate cached thumbnails

Steps to reproduce the error locally

1. Add S3 env vars to your .env file:

AWS_ACCESS_KEY_ID=<key>
AWS_SECRET_ACCESS_KEY=<secret>
MEDIA_BUCKET_NAME=stage.boost.org.media
AWS_S3_ENDPOINT_URL=https://s3.us-east-2.amazonaws.com/
AWS_S3_REGION_NAME=us-east-2

3. Add a temporary S3 media block to the bottom of config/settings.py :

if LOCAL_DEVELOPMENT and env("MEDIA_BUCKET_NAME", default=None):
    MEDIA_BUCKET_NAME = env("MEDIA_BUCKET_NAME")
    AWS_ACCESS_KEY_ID = env("AWS_ACCESS_KEY_ID")
    AWS_SECRET_ACCESS_KEY = env("AWS_SECRET_ACCESS_KEY")
    AWS_STORAGE_BUCKET_NAME = MEDIA_BUCKET_NAME
    AWS_S3_OBJECT_PARAMETERS = {"CacheControl": "max-age=86400"}
    AWS_DEFAULT_ACL = None
    AWS_S3_ENDPOINT_URL = env("AWS_S3_ENDPOINT_URL")
    AWS_S3_REGION_NAME = env("AWS_S3_REGION_NAME")
    STORAGES = {
        "default": {"BACKEND": "core.storages.MediaStorage"},
        "staticfiles": {"BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage"},
    }
    MEDIA_URL = f"{AWS_S3_ENDPOINT_URL}/{MEDIA_BUCKET_NAME}/"

5. Switch to gunicorn in docker/compose-start.sh:

gunicorn -c gunicorn.conf.py --log-level INFO --reload -b 0.0.0.0:$WEB_PORT config.wsgi
# $PYTHON manage.py runserver 0.0.0.0:$WEB_PORT

6. If you're on this branch, comment out AWS_S3_USE_THREADS = False in config/settings.py (the fix)
7. Restart docker with just down && just up
8. Log in and upload a profile photo at /users/me/. It should hang for ~30 seconds then error (locally I saw "Internal Server Error", not "upstream request timeout").

[herzog0]

Nice catch on those image references. Indeed, in production, when I go into a news details page the profile picture is not shown, while in the news list it is.

[herzog0]

Just something I found after the previous approval

[herzog0]

@gregjkal since you're already looking at this, can we just double check if this snippet is correct?
The if checks for entry.author.profile_image (or just image before), but the code below actually uses entry.author.image_thumbnail.url.

[gregjkal]

Good catch! This was pre-existing, but you're right that it's wrong. Changed to use get_thumbnail_url instead, which handles errors gracefully.

[jlchilders11]

Looks good to me. I am still curious as to how we were getting profile image uploads until the end of last year though.