Optimise images

[grischard]

Recompressing images losslessly once would save disk space and bandwidth. I saw about 30% improvement on jpeg sizes on my tile proxy after running mozjpeg.

I'm a fan of ImageOptim for macOS, but alternatives exist for other OSs.

[jochenklar]

Cool, thanks for the info. However I need to convert the files using gdal first. How do I combine mozjpeg? Any clue? The conversion is here: https://github.com/jochenklar/fis-broker/blob/master/2017/bin/convert.sh

[jochenklar]

Sorry, that did not make sense, the images are stroed as TIFF on the server, compression needs to happen with tilestache.

[grischard]

Ah! In that case, since the tiles are generated on the fly, piping them through mozjpeg would add more time than transmission would save. It's interesting that it's still significantly faster than the original wms.

[grischard]

Reading the tilestache documentation right now for geotiffs in Luxembourg, I see that it can cache rendered tiles, forever by default. Something like this could work with a tilestache.caches.disk:

#!/usr/bin/env python

"""
Optimise JPEGs in a tilestache cache with mozjpeg.

Remembers processed files so they are only processed once.

Find -mtime wouldn't work because optimising the files changes the mtime.
"""

import os
from subprocess import call

PATH = "/home/openstreetmap/tilestache/cache"
MOZJPEG = "/usr/local/bin/mozjpeg"

PROCESSED_FILES_FILE = os.path.join(PATH, "mozjpeg_optimised_tiles.txt")
PROCESSED_FILES = set(line.strip() for line in open(PROCESSED_FILES_FILE))

CACHE_DATA = os.path.join(PATH, "cache_data")

with open(PROCESSED_FILES_FILE, "a") as pff:
    for root, dirs, files in os.walk(CACHE_DATA):
        for tilefile in files:
            if tilefile.endswith(".jpeg"):
                tilepath = os.path.join(root, tilefile)
                if tilepath not in PROCESSED_FILES:
                    call([MOZJPEG,
                          "-copy", "none",
                          "-outfile", tilepath,
                          tilepath])
                    pff.write("%s\n" % tilepath)

[tordans]

@jochenklar do you think we could run this manually on the temp-folders that hold the png-files?
My understand is, that those only get deleted by a manual reset, so running it manually once would be fine. (Otherwise we could maybe run it once a week…)

I see good file size reduction on the new 2020 images as well:

For https://tiles.codefor.de/berlin-2020-truedop/18/140918/86125.png, using https://imageoptim.com/mac (https://github.com/ImageOptim/ImageOptim)

Since those images are used mobile as well, saving some bites add up.

---

@grischard the code you have above is for jpeg, right? As far as I see the cached files are saved as png. Do you have a recommendation on how to optimize those?

---

Btw, just for reference, ideally this issue would be moved over to https://github.com/codeforberlin/tilestache-config which is the repo that hold the tile-server-config and -code. This one is just a list of all available "layer".

[grischard]

For png I recommend pngwolf-zopfli. But why would you cache orthoimagery as png?

[jochenklar]

Hi @tordans @grischard years have passed and we have a completely new setup now. We use https://github.com/codeforberlin/gdal-tiles-api to create PNG, a better compression could be implemented there, but I am not sure if this makes sense.