migrate graphicsmagick to Pillow

[beudbeud]

No description provided.

[QSchulz]

Happy to see interest in using of Pillow instead of using subprocesses and imagemagick.

1. As a note, I've been testing on AARCH64, Intel processors with SIMD and without SIMD support. Pillow-simd will not be installed by pip on anything else than Intel processors supporting SIMD. So with the commit above you could be breaking a few users. I'd personally keep pillow there and add a note for users that installing pillow-simd instead might bring drastic improvement. To check if your CPU support SIMD instructions, run the following: cat /proc/cpuinfo | grep -e sse4 if it yields something, good. FWIW, pillow only support SSE4.x and AVX2 which are Intel-only. Other architectures also support SIMD (e.g. ARM with neon) but pillow-simd does not support them.

2. pillow is also able to read EXIF data just fine so you don't need an extra dependency:

    img = Image.open(path)
    exif = img._getexif()
    # Orientation EXIF metadata is stored in 0x0112 if it exists
    orient = exif.get(0x0112, 1) if exif else 1

3. Image.thumbnail() should be faster than Image.resize() as per documentation but I haven't tested resize, only thumbnail. Note that you need to do a copy of the original Image with Image.copy() before calling thumbnail() otherwise it overwrites the object. It's meaningful only if you do more than one thumbnail when opening the file, otherwise you can ignore this limitation.

4. With thumbnail() you don't need to compute the height as the ratio will be kept, you just need to provide one height that is ridiculously higher than width (I used 65596). I can check in pillow code what's happening for resize().

return float(im.size[0]) / float(im.size[1])

in python3 is unnecessary since division of two int give a float by default ( // if you want the result cast to an int).

6. For the ratio, the imagesize python package claims to be one of the fastest available. You can get the dimensions with: height, width = imagesize.get(path)

7. In rotate_jpeg(), I'm not entirely sure that keeping the exif in the image AND rotating/transposing it is going to work. It's either not both.

8. Also in rotate_jpeg, by calling im.save(filename) you overwrite the original file. IIUC you might actually do the rotating/transposing every time you have a thumbnail to create which would mean a few thumbnails would be oriented the wrong way.

9. BTW, Lanczos algorithm was used in imagemagick and not antialias AFAIR.

10. The DPI is lost IIRC when running im.save() without the dpi argument.

11. The subsampling by default I had was 4:2:0 whatever the input subsampling was. Unlike imagemagick which would keep the original subsampling. You can ask pillow to keep the subsampling for JPEG input file only, otherwise it raises an exception.

12. Do we need to force-convert the file to JPEG and not keep the original file format?

13. I've been experimenting quite a lot on reworking Prosopopee from the ground up (basically keeping only the templates) to make it the most efficient I could. The result is that the fastest build I could achieve was by creating one thread per CPU core (using Pool) and using Pillow-SIMD from there.

Here is my python test script for benchmarking resizing of images (github is not letting me add .py files as attachments):

import os
import time
from PIL import Image,ImageOps
from multiprocessing import Pool 

resize = [2000, 1920, 1366, 900, 800, 600, 450]
source = "test.jpg"
target = "test-prosopopee-x{resize}.jpg"
gm_switches = {
        "source": source,
        "auto-orient": "-auto-orient",
        "strip": "-strip",
        "quality": "-quality 95",
        }

total_ns = 0
for sz in resize:
    gm_switches.update({'resize': "-resize x" + str(sz)})
    gm_switches['target'] = target.format(resize=sz)
    command = "gm convert '{source}' {auto-orient} {strip} -interlace Line {quality} {resize} '{target}'".format(**gm_switches)
    prev = time.perf_counter()
    os.system(command)
    after = time.perf_counter()
    ns = after - prev
    total_ns = total_ns + ns
print("%02.15f secs. Finished sequential gm." % total_ns)

total_ns = 0
meh = ""
target = "test-prosopopee2-x{resize}.jpg"
for sz in resize[:-1]:
    meh = meh + " -resize x" + str(sz) + " -write " + target.format(resize=sz)
meh = meh + " -resize x" + str(sz) + " " + target.format(resize=resize[-1])
gm_switches.update({'resize': meh})
command = "gm convert '{source}' {auto-orient} {strip} -interlace Line {quality} {resize}".format(**gm_switches)
prev = time.perf_counter()
os.system(command)
after = time.perf_counter()
ns = after - prev
total_ns = total_ns + ns
print("%02.15f secs. Finished parallel gm." % total_ns)

total_ns = 0
target = "test-new-x{resize}.jpg"
prev = time.perf_counter()
im = Image.open(source)
for sz in resize:
    img = im.copy()
    img.thumbnail((65596,sz), Image.LANCZOS)
    img.save(target.format(resize=sz), "JPEG", quality=95, progressive=True, dpi=im.info['dpi'], subsampling="4:4:4")
after = time.perf_counter()
total_ns = after - prev
print("%02.15f secs. Finished pillow with img.copy()." % total_ns)

total_ns = 0
target = "test-new2-xx{resize}.jpg"
prev = time.perf_counter()
im = Image.open(source)
for sz in resize:
    im.thumbnail((65596,sz), Image.LANCZOS)
    im.save(target.format(resize=sz), "JPEG", quality=95, progressive=True, dpi=im.info['dpi'], subsampling="4:4:4")
after = time.perf_counter()
total_ns = after - prev
print("%02.15f secs. Finished pillow with consecutive thumbnail()." % total_ns)

total_ns = 0
target = "test-new3-xx{resize}.jpg"
def resizes(sz):
    im = Image.open(source)
    im.thumbnail((65596,sz), Image.LANCZOS)
    im.save(target.format(resize=sz), "JPEG", quality=95, progressive=True, dpi=im.info['dpi'], subsampling="4:4:4")

pool = Pool()
prev = time.perf_counter()
pool.map(resizes,resize)
after = time.perf_counter()
total_ns = after - prev
print("%02.15f secs. Finished threaded pillow with img.open()." % total_ns)

total_ns = 0
target = "test-new4-xx{resize}.jpg"
def resizes4(params):
    im,sz = params
    im.thumbnail((65596,sz), Image.LANCZOS)
    im.save(target.format(resize=sz), "JPEG", quality=95, progressive=True, dpi=im.info['dpi'], subsampling="4:4:4")

pool = Pool()
ims = list()
prev = time.perf_counter()
im = Image.open(source)
for i in range(len(resize)):
    ims.append(im)
pool.map(resizes4,zip(ims,resize))
after = time.perf_counter()
total_ns = after - prev
print("%02.15f secs. Finished threaded pillow with img passed as param." % total_ns)

IIRC though, there isn't substantial improvement when using pillow single threaded because it uses only one core of the CPU, while imagemagick is using all of them when calling subprocess.

The main reason why I stopped working on my "fork" of prosopopee is that I couldn't find a nice way to use Pillow in multi-threaded application with object-oriented programming. You can basically say bye to all your types (image, video, etc.) and use only a dict (from Manager) throughout the whole stack which is horrendous to maintain IMO. I'm happy to share my Proof-of-Concept and explain a bit what I've done and why if you're interested in it.