Problem with documentation on .getextrema()

[beruic]

What did you do?

from PIL import Image

grey_rgb = Image.new("RGB", (256,256), (128, 128, 128))
grey_greyscale = grey_rgb.convert("L")
print(grey_greyscale.getextrema())

What did you expect to happen?

I expected the output to be in floats, like the documentation at https://pillow.readthedocs.io/en/stable/reference/Image.html#PIL.Image.Image.getextrema mentions

What actually happened?

I got a tuple of integers: (128, 128)

What are your OS, Python and Pillow versions?

• OS: Ubuntu 24.04.2 LTS
• Python: 3.12.3
• Pillow: 11.2.1 (installed via pip in a virtualenv)

--------------------------------------------------------------------
Pillow 11.2.1
Python 3.12.3 (main, Feb  4 2025, 14:48:35) [GCC 13.3.0]
--------------------------------------------------------------------
Python executable is /home/jmk/.virtualenvs/layout-g4/bin/python3
Environment Python files loaded from /home/jmk/.virtualenvs/layout-g4
System Python files loaded from /usr
--------------------------------------------------------------------
Python Pillow modules loaded from /home/jmk/.virtualenvs/layout-g4/lib/python3.12/site-packages/PIL
Binary Pillow modules loaded from /home/jmk/.virtualenvs/layout-g4/lib/python3.12/site-packages/PIL
--------------------------------------------------------------------
--- PIL CORE support ok, compiled for 11.2.1
*** TKINTER support not installed
--- FREETYPE2 support ok, loaded 2.13.3
--- LITTLECMS2 support ok, loaded 2.17
--- WEBP support ok, loaded 1.5.0
*** AVIF support not installed
--- JPEG support ok, compiled for libjpeg-turbo 3.1.0
--- OPENJPEG (JPEG2000) support ok, loaded 2.5.3
--- ZLIB (PNG/ZIP) support ok, loaded 1.3, compiled for zlib-ng 2.2.4
--- LIBTIFF support ok, loaded 4.7.0
--- RAQM (Bidirectional Text) support ok, loaded 0.10.1, fribidi 1.0.13, harfbuzz 11.0.1
*** LIBIMAGEQUANT (Quantization method) support not installed
--- XCB (X protocol) support ok
--------------------------------------------------------------------

from PIL import Image

grey_rgb = Image.new("RGB", (256,256), (128, 128, 128))
grey_greyscale = grey_rgb.convert("L")
print(grey_greyscale.getextrema())

[radarhere]

Just for reference, here is the documentation you're referring to.

So you're talking about our type hints.

getextrema() sometimes returns a tuple of ints, but also sometimes tuple of floats. See this F mode example.

>>> from PIL import Image
>>> im = Image.new("F", (1,1))
>>> im.getextrema()
(0.0, 0.0)

float is being used as a superset of int.

PEP 484 – Type Hints states that

this PEP proposes a straightforward shortcut that is almost as effective: when an argument is annotated as having type float, an argument of type int is acceptable

[beruic]

Regarding the PEP 484 statement: I believe it will be confusing at first to many developers. Adding tuple[int, int] as well, will make this much more intuitively understandable. The statement from the PEP may apply to how machines read it, but this documentation is for humans.

I guess one of the things that has confused me as well, is this SO answer, where it is PIL is used instead of Pillow, which apparently means that mode "L" means floats: https://stackoverflow.com/questions/14041562/python-pil-detect-if-an-image-is-completely-black-or-white/14041871#14041871

It would be nice if the modes were mentioned and linked to in the documentation for .getextrema(), and that the mode documentation was updated to mention the difference on L from Pillow.
Maybe also add that the I and F modes are grayscale (which I am still unsure of, but it fits with my experimentation).

[wiredfool]

At a type level, Image is really an Image<T> where T is the modetype, where Image implies a Pixel, and therefore Image<T>.getextrema() -> tuple[Pixel<T>, Pixel<T>].

This indicates that if we have a Float image, we'll get out Float pixels. If we have an Int image, we'll get out Int pixels.

(But, it's really more complicated than that, because there's also the per band ones, so we'd have to add a | tuple[tuple[PixelComponent<T>, PixelComponent<T>], ...] on there as well)

However, we don't really have that level of specificity in the types, we just have an Image, and it has a mode, so as it stands we can't encode that the output type is the same as the pixel type.

Similarly to this -- bounding boxes appear to be specified as floats:

    def crop(self, box: tuple[float, float, float, float] | None = None) -> Image:
        """
        Returns a rectangular region from this image. The box is a
        4-tuple defining the left, upper, right, and lower pixel
        coordinate. See :ref:`coordinate-system`.
...
        return self._new(self._crop(self.im, box))

where we immediately turn around and throw away the floating point portion:

static PyObject *
_crop(ImagingObject *self, PyObject *args) {
    int x0, y0, x1, y1;
    if (!PyArg_ParseTuple(args, "(iiii)", &x0, &y0, &x1, &y1)) {
        return NULL;
    }

I think this is misleading as well.

Actually, that part does immediately round:

    def _crop(
        self, im: core.ImagingCore, box: tuple[float, float, float, float]
...
        x0, y0, x1, y1 = map(int, map(round, box))

[radarhere]

The types of data stored in different image modes is explained at https://pillow.readthedocs.io/en/stable/handbook/concepts.html#modes. Including that in all methods that return pixel values - load(), getpixel(), histogram(), getdata(), the PixelAccess class and probably others - sounds very repetitive to me.

Regarding float vs int | float, I could give in to the idea of using int | float wherever pixel data is concerned - but if you're not aware of the difference values stored in modes in the first place, I don't think the type hint is going to sufficently explain when to expect an int and when to expect a float. One of the ruff rules that we use deliberately wants the simpler float option - https://docs.astral.sh/ruff/rules/redundant-numeric-union/

a union that includes both int and float is redundant in the specific context of a parameter annotation, as it is equivalent to a union that only includes float. For readability and clarity, unions should omit redundant elements.

We added the rule in #7897, so I'm not alone in this thinking.

[wiredfool]

What if we started naming some of the types?

E.g. something like

class BoundingBox(NamedTuple):
     left: float | int
     top: float | int
     right: float | int
     bottom: float | int

class ExactBoundingBox(NamedTuple):
     left: int
     top:  int
     right: int
     bottom: int

class Coordinate(NamedTuple):
     x: int
     y: int


FloatPixel = float
IntPixel = int
PixelComponent = int

class RGBPixel(NamedTuple):
    r:PixelComponent
    g:PixelComponent
    b:PixelComponent
    x:PixelComponent = 0

class RGBAPixel(NamedTuple):
    r:PixelComponent
    g:PixelComponent
    b:PixelComponent
    a:PixelComponent

GenericMultiChannelPixel = tuple[PixelComponent, PixelComponent, PixelComponent, PixelComponent]

Pixel = FloatPixel | IntPixel | RGBPixel | RGBAPixel | GenericMultiChannelPixel
SingleChannelPixel = FloatPixel | IntPixel

We can name things. We can namespace things.

This would make the method signature look like

    def getextrema(self) -> tuple[SingleChannelPixel, SingleChannelPixel] | tuple[tuple[PixelComponent, PixelComponent], ...]:

This is (roughly) as concise as possible, and communicates the intent of the types.

PixelAccess could be:

__getitem__(self, xy: Coordinate) -> Pixel 

From the redundant numeric union page:

but to support common use cases

I'd argue here that we're not necessarily in the common use case. Our bands have ranges, we have distinctions between int32, int16, and uint8.

[radarhere]

I kind of like that idea, but from a maintenance point of view. I'm not sure if it would intimidate newcomers, and this is an issue where I think a developer relatively new to Pillow came in, and looked at the type hints without completely understanding image modes.

@beruic if you saw

def getextrema(self) -> tuple[SingleChannelPixel, SingleChannelPixel] | tuple[tuple[PixelComponent, PixelComponent], ...]:

would you appreciate clicking on the link to SingleChannelPixel and then landing on a page to explain that the value is a float for F mode images but otherwise an int,
or do you think that it would be off-putting, and cause more confusion?

[github-actions]

Closing this issue as no feedback has been received.