Contributions to the Firefox browser

Contributing to Mozilla Firefox development was a very pleasant and interesting experience and I’m happy that the best web browser is run as a friendly, open project. On this page, I’ve attempted to summarize my work on the Firefox codebase. My involvement was motivated by my FreeBSD + Wayland desktop adventures, starting with the touchpad.

Kinetic scrolling on GTK (Linux/Unix)

Kinetic (or inertial, or fling) scrolling is a big part of what makes touch UIs like on smartphones feel so natural and fun to use — the content gradually decelerating instead of suddenly stopping after you let go of the scroll gesture allows you to “fling” content around on the screen. On laptops with touchpads, Apple was first to support that with two-finger scrolling, and it worked as smoothly as on an iPhone. As a quick attempt to catch up, X11 touchpad drivers like xf86-input-synaptics (and Windows drivers too) have tried to handle that themselves, while still communicating emulated mouse wheel events to applications. This was never perfectly smooth and clearly was not a permanent solution.

With the adoption of evdev and Wayland however, Linux/Unix desktops gained proper support for touchpad scrolling — that is, applications started receiving precise pan gesture events instead of emulated scroll wheel events. But now, the responsibility to implement flings was moved into the applications. The Gtk UI toolkit was early to support them, and it did that pretty well (modulo the lack of framerate sync that made it a bit jittery on 90Hz touchpads). Regular Gtk applications gained this feature for free, but web browsers are special beasts. They barely use the toolkit and do almost everything by themselves. WebKitGTK was also quick to gain kinetic scrolling support, but Firefox was not.

So my first Firefox contribution was fixing bug 1213601, that is, integrating Gtk’s touchpad pan gesture events with Firefox’s APZ (async pan and zoom) system in patch D27983. This required supporting page units for deltas and simulating momentum events which Gtk did not emit.

Partial rendering on EGL (Wayland/X11/Android)

In 2019, an article from the GFX team at Mozilla came out about dramatically reducing power usage on macOS and the primary power saving method used there was partial rendering based on damage tracking — that is, preventing the GPU from rerendering things that haven’t changed between frames.

emersion has a detailed article explaining the concept, but basically: graphics APIs like OpenGL were designed for 3D cameras that change entirely from frame to frame, while flat UIs very often stay mostly the same for lots of frames in a row, so just naively using OpenGL for UIs results in a lot of pixels being wastefully rerendered all the time. On a composited desktop, this actually happens twice: in the application itself and in the compositor. To fix this, extra APIs are used that allow for only rendering what has changed.

After seeing the GFX article, I became really interested in bringing the power savings to my Wayland-powered desktops. (Later, when Firefox switched the X11 backend from GLX to EGL as well, X11 users started benefiting from this work as well.)

The first thing I’ve done was the compositor side, which is simpler: it’s just about passing buffer damage information (i.e. the changed regions) to the compositor. EGL provides the EGL_KHR_swap_buffers_with_damage (or EGL_EXT_swap_buffers_with_damage) extension for this purpose, and bug 1484812 existed for implementing support for it in Firefox. I have done that in patch D51517.

Then it was time to tackle the application side. There are two different EGL extensions for doing this. The one supported by desktop GPU drivers in Mesa is called EGL_EXT_buffer_age. As implied by the name, it lets the application know how many frames old the contents of the current buffer are, which enables the application to confidently skip drawing the regions that haven’t changed since. The WebRender implementation was tracked in bug 1620076, for which I authored patch D61062. WebRender has already supported partial rendering, but it was developed on Windows, which works differently: it was expecting the OS to handle copying the changed regions from the previous frame into the current buffer. For EGL, I’ve had to add an option to make WR render the union of the previous and current frames’ damage. After some interesting debugging, it worked!

Miscellaneous Wayland support improvements

As I’ve been using Firefox on Wayland daily, I was obviously interested in filling the little platform support gaps that I’ve been noticing.

The first one was Firefox not supporting the Wayland-native solution for idle wakelocks, the idle-inhibit protocol (bug 1587360). This is what makes it so that when you’re watching a video, you don’t have to constantly jiggle your mouse to prevent the system from blanking the screen and preparing to go to sleep. Firefox supported some D-Bus protocols for this (one freedesktop-branded, one GNOME-specific) in addition to XScreenSaver, but none of those worked for me as I’ve been using Wayfire. I’ve added the implementation in patch D49095.

Around the time I was working on the partial rendering patches, I also became interested in Firefox’s automatic enablement of GPU rendering. Namely, what was bothering me was that Xwayland was required for it (bug 1556301). To check that the GPU can be used on a system, Firefox forks off a glxtest process that attempts to establish its own connection to the display server, create a GL context and gather GL driver info. In patch D57474 I have added a Wayland-EGL code path to that process, dropping the X11 socket requirement.

Around that time, dma-buf support was being worked on, which is something that allows cross-process buffer sharing — necessary for hardware video decoding acceleration and for running WebGL in the content process without frame copy overhead. I have fixed one mistake in that code — an incorrect usage of the GBM API that was causing driver crashes (bug 1590832, patch D65239).

The next thing to tackle was lack of mouse pointer grabbing support (bug 1580595), which made things like 3D games and draggable number input controls in Figma unusable. All desktop platforms before Wayland just allowed applications to move the cursor around willy-nilly and the applications used that (constantly warping the cursor to the center of the window) to implement pointer grabbing, so there wasn’t a native API specific to this task. In Wayland, this is done via relative-pointer and pointer-constraints protocols. I have contributed patch D102114 that fixed this problem, adding Lock/UnlockNativePointer IPC messages from web content processes to the parent Firefox process and implementing their receiver using the aforementioned protocols.

Not actually Wayland-related, but I have also added the .desktop entry name to the MPRIS media control protocol implementation (bug 1676045, patch D96327), which helps desktop environments match the media playback session to the application.

Miscellaneous FreeBSD support improvements

Using Firefox on a Tier 3 platform like a BSD means that there will be some missing things and changes will be committed to the codebase that break the build on the platform. The most typical reason for accidental build breakage in a C/C++ project is of course header differences between platforms (often just in terms of transitive includes). Sometimes this is related to the OS itself (like bug 1615462) and sometimes to things like the C++ library (like bug 1594027). Firefox building but not working was never a situation I’ve encountered, however there were obscure corners like about:networking not working on Tier 3 platforms (bug 1477593, patch D58855). Two things however were not so obscure and not so trivial.

The first one was FIDO/U2F USB authentication token support (bug 1468349), which involved:

The second one was enabling the Firefox Profiler (bug 1634205). The resulting patch D73162 ended up mostly touching ifdefs (and adding platform knowledge like the ucontext_t structures for the supported architectures, amd64 and aarch64). But getting there was quite an adventure, with Google Breakpad being involved (but not as much as I initially thought), multiple ways of symbolication (the preferred way involving WASM-compiled tools which confusingly used to choke on debug builds of Firefox as they were over 2GB in size), having to figure out that dli_sname from the dladdr API is unusable on FreeBSD, and other interesting obstacles.

Speaking of supporting CPU architectures: I also made NSS, Firefox’s cryptography library, detect CPU features on FreeBSD/aarch64 (bug 1575843, patch D55386). Because I care about doubly obscure platforms and it’s really important for me that they use their cool hardware features when possible :)

Oh, there’s also one thing that’s actually relevant for Linux users as well, but unfortunately hasn’t landed (as of late 2022) because there sadly doesn’t seem to be anyone up for reviewing it. That is switching the gamepad support from the legacy Linux joystick interface to evdev, which is implemented by FreeBSD and DragonFlyBSD as well as Linux (bug 1680982, patch D98868). This patch is, however, currently shipped downstream in FreeBSD Ports, enabling gamepad support for all users of FreeBSD’s officially packaged Firefox out of the box.

Finally, I have experimented with enabling content process sandboxing using Capsicum (bug 1607980, patch D59253). Based on ideas from libpreopen, it uses an LD_PRELOAD hook to convert all the regular POSIX calls into something that does work with the strict Capsicum model of not having any global namespaces whatsoever and only allowing resources to be created by deriving them from existing ones (such as openat() below a directory you have an open file descriptor to). This is definitely quite a bit hacky and eventually I came to the conclusion that this kind of translation is better done in the kernel, and I’ve experimented in that direction — see the FreeBSD page.