Php8.1 with alternative malloc allocators

[ryancinsight]

Description

The following code:

LD_PRELOAD=libsnmallocshim.so php cron.php

Resulted in this output:

FREE():  invalid pointer
Aborted

But I expected this output instead:

111111111

more info is here:
microsoft/snmalloc#595

microsoft/mimalloc#377

I’m not an allocator expert but seems valgrind is showing memory leaks with alternative allocators and free is being called by malloc when replaced with LD_PRELOAD

PHP Version

PHP 8.1

Operating System

Debian Bullseye

[ryancinsight]

To clarify further I am using docker container and cron.php is for Nextcloud though same error occurs just writing “php” alone.

[nielsdos]

Executing php without extensions does not trigger the bug.
I can reproduce it if I enable the right extensions, so the problem is in the extensions and not in the core.
I found that enabling any of the following 3 extensions breaks php for me using snmalloc:

• curl
• mysqli
• zip

I'll now spend some time creating a debug build of PHP with these extensions to track down the bug.
EDIT: but only reproducible when the extensions are built as shared objects 🤔
EDIT2: it's probably all the extensions, it's just that the above three do some cleanup that calls to free upon the initialization, so that's why the error is triggered immediately.

Backtrace for curl:

#0  0x00007ffff7a068ec in ?? () from /usr/lib/libc.so.6
#1  0x00007ffff79b7ea8 in raise () from /usr/lib/libc.so.6
#2  0x00007ffff79a153d in abort () from /usr/lib/libc.so.6
#3  0x00007ffff79a229e in ?? () from /usr/lib/libc.so.6
#4  0x00007ffff7a10657 in ?? () from /usr/lib/libc.so.6
#5  0x00007ffff7a124bc in ?? () from /usr/lib/libc.so.6
#6  0x00007ffff7a14e63 in free () from /usr/lib/libc.so.6
#7  0x00007ffff7f7a7f2 in zend_string_release (s=0x7fbff4601b00) at /home/niels/php-src/Zend/zend_string.h:322
#8  0x00007ffff7f7ab42 in register_class_CURLFile () at /home/niels/php-src/ext/curl/curl_file_arginfo.h:69
#9  0x00007ffff7f7bd47 in curlfile_register_class () at /home/niels/php-src/ext/curl/curl_file.c:148
#10 0x00007ffff7f6b338 in zm_startup_curl (type=1, module_number=27) at /home/niels/php-src/ext/curl/interface.c:1212
#11 0x0000555555abcfc8 in zend_startup_module_ex (module=0x7fbff446a940) at /home/niels/php-src/Zend/zend_API.c:2199
#12 0x0000555555abd026 in zend_startup_module_zval (zv=0x7fbff4464440) at /home/niels/php-src/Zend/zend_API.c:2214
#13 0x0000555555ace385 in zend_hash_apply (ht=0x555556a21220 <module_registry>, 
    apply_func=0x555555abd003 <zend_startup_module_zval>) at /home/niels/php-src/Zend/zend_hash.c:1872
#14 0x0000555555abd669 in zend_startup_modules () at /home/niels/php-src/Zend/zend_API.c:2325
#15 0x0000555555a0fb3f in php_module_startup (sf=0x555556a07160 <cli_sapi_module>, additional_modules=0x0, num_additional_modules=0)
    at /home/niels/php-src/main/main.c:2260
#16 0x0000555555c295ff in php_cli_startup (sapi_module=0x555556a07160 <cli_sapi_module>)
    at /home/niels/php-src/sapi/cli/php_cli.c:409
#17 0x0000555555c2b7dd in main (argc=4, argv=0x7fbff44501b0) at /home/niels/php-src/sapi/cli/php_cli.c:1334

We can see that the freeing happens with the libc allocator indeed instead of snmalloc. So when the extension gets loaded the LD_PRELOAD isn't respected.

[nielsdos]

I identified the problem.
The problem is that extensions are loaded through dlopen, and dlopen does not respect LD_PRELOAD with the DEEPBIND option.

[nielsdos]

Removing RTLD_DEEPBIND from the DL_LOAD macro in Zend/zend_portability.h fixes this issue for me.
I don't know why we're using RTLD_DEEPBIND, but there's probably a good reason for it.

[nielsdos]

The flag was introduced by commit 601140cbe9a so I don't think we can easily get rid of it.

[NathanFreeman]

Maybe you can try the follow command to test it.

export USE_ZEND_ALLOC=0 && LD_PRELOAD=/your-path/libsnmallocshim.so

php -v

[nielsdos]

Maybe you can try the follow command to test it.

export USE_ZEND_ALLOC=0 && LD_PRELOAD=/your-path/libsnmallocshim.so

php -v

That won't work AFAIK, because it does not influence the dlopen flag. So that means the LD_PRELOAD has no effect on the extension and hence the libc allocator will be used instead of snmalloc.

[mjp41]

I believe I have fixed snmalloc to interact correctly with RTLD_DEEPBIND, based on the approach taken in jemalloc:

microsoft/snmalloc#598

@nielsdos if you still have the set up you were using to debug this issues, would you be able to check the latest main on snmalloc fixes it for you?

[nielsdos]

@mjp41 thanks for looking into this. I checked but it does not seem to fix the issue. I still get the invalid free message, and looking at the gdb backtrace it still calls free in libc.so.6

[nielsdos]

It actually looks like those hooks were deprecated and maybe it does not work because they are already removed in the glibc version I use (didnt check this)

[devnexen]

It actually looks like those hooks were deprecated and maybe it does not work because they are already removed in the glibc version I use (didnt check this)

You re right they are, the Mesh folks have been removed those.

[mjp41]

It actually looks like those hooks were deprecated and maybe it does not work because they are already removed in the glibc version I use (didnt check this)

This explains why some people observed jemalloc working and some didn't on the mimalloc issue.

@devnexen I don't think jemalloc is seeing the deprecation warning that Mesh had due to providing its own prototypes rather than include malloc.h:

https://github.com/jemalloc/jemalloc/blob/09e4b38fb1f9a9b505e35ac13b8f99282990bc2c/src/jemalloc.c#L3122-L3140

@interwq, @davidtgoldblatt, FYI: As glibc is updated the current approach you use to RTLD_DEEPBIND is going to stop working.