Fix GH-10229 http_build_query() skips Stringable params

[Girgias]

Did some refactoring to make the implementation more obvious as to what is going on and added some tests for some other weird types.

[cmb69]

BC alert! :) See e.g. https://3v4l.org/nH479, which might actually be used by some people, and behaves this way for ages. I don't think that it should behave that way, but we probably don't want to fix the behavior in stable versions (and maybe not even in a minor version). Maybe we should deprecate passing objects (or array containing objects) instead.

[nielsdos]

Hey, I'm just gonna chime in quickly regarding deprecating passing objects.
This function uses a key-value relationship to build a query string, and associative arrays and objects also follow a key-value relationship.
That's why I don't like deprecating passing objects: it fits the description nicely. In APIs, I also like to use objects in a plain-data-object way, the advantage over arrays is the type checking you can get because of typed properties. Now I could just pass in such an object to this function without converting to array first, which would no longer be possible if this gets deprecated.

[cmb69]

@nielsdos, thanks! Point taken. However, what should we do with Stringable objects?

[nielsdos]

One possible mitigation that works for the code that was posted in the issue is the following:

• If there are no public properties, and the object is Stringable, use __toString
• If there are no public properties, and the object is not Stringable, it should be the empty string just like is the case in current stable versions
• If there are public properties, always use the behaviour of current stable versions

I believe this way results in only minor BC break

[Girgias]

Arf, we're back to the annoying case where if PHP had proper struct this would be less of an issue as objects can be used in so many different ways.

I attempted to implement @nielsdos idea but somehow messed up the implementation in regard to recursive checks to prevent needed to recompute the visible (because if this function is called within a method it unmangle and adds protected/private) properties.

[Girgias]

Well I've tried something and spending way too long on something that is still buggy and having no idea why. So if someone has an either as to what I'm doing wrong feel free to let me know...

[nielsdos]

@Girgias I think I got it working, it's less engineered than your attempted solution (i.e. it does not do any caching related stuff) but I think it works (all tests pass and it implements my suggested behaviour): You can see my branch at https://github.com/nielsdos/php-src/commits/gh10229-attempt

I started from the commits of this PR, but dropped the last 3 commits (except that I squashed the added tests in the last commit to the "Add more tests with objects" commit. I also fixed a memleak which was in the PR. The last commit contains my code changes which seems to work.

[Girgias]

@Girgias I think I got it working, it's less engineered than your attempted solution (i.e. it does not do any caching related stuff) but I think it works (all tests pass and it implements my suggested behaviour): You can see my branch at https://github.com/nielsdos/php-src/commits/gh10229-attempt

I started from the commits of this PR, but dropped the last 3 commits (except that I squashed the added tests in the last commit to the "Add more tests with objects" commit. I also fixed a memleak which was in the PR. The last commit contains my code changes which seems to work.

Yeah me attempting to not do multiple passes of objects was the issue, so I think at least for a backport this makes more sense, as there is a lot of stuff that can (and should) be optimized in that function anyway (which might make a caching solution easier). @cmb69 what do you think about @nielsdos's solution?

[nielsdos]

Coincidentally I was just working on a small optimisation I just pushed: checking if __toString exists before attempting to determine if it has public properties.

[Girgias]

Coincidentally I was just working on a small optimisation I just pushed: checking if __toString exists before attempting to determine if it has public properties.

I don't know if all the internal objects that can cast to string actually implement __toString() :-/

[nielsdos]

Coincidentally I was just working on a small optimisation I just pushed: checking if __toString exists before attempting to determine if it has public properties.

I don't know if all the internal objects that can cast to string actually implement __toString() :-/

Oh wow that's so weird behaviour, you're right: https://3v4l.org/kD6HV
Okay then we should drop that one optimisation commit I guess...

[Girgias]

Maybe something to try to fix for PHP 8.3

[cmb69]

Hmm, I still don't see the current behavior strictly as bug. Unfortunately, the man page is underspecified, but it says:

If data is an object, then only public properties will be incorporated into the result.

While that doesn't talk about nested objects, the "Using http_build_query() with an object" examples clarifies this.

Given that @nielsdos' suggestion would prevent most BC breaks, we may not need to have to wait for a major version, or do some deprecation (or notices/warnings), but I'm still uncomfortable to change the behavior in PHP-8.1 (besides that the diff is quite large). I would suggest to target PHP-8.2, but the decision is up to the RMs. @ramsey, @patrickallaert, what do you think?

[nielsdos]

I don't know if all the internal objects that can cast to string actually implement __toString() :-/

Speaking about this again: I fixed the optimisation and added a test for it. The optimisation still holds if the zend_object uses zend_std_cast_object_tostring, so I added that condition.

[Girgias]

Hmm, I still don't see the current behavior strictly as bug. Unfortunately, the man page is underspecified, but it says:

If data is an object, then only public properties will be incorporated into the result.

While that doesn't talk about nested objects, the "Using http_build_query() with an object" examples clarifies this.

Given that @nielsdos' suggestion would prevent most BC breaks, we may not need to have to wait for a major version, or do some deprecation (or notices/warnings), but I'm still uncomfortable to change the behavior in PHP-8.1 (besides that the diff is quite large). I would suggest to target PHP-8.2, but the decision is up to the RMs. @ramsey, @patrickallaert, what do you think?

This makes sense, I wasn't expecting it to get that complicated. And I'm even leaning towards making this change only in master if we are doing it. But this is one of those edge cases that's hard to figure out what the "correct" behaviour even should be.

[nielsdos]

Imo this is a feature change instead of a bugfix, because docs say:

If data is an object, then only public properties will be incorporated into the result.

It says nothing about Stringable objects.

So imo it's best to get this in master.

[Girgias]

Well I've tried something and spending way too long on something that is still buggy and having no idea why. So if someone has an either as to what I'm doing wrong feel free to let me know...

So I've figured out the issue while rebasing and refactoring the function.

I was shadowing the new_prefix variable in some interior scope meaning it wasn't modifying the variable that actually needed to be changed... (-Wshadow would have been useful here but that's a no go because of the amount of macros that rely on this).

Thus my cached implementation should now work

[nielsdos]

The only thing I wonder now about your cached implementation vs my non-cached one is whether the cached one is actually faster as it does more memory allocations and hashtable additions I think. I wonder whether we can easily benchmark that somehow?

[Girgias]

The only thing I wonder now about your cached implementation vs my non-cached one is whether the cached one is actually faster as it does more memory allocations and hashtable additions I think. I wonder whether we can easily benchmark that somehow?

Ideally I would not do a complete copy of the properties array, but I don't understand why when I try to just create a new array and add key/value pairs I getting use after frees within the shutdown sequence, as it seems to want to destroy the properties HashTable twice...

The main other reason as to why I'm returning a new array is that I would expect that other extensions may want to get all the visible/public properties and having this as a ZEND_API may make sense.