-
Notifications
You must be signed in to change notification settings - Fork 7.9k
Fix #37676 : Add E_WARNING when using array-index on non valid container #2031
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
The failed tests appear to be unrelated to this change |
Existing PRs: #1269, #866 -- I think there might even be one more. The tl;dr is that this should go through the RFC process, because it will likely have fallout (especially dereferencing of null/false). I'm definitely in favor of the change (and think most other people are as well), but I don't think this can be merged directly. |
I suspect you're right. I guess I didn't look close enough to existing PR's, as none were attached to the bug tickets. |
Yes, an RFC would be required, see the rather lengthy and controversial discussion about this issue from last year. Are you interested in authoring such an RFC, David? |
Thanks Chris. This is exactly what I was looking for before I start to
|
Any updates on this? At least add the behavior to the documentation in the meanwhile! This is really unexpected. |
hi @abiusx I have a draft RFC https://wiki.php.net/rfc/notice-for-non-valid-array-container which is currently under discussion. There hasn't been a whole lot of communication on it, but I do plan to push it into voting within the week or two. |
Hi
|
Maybe, maybe not. :-) Anyhow, there's no need to hurry, because the RFC is way to late for PHP 7.1 anyway, unfortunately. |
I concur. I didn't think about the differences in execution when assigning by reference rather than by value (Sorry I'm pretty green on internals). Your bug does raise some interesting points, and I'm a bit concerned that I would be inclined to attempt to standardize the differences when accessing containers for read or for write. Making it so the container does not magically become an array if the variable is undefined/null/false. |
Well, and here's where my lack of experience probably hinders, when doing an assignment of an arrayesque-element by reference and doing a set of an array to a reference we're doing the same opcode.
Are both doing a |
The FETCH_DIM_W is the parser struct, anything that is accessed using array operator [] is FETCH_DIM_W,
|
@bp1222 References are bidirectional. Btw, don't forget that as of PHP 7.1 |
That should explain the weird null behavior.
|
Zend/zend_execute.c
Outdated
if (type != BP_VAR_IS) { | ||
if (UNEXPECTED(Z_TYPE_P(container) == IS_UNDEF)) { | ||
zval_undefined_cv(EG(current_execute_data)->opline->op1.var, EG(current_execute_data)); | ||
} else if (EXPECTED((EG(current_execute_data)->opline->opcode == ZEND_FETCH_LIST && Z_LVAL_P(dim) == 0 && Z_TYPE_P(container) != IS_FALSE && (Z_TYPE_P(container) != IS_STRING || (Z_TYPE_P(container) == IS_STRING && Z_STRLEN_P(container) != 0))) || (EG(current_execute_data)->opline->opcode != ZEND_FETCH_LIST && (EG(current_execute_data)->opline->op1_type != IS_VAR || (Z_TYPE_P(container) != IS_UNDEF && Z_TYPE_P(container) != IS_NULL && (support_strings && Z_TYPE_P(container) != IS_STRING)))))) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you please extract this condition into a function? It's pretty hard to understand what is really going on here...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was planning on doing that before all said and done. I'm trying to limit the sheer quantity of notices that could be expelled for many cases. Specifically
foreach(list($a, $b) = each($arr))
was notice'ing when attempting to assign the final null value of each.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure special-casing each() is a good idea. At least, the way you do it (not specifically tailored to each()) will have side-effects on other cases as well (and tailoring it is even more crazy). This condition looks like a large assembly of various special-cases, which will introduce a bunch of extra inconsistencies resulting in different behavior based on minor differences in code -- like whether something reads a simple variable or an array offset/function call result.
(I'd actually consider it a benefit if uses of each() in that way started to generate notices -- serves as a natural deprecation. This pattern needs to die.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess that'd be left to debate/argue, but I'm trying to introduce fewest warnings for code that isn't a user accessing an element: like list(), and each iin a foreach/while. Although there may be better error types for seeing that pattern and raising an E_DEPRECEATED or something more targeted to that behavior.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In that case, might it be preferable to exclude list() entirely?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wouldn't be opposed to that. Might want to have list() continue to exist as is without modification
@nikic - I had forgotten @abiusx - That would explain the null behavior. Not sure if that'd be something worth changing in this RFC, or not. |
I removed list from raising warnings, and would want to treat this change as a read-operation warning, rather than read/write. Barring any comments here (@Internals has been quiet on my thread) I'm thinking this change for what it's purpose is, is just about done. |
Can you try the following code snippet and post the results of your version: $index_v=5; echo "Null:\n"; // die(); echo "Bool:\n"; echo "Float:\n"; echo "Null multidim:\n"; |
|
My only 'problem' with this, is that null values used in a reference assignment are converted to array, rather than failing as they do in the case of int/bool/float. It might be out of scope for this RFC, but I'd rather see only undef's be mutated. |
Good.
|
0bf495e
to
c8d7077
Compare
Zend/zend_execute.c
Outdated
@@ -1795,6 +1795,18 @@ static zend_never_inline void zend_fetch_dimension_address_UNSET(zval *result, z | |||
zend_fetch_dimension_address(result, container_ptr, dim, dim_type, BP_VAR_UNSET); | |||
} | |||
|
|||
static zend_always_inline int zend_fetch_dimension_address_read_valid_container(zval *container, int support_strings) { | |||
if (EG(current_execute_data)->opline->opcode != ZEND_FETCH_LIST) { | |||
if (EG(current_execute_data)->opline->op1_type != IS_VAR) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why do we need the check for IS_VAR?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The check here for not IS_VAR is to aide in preventing raising multiple warnings for
$a = null;
$a[1][2][3];
When accessing $a[1], the container op1_type is a CV, but the return of that for access of [2] is VAR. This was the attempt to only show the warning for user based code like this.
Although since you made me double check that I have found that it's not necessarily perfect.
function foo() {
return null;
}
foo()[1];
Does not properly throw any warning, since the return of a function IS_VAR
I'll change it around to look if IS_VAR the previous op wasn't a function call, so we can properly throw warning on the first access post-call.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bp1222 Won't that still incorrectly suppress the warning for the case of $a = [null]; $a[0][1];
? The [1]
is on null.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It does. Needs to check more previous op's to determine if we ought to raise the error or not when container is null, but a valid null.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Our standard approach to suppressing duplicate warnings is use of ERROR zvals. Doing this will need some care to ensure there is no impact on the fast-path though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah. I looked into using ZVAL_ERROR(), however, it's at the end of zend_fetch_dimension_address_read
that it sets the return to NULL if we hit error cases. This could have some cascading effects which I notices, for example:
$a = null;
var_dump($a[0]);
would not be NULL
, rather it'd be UNDEFINED:0
. This causes many many problems in existing tests where it's expecting the result of a failed lookup to be NULL. I assume internally they are functionally similar but I haven't looked much deeper.
My thought to accommodate your example would be to look to the previous opcode and determine if a null in this context is error-able or not. So if we're not looking at a compiled-var as the container, if it's null, but the previous op-code wasn't a ZEND_FETCH_DIM_R we should error. This would limit the erroring on a null -only- to if the container wasn't a result of a fetch already.
Confirmed: php-src/ext/opcache/zend_accelerator_module.c Lines 600 to 606 in d87fdd9
We should document that. |
Thanks! Done, (might need further clarification, though) |
Cf. <php/php-src#2031 (comment)>. git-svn-id: https://svn.php.net/repository/phpdoc/en/trunk@345790 c90b9560-bf6c-de11-be94-00142212c4b1
Cf. <php/php-src#2031 (comment)>. git-svn-id: http://svn.php.net/repository/phpdoc/en@345790 c90b9560-bf6c-de11-be94-00142212c4b1
Hey guys! It's really great to see work on bug aged 12+ years. Any chance to deliver it with v7.4? |
I would continue working on this, however, the last time it was in a merge-ready state the discussion was about the implementation relying on examining next op-codes in a delicate way. So even resolving the opcache, and current conflict, would still leave the major issue unaddressed. I wouldn't want to see any more effort going into this 2+year old PR without a solid end goal in place. |
Good point. @dstogov @nikic Since you were pinged about this before, what do you think about the delicate checking of op-codes? Is there a better way to check? I assume @cmb69 was talking about this line: |
ACK @bugreportuser Yes. See also #2031 (comment) f |
Hi again :) Is there any news for past 3 months? Thanks! |
Thanks for the ping. I believe that at this point, we mainly need to resolve the question of To recap, some people think that writing I think the possible options we have to resolve this are:
I'm not really sure what to do about this. I hate the idea of special-casing list() here, but if we want to do it we should probably exclude it entirely (the argument for null can just as well be made for false). If we can't reach a consensus here, this will need to go back to the internals list. |
A possible middle ground is to rely on
|
I believe a lot of developers would love approach to treat as a normal array accesses. However, as a compromise solution that looks not bad:
I mean, there should be a way to treat things strict at least with defined |
Please don't propose to abuse strict_types for stuff it's not intended for - strict_types are about type coercion, not array accesses. |
I believe this is also related to #54556. |
I don't see why list() accesses should be treated differently than normal array accesses, moreover it is pretty trivial to do a null check within the same condition. |
Hey guys, another half of year past. Appreciate your work, but if there any chance to make this working and merged? Thanks! |
New PR up at #4386. |
Wow! So this should be closed since #4386 is merged already :) |
Yeah, sorry this took so long, should have merged a reduced variant long ago. I think as far as list() is concerned, we might consider "blessing" the if (list() = $x) pattern by actually doing different codegen if list() is used in a conditional context and make it a first-class refutable binding. But that's a bigger discussion, so for now list() is just excluded from this check. |
Cf. <php/php-src#2031 (comment)>. git-svn-id: https://svn.php.net/repository/phpdoc/en/trunk@345790 c90b9560-bf6c-de11-be94-00142212c4b1
Cf. <php/php-src#2031 (comment)>. git-svn-id: https://svn.php.net/repository/phpdoc/en/trunk@345790 c90b9560-bf6c-de11-be94-00142212c4b1
This bugfix attempts to resolve 37676. It will make sure that the container is either compiled, or if it's a return-variable make sure it's something that we know is error-able.
Updated tests to account for the E_NOTICE that will go out with this fix.
Other related bugs:
39915
72636