Skip to content

Delayed notice again #12805

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from
Draft

Delayed notice again #12805

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
fa475ea
Delay notice emission until end of opcode
nikic Apr 23, 2021
198b22a
Add support for INDIRECT delaying
iluuu1994 Aug 29, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Zend/tests/bug52041.phpt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
--TEST--
Bug #52041 (Memory leak when writing on uninitialized variable returned from function)
--INI--
opcache.jit=0
Comment on lines +3 to +4
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you disable JIT. Its not supported yet?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, JIT doesn't seem to work but I have not looked into where it goes wrong.

--FILE--
<?php
function foo() {
Expand Down
2 changes: 2 additions & 0 deletions Zend/tests/bug78598.phpt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
--TEST--
Bug #78598: Changing array during undef index RW error segfaults
--INI--
opcache.jit=0
--FILE--
<?php

Expand Down
43 changes: 43 additions & 0 deletions Zend/tests/delayed_error_001.phpt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
--TEST--
Delayed error 001
--INI--
opcache.jit=0
--FILE--
<?php
$array[0][1] .= 'foo';
$array[2][3]++;
$array[3][4]--;
++$array[5][6];
--$array[7][8];
$array[9][10] += 42;
?>
--EXPECTF--
Warning: Undefined variable $array in %s on line %d

Warning: Undefined array key 0 in %s on line %d

Warning: Undefined array key 1 in %s on line %d

Warning: Undefined array key 2 in %s on line %d

Warning: Undefined array key 3 in %s on line %d

Warning: Decrement on type null has no effect, this will change in the next major version of PHP in %s on line %d

Warning: Undefined array key 3 in %s on line %d

Warning: Undefined array key 4 in %s on line %d

Warning: Undefined array key 5 in %s on line %d

Warning: Undefined array key 6 in %s on line %d

Warning: Decrement on type null has no effect, this will change in the next major version of PHP in %s on line %d

Warning: Undefined array key 7 in %s on line %d

Warning: Undefined array key 8 in %s on line %d

Warning: Undefined array key 9 in %s on line %d

Warning: Undefined array key 10 in %s on line %d
10 changes: 8 additions & 2 deletions Zend/tests/undef_index_to_exception.phpt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
--TEST--
Converting undefined index/offset notice to exception
--INI--
opcache.jit=0
--FILE--
<?php

Expand Down Expand Up @@ -37,10 +39,14 @@ try {
?>
--EXPECT--
Undefined array key 0
array(0) {
array(1) {
[0]=>
string(3) "xyz"
}
Undefined array key "key"
array(0) {
array(1) {
[0]=>
string(3) "xyz"
}
Undefined global variable $test
Undefined variable $test
38 changes: 38 additions & 0 deletions Zend/zend.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -672,6 +672,18 @@ static void zend_init_exception_op(void) /* {{{ */
}
/* }}} */

static void zend_init_delayed_error_op(void) /* {{{ */
{
memset(EG(delayed_error_op), 0, sizeof(EG(delayed_error_op)));
EG(delayed_error_op)[0].opcode = ZEND_HANDLE_DELAYED_ERROR;
ZEND_VM_SET_OPCODE_HANDLER(EG(delayed_error_op));
EG(delayed_error_op)[1].opcode = ZEND_HANDLE_DELAYED_ERROR;
ZEND_VM_SET_OPCODE_HANDLER(EG(delayed_error_op)+1);
EG(delayed_error_op)[2].opcode = ZEND_HANDLE_DELAYED_ERROR;
ZEND_VM_SET_OPCODE_HANDLER(EG(delayed_error_op)+2);
}
/* }}} */

static void zend_init_call_trampoline_op(void) /* {{{ */
{
memset(&EG(call_trampoline_op), 0, sizeof(EG(call_trampoline_op)));
Expand Down Expand Up @@ -786,6 +798,7 @@ static void executor_globals_ctor(zend_executor_globals *executor_globals) /* {{
zend_copy_constants(executor_globals->zend_constants, GLOBAL_CONSTANTS_TABLE);
zend_init_rsrc_plist();
zend_init_exception_op();
zend_init_delayed_error_op();
zend_init_call_trampoline_op();
memset(&executor_globals->trampoline, 0, sizeof(zend_op_array));
executor_globals->capture_warnings_during_sccp = 0;
Expand Down Expand Up @@ -1025,6 +1038,7 @@ void zend_startup(zend_utility_functions *utility_functions) /* {{{ */
#ifndef ZTS
zend_init_rsrc_plist();
zend_init_exception_op();
zend_init_delayed_error_op();
zend_init_call_trampoline_op();
#endif

Expand Down Expand Up @@ -1709,6 +1723,30 @@ ZEND_API void zend_free_recorded_errors(void)
EG(num_errors) = 0;
}

ZEND_API ZEND_COLD void zend_error_delayed(int type, const char *format, ...)
{
ZEND_ASSERT(!(type & E_FATAL_ERRORS) && "Cannot delay fatal error");
zend_error_info *info = emalloc(sizeof(zend_error_info));
info->type = type;
get_filename_lineno(type, &info->filename, &info->lineno);
zend_string_addref(info->filename);

va_list args;
va_start(args, format);
info->message = zend_vstrpprintf(0, format, args);
va_end(args);

zend_hash_next_index_insert_ptr(&EG(delayed_errors), info);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess a single linked list in zend_error_info would be more straightforward than using a hashtable for that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's used as a stack, essentially, and avoids an allocation for each warning. Should I use zend_stack instead?


if (EG(current_execute_data)->opline != EG(delayed_error_op)) {
EG(opline_before_exception) = EG(current_execute_data)->opline;
EG(current_execute_data)->opline = EG(delayed_error_op);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you like to implement this branch in JIT?
Note that you might need to save all data kept in CPU registers before branching.
The jump back from delayed warning to normal control flow is not possible at all.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh... I assumed JIT handled exception the same way as the VM, checking EX(opline) rather than EG(exception). It also calls the EG(exception_op)->handler directly, rather than looking up EX(opline)->handler. That indeed will not work here.

The jump back from delayed warning to normal control flow is not possible at all.

I will have to look more closely how this works for exceptions to understand what changes are required. I assumed exceptions trigger deoptimization but I guess that's not correct.

/* Reset to ZEND_HANDLE_DELAYED_ERROR */
EG(delayed_error_op)[0] = EG(delayed_error_op)[2];
EG(delayed_error_op)[1] = EG(delayed_error_op)[2];
}
}

ZEND_API ZEND_COLD void zend_throw_error(zend_class_entry *exception_ce, const char *format, ...) /* {{{ */
{
va_list va;
Expand Down
1 change: 1 addition & 0 deletions Zend/zend.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -352,6 +352,7 @@ ZEND_API ZEND_COLD void zend_error_at(int type, zend_string *filename, uint32_t
ZEND_API ZEND_COLD ZEND_NORETURN void zend_error_at_noreturn(int type, zend_string *filename, uint32_t lineno, const char *format, ...) ZEND_ATTRIBUTE_FORMAT(printf, 4, 5);
ZEND_API ZEND_COLD void zend_error_zstr(int type, zend_string *message);
ZEND_API ZEND_COLD void zend_error_zstr_at(int type, zend_string *filename, uint32_t lineno, zend_string *message);
ZEND_API ZEND_COLD void zend_error_delayed(int type, const char *format, ...) ZEND_ATTRIBUTE_FORMAT(printf, 2, 3);

ZEND_API ZEND_COLD void zend_throw_error(zend_class_entry *exception_ce, const char *format, ...) ZEND_ATTRIBUTE_FORMAT(printf, 2, 3);
ZEND_API ZEND_COLD void zend_type_error(const char *format, ...) ZEND_ATTRIBUTE_FORMAT(printf, 1, 2);
Expand Down
25 changes: 24 additions & 1 deletion Zend/zend_execute.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2226,6 +2226,28 @@ ZEND_API ZEND_COLD zval* ZEND_FASTCALL zend_undefined_offset_write(HashTable *ht
return zend_hash_index_add_new(ht, lval, &EG(uninitialized_zval));
}

ZEND_API ZEND_COLD void ZEND_FASTCALL zend_undefined_offset_delayed(zend_long lval)
{
zend_error_delayed(E_WARNING, "Undefined array key " ZEND_LONG_FMT, lval);
}

ZEND_API void ZEND_FASTCALL zend_handle_delayed_errors(void)
{
/* Clear EG(delayed_errors), as more errors may be delayed while we are handling these. */
HashTable ht;
memcpy(&ht, &EG(delayed_errors), sizeof(HashTable));
zend_hash_init(&EG(delayed_errors), 0, NULL, NULL, 0);

zend_error_info *info;
ZEND_HASH_FOREACH_PTR(&ht, info) {
zend_error_zstr_at(info->type, info->filename, info->lineno, info->message);
zend_string_release(info->filename);
zend_string_release(info->message);
efree(info);
} ZEND_HASH_FOREACH_END();
zend_hash_destroy(&ht);
}

ZEND_API ZEND_COLD zval* ZEND_FASTCALL zend_undefined_index_write(HashTable *ht, zend_string *offset)
{
zval *retval;
Expand Down Expand Up @@ -2495,7 +2517,8 @@ static zend_always_inline zval *zend_fetch_dimension_address_inner(HashTable *ht
retval = &EG(uninitialized_zval);
break;
case BP_VAR_RW:
retval = zend_undefined_offset_write(ht, hval);
zend_undefined_offset_delayed(hval);
retval = zend_hash_index_add_new(ht, hval, &EG(uninitialized_zval));
break;
}
} else {
Expand Down
3 changes: 3 additions & 0 deletions Zend/zend_execute.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,9 @@ ZEND_API ZEND_COLD void ZEND_FASTCALL zend_invalid_class_constant_type_error(uin
ZEND_API ZEND_COLD void ZEND_FASTCALL zend_object_released_while_assigning_to_property_error(const zend_property_info *info);

ZEND_API ZEND_COLD void ZEND_FASTCALL zend_cannot_add_element(void);
ZEND_API ZEND_COLD void ZEND_FASTCALL zend_undefined_offset_delayed(zend_long lval);

ZEND_API void ZEND_FASTCALL zend_handle_delayed_errors(void);

ZEND_API bool zend_verify_scalar_type_hint(uint32_t type_mask, zval *arg, bool strict, bool is_internal_arg);
ZEND_API ZEND_COLD void zend_verify_arg_error(
Expand Down
3 changes: 3 additions & 0 deletions Zend/zend_execute_API.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,9 @@ void init_executor(void) /* {{{ */

zend_max_execution_timer_init();
zend_fiber_init();

zend_hash_init(&EG(delayed_errors), 0, NULL, NULL, 0);

zend_weakrefs_init();

EG(active) = 1;
Expand Down
3 changes: 3 additions & 0 deletions Zend/zend_globals.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -246,6 +246,8 @@ struct _zend_executor_globals {
zend_object *exception, *prev_exception;
const zend_op *opline_before_exception;
zend_op exception_op[3];
zend_op delayed_error_op[3];
zval delayed_error_consts[3];

struct _zend_module_entry *current_module;

Expand Down Expand Up @@ -303,6 +305,7 @@ struct _zend_executor_globals {
pid_t pid;
struct sigaction oldact;
#endif
HashTable delayed_errors;

void *reserved[ZEND_MAX_RESERVED_RESOURCES];
};
Expand Down
72 changes: 72 additions & 0 deletions Zend/zend_vm_def.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8022,6 +8022,10 @@ ZEND_VM_HANDLER(149, ZEND_HANDLE_EXCEPTION, ANY, ANY)
{
const zend_op *throw_op = EG(opline_before_exception);

if (zend_hash_num_elements(&EG(delayed_errors))) {
zend_handle_delayed_errors();
}

/* Exception was thrown before executing any op */
if (UNEXPECTED(!throw_op)) {
ZEND_VM_DISPATCH_TO_HELPER(zend_dispatch_try_catch_finally_helper, try_catch_offset, -1, 0, 0);
Expand Down Expand Up @@ -8091,6 +8095,74 @@ ZEND_VM_HANDLER(149, ZEND_HANDLE_EXCEPTION, ANY, ANY)
ZEND_VM_DISPATCH_TO_HELPER(zend_dispatch_try_catch_finally_helper, try_catch_offset, current_try_catch_offset, op_num, throw_op_num);
}

ZEND_VM_HANDLER(204, ZEND_HANDLE_DELAYED_ERROR, ANY, ANY)
{
const zend_op *prev_op = EG(opline_before_exception);
bool delay = false;
switch (prev_op->opcode) {
case ZEND_FETCH_W:
case ZEND_FETCH_RW:
case ZEND_FETCH_FUNC_ARG:
case ZEND_FETCH_UNSET:
case ZEND_FETCH_DIM_W:
case ZEND_FETCH_DIM_RW:
case ZEND_FETCH_DIM_UNSET:
case ZEND_FETCH_LIST_W:
case ZEND_FETCH_OBJ_W:
case ZEND_FETCH_OBJ_RW:
case ZEND_FETCH_OBJ_UNSET:
delay = true;
break;
}

// FIXME: Is this guaranteed to be there?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the locations where a delayed error may be thrown currently, yes.
We might want to simply ZEND_ASSERT(next_op < EX(func)->op_array.opcodes + EX(func)->op_array.last)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's probably ok to just check, this handler is part of a slow path, and not intended to be particularly fast.

const zend_op *next_op = prev_op + 1;
if (next_op->opcode == ZEND_OP_DATA) {
next_op++;
}

if (delay) {
zend_op *delayed_op = &EG(delayed_error_op)[0];
*delayed_op = *next_op;
Comment on lines +8125 to +8126
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case you have few FETCH instructions in row and few of them produce warnings, then EG(delayed_error_op)[0] is going to be overridden few times. Is it OK?
Will this work in conjunction with magic __get() and different warnings in main code and magic methods?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case you have few FETCH instructions in row and few of them produce warnings, then EG(delayed_error_op)[0] is going to be overridden few times. Is it OK?

Yes. The idea is that each instruction producing indirect values only delays to the next opcode. If that opcode also produces an indirect value, then it wil once again set EG(delayed_error_op)[0] to the instruction after that, until eventually the indirect value is used and the error can be emitted.

Note that this happens only if the first FETCH emits a warning. The error is still delayed until the fetch+assign chain.

Will this work in conjunction with magic __get() and different warnings in main code and magic methods?

I didn't test this, but I think one unexpected things could be that we don't associate EG(delayed_errors) with a particular stack frame. If the magic method produces a warning, pending warnings from the outer VM call will also be handled. This could be solved by storing the execute_data on the delayed error, and only handling those belonging to the current one. Apart from that I believe magic methods should work correctly with this approach.

if (delayed_op->op1_type == IS_CONST) {
ZVAL_COPY_VALUE(&EG(delayed_error_consts)[0], RT_CONSTANT(next_op, next_op->op1));
delayed_op->op1.num = (char *)&EG(delayed_error_consts)[0] - (char *)delayed_op;
}
if (delayed_op->op2_type == IS_CONST) {
ZVAL_COPY_VALUE(&EG(delayed_error_consts)[1], RT_CONSTANT(next_op, next_op->op2));
delayed_op->op2.num = (char *)&EG(delayed_error_consts)[1] - (char *)delayed_op;
}
// FIXME: Is this guaranteed to be there?
if (next_op[1].opcode == ZEND_OP_DATA) {
const zend_op *next_opdata = &next_op[1];
zend_op *delayed_opdata = &EG(delayed_error_op)[1];
*delayed_opdata = *next_opdata;
if (delayed_opdata->op1_type == IS_CONST) {
ZVAL_COPY_VALUE(&EG(delayed_error_consts)[2], RT_CONSTANT(next_opdata, next_opdata->op1));
delayed_opdata->op1.num = (char *)&EG(delayed_error_consts)[2] - (char *)delayed_opdata;
}
} else {
/* Reset to ZEND_HANDLE_DELAYED_ERROR */
EG(delayed_error_op)[1] = EG(delayed_error_op)[2];
}
EG(opline_before_exception) = next_op;

ZEND_VM_SET_NEXT_OPCODE(delayed_op);
} else {
EX(opline) = prev_op;
zend_handle_delayed_errors();

if (EG(exception)) {
HANDLE_EXCEPTION();
}

EG(opline_before_exception) = NULL;
ZEND_VM_SET_NEXT_OPCODE(next_op);
}

ZEND_VM_CONTINUE();
}

ZEND_VM_HANDLER(150, ZEND_USER_OPCODE, ANY, ANY)
{
USE_OPLINE
Expand Down
Loading