Fix GH-10092 (Internal stream casting should not emit lost bytes warning twice)

[Girgias]

The reason why the warning was emitted is that it was not guarded via the show_err parameter.

This parameter is set to false/0 via the macro function php_stream_can_cast() to check this silently.

The warning is completely removed as this function in particular sets this parameter to 0 when it casts the streams, it "performs a cast" just to retrieve the underlying file number.

[arnaud-lb]

I feel that show_err is meant to control errors related to cast failures, and that errors that do not result in a cast failure should not be silenced. That's because php_stream_cast(show_err: 0) callers would probably display a warning themselves or fallback to something else if php_stream_cast fails (so the php_stream_cast errors would be noise), but the "non-emtpy buffer" warning would still be useful in case php_stream_cast() succeeded.

stream_select() uses the PHP_STREAM_CAST_INTERNAL flag to silence this warning. streamfuncs.c has a comment mentioning this:

php-src/ext/standard/streamsfuncs.c

Lines 634 to 636 in b9cd1cd

	* NB: Most other code will NOT use the PHP_STREAM_CAST_INTERNAL flag
	* when casting. It is only used here so that the buffered data warning
	* is not displayed.

This appears to be its only effect, as it's only referenced here:

php-src/main/streams/cast.c

Line 324 in b9cd1cd

(flags & PHP_STREAM_CAST_INTERNAL) == 0

In case you decide to use PHP_STREAM_CAST_INTERNAL, there are a few other php_stream_cast(show_err: 0) callers that would need this flag (at least posix_isatty() and sapi_windows_vt100_support()).

[bukka]

I don't think this warning should be silenced as it signals that the stream lost some data during casting. However that check doesn't seem really correct to me for all cases. I have done some debugging for the reported case (http wrapper) the readpos (354) and writepos (1610) are the same before and after the cast so nothing changed during the cast (no data lost) and thus the warning doesn't really make any sense here. Basically this logic might be correct for some streams but doesn't not apply to all. It might need some thorough investigation considering all supported streams and possibly some refactoring to better signal when the stream actually lost some data.

There's of course another issue with emitting 2 warning which is more a bug in stream_isatty and can be easily addressed by

diff --git a/ext/standard/streamsfuncs.c b/ext/standard/streamsfuncs.c
index 13f3ead5ae..c73c8fbac9 100644
--- a/ext/standard/streamsfuncs.c
+++ b/ext/standard/streamsfuncs.c
@@ -1632,11 +1632,10 @@ PHP_FUNCTION(stream_isatty)
 
        php_stream_from_zval(stream, zsrc);
 
-       if (php_stream_can_cast(stream, PHP_STREAM_AS_FD_FOR_SELECT) == SUCCESS) {
-               php_stream_cast(stream, PHP_STREAM_AS_FD_FOR_SELECT, (void*)&fileno, 0);
-       } else if (php_stream_can_cast(stream, PHP_STREAM_AS_FD) == SUCCESS) {
-               php_stream_cast(stream, PHP_STREAM_AS_FD, (void*)&fileno, 0);
-       } else {
+       if (
+               php_stream_cast(stream, PHP_STREAM_AS_FD_FOR_SELECT, (void*)&fileno, 0) == FAILURE &&
+               php_stream_cast(stream, PHP_STREAM_AS_FD, (void*)&fileno, 0) == FAILURE
+       ) {
                RETURN_FALSE;
        }

In general doing can_cast directly followed by cast with suppressed warning is an anti-pattern as it is doing exactly the same thing twice... I think this should be fixed / improved independently but with a better tests that would not be online like the one here. We might need to do first the investigation to see if there are actually cases when data can be lost and possibly recreate such case in the test. Depending on the result of that, the change in stream_isatty will be either a bug fix (target PHP-8.1) or a feature (master branch target).

[Girgias]

I don't think this warning should be silenced as it signals that the stream lost some data during casting. However that check doesn't seem really correct to me for all cases. I have done some debugging for the reported case (http wrapper) the readpos (354) and writepos (1610) are the same before and after the cast so nothing changed during the cast (no data lost) and thus the warning doesn't really make any sense here. Basically this logic might be correct for some streams but doesn't not apply to all. It might need some thorough investigation considering all supported streams and possibly some refactoring to better signal when the stream actually lost some data.

There's of course another issue with emitting 2 warning which is more a bug in stream_isatty and can be easily addressed by

diff --git a/ext/standard/streamsfuncs.c b/ext/standard/streamsfuncs.c
index 13f3ead5ae..c73c8fbac9 100644
--- a/ext/standard/streamsfuncs.c
+++ b/ext/standard/streamsfuncs.c
@@ -1632,11 +1632,10 @@ PHP_FUNCTION(stream_isatty)
 
        php_stream_from_zval(stream, zsrc);
 
-       if (php_stream_can_cast(stream, PHP_STREAM_AS_FD_FOR_SELECT) == SUCCESS) {
-               php_stream_cast(stream, PHP_STREAM_AS_FD_FOR_SELECT, (void*)&fileno, 0);
-       } else if (php_stream_can_cast(stream, PHP_STREAM_AS_FD) == SUCCESS) {
-               php_stream_cast(stream, PHP_STREAM_AS_FD, (void*)&fileno, 0);
-       } else {
+       if (
+               php_stream_cast(stream, PHP_STREAM_AS_FD_FOR_SELECT, (void*)&fileno, 0) == FAILURE &&
+               php_stream_cast(stream, PHP_STREAM_AS_FD, (void*)&fileno, 0) == FAILURE
+       ) {
                RETURN_FALSE;
        }

In general doing can_cast directly followed by cast with suppressed warning is an anti-pattern as it is doing exactly the same thing twice... I think this should be fixed / improved independently but with a better tests that would not be online like the one here. We might need to do first the investigation to see if there are actually cases when data can be lost and possibly recreate such case in the test. Depending on the result of that, the change in stream_isatty will be either a bug fix (target PHP-8.1) or a feature (master branch target).

Thanks for the detailed explanation, I will try and figure out a test that is not online, but I'm also wondering if doing both casts isn't already a bug in itself? Because I would expect that if the cast to PHP_STREAM_AS_FD_FOR_SELECT fails then the one for PHP_STREAM_AS_FD would also fail, and vice versa if the PHP_STREAM_AS_FD_FOR_SELECT cast succeeds then necessarily the PHP_STREAM_AS_FD one would too.

[bukka]

I'm also wondering if doing both casts isn't already a bug in itself? Because I would expect that if the cast to PHP_STREAM_AS_FD_FOR_SELECT fails then the one for PHP_STREAM_AS_FD would also fail, and vice versa if the PHP_STREAM_AS_FD_FOR_SELECT cast succeeds then necessarily the PHP_STREAM_AS_FD one would too.

This is mostly true for all core streams. The only exception is the ssl one:

php-src/ext/openssl/xp_ssl.c

Lines 2534 to 2557 in 383053c

	case PHP_STREAM_AS_FD_FOR_SELECT:
	if (ret) {
	size_t pending;
	if (stream->writepos == stream->readpos
	&& sslsock->ssl_active
	&& (pending = (size_t)SSL_pending(sslsock->ssl_handle)) > 0) {
	php_stream_fill_read_buffer(stream, pending < stream->chunk_size
	? pending
	: stream->chunk_size);
	}
	*(php_socket_t *)ret = sslsock->s.socket;
	}
	return SUCCESS;
	case PHP_STREAM_AS_FD:
	case PHP_STREAM_AS_SOCKETD:
	if (sslsock->ssl_active) {
	return FAILURE;
	}
	if (ret) {
	*(php_socket_t *)ret = sslsock->s.socket;
	}
	return SUCCESS;

It can theoretically fail for PHP_STREAM_AS_FD and succeed for PHP_STREAM_AS_FD_FOR_SELECT if the ssl is not active. This logic is a bit suspicious though as I'm not sure if it makes much sense (added it to my notes and will look into it later). In any case this still makes the second PHP_STREAM_AS_FD case not that useful as it is behind PHP_STREAM_AS_FD_FOR_SELECT. However there might be also 3rd party extensions streams or possible future use cases in core where PHP_STREAM_AS_FD_FOR_SELECT would fail but PHP_STREAM_AS_FD succeed as some file descriptors might be potentially not selectable / pollable (e.g. /dev/net/tun and some other special fds). Personally I think that the opposite case that is happening in xp_ssl should be eventually eliminated as it really doesn't make any sense to me.

I think that in case of stream_isatty it should just use PHP_STREAM_AS_FD because even for xp_ssl it would ended up as false on isatty check. I think that using PHP_STREAM_AS_FD_FOR_SELECT should be done only if it needs to do poll on the fd later as the purpose of that is to do some extra preparation (often reading data to the buffer) for select / poll.

[arnaud-lb]

the readpos (354) and writepos (1610) are the same before and after the cast so nothing changed during the cast (no data lost) and thus the warning doesn't really make any sense here

Data loss or corruption would happen if we attempted to read or write from the file descriptor after casting, because this would not reflect the buffered data.

For instance, this script loses buffered data.

I think that the warning is legit, but the wording could be improved.

Regarding posix_isatty(), it's correct to silence the warning because we do not read or write from the file descriptor. stream_select() silences the warning for the same reason.

[bukka]

@arnaud-lb Ah ok I got it now. I completely misunderstood it and thought that it was about loosing data when doing the conversion. I think the during confused me here. I agree that it would be good to update the message and suppress the warning in such case.

[Girgias]

Closing in favour of #10173