Add --progress option to run-tests.php

[iluuu1994]

The option is enabled in CLI but disabled in CI by default. Previously,
adding the -g argument would disable progress, even locally.

[cmb69]

Previously, adding the -g argument would disable progress, even locally.

Hmm, doesn't that depend on the values passed to -g? I mean -g PASS,FAIL,XFAIL,XLEAK,SKIP,BORK,WARN,LEAK,REDIRECT should show all tests.

[iluuu1994]

@cmb69 By progress I mean the following:

TEST 1/1000 [14/14 concurrent test workers running]

The test results are shown depending on what you specify with -g, as before.

[cmb69]

Ah, I see! Then it makes sense to me, but I wouldn't bake-in CI detection, but rather change the CI scripts to pass the --no-progress option (the BC break seems to be acceptable).

[iluuu1994]

@cmb69 Do you have concerns with auto-disabling progress in CI? It would be useful there but only if we massively turn down the frequency, maybe every minute or so. Most CIs do not support \r without \n and thus will just fill the logs.

[cmb69]

Do you have concerns with auto-disabling progress in CI?

Not generally, but I don't like a hard-coded list in run-tests.php. There is no chance to catch all CI providers anyway.

[iluuu1994]

@cmb69 Isn't 95% better than nothing? If a specific CI isn't covered one can pass the --no-progress manually. This is how most tools seem to use it. Either way, I am not set on it, I can remove it 🙂

[krakjoe]

Isn't 95% better than nothing?

I have some words ...

It would be nice if the concerns of run-tests.php were restricted to running tests, it's already a very messy place to make changes.

How run-tests is configured should be determined by the caller of the script by the options we provide.

Note that, we already went down the special casing route with travis-ci, now the script is littered with special cases for travis, a thing we only use for exotics. The best thing to do now, is remove this special casing and move the configuration to calls from travis scripts.

The best thing to do moving forward, imo, is avoid adding more of these special cases, even if they look useful, even when they cover the 95% :)

[iluuu1994]

Works for me, I'll remove it then :)