CI: Use Cython nightly wheel on free-threaded CI

[andfoy]

Reference issue

See #20669

What does this implement/fix?

Since Cython nightly wheels are now being published in the Scientific Python Anaconda channel, SciPy free-threaded CIs now can use them as opposed to building them in-place, this will reduce overall execution times.

Additional information

See scientific-python/upload-nightly-action#80

[tylerjereddy]

You could also just add NO_CYTHON_COMPILE=true to the start of these lines to accomplish the same thing, if the only objective is to reduce CI time, as noted in the linked discussion.

I don't think the distinction matters much, depends on how closely you want to match the frequency of changes/breaks in Cython source vs. less frequently updated wheels.

Since both free-threaded CI jobs I'm aware of (the wheel one and regular one) both look fine here, it is probably just fine to merge this.

[andfoy]

Well, the upside of compiling Cython in-place is that issues such as #20717 (comment) could be caught earlier, however, I don't know if the frequency of source changes in Cython is something that SciPy wants to keep track after 3.13 comes up and Cython wheels can be installed directly from PyPi

[tylerjereddy]

compiling Cython in-place

That issue was specific to a compiled version of Cython? I'm just saying that the new wheels you propose to use here are not compiled Cython IIUC, just the raw Python source (which is mostly fine), and the purpose of the PR to

reduce overall execution times

could be accomplished by either pulling in the non-compiled wheels or by pulling in the raw source from GitHub and specifying to pip not to compile it, and the only difference is the frequency of change. Maybe using the wheels is less likely to pick up noise.

[rgommers]

I don't think regressions are happing often. I like the idea of using Cython nightly wheels. The trade-off is that fixes in Cython master may take a little longer to materialize. But so far we haven't had any problems in the free-threaded build with Cython, so I think choosing faster install times and a bit more stability makes sense.

[tylerjereddy]

choosing faster install times

Ok, I don't think this one is true if you disable the compile on the pip install, but I think we agree on the rest.

[rgommers]

Thanks @andfoy. This mostly looks good to me, however you forget to update cibw_before_build_linux.sh - could you make the same change there?

[rgommers]

Thanks @andfoy, let's give this a go. And thanks for the review @tylerjereddy.

[rgommers]

As a next step here, now that CPython 1.13.0b2 is out and NumPy has free-threaded macOS wheels, I think you could try adding macOS nightly wheels for SciPy as well?