TST: Added Airspeed Velocity benchmarks for SphericalVoronoi

[tylerjereddy]

Purpose

This PR adds airspeed velocity benchmarks to the scipy benchmarks suite for scipy.spatial.SphericalVoronoi. The latter class has so far not been covered by performance benchmarks at all after Nikolai and I contributed it. The idea is to prevent performance regressions and also to lower the barrier to performance improvements by providing a convenient platform for benchmark comparison if / when I or others improve that code.

Testing these new benchmarks

Here is how I tested the new benchmarks to check for issues:
At path scipy/benchmarks:

asv run --bench SphericalVor* -e
asv publish
asv preview

This is a very limited test, but allowed me to squash the issues that cropped up. The graphical output looked ok and the text output was:

[ 50.00%] ··· Running spatial.SphericalVor.time_spherical_voronoi_calculation                                                                                                                            ok
[ 50.00%] ···· 
               ============ ==========
                num_points            
               ------------ ----------
                    10       742.22μs 
                   100        2.29ms  
                   1000      45.83ms  
                   5000      596.71ms 
                  10000       2.22s   
               ============ ==========

[100.00%] ··· Running spatial.SphericalVorSort.time_spherical_voronoi_calculation                                                                                                                        ok
[100.00%] ···· 
               ============ ==========
                num_points            
               ------------ ----------
                    10       764.01μs 
                   100       10.36ms  
                   1000      106.75ms 
                   5000      528.48ms 
                  10000       1.12s   
               ============ ==========

Considerations

• It is slightly awkward that I used the same test method name in each of the new classes -- this was to allow inheritance / recycling of code without re-running the method of the base class. Better approach?
• The 10k point test is rather slow compared to the 0.25 s goal time. Ideally, I'd want to go up to even more points so that any future PRs improving the performance of the class could basically just use asv compare to conveniently measure progress between commit hashes over a broad range of values, but I suspect that the time cost would be too great over the span of the project.
• I assume someone else (looks like Pauli here: https://pv.github.io/scipy-bench/) runs the benchmark suite periodically and publishes those online now and then; it certainly seems to take a long time on my laptop to do the full run though

[rgommers]

More that 10k points doesn't seem necessary, because the scaling of the timing with number of points is almost perfectly linear.

I would get rid of the inheritance completely. The code will be shorter without it, and you don't need constraints like using the same time_ method names.

[rgommers]

this teardown method should not be necessary

[rgommers]

and the one below shouldn't be needed either. Did you see a problem without those?

[rgommers]

if it's due to the use of super in the setup method of the subclass, why not make the generation of points on a sphere a small standalone function and call those. stuff like super is best avoided unless really needed.

[rgommers]

pep8 nitpick: no spaces around =

[tylerjereddy]

Regarding the 10k point limit, I think the near-linear behaviour is deceptive. The algorithm is O(n**2), and if we ever improve it to the theoretical limit -- which is the limit of sorting proper -- n * log(n), asv would possibly show that the performance up to 10k points worsens even though the worst-case optimal performance has been achieved.

This is perhaps clarified from some benchmarks I did a while ago (ignore the biology labels), where the best algorithms might be represented by the green loglinear fit line, and current SphericalVoronoi is purple:

Is it not a valid concern that asv might detect a regression for a superior algorithm because the generator count is too low to be sensitive to the true time complexity?

As an aside, I wonder how practical it is to have a separate algorithm for a different number of input generators -- I'm not sure if conventional wisdom is to simply take the performance hit at lower generator counts if you eventually mange to implement the worst-case optimal algorithm.

The other suggestions all make sense.

[rgommers]

Is it not a valid concern that asv might detect a regression for a superior algorithm because the generator count is too low to be sensitive to the true time complexity?

That could happen. It's always possible to fix the benchmark if that does happen. But if you want to include a longer-running test, you can use SCIPY_XSLOW. See in benchmarks/optimize how that's used.

[tylerjereddy]

Ok, I think I've addressed all the comments so far.

I didn't add a larger / slower test, but if the algorithm is substantially improved we can indeed expand to more points if that proves necessary. As it stands, this can still guard against regressions and facilitate demonstration of improvement via i.e., cythonization of the current algorithm.

Note that 60k points causes a hard crash on a machine with 16 GB of RAM when performed in the context of the asv testing suite, and the additional memory footprint is another motivation for avoiding these larger tests with the current algorithm.