Improve Panel slicing speed

[wesm]

from mailing list:

To follow up:

I used timeit.Timer() to time this two different ways:
a) return a slice of a Panel using panel.ix[foo, bar, qux] (where foo,
bar, and qux are arrays)
b) i) iterate through the items of the panel that are in foo
   ii) do a slice on each DataFrame with data.ix[bar, qux]
   iii) save the slice
   iv) reassemble the slices into a new, smaller Panel
   v) return the new Panel

For the data I was using (10 items x 768432 major x 9 minor), the
results I got were:
   t = timeit.Timer(a, setup=setup)
   print min(t.repeat(repeat=3, number=10))
   2.52914

   t = timeit.Timer(b, setup=setup)
   print min(t.repeat(repeat=3, number=10))
   0.54757

This seems odd to me. Approach b is not only faster than a, but almost
5-fold faster. It appears that Panel.ix[foo, bar, qux] is doing
something even slower than the dumb approach embodied in B - either
that or I'm doing something horribly wrong! :-)


Iain

[wesm]

I was able to speed this up by almost 5x:

Test case:

p = Panel(np.random.randn(10, 768432, 9))
a0 = range(8)
a1 = np.random.permutation(768432)[:100000]
a2 = range(6)
p.ix[a0, a1, a2]

Before:

In [2]: timeit p.ix[a0, a1, a2]
1 loops, best of 3: 285 ms per loop

After:

In [2]: timeit p.ix[a0, a1, a2]
10 loops, best of 3: 67.9 ms per loop