Groupby all levels except the specified ones

[harisbal]

Inspired by

http://stackoverflow.com/questions/25606478/grouping-by-everything-except-for-one-index-column-in-pandas

I was thinking if it would be useful to add a parameter to the groupby function (e.g. allbut=True) so that pandas will group all levels except the specified ones) .

I suggest so because in a multilevel index with several levels, eliminating just one level requires a quite verbose approach

[jreback]

Actually this would be quite trivial to implement. We currently support a union (set ops) for FrozenList, e.g. df.index.names + ['foo'], so supporting difference is easy.

This might break other things, so need some tests. Also would be nice to support intersection, & as well.

In [3]: df.groupby(level=df.index.names-['name']).mean()
Out[3]: 
                 accuracy  velocity
condition trial                    
a         1      0.661597  0.277299
          2      0.953187 -0.875767
b         1     -0.836886  0.944629
          2     -0.251443  0.799134

In [4]: quit()
(pandas) bash-3.2$ git diff
diff --git a/pandas/core/base.py b/pandas/core/base.py
index 92ec6bb..b8a4c1d 100644
--- a/pandas/core/base.py
+++ b/pandas/core/base.py
@@ -711,6 +711,12 @@ class FrozenList(PandasObject, list):
 
     __iadd__ = __add__
 
+    def __sub__(self, other):
+        if isinstance(other, tuple):
+            other = list(other)
+        return self.__class__([ e for e in self if e not in set(other)])
+
+
     # Python 2 compat
     def __getslice__(self, i, j):
         return self.__class__(super(FrozenList, self).__getslice__(i, j))

[harisbal]

Great idea. Operations on Frozenlists would be really helpful indeed.
Thanks for the quick reply

[jreback]

going to merge this soon: #15477

should provide a bit more runway for expanding / testing these structures.