MultiIndex smart repr fmt doesn't work well for complex examples

In [42]: labels = [('foo', 1),
    ...:  ('foo', 2),
    ...:  ('bar', 3),
    ...:  ('bar', 4)]
    ...: pd.MultiIndex.from_tuples(labels)
Out[42]: 
MultiIndex
[foo  1,      2, bar  3,      4]

hiding the repeated foo and bar is nice for small multindices, but
for more levels and more complex structures, it can becomes
harder to read. The blanks also keep their width, and the result
isn't very attractive when presented as a single line:

labels = [('foo', '2012-07-26T00:00:00', 'b5c2700'),
    ...:  ('foo', '2012-08-06T00:00:00', '900b2ca'),
    ...:  ('foo', '2012-08-15T00:00:00', '07f1ce0'),
    ...:  ('foo', '2012-09-25T00:00:00', '5c93e83'),
    ...:  ('foo', '2012-09-25T00:00:00', '9345bba')]
    ...: pd.MultiIndex.from_tuples(labels)

MultiIndex
[append_frame_single_homogenous  2012-07-26  b5c2700,                     2012-08-06  00b2ca,                                 2012-08-15  07f1ce0,                                 2012-09-25  5c93e83,                                             9345bba]

[cpcloud]

Why not have it repr as if it were the columns of a DataFrame?

Interesting. Along with a couple of similar changes queued for 0.12 though,
I'd opt for having a repr() output be valid python code to recreate the object. That's
pythonic and useful.

If that sort of view is wanted, it's not difficult to convert the multindex tuples
into a dataframe explicitly.

[cpcloud]

@y-p Oh I meant as if it were the actual column index of a DataFrame, (e.g., a DataFrame with a column MultiIndex only just showing the column index), but I like your suggestion too.

I've done that in the past with DataFrame(np.row_stack(mi), columns=mi.names) where mi is a MultiIndex.

[hayd]

+1 for repr being valid and give back the output to recreate object.

In [1]: pd.Index([1, '2'])
Out[1]: Index([1, 2], dtype=object)

In [2]: pd.Index([1, 2])
Out[2]: Int64Index([1, 2], dtype=int64)

I suppose the possible caveat could be if the index is very long (the ...s are kind of useful?):

In [3]: pd.Index(xrange(100000))
Out[3]: Int64Index([0, 1, 2, ..., 99997, 99998, 99999], dtype=int64)

[jreback]

@jtratner didn't we have a recent issue related to multi-index formatting? can you link it here (and possibly close this one if a dupe)

[cpcloud]

related to #3941

[jtratner]

how about we just change the MultiIndex repr to just look like how it looks with columns?

So if you have a Series with a MultiIndex like this:

C0       R0
C_l0_g0  R_l0_g0    R0C0
         R_l0_g1    R1C0
         R_l0_g2    R2C0
         R_l0_g3    R3C0
         R_l0_g4    R4C0
C_l0_g1  R_l0_g0    R0C1
         R_l0_g1    R1C1
         R_l0_g2    R2C1
         R_l0_g3    R3C1
         R_l0_g4    R4C1
C_l0_g2  R_l0_g0    R0C2
         R_l0_g1    R1C2
         R_l0_g2    R2C2
         R_l0_g3    R3C2
         R_l0_g4    R4C2
C_l0_g3  R_l0_g0    R0C3
         R_l0_g1    R1C3
         R_l0_g2    R2C3
         R_l0_g3    R3C3
         R_l0_g4    R4C3
C_l0_g4  R_l0_g0    R0C4
         R_l0_g1    R1C4
         R_l0_g2    R2C4
         R_l0_g3    R3C4
         R_l0_g4    R4C4
dtype: object

then the MultiIndex repr should just be this (or maybe it's transpose?)

C0       R0
C_l0_g0  R_l0_g0
         R_l0_g1
         R_l0_g2
         R_l0_g3
         R_l0_g4
C_l0_g1  R_l0_g0
         R_l0_g1
         R_l0_g2
         R_l0_g3
         R_l0_g4
C_l0_g2  R_l0_g0
         R_l0_g1
         R_l0_g2
         R_l0_g3
         R_l0_g4
C_l0_g3  R_l0_g0
         R_l0_g1
         R_l0_g2
         R_l0_g3
         R_l0_g4
C_l0_g4  R_l0_g0
         R_l0_g1
         R_l0_g2
         R_l0_g3
         R_l0_g4

dtype: object

Way better.

[hayd]

Can we have a str/unicode as @jtratner suggests, but a repr which is valid python to recreate itself?

[jtratner]

sure, but repr should just be this:

>>> mi = MultiIndex.from_arrays([[1, 1, 1, 1], [1, 3, 5, 7], [9, 9, 1, 1]])
>>> print repr(mi)
MultiIndex(levels=[[1], [1, 3, 5, 7], [1, 9]],
           labels=[[0, 0, 0, 0], [0, 1, 2, 3], [1, 1, 0, 0]],
           names=[None, None, None],
           sortorder=None )

But probably wouldn't show the ones that are all None.

[jtratner]

oh how nice, we can already just use MultiIndex.format to do this. and if it's beyond max rows, add a ...

So if max rows were 10, it'd be:

C0       R0
C_l0_g0  R_l0_g0
         R_l0_g1
         R_l0_g2
         R_l0_g3
         R_l0_g4
C_l0_g1  R_l0_g0
         R_l0_g1
         ...
         R_l0_g4
C_l0_g2  R_l0_g0

Are we going to do something special for HTML?

[jtratner]

Got this working - yay!

In [1]: import pandas as pd

In [2]: pd.MultiIndex.from_arrays([[1, 1, 1, 1], [1, 3, 5, 7], [9, 9, 1, 1]])
Out[2]:
MultiIndex(levels=[[1], [1, 3, 5, 7], [1, 9]]
           labels=[[0, 0, 0, 0], [0, 1, 2, 3], [1, 1, 0, 0]])

In [3]: mi = _

In [4]: mi.names = list('abc')

In [5]: print mi
a  b  c
1  1  9
   3  9
   5  1
   7  1

And with too many rows:

In [10]: pd.set_option('display.max_rows', 15)

In [11]: lst1 = [1] * 3 + [2] * 5 + [3] * 2

In [12]: lst1
Out[12]: [1, 1, 1, 2, 2, 2, 2, 2, 3, 3]

In [13]: lst2 = ['a'] * 6 + ['b'] * 3 + ['c'] * 1

In [14]: lst2
Out[14]: ['a', 'a', 'a', 'a', 'a', 'a', 'b', 'b', 'b', 'c']

In [15]: mi = pd.MultiIndex.from_arrays([lst1 * 10, lst2 * 10, range(100)]); mi
Out[15]:
MultiIndex(levels=[[1, 2, 3], [u'a', u'b', u'c'], [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99]]
           labels=[[0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2], [0, 0, 0, 0, 0, 0, 1, 1, 1, 2, 0, 0, 0, 0, 0, 0, 1, 1, 1, 2, 0, 0, 0, 0, 0, 0, 1, 1, 1, 2, 0, 0, 0, 0, 0, 0, 1, 1, 1, 2, 0, 0, 0, 0, 0, 0, 1, 1, 1, 2, 0, 0, 0, 0, 0, 0, 1, 1, 1, 2, 0, 0, 0, 0, 0, 0, 1, 1, 1, 2, 0, 0, 0, 0, 0, 0, 1, 1, 1, 2, 0, 0, 0, 0, 0, 0, 1, 1, 1, 2, 0, 0, 0, 0, 0, 0, 1, 1, 1, 2], [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99]])

In [16]: mi = _

In [17]: print mi

1  a  0
      1
      2
2  a  3
      4
      5
  ...
2  a  93
      94
      95
   b  96
      97
3  b  98
   c  99

[jtratner]

Not sure how to handle too wide -- wrap? format() doesn't try to sparsify after a column with values everywhere.

In [19]: mi = pd.MultiIndex.from_arrays([lst1 * 10, lst2 * 10, range(100)] * 10)
In [20]: print mi

1  a  0   1  a  0   1  a  0   1  a  0   1  a  0   1  a  0   1  a  0   1  a  0   1  a  0   1  a  0
      1   1  a  1   1  a  1   1  a  1   1  a  1   1  a  1   1  a  1   1  a  1   1  a  1   1  a  1
      2   1  a  2   1  a  2   1  a  2   1  a  2   1  a  2   1  a  2   1  a  2   1  a  2   1  a  2
2  a  3   2  a  3   2  a  3   2  a  3   2  a  3   2  a  3   2  a  3   2  a  3   2  a  3   2  a  3
      4   2  a  4   2  a  4   2  a  4   2  a  4   2  a  4   2  a  4   2  a  4   2  a  4   2  a  4
      5   2  a  5   2  a  5   2  a  5   2  a  5   2  a  5   2  a  5   2  a  5   2  a  5   2  a  5
                                               ...
2  a  93  2  a  93  2  a  93  2  a  93  2  a  93  2  a  93  2  a  93  2  a  93  2  a  93  2  a  93
      94  2  a  94  2  a  94  2  a  94  2  a  94  2  a  94  2  a  94  2  a  94  2  a  94  2  a  94
      95  2  a  95  2  a  95  2  a  95  2  a  95  2  a  95  2  a  95  2  a  95  2  a  95  2  a  95
   b  96  2  b  96  2  b  96  2  b  96  2  b  96  2  b  96  2  b  96  2  b  96  2  b  96  2  b  96
      97  2  b  97  2  b  97  2  b  97  2  b  97  2  b  97  2  b  97  2  b  97  2  b  97  2  b  97
3  b  98  3  b  98  3  b  98  3  b  98  3  b  98  3  b  98  3  b  98  3  b  98  3  b  98  3  b  98
   c  99  3  c  99  3  c  99  3  c  99  3  c  99  3  c  99  3  c  99  3  c  99  3  c  99  3  c  99

[jtratner]

And here's @y-p's more complicated thing:

In [9]: labels = [('foo', '2012-07-26T00:00:00', 'b5c2700'),
   ...: ('foo', '2012-08-06T00:00:00', '900b2ca'),
   ...: ('foo', '2012-08-15T00:00:00', '07f1ce0'),
   ...: ('foo', '2012-09-25T00:00:00', '5c93e83'),
   ...: ('foo', '2012-09-25T00:00:00', '9345bba')]

In [10]: print pd.MultiIndex.from_tuples(labels)

foo  2012-07-26T00:00:00  b5c2700
     2012-08-06T00:00:00  900b2ca
     2012-08-15T00:00:00  07f1ce0
     2012-09-25T00:00:00  5c93e83
                          9345bba