Unstack performance regression

[TomAugspurger]

http://pandas.pydata.org/speed/pandas/#reshape.SparseIndex.time_unstack

#18460 (comment)

cc @toobaz

[TomAugspurger]

@toobaz do you have time to take a look at this for 0.23?

[toobaz]

I'll try to look at this, and hopefully fix it, next week.

[toobaz]

@TomAugspurger .unstack() needs to find unused levels. The problem is that since #18460, this is done twice for each level, once when building the index, and once when building the values (previously, it was done twice only for the level which was being unstacked). There is room for improving the code, but it's not simple, I won't have time soon.

#20703 recovers the performance drop by just making MultiIndex.remove_unused_levels more performant when there are no unused levels. It does not tackle the problem of checking twice, nor brings any improvement when all levels have unused items. We can leave this bug open if you want to keep a reminder for a more general refactoring.