BUG: upcasting on reshaping ops

[jennolsen84]

Code Sample, a copy-pastable example if possible

import pandas as pd
import numpy as np

pn1 = pd.Panel(np.array([1.0], dtype=np.float32, ndmin=3)) # panel of 1.0
pn2 = pd.Panel(np.array([np.nan], dtype=np.float32, ndmin=3)) # panel of nan

print(pd.concat([pn1, pn1]).values.dtype)
print(pd.concat([pn2, pn2]).values.dtype)

Current Output

float32
float64

Expected Output

float32
float32

output of pd.show_versions()

INSTALLED VERSIONS
------------------
commit: None
python: 3.5.1.final.0
python-bits: 64
OS: Darwin
OS-release: 14.5.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

pandas: 0.18.1
nose: 1.3.7
pip: 8.1.0
setuptools: 20.2.2
Cython: 0.23.4
numpy: 1.11.0
scipy: 0.17.0
statsmodels: None
xarray: None
IPython: 4.0.1
sphinx: 1.3.1
patsy: None
dateutil: 2.4.2
pytz: 2015.7
blosc: None
bottleneck: 1.1.0dev
tables: 3.2.2
numexpr: 2.5.1
matplotlib: 1.5.1
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
httplib2: 0.9.2
apiclient: None
sqlalchemy: 1.0.12
pymysql: 0.6.7.None
psycopg2: 2.6.1 (dt dec pq3 ext)
jinja2: 2.8
boto: None
pandas_datareader: None

[sinhrks]

Thanks for the report. Though Panel is being replaced with xarray near future, sending a PR is appreciated.

[jennolsen84]

@sinhrks same behavior with DataFrame, is that going away too?

[jennolsen84]

In [7]: pn1 = pd.DataFrame(np.array([1.0], dtype=np.float32, ndmin=2)) # df of 1.0

In [8]: pn2 = pd.DataFrame(np.array([np.nan], dtype=np.float32, ndmin=2)) # df of nan

In [9]: print(pd.concat([pn2, pn2]).values.dtype)
float64

In [10]: print(pd.concat([pn1, pn1]).values.dtype)
float32

[jreback]

yeah, I suspect the null forces immediate upcast to float64. This should be cognizant that floats are available in their native itemsize even w/NaN's (and so should take a size that can deal).

I don't think this is very hard to fix in a general way.

care to take a stab?

[jennolsen84]

@jreback how's this for starters? jennolsen84@3b3797a . I can add the tests, whatsnew, etc. if it looks good to you.

I did some tests and it seems to work for me for float types. I did a quick check with ints, and that seemed to work just fine as well (I am guessing it just uses np rules there).

I also checked concating int32 + float16, and in that case we use float64. In that case, one might want a float64, so I left that alone.