Skip to content

pd.Series.sum() returns nan for Int64 with pd.NA #30958

Closed
@koizumihiroo

Description

@koizumihiroo

Code Sample, a copy-pastable example if possible

I'm not sure this is a bug or intended behavior. If intended, I'm glad to be pointed to how API design was discussed and decided.

On 1.0.0 rc0, pd.Series.sum()returns nan when using skipna=False for 'Int64' with pd.NA.

>>> pd.Series([1, 2, pd.NA]).sum(skipna=False)
<NA>
>>> pd.Series([1, 2, pd.NA], dtype='Int64').sum(skipna=False)
nan

>>> pd.Series([1, 2, np.nan]).sum(skipna=False)
nan
>>> pd.Series([1, 2, np.nan], dtype='Int64').sum(skipna=False)
nan

>>> pd.Series([1, 2, None]).sum(skipna=False)
nan
>>> pd.Series([1, 2, None], dtype='Int64').sum(skipna=False)
nan

Problem description

I understand the second and third example returns nan for backward compatibility, but the first example of dtype='Int64' seems natural to return pd.NA.

Expected Output

>>> pd.Series([1, 2, pd.NA], dtype='Int64').sum(skipna=False)
<NA>

# When breaking backward compatibility is allowed, this is also expected (on and after pandas 2.0.0?)
>>> pd.Series([1, 2, np.nan], dtype='Int64').sum(skipna=False)
<NA>
>>> pd.Series([1, 2, None], dtype='Int64').sum(skipna=False)
<NA>

Output of pd.show_versions()

>>> pd.show_versions()

INSTALLED VERSIONS

commit : None
python : 3.7.6.final.0
python-bits : 64
OS : Linux
OS-release : 4.9.184-linuxkit
machine : x86_64
processor :
byteorder : little
LC_ALL : None
LANG : C.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.0.0rc0
numpy : 1.18.1
pytz : 2019.3
dateutil : 2.8.1
pip : 19.3.1
setuptools : 44.0.0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : None
matplotlib : 3.1.2
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
pytest : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
xlsxwriter : None
numba : None

# Dockerfile
FROM python:3.7.6
WORKDIR /home
RUN pip install pandas==1.0.0rc0
CMD ["/bin/bash"]

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugExtensionArrayExtending pandas with custom dtypes or arrays.Missing-datanp.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions