Use data argument for scatter plotting timestamps from pandas

[ImportanceOfBeingErnest]

Bug report

Bug summary

This may be either a bug report or a feature request, depending how you view things. The issue is that you cannot use the data argument to scatter to plot a pandas dataframe Timestamp column.

Code for reproduction

Consider the following data

import matplotlib.pyplot as plt
import pandas as pd
import numpy as np; np.random.seed(42)

df = pd.DataFrame(np.random.rand(10,2), columns=["x","y"])
df["time"] = pd.date_range("2018-01-01", periods=10, freq="D")

Using the data argument, you may plot

plt.plot("x", "y", data=df)

You may equally plot

plt.plot("time", "y", data=df)

You may also scatter

plt.scatter(x="x", y="y", data=df)

However, you may not scatter the timestamps

plt.scatter(x="time", y="y", data=df)

This fails with a TypeError: invalid type promotion

Traceback (most recent call last):
  File "D:/.../test/GI_scatterdataarg.py", line 40, in <module>
    plt.scatter(x="time", y="y", data=df)
  File "d:\...\matplotlib\lib\matplotlib\pyplot.py", line 2623, in scatter
    verts=verts, edgecolors=edgecolors, data=data, **kwargs)
  File "d:\...\matplotlib\lib\matplotlib\__init__.py", line 1775, in inner
    return func(ax, *args, **kwargs)
  File "d:\...\matplotlib\lib\matplotlib\axes\_axes.py", line 3982, in scatter
    offsets = np.column_stack([x, y])
  File "D:\...\matplotlib\lib\site-packages\numpy\lib\shape_base.py", line 369, in column_stack
    return _nx.concatenate(arrays, 1)
TypeError: invalid type promotion

This is a bit surprising since you may well plot timestamps through scatter

plt.scatter(x=df.time.values, y=df.x.values)

just not through the data argument.

Expected outcome

Given that one may plot the timestamps correctly with plot and timestamps with scatter by providing the values, it would be nice to be able to use the data argument with scatter as well.

... and ideally with bar, fill_between etc. ;-)

Matplotlib version

• Operating system: Win 8.1
• Matplotlib version: master 2.2.2.post1270+g6665bc739
• Matplotlib backend: Qt5Agg
• Python version: 3.6
• Jupyter version (if applicable):
• Other libraries: Pandas 0.22.0

[Sdoof]

Fully agree, some consitency would be nice

[jklymak]

This seems to be a bug in that the data isn’t being converted to the right form early enough.

[jklymak]

I actually think this is likely downstream in pandas. The units converter called is pandas.plotting._converter.DatetimeConverter, and it doesn't convert to matplotlib datenums. If you call ax.scatter(x=df.time.values, y=df.x.values) then it does convert the values to matplotlib datenums.

I'm going to close as needing a downstream fix, but I may be just passing the buck and not understanding something.

[ImportanceOfBeingErnest]

Yep, I guess the problem is more that ax.scatter(x=df.time, y=df.x) doesn't work either.

[tacaswell]

What is the type that is failing to be promoted?

I suspect that this is going to boil down to a datetime64 issue...

[jklymak]

Again, if you call ax.scatter(x=df.time, y=df.x) you get the Pandas datetime converter: pandas.plotting._converter.DatetimeConverter.

As far as I can tell, the DatetimeConverter doesn't do any conversion on the Pandas series object:

Adding the following print statements to convert_units in axis.py

        ret = self.converter.convert(x, self.units, self)
        print(self.converter)
        print('x', x)
        print('\n x type', type(x))
        print('\nret', ret)
        print('\nret type', type(ret))

<pandas.plotting._converter.DatetimeConverter object at 0x1147f7d68>
x 0   2018-01-01
1   2018-01-02
2   2018-01-03
3   2018-01-04
4   2018-01-05
5   2018-01-06
6   2018-01-07
7   2018-01-08
8   2018-01-09
9   2018-01-10
Name: time, dtype: datetime64[ns]

 x type <class 'pandas.core.series.Series'>


ret 0   2018-01-01
1   2018-01-02
2   2018-01-03
3   2018-01-04
4   2018-01-05
5   2018-01-06
6   2018-01-07
7   2018-01-08
8   2018-01-09
9   2018-01-10
Name: time, dtype: datetime64[ns]

ret type <class 'pandas.core.series.Series'>

As you can see, the Pandas unit conversion doesn't do anything to the series.