Skip to content

BUG date_parser not accurate with Timestamp #2932

@hayd

Description

@hayd

When reading in this DataFrame (with parse_dates=[[0,1]]) we see the datetimes have (29000) microseconds but when using date_parser=pd.Timestamp we lose this information:

from StringIO import StringIO
data = """05/31/2012,15:30:00.029,1306.25,1,E,0,,1306.25
05/31/2012,15:30:00.029,1306.25,8,E,0,,1306.25"""
In [3]: pd.read_csv(StringIO(data), sep=',', header=None, parse_dates=[[0,1]])
Out[3]: 
                          0_1        2  3  4  5   6        7
0  2012-05-31 15:30:00.029000  1306.25  1  E  0 NaN  1306.25
1  2012-05-31 15:30:00.029000  1306.25  8  E  0 NaN  1306.25

In [4]: pd.read_csv(StringIO(data), sep=',', header=None, parse_dates=[[0,1]], date_parser=pd.Timestamp)
Out[4]: 
                   0_1        2  3  4  5   6        7
0  2012-05-31 00:00:00  1306.25  1  E  0 NaN  1306.25
1  2012-05-31 00:00:00  1306.25  8  E  0 NaN  1306.25

Note: This works accurately individually:

In [5]: pd.Timestamp('05/31/2012,15:30:00.029')
Out[5]: <Timestamp: 2012-05-31 15:30:00.029000>

Migrated from StackOverflow.

Metadata

Metadata

Assignees

Labels

BugDatetimeDatetime data dtypeIO DataIO issues that don't fit into a more specific label

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions