API: make Day preserve time-of-day across DST transitions

[jbrockmendel]

• closes API: Make 'D' & offsets.Day always operate as calendar day instead of 24 Hour #41943 (Replace xxxx with the GitHub issue number)
• Tests added and passed if fixing a bug or adding a new feature
• All code checks passed.
• Added type annotations to new arguments/methods/functions.
• Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

Updated version of #51874.

I'm trying to find things to split off of this or do as bite-size deprecations. Not much optimism on those fronts. I expect we'll need to do this all-at-once in 3.0 or as a giant deprecation in 2.x with a future option.

update: ideas for bits to split off or deprecate

• Day.__truediv__ and multiplication by non-int could be deprecated, this shouldn't be too painful for users since i doubt anyone actually uses it
• deprecate allowing Timedelta(day_obj). Again should be rare
• deprecate allowing Day objects for freq in TDI constructor

[github-actions]

This pull request is stale because it has been open for thirty days with no activity. Please update and respond to this comment if you're still interested in working on this.

[MarcoGorelli]

I'm trying to find things to split off of this or do as bite-size deprecations. Not much optimism on those fronts. I expect we'll need to do this all-at-once in 3.0 or as a giant deprecation in 2.x with a future option.

all at once sounds fine to be honest, I'd consider it a bug that

In [44]: pd.Series([datetime(2020, 10, 25)]).dt.tz_localize('Europe/London') + pd.tseries.offsets.Day(1)
Out[44]:
0   2020-10-25 23:00:00+00:00
dtype: datetime64[ns, Europe/London]

happens

Please do ping when it's ready for review (looks like there's lots of conflicts atm)

[jbrockmendel]

@jorisvandenbossche can i get you to take a look here pre-merge (which is not imminent)?

[mroeschke]

if self.n == 0 do we still need to show "0D"?

[jbrockmendel]

That’s what it is ATM. What else would you suggest?

[mroeschke]

Ah OK if it's the current behavior I suppose that's fine to keep it

[mroeschke]

I noticed there's quite a few places still where we convert Day to its hour equivalent and allow operations to still continue. IIRC back in the day I was hoping to converting Day to a non-fixed frequency and living with the consequences of Day and "D" having non-fixed frequency semantics throughout other operations (which I would imagine be for things to raise more)

Are the conversions of Day to hours needed for this change to be less breaking?

[jbrockmendel]

Are the conversions of Day to hours needed for this change to be less breaking?

IIRC they were for either a) cases where we went DatetimeIndex->TimedeltaIndex (e.g. subtracting a scalar from DatetimeIndex) or b) to allow users to continue to pass freq="2D" to TimedeltaIndex-like functions. The latter could be disallowed either directly or via deprecation.

I'll take another pass and see if any of these usages can be removed.