Shifting a datetime column with timezone after groupby loses the timezone.

[borispf]

Code Sample, a copy-pastable example if possible

weeks = pd.Series(pd.date_range('2018-01', '2018-02', freq='7D', tz='America/New_York'))
week_ago = weeks.groupby([1, 1, 1, 1, 1]).shift()
print(week_ago)

Outputs:

0                   NaT
1   2018-01-01 05:00:00
2   2018-01-08 05:00:00
3   2018-01-15 05:00:00
4   2018-01-22 05:00:00
dtype: datetime64[ns]

Problem description

Shifting a groupby'd datetime column removes the timezone. It's not mentioned in the documentation so either the timezone should be preserved or the documentation should mention that timezone is removed (by converting to UTC it seems).

I had a quick look and couldn't find another issue for this. I glanced at the GroupBy shift implementation and my guess is that it's due to cython/numpy not supporting timezones.

I'm happy to work on a fix if someone says what the preferred solution is.

Expected Output

0                         NaT
1   2018-01-01 00:00:00-05:00
2   2018-01-08 00:00:00-05:00
3   2018-01-15 00:00:00-05:00
4   2018-01-22 00:00:00-05:00
dtype: datetime64[ns, America/New_York]

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.3.final.0
python-bits: 64
OS: Linux
OS-release: 3.10.0-693.el7.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: en_US.UTF-8
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.21.0
pytest: 3.4.1
pip: 9.0.1
setuptools: 38.5.1
Cython: None
numpy: 1.13.3
scipy: 1.0.0
pyarrow: 0.8.0
xarray: None
IPython: 6.2.1
sphinx: None
patsy: 0.5.0
dateutil: 2.6.1
pytz: 2018.3
blosc: None
bottleneck: 1.2.1
tables: None
numexpr: 2.6.4
feather: 0.4.0
matplotlib: 2.1.2
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: 0.9999999
sqlalchemy: 1.2.1
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: 0.1.4
pandas_gbq: None
pandas_datareader: None

[jreback]

duplicate of #15884

[akdor1154]

@jreback the linked issue is closed as fixed but shift still has this behaviour (only first and last were fixed), can this be reopened?

[TomAugspurger]

Thanks @akdor1154.