Skip to content

DEPR fill_method and limit keywords in pct_change #53520

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 16 commits into from
Jun 9, 2023
Merged

DEPR fill_method and limit keywords in pct_change #53520

merged 16 commits into from
Jun 9, 2023

Conversation

Charlie-XIAO
Copy link
Contributor

@Charlie-XIAO Charlie-XIAO requested a review from rhshadrach as a code owner June 4, 2023 16:10
rhshadrach
rhshadrach requested changes Jun 4, 2023
View reviewed changes
Copy link
Member

@rhshadrach rhshadrach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be good if the default for fill_method was None. However, it is currently "pad". When the deprecation is enforced, I think we'll want the behavior to be None instead. This means we first need to deprecate/change the result, and then once that's done we can deprecate the argument.

@Charlie-XIAO
Copy link
Contributor Author

Can these be done in a single PR or in separate PRs?

@rhshadrach
Copy link
Member

I don't think these can be done in a single major version. In 2.x we deprecate the default; in 3.x we can then change the default to None. Finally in 3.x we can also deprecate the arguments and in 4.x we can remove them.

@Charlie-XIAO
Copy link
Contributor Author

Charlie-XIAO commented Jun 5, 2023 •
edited
Loading

@rhshadrach I have updated the PR to deprecate only the default value of fill_method. Not sure if I'm doing correctly, so please let me know if I should make changes.

It also seems that deprecating this behavior is raising warnings in so many tests. I've already fixed some of those, do I need to fix the rest as well (like what I'm currently doing), or is there a better resolution?

@Charlie-XIAO Charlie-XIAO changed the title DEPR fill_method and limit keywords in pct_change DEPR default value of fill_method in pct_change Jun 5, 2023
@jbrockmendel
Copy link
Member

I disagree with @rhshadrach that this needs to be a two-major-version process. This is just not that big of a deal.

@rhshadrach
Copy link
Member

@jbrockmendel - what's your recommended way forward here?

@jbrockmendel
Copy link
Member

  1. warn if method is explicitly passed
  2. otherwise warn if there are NAs present

@rhshadrach
Copy link
Member

Ah - thanks; that sounds great to me.

@Charlie-XIAO
Copy link
Contributor Author

Okay, so in other words, revert to my original version, and in addition warn when has NA but method is not explicit passed.

I will make the changes soon. Thanks for your discussion.

@mroeschke mroeschke added the Deprecate Functionality to remove in pandas label Jun 7, 2023
@Charlie-XIAO Charlie-XIAO changed the title DEPR default value of fill_method in pct_change DEPR fill_method and limit keywords in pct_change Jun 8, 2023
rhshadrach
rhshadrach approved these changes Jun 8, 2023
View reviewed changes
Copy link
Member

@rhshadrach rhshadrach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@rhshadrach rhshadrach added the Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate label Jun 8, 2023
@rhshadrach rhshadrach added the Transformations e.g. cumsum, diff, rank label Jun 8, 2023
@rhshadrach rhshadrach added this to the 2.1 milestone Jun 8, 2023
@mroeschke
Copy link
Member

Could you merge main once more?

@mroeschke mroeschke merged commit 6e3c37c into pandas-dev:main Jun 9, 2023
@mroeschke
Copy link
Member

Thanks @Charlie-XIAO

@Charlie-XIAO Charlie-XIAO deleted the depr-pct-change branch June 28, 2023 05:41
Daquisu pushed a commit to Daquisu/pandas that referenced this pull request Jul 8, 2023
@Charlie-XIAO
DEPR fill_method and limit keywords in pct_change (pandas-dev#5…
0b86e3e 
…3520)

* DEPR fill_method and limit keywords in DataFrame/Series pct_change

* DEPR fill_method and limit keywords in GroupBy pct_change

* changelog added

* DataFrame/Series pct_change only deprecate default

* typo

* GroupBy pct_change only deprecate default

* changelod updated correspondingly

* reverting

* Revert "reverting"

This reverts commit 2cb449b.

* resolved conversation: also warn if nan and not specified

* modify msgs since fillna method deprecated; tests updated

* changelog updated

* docstring use ffill instead of fillna
@holymonson
Copy link
Contributor

@rhshadrach @jbrockmendel Hi, we need fill_method='None' to force pct_change not to handle NAs, but this PR cause a lot of deprecated warning and we can't fix it. Before you provide an alternative to do that, deprecatingfill_method is unwise.

@Charlie-XIAO
Copy link
Contributor Author

Does obj.fillna(...).pct_change not work, as is also mentioned in the deprecation message? @holymonson

@holymonson
Copy link
Contributor

Does obj.fillna(...).pct_change not work, as is also mentioned in the deprecation message? @holymonson

We don't want fillna() at all, on the contrary we want keeping NAs, while your deprecation message doesn't tell howto NOT fillna.

@rhshadrach
Copy link
Member

@holymonson - I've responded in #53491

@rhshadrach rhshadrach mentioned this pull request Mar 6, 2024
Merged
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mroeschke mroeschke mroeschke approved these changes

@rhshadrach rhshadrach rhshadrach approved these changes

Assignees
No one assigned
Labels
Deprecate Functionality to remove in pandas Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate Transformations e.g. cumsum, diff, rank
Projects
None yet
Milestone
2.1
Development

Successfully merging this pull request may close these issues.

DEPR: pct_change method/limit keyword
5 participants
@Charlie-XIAO @rhshadrach @jbrockmendel @mroeschke @holymonson