Skip to content

PERF: BaseMaskedArray.__iter__ #49851

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Nov 23, 2022
Merged

PERF: BaseMaskedArray.__iter__ #49851

merged 4 commits into from
Nov 23, 2022

Conversation

lukemanley
Copy link
Member

Faster iteration through EA's backed by BaseMaskedArray.

ASV added for iterating through a Series backed by various dtypes:

       before           after         ratio
     [b6736b44]       [b9193932]
                      <masked-array-iter>
-      16.8±0.9ms      5.14±0.06ms     0.31  series_methods.Iter.time_iter('Float64')
-        17.2±1ms       5.15±0.1ms     0.30  series_methods.Iter.time_iter('Int64')
-        16.8±1ms      3.50±0.02ms     0.21  series_methods.Iter.time_iter('boolean')

@lukemanley lukemanley added Performance Memory or execution speed performance ExtensionArray Extending pandas with custom dtypes or arrays. labels Nov 23, 2022
jbrockmendel
jbrockmendel reviewed Nov 23, 2022
View reviewed changes
mroeschke
mroeschke reviewed Nov 23, 2022
View reviewed changes
@mroeschke mroeschke added this to the 2.0 milestone Nov 23, 2022
mroeschke
mroeschke approved these changes Nov 23, 2022
View reviewed changes
Copy link
Member

@mroeschke mroeschke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM cc @jbrockmendel merge when ready

@jbrockmendel jbrockmendel merged commit 0a4440d into pandas-dev:main Nov 23, 2022
@jbrockmendel
Copy link
Member

thanks @lukemanley

mliu08 pushed a commit to mliu08/pandas that referenced this pull request Nov 27, 2022
@lukemanley @mliu08
PERF: BaseMaskedArray.__iter__ (pandas-dev#49851)
c0b2e8a 
* BaseMaskedArray.__iter__ perf

* fix

* gh ref

* clarify whatsnew
@lukemanley lukemanley deleted the masked-array-iter branch December 20, 2022 00:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jbrockmendel jbrockmendel jbrockmendel left review comments

@mroeschke mroeschke mroeschke approved these changes

Assignees
No one assigned
Labels
ExtensionArray Extending pandas with custom dtypes or arrays. Performance Memory or execution speed performance
Projects
None yet
Milestone
2.0
Development

Successfully merging this pull request may close these issues.
3 participants
@lukemanley @jbrockmendel @mroeschke