Skip to content

Conversation

@manuzhang
Copy link
Member

@manuzhang manuzhang commented Nov 5, 2023

This fixes #8990

@manuzhang
Copy link
Member Author

@yabola @nastra PTAL, thanks!

@manuzhang manuzhang changed the title Spark 3.5: Don't throw exception when decoding dictionary of type INT96 Parquet: don't throw exception on row group filters when reading INT96 column Nov 6, 2023
@manuzhang manuzhang changed the title Parquet: don't throw exception on row group filters when reading INT96 column Parquet: Support reading INT96 column in row group filter Nov 7, 2023
Copy link
Contributor

@nastra nastra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with one suggestion and once CI passes

@nastra
Copy link
Contributor

nastra commented Nov 7, 2023

@manuzhang does that also fix #8949 or is that a different root cause?

@manuzhang
Copy link
Member Author

@nastra #8949 is a different RC.

@manuzhang
Copy link
Member Author

Any more comments?

@nastra
Copy link
Contributor

nastra commented Nov 9, 2023

I think it would be good to also get the opinion of @RussellSpitzer or @aokolnychyi on this one

@manuzhang
Copy link
Member Author

@RussellSpitzer @aokolnychyi could you please take a look?

@manuzhang
Copy link
Member Author

gentle ping @nastra @RussellSpitzer @aokolnychyi

@manuzhang
Copy link
Member Author

I submitted #9408 to fix failed flaky test.

@manuzhang
Copy link
Member Author

@nastra can we move forward with this PR?

@manuzhang manuzhang force-pushed the fix_decode_int96 branch 2 times, most recently from dce3a37 to 12c2e6e Compare January 6, 2024 14:30
@manuzhang manuzhang requested a review from nastra January 6, 2024 15:43
@manuzhang manuzhang force-pushed the fix_decode_int96 branch 3 times, most recently from 2474e37 to cc8b63b Compare January 7, 2024 15:28
@nastra nastra merged commit 4a99678 into apache:main Jan 8, 2024
@manuzhang manuzhang deleted the fix_decode_int96 branch January 8, 2024 11:08
geruh pushed a commit to geruh/iceberg that referenced this pull request Jan 26, 2024
adnanhemani pushed a commit to adnanhemani/iceberg that referenced this pull request Jan 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot decode dictionary of type INT96 when reading imported Spark parquet table

2 participants