-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Bump parquet from 1.16.0 to 1.17.0 #17504
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR upgrades the Parquet library from version 1.16.0 to 1.17.0 and addresses a breaking change in how INT96 fields are identified. The upgrade was necessary due to a bug fix in parquet-java (PR #3311) that changed the field name representation for INT96 types.
Changes:
- Bumped parquet version from 1.16.0 to 1.17.0 in the root POM
- Updated INT96 field identification logic to check fixed-size byte arrays with documentation instead of relying on field names
- Improved code consistency by removing unnecessary fully qualified class names and adding defensive assertions
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| pom.xml | Updates parquet dependency version to 1.17.0 |
| ParquetAvroRecordExtractor.java | Refactors INT96 detection logic to use fixed-size and documentation checks instead of field name comparison, removes unused import, and simplifies Schema type references |
...et/src/main/java/org/apache/pinot/plugin/inputformat/parquet/ParquetAvroRecordExtractor.java
Show resolved
Hide resolved
...et/src/main/java/org/apache/pinot/plugin/inputformat/parquet/ParquetAvroRecordExtractor.java
Show resolved
Hide resolved
...et/src/main/java/org/apache/pinot/plugin/inputformat/parquet/ParquetAvroRecordExtractor.java
Show resolved
Hide resolved
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #17504 +/- ##
=========================================
Coverage 63.25% 63.26%
+ Complexity 1477 1476 -1
=========================================
Files 3167 3167
Lines 189175 189177 +2
Branches 28948 28950 +2
=========================================
+ Hits 119661 119678 +17
+ Misses 60235 60202 -33
- Partials 9279 9297 +18
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Fix the way to identify the
INT96fieldThe old way breaks because of this bug fix: apache/parquet-java#3311
The name of the field is no longer
"INT96"