Core: Store split offset for delete files #7011
Conversation
|
Overall this looks fine, but I see CI failures. |
5e4868d to
0d1f29b
Compare
jackye1995
left a comment
There was a problem hiding this comment.
Overall looks good to me, I checked all existing Util classes, did not find a good one to put this method either, so I am good with adding a KryoUtil class as well, unless there are other better suggestions.
szehon-ho
left a comment
There was a problem hiding this comment.
Thanks @singhpk234 for this, it is great, great to see the KryoUtil refactoring as well, just one suggestion for test for consideration.
amogh-jahagirdar
left a comment
There was a problem hiding this comment.
Thanks @singhpk234 this is great, minor comments and a question.
spark/v3.3/spark/src/test/java/org/apache/iceberg/spark/source/TestPositionDeletesTable.java
Show resolved
Hide resolved
|
This looks fine to me overall, but I'd prefer to use |
40ee7ed to
19b35b3
Compare
Ack made the changes |
amogh-jahagirdar
left a comment
There was a problem hiding this comment.
Overall looks good to me @singhpk234 !
19b35b3 to
733a8c1
Compare
733a8c1 to
3fd1781
Compare
szehon-ho
left a comment
There was a problem hiding this comment.
Looks great, thanks @singhpk234
|
Approved for test run. |
|
Thanks @singhpk234 for change, and @rdblue @jackye1995 @amogh-jahagirdar for extra review! |
| private Map<Integer, ByteBuffer> upperBounds = null; | ||
| private ByteBuffer keyMetadata = null; | ||
| private Integer sortOrderId = null; | ||
| private List<Long> splitOffsets = null; |
There was a problem hiding this comment.
@singhpk234 I think we would need to copy/reset this field in clear() and in copy()?
There was a problem hiding this comment.
Agree @nastra, i think it go missed, let me raise a pr for this along with tests
Fixes #6659