-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rework Block and Index cache #1473
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Most of the changes are refactoring.
Reviewed 6 of 12 files at r1, 6 of 6 files at r2.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @jarifibrahim and @manishrjain)
table/table.go, line 116 at r2 (raw file):
blockIndex []*pb.BlockOffset ref int32 // For file garbage collection. Atomic. bf *z.Bloom // Nil if BfCache is set.
Undo this.
Co-authored-by: Naman Jain <[email protected]>
Co-authored-by: Naman Jain <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 2 of 2 files at r3.
Reviewable status: 11 of 12 files reviewed, 5 unresolved discussions (waiting on @ashish-goswami, @jarifibrahim, and @manishrjain)
…o/badger into ibrahim/cache-changes-1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 5 of 12 files at r1, 4 of 6 files at r2, 1 of 2 files at r3, 1 of 1 files at r4, 1 of 1 files at r5.
Reviewable status: all files reviewed, 5 unresolved discussions (waiting on @jarifibrahim, @manishrjain, and @NamanJain8)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: all files reviewed, 5 unresolved discussions (waiting on @jarifibrahim, @manishrjain, and @NamanJain8)
Review comments addressed
This PR separates the block cache and the index cache. Earlier we had a mix of both the caches. Block Cache -> Stores block. Should be used when running badger with compression/encryption Index Cache -> Used to limit the memory used by table indices and bloom filters. This PR also remove the `KeepBlocksInCache` and `KeepBlockIndicesInCache` options. (cherry picked from commit c3b1582)
This PR separates the block cache and the index cache. Earlier we had a mix of both the caches. Block Cache -> Stores block. Should be used when running badger with compression/encryption Index Cache -> Used to limit the memory used by table indices and bloom filters. This PR also remove the `KeepBlocksInCache` and `KeepBlockIndicesInCache` options. (cherry picked from commit c3b1582)
This PR separates the block cache and the index cache. Earlier we had a mix of both the caches. Block Cache -> Stores blocks. Should be used when running badger with compression/encryption Index Cache -> Used to limit the memory used by table indices and bloom filters. This PR also remove the `KeepBlocksInCache` and `KeepBlockIndicesInCache` options. (cherry picked from commit c3b1582)
This PR separates the block cache and the index cache. Earlier we had a mix of both the caches. Block Cache -> Stores block. Should be used when running badger with compression/encryption Index Cache -> Used to limit the memory used by table indices and bloom filters. This PR also remove the `KeepBlocksInCache` and `KeepBlockIndicesInCache` options.
This PR separates the block cache and the index cache. Earlier we had a mix of both the caches.
Block Cache -> Stores block. Should be used when running badger with compression/encryption
Index Cache -> Used to limit the memory used by table indices and bloom filters.
This PR also remove the
KeepBlocksInCache
andKeepBlockIndicesInCache
options.This change is