Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve error for key value size limits #237

Merged
merged 7 commits into from
Sep 19, 2017

Conversation

peterstace
Copy link

@peterstace peterstace commented Sep 19, 2017

We now have two separate errors, one for keys and one for values. The error will contain the first 1KB of the offending key/value, and show what the limit was.

Also, the max value size is now the same as the value log size. It's not checked when replaying the value log, since the setting could have been different for a previous run of badger.

I increased the limit for the key size from 1MB to 16MB (found some data in the graph overflow dataset that possibly violates the 1MB limit).


This change is Reviewable

@manishrjain
Copy link
Contributor

:lgtm: Address the comment before submitting.


Reviewed 3 of 3 files at r1.
Review status: all files reviewed at latest revision, 1 unresolved discussion, some commit checks failed.


value.go, line 83 at r1 (raw file):

)

const maxKeySize = 16 << 20

I think 1MB is good for keys. In comparison, our tables are only 64MB each. If we have some data in SO which violates this, we should avoid the exact index.


Comments from Reviewable

@peterstace
Copy link
Author

Review status: all files reviewed at latest revision, 1 unresolved discussion, some commit checks failed.


value.go, line 83 at r1 (raw file):

Previously, manishrjain (Manish R Jain) wrote…

I think 1MB is good for keys. In comparison, our tables are only 64MB each. If we have some data in SO which violates this, we should avoid the exact index.

Done. Reverted to use 1MB.


Comments from Reviewable

@peterstace peterstace merged commit b27e3d8 into master Sep 19, 2017
srh pushed a commit that referenced this pull request Sep 21, 2017
@deepakjois deepakjois deleted the improve_error_for_key_value_size_limits branch October 30, 2017 10:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants