Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

release(v20.11): merge master to bring in latest changes #7076

Merged
merged 39 commits into from
Dec 7, 2020

Conversation

NamanJain8
Copy link
Contributor

@NamanJain8 NamanJain8 commented Dec 7, 2020

This PR brings in the latest master to release/v20.11.
Important ones among them are:

There were a bunch of improvements in debug tool and test scripts. A bunch of doc updates is there.

The following are improvements and fixes from badger.
Performance:

Fixes:


This change is Reviewable

rahulgurnani and others added 30 commits December 2, 2020 12:03
#7018)

* Add test for old restore to a cluster

* Fix author query

* fix(restore): Use correct version after marshalling

The marshalPostingList function returns a KV without the version set and
this causes issues if you restore a backup on top of an existing data.
This PR fixes this issue by explicitly setting the version for the KV
returned by the MarshalPostingList function.

* Fix test for restoring old backup incremental

* Address review comments

* Add old backup dir

* Add wait for restore, address review comments

* Make test use tls, refactor out waitforrestore

Co-authored-by: Ibrahim Jarif <[email protected]>
…5) (#7016)

This PR does the following things:
* adds a counter in `/probe/graphql` response which indicate how many time the GraphQL schema has been updated.
* adds some common test methods to update the GQL schema which ensure schema update using the counter.
* Refactors existing tests to use those methods in order to update the GQL schema.
* adds the best practice of using `common.RequireNoGQLErrors` instead of `require.Nil` for verifying that the `GraphQLResponse` has no errors.
* enables some skipped tests in `graphql/e2e/admin_auth`.

With this PR, all the flakiness around GraphQL schema updates should no longer be there.
The added directory is used in systest/backup/filesystem/docker-compose.yml.
This PR brings in the change of an increasing number of buffers from 32 to 64 in z.Allocator.
Update Badger to bring in fixes to reduce memory usage during snapshot streaming.
dgraph-io/badger@3d225d7...70088c6

- Keep the cheaper parts of the index within table struct. (dgraph-io/badger#1608)
- Fix(OOM): Reuse pb.KVs in Stream (dgraph-io/badger#1609)
…6981)

* Note proper use and meaning of --replicas flag, minor rewording for style
Fix typo in  "Variables" of query: "Example: Add mutation on single type using variables".
…#7051)

This adds a new metric called dgraph_memory_alloc_bytes that
tracks the jemalloc memory reported by z.NumAllocBytes().

Changes

* Add dgraph_memory_alloc_bytes Prometheus metric.
* Add a comment about runtime.ReadMemStats stopping the world.
* rephrase + added expample to query logging

* rephrase + typos
The debug tool was using normal iterators which are very slow. This PR uses 
the stream framework in debug tool instead of the iterators.
…7005)

These metrics should be using LastCount to get the measured metric, not just the
number of counted events.
Add --rollup flag in debug tool which allows key rollup.
* Use z.Allocator again
* Update go.mod
)

Add metrics to track Raft:

* dgraph_pending_proposal_bytes
* dgraph_raft_applych_size
Accessible via /jemalloc at Zero's HTTP port.

http://localhost:6080/jemalloc

    Num Allocated Bytes: 0 B [0]
    Allocators:
    
    NO leaks found.

Also, update to the latest Badger to fix OOMs during compaction.
danielmai and others added 7 commits December 5, 2020 13:34
When retrieving Uids, if we are not doing any filtering or ordering, then we should respect the first: N argument and only retrieve what's being asked for. This really speeds up the query execution.

* We should respect first N argument
* Deal with negative first
… type which are present in interface and also allow to inherit field of same name of type ID from multiple interfaces. (#7053)

Fixes GRAPHQL-672
Currently, we don't allow the repetition of fields in implementing types that are in the interface. But according to GraphQl specs, we should allow them. Not allowing them can also result in unexpected behavior in some scenarios.
For example, in the below example Banana type doesn't have any fields apart from that are in the Fruit interface. And we don't allow empty types. So allowing repetitive fields solve this problem.

interface Fruit {
    id: ID!
    price: Int!
}

type Apple implements Fruit {
    id: ID!
    price: Int!
    color: String!
}

type Banana implements Fruit {
    id: ID!
    price: Int!
}

Also, currently, if we have two interfaces and a type that implements both then we don't allow a field of the same name in both of them. We are allowing it for ID type fields because it's required in some use cases and there is no problem with it as ID type field is not actually a predicate.

interface I1 {
  f1: ID!
}
interface I2 {
 f1: ID!
}
type T implements I1 & I2 {
 f2: Int!
}
@github-actions github-actions bot added area/documentation Documentation related issues. area/graphql Issues related to GraphQL support on Dgraph. area/testing Testing related issues labels Dec 7, 2020
@netlify
Copy link

netlify bot commented Dec 7, 2020

Deploy preview for dgraph-docs ready!

Built with commit 0d85513

https://deploy-preview-7076--dgraph-docs.netlify.app

Copy link
Contributor

@pawanrawal pawanrawal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sure that CI passes and then merge. Changes look fine here.

* remove <X> <uid> <X> type of triplets from the output

* fix output for recurse queries

* fix for recurse queries

* fix test

* add Test for ignoring uid

* minor modification

* fix solution for recurse

(cherry picked from commit 17df3f5)
…onfig (#7013)

Pass the encryption key to `raftwal.InitEncrypted()` instead of directly reading from the workerConfig.
Fix the debug tool and raftwal migrate, to allow using them with encryption enabled.

(cherry picked from commit cf4c796)
@danielmai danielmai merged commit 0d85513 into release/v20.11 Dec 7, 2020
@NamanJain8 NamanJain8 deleted the naman/merge-master-rc3 branch December 7, 2020 19:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/documentation Documentation related issues. area/graphql Issues related to GraphQL support on Dgraph. area/testing Testing related issues
Development

Successfully merging this pull request may close these issues.