Support async iteration with query cursors and aggregation cursors

[vkarpov15]

for/await/of is natively supported in Node.js 10.x, so ideally you should be able to do something like:

for await (const doc of Model.find().cursor({ asyncIterable: true })) {}

or

for await (const doc of Model.find().cursor().asyncIterable()) {}

@lineus any thoughts?

I'm SUPER inexperienced with async iterators, so I had to do some research and test this out (below).

If I understand correctly, for-await-of converts the the synchronous iterable to an async iterable for us. I found this in a blog post but haven't been able to find the explicit language that would guarantee this behavior in the spec yet.

Is there a performance benefit to be gained by explicitly creating the async iterator ourselves?

I tested this out with Node 10.6

test.js

#!/usr/bin/env node --no-warnings
'use strict';

const mongoose = require('mongoose');
mongoose.connect('mongodb://localhost:27017/test', { useNewUrlParser: true });
const conn = mongoose.connection;
const Schema = mongoose.Schema;
var called = 0;
const schema = new Schema({
  name: String
});

const Test = mongoose.model('test', schema);

const tests = [];
for (let i = 0; i < 10000; i++) {
  tests.push({ name: `name${i}`});
}

async function run() {
  await conn.dropDatabase()
  await Test.collection.insertMany(tests)

  for await (const doc of Test.find().cursor()) {
    if (doc.name === 'name5000') {
      console.log(`called: ${called}`);
      process.exit(0)
    } else {
      called++
    }
  }
}

run()

Output:

Node: ./test.js
called: 5000
Node:

heh, it figures that not even 5 minutes after my last comment, I would check my rss client and see this post. I understand now.

It seems like must-add feature to me.

[vkarpov15]

Yeah we should be able to just add a Symbol.asyncIterator function to queries that means you can use for/await/of with Model.find(). Will be very nice to have

[sibelius]

it worked well for, just got this warning

ExperimentalWarning: Readable[Symbol.asyncIterator] is an experimental feature. This feature could change at any time

should we add this to the docs?

[vkarpov15]

@sibelius that's strange, I've never seen that warning before. Can you show me the code you used to get that warning?

[sibelius]

This only happens on node 10, the same code as above in the comments

[vkarpov15]

It works for query cursors, still need to add support for aggregation cursors.