Cannot pass custom config / objects to steps

[neekfenwick]

I've tried using mongration but found I wanted my steps to be able to use the winston logger module. When doing things programatically, where you initialise your migration from our own top level .js file, you provide mongration with where to find your steps, then it instantiates and runs them completely out of your control.

I think other users may have all sorts of other similar requirements to pass in custom objects or config to their steps, that were constructed in the main .js file. In my case, this looked like this:

'use strict';

var path = require('path'),
    Migration = require('mongration').Migration,
    winston = require('winston');

var config = {
    hosts: 'localhost',
    db: 'WIN',
    migrationCollection: 'ratesMigration',
    stepConfig: {  // <-- this stepConfig argument is new
        log: winston
    }
}, log = winston;

winston.level = 'debug';

var migration = new Migration(config);

migration.addAllFromPath(path.join(__dirname, './migration-steps/'));

migration.migrate(function(err, results){
    log.error(err, results);
});

My Step wants to use it like this:

up: function (db, cb) {
    this.config.log.debug('up has started.');
}

The winston logger instantiated in the top level code may have all sorts of appenders added (file, database, Amazon SNS, whatever) and the steps should each log to the same log stream. Other users requirements may vary.

[wmartins]

Hey @neekfenwick, really nice work done here! I'm discussing it with @eberhara to see what he thinks about that.

Just to understand a simple thing here, do you really need to use the same log object (and by the same I mean, they are really the same, even the same reference) inside the migration? Can't you just require('winston') inside your migration files and use this? Or, if you really need that to be the same object (or at least an object with same configuration), you can require('./config/logger') or something like that, and this will ensure you're using the same configuration (this will ensure the same log level, for example).

I'm just proposing another solution because adding this stepConfig will make us handle two extra variables - maybe you can rename them to be the same -, the stepConfig and config. And we should write some tests to it. Also, if is this a thing that you really can't extend without modifying codebase, maybe mongration should have an option to be extensible, and then we can accept your PR! :)

What do you think @eberhara?

[neekfenwick]

Thank you @wmartins .. I might have missed a trick here, you are right. Since require('winston') ought to return the same module object, regardless of how many times or from where it's called, I could do that in order to get the top level logger created by that module. The way winston happens to work, it always gives you a top level Logger, and in my case I'm happy to use that. Some people might have a requirement to create specific loggers for various purposes (although with winston they might be misusing it where creating custom levels of logging with specific transports per level would achieve their aim...)

About the two variables you mention, I think they're actually the same. I passed stepConfig to the mongration constructor, because at that level the concept of 'step' needs to be specified. Just calling it config would be misleading. However once it's apssed into a step, I store it as config on the step. It's exactly the same object that was passed as stepConfig to mongration.

For my case, I think I could live with require('winston') and rely on the require mechanism but for other people I think the ability to pass custom config into the step objects could be useful, might be something driving away some trying to use the library.

If you're going to accept, I do recommend we change a couple of things, as I pointed out.. I just didn't want to spend time polishing or changing it without some feedback from the people closer to the project.

[neekfenwick]

I started to rework the code a little making a few changes:

• second argument to Migration constructor is 'params' and contains optional 'stepConfig' object.
• 'stepConfig' is passed into steps by step-file-reader as 'config' and held on each Step instance as 'config'.

I ran into trouble trying to add to the test, I had to read up on describe and mocha, it seems to run the test you just run: npm test

But although I find the 1.js test is loaded, its up() function doesn't seem to be invoked, despite the output looking good. I put a 'throw' statement in 1.js up() and the output was still:

> mongration@0.0.14 test /home/neek/src/mongration
> mocha --recursive


  Run migrations
    ✓ Must not have errors


  1 passing (72ms)

Is the current test framework reliable, and am I running it the right way?

[eberhara]

Hey @neekfenwick , how you doing?
The current tests are pretty simple, but I believe you could rely on them.

Anyway... Let me try to figure where we are at now, @wmartins and @neekfenwick:
We are no longer trying to solve neek's logging problem, however we are going to allow developers to pass a custom object (as a prop of config) - so developers would have access to that same custom object during each migration step. Is that it? :)

[neekfenwick]

Correct @eberhara , I have pushed my latest changes to neekfenwick/mongration on branch step-config, including a basic change to the single test that's in there. I was hoping this first attempt would fail, then I would correct the '123' to '1234' and see the test pass. However I cannot make the test fail with a throw() statement. Indeed, even my console.log call in up() doesn't seem to output anything. I'm not very familiar with npm test or mocha so am not sure how to run the code in a debugger. Hoping you guys can fix it from here or point out what I'm doing wrong.