Add pipeline visualizer?

[amueller]

Working with more complex pipelines and ColumnTransformer it can be a bit hard to visualize what's going on.
I think it would be nice to have a way to visualize the estimator graph.
Unfortunately this will be tricky without using graphviz.
With graphviz, it's pretty easy though.

on the column transformer example this produces:

or (with graph_attr={'rankdir': 'LR'}):

I'll put this into dabl for sure, but might also be interesting for sklearn, I think.
This is obviously just a hack and we could make this much nicer, for example with graphviz records. But even better would be avoiding graphviz.

def pipe_to_viz(est, name_from_parent=None, parent=None, **kwargs):
    # graph is not actually used in the singleton case
    # unless there's no parent (and it's not actually a graph)
    if name_from_parent is None:
        graph = Digraph("Some Pipeline", **kwargs)
    else:
        graph = Digraph("cluster " + name_from_parent, **kwargs)
        graph.attr(color='grey')
        graph.attr(label=name_from_parent)
    graph.attr(compound='true')

    if isinstance(est, Pipeline):
        prev_step = None
        prev_sub = None
        for step in est.steps:
            sub, node = pipe_to_viz(step[1], name_from_parent=step[0], parent=graph)
            graph.subgraph(sub)
            if prev_step is not None:
                ltail = getattr(prev_sub, 'name', None)
                lhead = getattr(sub, 'name', None)
                graph.edge(prev_step, node, ltail=ltail, lhead=lhead)


            prev_step = node
            prev_sub = sub
        return graph, node
    elif isinstance(est, ColumnTransformer):
        graph.node("out", shape="point")
        for trans in est.transformers:
            sub, node = pipe_to_viz(trans[1], name_from_parent=trans[0], parent=graph)
            graph.subgraph(sub)
            ltail = getattr(sub, 'name', None)
            graph.edge(node, 'out', ltail=ltail)
        return graph, "out"
    else:
        label = est.__class__.__name__
        node = label + "_" + str(np.random.randint(10000))
        if parent is None:
            graph.node(node, label=label, shape='box')
            return graph, node
        # we could draw boxes around each estimator with the name
        # or we could use the name in  the label? hm
        parent.node(node, label=label, shape='box')
        return None, node

[jnothman]

I like this. I have considered such visualisations over the years and think it is a helpful way to demonstrate pipeline flows. With ColumnTransformer it would be more important to display the column selection than the name in some cases. FeatureUnions should also be shown. I think I had previously expected such a visualisation to show (changed) parameters other than steps, or at least other metaestimator structures. Parameters are hard because graph is gets ugly with lots of text. In the world of notebooks and tensorflow, this should be considered an alternative to repr display, but it's hard to make it as informative.

[mitar]

cc @dmartinez05

[amueller]

This was a half-hour hack, yes, meta-estimators and feature-union are missing but easy to add.
With the "changed only" it might be possible to have the parameters look reasonable.
I was also wondering if the names of the steps should be shown (also easy).

Maybe we need a designer for this lol?

An alternative to graphviz would be using JS/d3 if we mostly target Jupyter. probably don't want that inside sklearn but could be nice?

[dmartinez05]

Is there any reason for trying to avoid graphviz?

[amueller]

[amueller]

@dmartinez05 it's hell to install, in particular without conda. Even in conda, they do evil tricks with a binary because they can't actually compile it.

[dmartinez05]

Yeah, I agree, I implemented something like this (even more basic) for D3M, and there was a lot of challenges even using docker images.

[mitar]

Yea, probably a HTML/JS output would be easier to make. I have seen some profilers do that.

[amueller]

yeah I think html + css might be enough. Without javascript it's easier to embed in a notebook.
I feel like I'm not good enough at html + css and UX/UI design to actually prototype this. I tried and it looked terrible lol.

[thomasjpfan]

Here is a 20 minute hacky version in HTML and CSS, just to see if its possible: https://codepen.io/anon/pen/agONqE

[TimothyWillard]

What about a plain stdout version of this? Something similar to Kera's print_summary. Might be useful when using python scripts instead of notebooks for longer running models or models that need to be run remotely but still want a detailed log of the model.

[thomasjpfan]

In visualization from PR #14180, can be demoed here: https://thomasjpfan.github.io/sklearn_viz_html/index.html

[benjamin-ny]

I've also worked on this in the past, though it is a little hacky:

https://nbviewer.jupyter.org/github/ottogroup/dstoolbox/blob/master/notebooks/Examples_visualization.ipynb

I believe it's helpful to show the name of the step, e.g. to better spot which hyper-parameters to tune.