AssertionError: Tree consistency failed in TSNE

[yanshuaicao]

Description

TSNE learning on digit dataset throws error for perplexity=8

Steps/Code to Reproduce

from sklearn import manifold, datasets
data = datasets.load_digits()
X = data['data']
n_components = 2
tsne = manifold.TSNE(n_components=n_components, init='pca', perplexity=8, 
                         random_state=0, verbose=0)
Y = tsne.fit_transform(X)

Expected Results

No error.

Actual Results

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-9-553b1868b3c8> in <module>()
     11 tsne = manifold.TSNE(n_components=n_components, init='pca', perplexity=8, 
     12                          random_state=0, verbose=0)
---> 13 Y = tsne.fit_transform(X)

/home/owner/py27_env/local/lib/python2.7/site-packages/sklearn/manifold/t_sne.pyc in fit_transform(self, X, y)
    895             Embedding of the training data in low-dimensional space.
    896         """
--> 897         embedding = self._fit(X)
    898         self.embedding_ = embedding
    899         return self.embedding_

/home/owner/py27_env/local/lib/python2.7/site-packages/sklearn/manifold/t_sne.pyc in _fit(self, X, skip_num_points)
    792                           X_embedded=X_embedded,
    793                           neighbors=neighbors_nn,
--> 794                           skip_num_points=skip_num_points)
    795 
    796     @property

/home/owner/py27_env/local/lib/python2.7/site-packages/sklearn/manifold/t_sne.pyc in _tsne(self, P, degrees_of_freedom, n_samples, random_state, X_embedded, neighbors, skip_num_points)
    868         opt_args['it'] = it + 1
    869         params, kl_divergence, it = _gradient_descent(obj_func, params,
--> 870                                                       **opt_args)
    871 
    872         if self.verbose:

/home/owner/py27_env/local/lib/python2.7/site-packages/sklearn/manifold/t_sne.pyc in _gradient_descent(objective, p0, it, n_iter, objective_error, n_iter_check, n_iter_without_progress, momentum, learning_rate, min_gain, min_grad_norm, min_error_diff, verbose, args, kwargs)
    386 
    387     for i in range(it, n_iter):
--> 388         new_error, grad = objective(p, *args, **kwargs)
    389         grad_norm = linalg.norm(grad)
    390 

/home/owner/py27_env/local/lib/python2.7/site-packages/sklearn/manifold/t_sne.pyc in _kl_divergence_bh(params, P, neighbors, degrees_of_freedom, n_samples, n_components, angle, skip_num_points, verbose)
    289     error = _barnes_hut_tsne.gradient(sP, X_embedded, neighbors,
    290                                       grad, angle, n_components, verbose,
--> 291                                       dof=degrees_of_freedom)
    292     c = 2.0 * (degrees_of_freedom + 1.0) / degrees_of_freedom
    293     grad = grad.ravel()

sklearn/manifold/_barnes_hut_tsne.pyx in sklearn.manifold._barnes_hut_tsne.gradient (sklearn/manifold/_barnes_hut_tsne.c:8155)()

AssertionError: Tree consistency failed: unexpected number of points=1796 at root node=1797

Versions

Linux-4.4.0-78-generic-x86_64-with-Ubuntu-16.04-xenial
('Python', '2.7.12 (default, Jul 1 2016, 15:12:24) \n[GCC 5.4.0 20160609]')
('NumPy', '1.12.1')
('SciPy', '0.18.1')
('Scikit-Learn', '0.19.dev0')

Note:

No error encountered for perplexity = 7 or 9, or any other values tried.

[amueller]

Thanks for the report. That's... odd... to say the least.

[lesteve]

Hmmm I can reproduce on master. Note this works fine on 0.18.1.

[jnothman]

(Time to do randomised testing?)

…

On 6 June 2017 at 21:41, Loïc Estève ***@***.***> wrote: Hmmm I can reproduce on master. Note this works fine on 0.18.1. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#8992 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEz642RO8637Zj0wOi3h3YGE2NiJ_Y5ks5sBTsEgaJpZM4Nw4YF> .

[Sentient07]

Hi, I am not sure what the "blocker" tag is, but can i work on this ? If so, could someone please provide me an idea of where to start looking from?
Thanks,

[jnothman]

blocker means we can't release a version without fixing this. I think that's an exaggeration for this issue, but it's something that really needs attention (in part because we don't understand its cause). Debugging this may be hard, but you've got at least two good clues: * It only stopped working recently so you may be able to find the commit that broke it. * It breaks as a function of perplexity You could try other values of perplexity, e.g. 1 to 50 just to check if it fails on another.

…

On 8 Jun 2017 8:44 am, "Ramana Subramanyam" ***@***.***> wrote: I am not sure what the "blocker" tag is, but can i work on this ? If so, could someone please provide me an idea of where to start looking from? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#8992 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEz63eM9MOkueKRkgGR1X857Xqvz5BKks5sByfNgaJpZM4Nw4YF> .

[amueller]

@jnothman do you think this is realistic to tackle for 0.19?

[jnothman]

seeing as it's not been diagnosed fully and we have basically a rewrite of tsne in the works for 0.20, i think we have to say no. we do have the option of reverting any fixes since 0.17 so that at least it is consistently broken

…

On 21 Jun 2017 2:19 am, "Andreas Mueller" ***@***.***> wrote: @jnothman <https://github.com/jnothman> do you think this is realistic to tackle for 0.19? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#8992 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEz65ChrBv9p3ShjNLORvH_v5qRdiNUks5sF_EJgaJpZM4Nw4YF> .

[ogrisel]

Actually I am pretty confident that #9032 fixes this issue and is not behaving correctly.

[ogrisel]

seeing as it's not been diagnosed fully

@tomMoral and I diagnosed that this was caused by the master implementation of the QuadTree datastructure. Contiguous cells (or tiles) did not always have exactly matching boundaries due to floating point rounding. For deep enough trees with small cells, you get a non-zero chance to insert a point in between consecutive tiles in the QuadTree...

The reimplementation of the QuadTree datastructure in #9032 does not have this issue anymore. The max boundary of a cell is exactly the min boundary of the following cell.

[jnothman]

Should be fixed in #9032