IsolationForest(max_features=0.8).predict(X) fails input validation

[betatim]

When subsampling features IsolationForest fails the input validation when calling predict().

from sklearn.ensemble import IsolationForest
from sklearn import datasets

iris = datasets.load_iris()
X = iris.data
y = iris.target

clf = IsolationForest(max_features=0.8)
clf.fit(X, y)
clf.predict(X)

gives the following:

scikit-learn/sklearn/tree/tree.pyc in _validate_X_predict(self, X, check_input)
    392                              " match the input. Model n_features is %s and "
    393                              " input n_features is %s "
--> 394                              % (self.n_features_, n_features))
    395
    396         return X

ValueError: Number of features of the model must  match the input. Model n_features is 3 and  input n_features is 4

In predict one of the individual fitted estimators is used for input validation: self.estimators_[0]._validate_X_predict(X, check_input=True) but it is passed the full X which has all the features. After looking into it a bit, bagging.py sub-samples the features itself, where as forest.py delegates it to the underlying DecisionTree.

[agramfort]

ping @ngoix

[ngoix]

Yes we can't apply X_test to the base estimators self.estimators_ because X_test has a full number of features, whereas base estimators are built on a smaller number of features.

The original Isolation Forest algorithm does not consider sub-sampling features. We let this argument because it was supposed to be an easy extension. Seems that it is not the case. I vote for removing max_features argument.

[glouppe]

To me the issue comes from the fact that the validity of X is checked through one its sub-estimators
(See X = self.estimators_[0]._validate_X_predict(X, check_input=True) at line 194). This is not the correct way to do it.

[ngoix]

Yes but even if you replace this line by just X = check_array(X), you get error in predict when doing (L202)

for i, tree in enumerate(self.estimators_):
         leaves_index = tree.apply(X)

[glouppe]

Yes, because you should not do that either. Since you inherit form BaseBagging, you need to iterate over zip(self.estimators_, self.estimators_features_) and then compute leaves_index = tree.apply(X[:, features]). Sorry for not catching that in the initial review...

[ngoix]

Ok great! I didn't know about self.estimators_features_. So it is indeed an easy extension, and I withdraw my vote for removing max_features ;)