[MRG] Sample weights for ElasticNetCV

[lorentzenchr]

Reference Issues/PRs

Partially solves #3702: Adds sample_weight to ElasticNetCV and LassoCV, but only for dense feature array X.
It is a follow up of PR #15436.

Any other comments?

DO NOT MERGE BEFORE #15436 as it is based on that branch.

[lorentzenchr]

@agramfort How would you like to calculate the cross-validated mean square error? Weighted by sample_weight or unweighted. I think this was discussed somewhere else. I would prefer the weighted version.

Furthermore, just as info, the current approach to rescale X by sqrt(sw) might use more memory copies as the unweighted version.

[rth]

The diff on Github doesn't seem to take into account the merge #15436 PR, maybe merging master would help?

How would you like to calculate the cross-validated mean square error? Weighted by sample_weight or unweighted. I think this was discussed somewhere else. I would prefer the weighted version.

I couldn't find the corresponding issue, maybe it could be worth opening one. There was #15651 but it's a different topic.

In the short term, I think we want to be consistent with GridSearchCV(ElasticNet()).fit(X, y, sample_weight). As far as I can tell from

scikit-learn/sklearn/model_selection/_search.py

Line 682 in b194674

out = parallel(delayed(_fit_and_score)(clone(base_estimator),

and the corresponding _fit_and_score function, sample_weight is not used for scoring there. We could discuss what is the right thing to do (or whether there should be an option to allow taking into account sample weight for scoring) in a separate issue.

[lorentzenchr]

@jjerphan If you feel comfortable, I'd appreciate your review approval very much:smirk:

[jjerphan]

Hi @lorentzenchr,

Here are a few lasts suggestions before approval. 🙂

[jjerphan]

LGTM, thanks @lorentzenchr!

[lorentzenchr]

Decision in yesterday's dev meeting (2021-05-25): Remove use_weights_in_cv. This functionality will come with SLEP006 (hopefully) and makes this PR non-controversial.

[lorentzenchr]

To have it easily referenced, here is the code that is needed for use_weights_in_cv:

• LinearModelCV

  """
  use_weights_in_cv : bool, default=False
      If `True`, the MSE over test folds is calculated as a weighted average,
      weighted by the sum of `sample_weight` of each test fold. Here,
      `sample_weight=None` acts like `sample_weight=1`, which means the sum
      of weights in the test fold is the number of observations.
  """
• fit

  if not self.use_weights_in_cv:
      # The mean is computed over folds.
      mean_mse = np.mean(mse_paths, axis=1)
  else:
      if sample_weight is None:
          # Note that both len(test) and sample_weight[test].sum() can
          # have different values for different folds.
          sw_paths = [len(test) for train, test in folds]
      else:
          sw_paths = [sample_weight[test].sum() for train, test in folds]
      # The average is computed over folds.
      mean_mse = np.average(mse_paths, axis=1, weights=sw_paths)
• test_enet_cv_sample_weight_correctness

  @pytest.mark.parametrize("use_weights_in_cv", [True, False])
  @pytest.mark.parametrize("fit_intercept", [True, False])
  def test_enet_cv_sample_weight_correctness(use_weights_in_cv, fit_intercept):
      """Test that ElasticNetCV with sample weights gives correct results."""
      ...
      if use_weights_in_cv:
        # Note: The order of groups from LeaveOneGroupOut is the same for both.
        assert_allclose(reg_sw.mse_path_, reg.mse_path_)

[agramfort]

besides LGTM

thx a lot @lorentzenchr for taking the time to dive deep into this

[lorentzenchr]

@rth Do you want to merge? @agramfort and @jjerphan have already approved. This PR only adds sample_weight to some fit methods, nothing more API-wise. test_enet_cv_grid_search assures equivalency with GridSearchCV.

[agramfort]

@lorentzenchr you need to rebase this one

[rth]

Thanks a lot! LGTM. Please merge main in to resolve conflicts.

[lorentzenchr]

@rth Main merged and all CI green.