Fix/ngboost col sample lt 1

[CloseChoice]

Overview

Closes #3261

Description of the changes proposed in this pull request:
This PR only solves the problem if we have just 1 estimator. So for this script:

import shap 
import ngboost as ngb
import numpy as np

X, y = shap.datasets.california(n_points=500)
model = ngb.NGBRegressor(n_estimators=1, col_sample = 0.9).fit(X, y)

predicted = model.predict(X)

# explain the model's predictions using SHAP values
explainer = shap.TreeExplainer(model, model_output=0)

explanation = explainer(X_predict_explain)

# check that SHAP values sum to model output
assert (
    np.abs(explanation.values.sum(1) + explanation.base_values - predicted).max()
    < 1e-5
)

Checklist

• All pre-commit checks pass.
• Unit tests added (if fixing a bug or adding a new feature)

Here is a breakdown of the issue: If we use ngboost with col_sample < 1 some columns of X get dropped, the rest is reordered, see here. Unfortunately, this might be different for each estimator. Which means, we would need to rewrite the shap_values function or predict function, not quite sure though.

[thatlittleboy]

Thanks for the PR @CloseChoice , I've not yet taken a close look at this, but can you add your script as a test that reproduces the error so we make sure we don't regress next time?

Maybe Connor's PR #3262 might be of inspiration as well.

[CloseChoice]

@thatlittleboy so I found a better fix that also fixes the problem for n estimators. The problem was, that due to the reordering of columns in ngboost, the predictions of the ngboost model do not match the predictions that we get from the treeensemble model, that we directly construct from the ngboost trees. I fixed this by reordering the features correctly. I also added a test that checks that the predictions are always identical

[connortann]

This is a great PR - thanks for the fix and the test!