IsotonicRegression results differ between fit/transform and fit_transform with ties in X

[mjbommar]

Per conversation in issue #2507, IsotonicRegression appears to have regressed due to commit a9ea55f.

This IPython notebook demonstrates the failure on HEAD.

I tested the following two commits with the notebook:

• d255866: no difference, SUCCESS

• a9ea55f: difference, FAILURE

  In other words, I think we can blame the switch for interp1d from "linear" to "slinear"; first thought is that 1-d spline "slinear" matrix formulation is ill-posed for x-ties, whereas the piecewise "linear" implementation is unaffected?

Small additional note: confirmed failure with test case where x-values are all non-zero, e.g., [1, 1, 2, 3, 4] instead of [0, 0, 1, 2, 3], so x=0 isn't part of the cause.

[mjbommar]

Including @ragv and @amueller.

[raghavrv]

Ah extremely sorry that I broke that :/
Will see ASAP what can be done to keep slinear intact and still avoid this issue...

note to self: need NRTs after fix.

[raghavrv]

thanks for the report!

[mjbommar]

No worries, @ragv! I have a simple pull request for a basic x-ties unit test in PR #4185, which shows the failure in Travis.

[mjbommar]

As a workaround, I tested using scipy.interpolate.UnivariateSpline as follows and got acceptable stuff:

# Sample with x ties
data_with_ties = pandas.DataFrame()
data_with_ties["feature"] = [1, 1, 2, 3]
data_with_ties["target"] = [0.1, 0.05, 0.15, 0.2]

sp1 = scipy.interpolate.UnivariateSpline(data_with_ties["feature"],
                                   data_with_ties["target"],
                                   k=1,
                                   ext=3)
sp1(data_with_ties["feature"])
array([ 0.07727273,  0.07727273,  0.14090909,  0.20454545])

vs.

sp2 = scipy.interpolate.interp1d(data_with_ties["feature"],
                                data_with_ties["target"],
                                kind="slinear")
sp2(data_with_ties["feature"])

array([ 0.  ,  0.  ,  0.15,  0.2 ])

[amueller]

At d255866
I get

AssertionError: 
Arrays are not equal

(mismatch 14.2857142857%)
 x: array([ 0.,  1.,  1.,  3.,  4.,  5.,  6.])
 y: array([ 0.,  1.,  2.,  3.,  4.,  5.,  6.])

[amueller]

I get the same failure in 0.15.2

[amueller]

(that is using your regression tests, should have posted in the PR I guess)

[mjbommar]

Hm, I had only tested 0.15.2 with the simple duplicate zero minimum value case. It goes deeper!

[mjbommar]

Just updated PR #4185 to include comparison to R's isotone with gpava(ties="primary"). Discussion there on ties= methods is probably one of the first questions to answer...

[ghost]

Possibly related, also in sklearn 0.15.2

Given dataset X

nmf = sklearn.decomposition.nmf.NMF(n_components=3, random_state=long(1))
A = nmf.fit_transform(X)
B = nmf.fit(X).transform(X)

Provides:

>>> A[0]
[  1.76089236e-32   2.60906610e-05   3.82422063e-06]
>>> B[0]
[  3.87175453e-33   2.60907250e-05   3.79032621e-06]

Although not huge differences, the two approaches to fit>transform are no longer equivalent

[amueller]

@FixLaurens not really related to the issue here. I am think this outcome is fine, as the result is quite close. You can't entirely guarantee exactly similar results from numeric optimization. We could try to get closer in NMF but I'm not sure this is worth the extra computation.
Did this cause any issues in your application?