Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: optimize._hessian: use _jacobian to compute accurate Hessian #20708

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

mdhaber
Copy link
Contributor

@mdhaber mdhaber commented May 13, 2024

Reference issue

Follow-up to gh-20630

What does this implement/fix?

Adds a private function to compute accurate Hessians using a finite difference approximation. Vectorized to compute Hessian at an arbitrary number of points in a single call.

Additional information

To do eventually:

  • Test (and fix, where applicable) attributes of return object (for this and _jacobian). It doesn't look like nfev is correct. Fixed: only sum the inner nfev counts for the first nit outer iterations.
  • Improve error estimate by considering error of _jacobian. Just tighten tolerance of inner Jacobian. In the future we can use a legit 2-D stencil.
  • Consider adding features to help user vectorize with respect to multiple parameters. (Separate PR; not essential.)
  • Consider using first Jacobian evaluation to estimate appropriate step size. This could reduce number of iterations required in subsequent evaluations. (Separate PR; not essential.)

@mdhaber mdhaber added enhancement A new feature or improvement scipy.optimize labels May 13, 2024
del res.df # this is renamed to ddf
del res.nit # this is only the outer-jacobian nit

i = res.error <= np.swapaxes(res.error, 0, 1)
Copy link
Contributor Author

@mdhaber mdhaber Jun 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the failure occurs when the errors are equal; there would need to be an additional condition to ensure that corresponding off-diagonals of i are mutually exclusive.

An unrelated issue that is probably rare but could happen: what if the lower error corresponds with a nonzero status. Probably need to make the condition contingent on success.

I'm conflicted about what to do here, so I'll wait for feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement A new feature or improvement scipy.optimize
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant