Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MeanAveragePrecision backend #2034

Merged
merged 13 commits into from
Sep 8, 2023
Merged

MeanAveragePrecision backend #2034

merged 13 commits into from
Sep 8, 2023

Conversation

SkafteNicki
Copy link
Member

@SkafteNicki SkafteNicki commented Aug 29, 2023

What does this PR do?

Fixes #1153
Fixes #2035
Adds a backend argument to MeanAveragePrecision. The official pycocotools implementation we use is flawed as there are certain corner cases were the implementation does not work. Sadly the implementation is not really maintained any more. However, it is still considered the "official" implementation and therefore we cannot do so much about this.

This PR adds the possibility of other backends for computation, starting with https://github.com/MiXaiLL76/faster_coco_eval which is a faster version and fixes at least the corner case identified in the linked issue. I still think we need to keep pycocotools as the default backend even if all the testing I have done shows that the two backends produce identical results.

The goal for the future would be to reintroduce our old pytorch implementation with backend="pytorch"

Before submitting
  • Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?
PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃


📚 Documentation preview 📚: https://torchmetrics--2034.org.readthedocs.build/en/2034/

Sorry, something went wrong.

Unverified

This commit is not signed, but one or more authors requires that any commit attributed to them is signed.
@SkafteNicki SkafteNicki added the enhancement New feature or request label Aug 29, 2023
@codecov
Copy link

codecov bot commented Aug 29, 2023

Codecov Report

Merging #2034 (6e3930c) into master (1caaf28) will increase coverage by 0%.
The diff coverage is 93%.

Additional details and impacted files
@@          Coverage Diff           @@
##           master   #2034   +/-   ##
======================================
  Coverage      87%     87%           
======================================
  Files         285     285           
  Lines       15993   16021   +28     
======================================
+ Hits        13871   13916   +45     
+ Misses       2122    2105   -17     
📢 Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Lightning-AI).

@mergify mergify bot added the ready label Aug 31, 2023
SkafteNicki and others added 2 commits September 6, 2023 08:01
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
@github-actions github-actions bot added documentation Improvements or additions to documentation topic: Detect labels Sep 6, 2023
@mergify mergify bot added the has conflicts label Sep 6, 2023
@mergify mergify bot removed the has conflicts label Sep 7, 2023
@SkafteNicki SkafteNicki merged commit 6960602 into master Sep 8, 2023
@SkafteNicki SkafteNicki deleted the map_backend branch September 8, 2023 08:50
matsumotosan pushed a commit to matsumotosan/torchmetrics that referenced this pull request Sep 19, 2023
* implementation

* imports + requirements + links

* add tests

* clarify language

* Apply suggestions from code review

* fix line length

---------

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request ready topic: Detect
Projects
None yet
3 participants