What is Public API

[nstarman]

Description

Astropy has a very ad-hoc definition and adherence to any standard for defining public API.
I think we should be more rigorous and make the necessary changes to have one definition.

I really like Python's reference (https://github.com/python/typing/blob/master/docs/source/libraries.rst#how-much-of-my-library-needs-types) and scipy's reference on what is public scoping (https://docs.scipy.org/doc/scipy/reference/index.html#importing-from-scipy). Were we to adopt these compatible standards, we would need to overhaul most __all__ in Astropy, but then we'd have an authoritative definition of what is and isn't public.
This is also a good opportunity to enforce that we want users to import from specific places, like from astropy.coordinates import ICRS not from astropy.coordinates.builtin_frames.....

So a module might look like this:

__init__.py
    __all__ = ["Foo"]
foo.py
    __all__ = []  # not public!  only the module-level import is public.
    class Foo: ...

[pllim]

Thanks! Might be a good topic to bring up at the next coordination meeting.

[mhvk]

I think this would start with formalizing what we now have, which is that the sub-module is the level one imports from (with utils as the only exception).

[WilliamJamieson]

This maybe a separate issue, but I would also like to have a discussion about the supported API for function calls. So I am piggy-backing off this issue for now.

Namely for packages like modeling, we have a very informal method of defining what is an acceptable input to an astropy model. Basically, we attempt to support anything that ever worked or someone feels should work (in terms of shape, value, etc). This is extremely hard to support in the long run as attempting to "fix" one thing breaks something else, which we are unaware of. For example all the discussion in #12021.

[pllim]

This came up in the discussions (Sphinx unpin campaign):

• Unpin sphinx alternative #14140
• DOC: Warnings about duplicate object description of astropy.coordinates.builtin_frames #11723
• Add option to set :noindex: sphinx-automodapi#150

There are also quite a few other things linked to the issues/PRs above.

[pllim]

As discussed on 2023-04-07, @nstarman would draft up an APE on the policy and accompany it with a small proof-of-concept PR on what change is actually needed to comply with the new policy (maybe start with coordinates built-in frames since they were mentioned above).

[mhvk]

Darn, forgot that meeting! But just to mention it here: scipy has gone through a change of how they deal with public vs private API, essentially leading to a lot of modules prefixed with underscores. I'm not a great fan, but it is obviously relevant to see what the ecosystem does, especially as numpy may be moving in the same direction (see the draft NEP at numpy/numpy#23537).

I should add that I think both scipy and numpy were in rather messy states, where a drastic cleanup was warranted (for numpy, this is for version 2.0, so they are allowing breaking changes, though with the intent that anything that relied on top-level API would be essentially unaffected).

I feel astropy's way of laying things out has been much more logical and consistent, so I do not see a need for renaming most files, but would rather prefer to formalize the system we have been following. Anyway, for the APE I guess!