Provide a way to nicely control fixture execution order

[astraw38]

We sometimes have fixtures that need to execute in a specific order, without having them include one another as an argument. This is to provide flexibility with fixtures, so they aren't always linked to one another.

Quick example:

@pytest.fixture()
def a():
    print "In a"


# Fixture in conftest.py, used in other contexts
@pytest.fixture()
def multi_use():
    # Don't always want to use this fixture with 'a', so isn't included as an argument.
    print "Should always be executed after 'a', if 'a' is included with the test via usefixtures"

I know there's ways to do this w/ using fixture arguments, like having intermediary fixtures that exist purely to order the other fixtures, but it's not very pretty.

[RonnyPfannschmidt]

the idea of declaring indirect dependency orders is tricky to solve,
i'd like to avoid a quick intermediate solution and focus on making the setupstate/di framework more self contained first

[astraw38]

I was thinking of was something similar to what was done w/ plugin order via tryfirst/trylast ... but no idea how quick that would be. Perhaps a sorting system similar to how logging levels work, where fixture order within a level would be indeterminate?

[hpk42]

i guess there could be another parameter to the "pytest.fixture" decorator which influences ordering like "after='a''" but not sure how easy it would be to implement. It would also need to provide proper error messages when ordering can not be established according to fixture configuration. And i somewhat agree to the point Ronny is making that we might need to do some internal refactoring first. That being said, it's always good to have a guiding use case that should be enabled by a refactoring (apart from the need for full backward compat of course). FWIW myself i probably won't try to implement this on unpaid time.

[ktosiek]

For a more concrete use case: I'm trying to implement shared DB state fixtures based on layering subtransactions here: pytest-dev/pytest-django#258, and I need to make sure db (which is function-level) only runs after all users of shared_db_wrapper.

For this case something like inherited before="db" would be needed.

[nipunn1313]

I also have a concrete use case.

A top level conftest.py has a couple of fixtures

def user():
    return new_user

def application(user):
    return new_application(user)

A subdirectory with some special tests needs to create some special state for user before loading the application. Ideally we could do.

@pytest.usefixtures(before='application')
def special_user_setup(user):
    prepare_some_special_stuff_for(user)

We could override the user fixture, but that would be unfortunate if the user fixture was doing a lot of work on its own (we'd have to copy paste).
In this case, application cannot inherit from special_user_setup.

Is there a way to do this today, or will it require such a before= argument?

[The-Compiler]

@nipunn1313 I wonder if you could just do something like:

@pytest.fixture
def user(user):
    pass

Kind of a hack, it might or might not work.

[aurzenligl]

I had the same problem (wanted bar to execute before foo) and used following hack:

@pytest.yield_fixture
def foo():
    ...

@pytest.mark.early
@pytest.yield_fixture
def bar():
    ...

def reorder_early_fixtures(metafunc):
    for fixturedef in metafunc._arg2fixturedefs.values():
        fixturedef = fixturedef[0]
        if getattr(fixturedef.func, 'early', None):
            order = metafunc.fixturenames
            order.insert(0, order.pop(order.index(fixturedef.argname)))

def pytest_generate_tests(metafunc):
    reorder_early_fixtures(metafunc)

def test_baz(foo, bar):

Whereas I suppose marking fixtures - while not intended - is innocuous, I have doubts as to how foolish assumptions of reorder_early_fixtures may be:

1. presence of metafunc._arg2fixturedefs as dictionary with tuples with FixtureDef as first elem,
2. shuffling order of names in metafunc.fixturenames is legal and influences order of execution.

How long would these assumptions remain to be true (I'm using pytest 3.0.3)?
Is there any safer way to achieve this?

[fruch]

@aurzenligl looks like a good idea, question, can we implement the marker with the before param mentioned above, sounds like a great way to enforce the order

[ssalbdivad]

While I do suggest caution in ordering fixtures, I've written up a flexible way to do so when the situation calls for it:

def order_fixtures(metafunc):
    metafunc.fixturenames[:] = []
    orders = {name: getattr(definition[0].func, "order", None)
              for name, definition in metafunc._arg2fixturedefs.items()}
    ordered = {name: getattr(order, "args")[0] for name, order in orders.items() if order}
    unordered = [name for name, order in orders.items() if not order]
    first = {name: order for name, order in ordered.items() if order and order < 0}
    last = {name: order for name, order in ordered.items() if order and order > 0}
    merged = sorted(first, key=first.get) + unordered + sorted(last, key=last.get)
    metafunc.fixturenames.extend(merged)

def pytest_generate_tests(metafunc):
    order_fixtures(metafunc)

Fixtures can then be ordered so that those with a negative order are run before unordered fixtures in ascending order, then unordered fixtures are run, then those with a positive order are run in ascending order, as in the following example:

@pytest.mark.order(-2)
@pytest.fixture
def first():
    print("I go first.")

@pytest.mark.order(-1)
@pytest.fixture
def second():
    print("I go second.")

@pytest.fixture
def third():
    print("I go third.")

@pytest.mark.order(1)
@pytest.fixture
def fourth():
    print("I go fourth.")

@pytest.mark.order(2)
@pytest.fixture
def fifth():
    print("I go fifth.")

def test_something(fifth, fourth, third, second, first):
    print("Running your test.")

"""
Output:
I go first.
I go second.
I go third.
I go fourth.
I go fifth.
Running your test.
"""

[RonnyPfannschmidt]

in general marks are not supported on fixtures, and its not clear how marks of fixtures shouzld map to tests,
so the solution cant use marks

[ssalbdivad]

Hey @RonnyPfannschmidt,
The intent of this solution was not to propose its adoption by pytest but rather to provide a flexible and seemingly reliable way for folks to specify the execution order of their fixtures expressively while a decision is made regarding how best to integrate first-class support for doing so. Based on my testing this seems to be fairly robust, but if there are situations in which relying on fixture marking in the way that I have will cause things to break it'd be really useful to know what those are.
Thanks!

[claweyenuk]

What about using something like z-index from CSS?

@pytest.fixture(teardown_index=-9999)
def first():
    pass

@pytest.fixture(teardown_index=9999)
def last:
    pass

just go in order of teardown index, if 2 items have the same value, no guarantees. The default index can be 0. If teardown should be early, use a negative, if they should be later, use a positive number.

[RonnyPfannschmidt]

that doesn't take scopes into account ^^

[claweyenuk]

Let's say scope takes priority, it doesn't make sense to teardown class fixtures after module fixtures. The kwarg could even by scope_teardown_index, though that is getting long

[RonnyPfannschmidt]

@claweyenuk that still just imposes a random artificial order - which in turn is very likely to bite someone

[romuald]

To complete @aurzenligl solution (that broke with pytest 4.1), here is an updated example:

def reorder_early_fixtures(metafunc):
    """
    Put fixtures with `pytest.mark.early` first during execution

    This allows patch of configurations before the application is initialized

    """
    for fixturedef in metafunc._arg2fixturedefs.values():
        fixturedef = fixturedef[0]
        for mark in getattr(fixturedef.func, 'pytestmark', []):
            if mark.name == 'early':
                order = metafunc.fixturenames
                order.insert(0, order.pop(order.index(fixturedef.argname)))
                break


def pytest_generate_tests(metafunc):
    reorder_early_fixtures(metafunc)


@pytest.fixture
@pytest.mark.early
def early_stuff(self, another_fixture):
    # ...

[GabrieleCalarota]

Is there any approved solution to be working and safely implemented?

[The-Compiler]

@GabrieleCalarota For simple cases, using the other fixture as an argument - for more complex cases (or what the OP described), no, otherwise this would be closed.

[GabrieleCalarota]

@The-Compiler Yep, the problem I had was related to the order but I import multiple fixture in pytest.

Example:

@pytest.fixture()
def a():
  return fetch_something()

@pytest.fixture()
def b():
  setup_user()

@pytest.fixture()
def c(a, b):
   do_something_with_a_and_b()
   return a


def test_something_with_a_and_b_changed_with_c(c, a):
    """
    # FIXME: the problem is I would like to call first c and secondly fetch again the value in fixture `a`, 
but this seems not to be possible related to order of fixture and caching the value 
fetched in fixture `a` that was also called in fixture `c` imported into the test function
    """
    here I use c and a

Is this achievable in some way to get a newly fetched fixture a ?

[SaturnIC]

To complicate matters even more,
is to look at the fixture teardown order.

Now it seem the order for the teardown is a reverse of the order of the setup.

Assuming this is true,
is there a way to set the fixture order independently for setup and teardown?

[RonnyPfannschmidt]

Tear down as reverse of the setup it's the most solid pattern for preventing loads of foot guns

If you need a different resources cleanup pattern outlining the use case in a discussion would be a good start

[SaturnIC]

It seems changing the order of item.fixturenames in pytest_runtest_teardown hook has no effect?

[SaturnIC]

If you need a different resources cleanup pattern outlining the use case in a discussion would be a good start

An obvious workaround would be to split a single fixture with teardown into two fixtures,
one for teardown and one for setup with different metafunc.fixturenames orderings and thus different priorities.

[RonnyPfannschmidt]

I'm not going to keep discussing foot guns in a XY problem Style,im out

[hpk42]

Who was dumb enough to make an obvious ClusterB disordered "human" into a maintainer?

That probably was me -- and ronny has been one of the key collaborators on pytest from the start FYI.
While i am only a past maintainer i suggest to stop the insulting style of your commenting.

I am overall not sure that this issue should be kept open, btw. Several people pointed out workarounds/ways to implement some sort of fixture ordering and putting support into pytest itself would be a lot of work as far as i can still tell.

[RonnyPfannschmidt]

i believe best to close this one, and revisit the idea at a later point when the fixture system is in better shape

recently a new person started to do great work towards cleaning things up and eventually enabling more

[nicoddemus]

Agree, closing.