Expectations are not respected when a method is stubbed out

[saikat]

If you stub a method on a class and then set an expectation on that method, the expectation is not actually respected.

Reproduction:

class Foo
  def self.bar
    puts 'hi'
  end
  def self.car
    puts 'bye'
  end
end

Foo.bar
# => hi

require 'mocha'

Foo.stubs(:bar => 'stub')
Foo.bar
# => "stub"

Foo.expects(:bar).never
Foo.bar
# => "stub"

Foo.expects(:car).never
Foo.car

# Mocha::ExpectationError: unexpected invocation: Foo.car()
# unsatisfied expectations:
# - expected never, invoked once: Foo.car(any_parameters)
# satisfied expectations:
# - expected never, not yet invoked: Foo.bar(any_parameters)
# - allowed any number of times, invoked twice: Foo.bar(any_parameters)

[floehopper]

This is the intended behaviour. The important thing to realise is that Foo.expects(:bar).never does not entirely override Foo.stubs(:bar => 'stub'), but is merely considered first when receiving a call to Foo.bar. Expectations are matched in reverse order and then, only if none match, an unexpected invocation error is raised.

In this case Foo.expects(:bar).never is considered to have stopped matching immediately, so then the next expectation for the method Foo.bar is considered i.e. Foo.stubs(:bar => 'stub'). This will match and will continue to match for any number of invocations of the method Foo.bar, and so an unexpected invocation error will never be raised.

What is the behaviour you actually want to test? Perhaps there is another way of doing it?

I hope you don't mind, but I reformatted the code in your initial post to make it a bit clearer what's going on. This should make it more useful to other people as a reference.

[saikat]

Thanks for the quick response (and thanks for fixing up the code, definitely much easier to read now!)

I think I understand your reasoning for why the current behavior exists, but not sure I know the motivation behind the current behavior. The use case here I have is that I have some module A around which I have tests, and A calls into methods on B. For most tests on A, I simply want to stub out behavior on B so I can unit test the behavior on A given known returns from B (the typical use case for stubbing). As a result, I simply stub out a lot of the behavior on B in a setup method. But then for certain tests for A, I need to test the behavior in A by checking to see if it actually calls into B. So, I end up in situations where B.foo is stubbed out but then I have a specific test in the module A tests that wants to make sure that B.foo never gets called.

I think my conceptual model (which may be completely wrong) is:

• Stubs are a way to modify the behavior of a method external to the SUT to use it for the purposes of your tests around a SUT
• Expectations are a way to check the actual behavior of the SUT (i.e. whether or not it calls certain methods in code external to the SUT)

In this model, stubs and expectations don't affect each other and serve separate purposes. What is the motivation to treat stubs and expectations as part of the same system as opposed to having stubs simply be a way to modify the behavior of a called method and expectations be a way to simply determine whether or not a method is called?

Again, thanks so much for the response and being patient with me =). I should add that I'd be willing to look into modifying the current behavior if we actually agree to change it.

[saikat]

I think a clearer/more concise way to put this -- I think my issue here may be simply with the fact that in certain cases, all expectations seem to need to pass for a test to pass, but in others, it seems like certain expectations can fail but the overall test will still pass. I believe in the first part of that last sentence because, for example, the following code will throw an exception:

Foo.expects(:a).twice
Foo.expects(:a).once
Foo.a('bye')

Here, though the second expectation is satisfied, the first is not. A failure here makes sense to me. Similarly, if I do:

Foo.stubs(:a)
Foo.expects(:a).once

This will fail if I never call Foo.a because the second expectation is never satisfied. This model of "every expectation that is set up should be satisfied" seems like a reasonable statement to make. As a result, I think the following code should fail:

Foo.stubs(:a)
Foo.expects(:a).never
Foo.a

Here, even though Foo.a satisfies the first expectation, it does not satisfy the second. Similarly, the following code will pass:

Foo.stubs(:a)
Foo.expects(:a).once
Foo.a
Foo.a

Here, the second expectation should fail because Foo.a was called twice, not once. The first expectation is satisfied though. The issue here seems to be specific to what happens when extraneous calls to a method is made in the case of multiple expectations being set up. Basically, in the case that I just have Foo.expects(:a).once; Foo.a; Foo.a, the expectation fails, so similarly, I believe that expectation should always fail if multiple calls to Foo are made even if other expectations are set up on Foo.a.

Thanks again for taking the time to look and respond to these. If the above logic makes sense to you, I'd be happy to write a pull request to fix this issue (otherwise, I can just workaround this in my code).

This results in a failure (correctly).

[saikat]

In the meantime, I've added a workaround for this in my test suite with the following monkeypatch:

https://gist.github.com/1963537

The idea here is that on every invocation, we simply find all matching methods and and invoke them. Then, we just check at the end of a test whether or not the number of invocations violates our expectations. We don't treat never expectations differently -- they just get checked at the end of the method the same way everything else does (so no more immediate UnexpectedInvocation). This allows the examples above to do the right thing. It will also end up breaking some tests that would previously pass. For example, before, the following would create a passing test:

Foo.expects(:a).once
Foo.expects(:a).twice
Foo.a
Foo.a
Foo.a

The expectations above, however, seem to me to be stating that there are two conflicting expectations of the number of times 'a' should be invoked (and neither is saying a should be invoked three times). So breaking in this case now seems like it may be the correct thing to do.

The only thing that is a bit unclear with this patch is if I set up multiple expectations that are all matching but that each specify different return values. So, let us take the following example:

Foo.expects(:a).twice.returns(1)
Foo.expects(:a).twice.returns(2)
Foo.a
Foo.a

Previously, this would return 2 followed by 1, whereas now it returns 2 twice. I also thought this may be an ok tradeoff in that, I think, the previous behavior was not really defined. In most cases when people set up multiple expectations like this that will all match, there isn't (or shouldn't be) a lot of logic about the order of returns for multiple invocations. To set up ordering of return values for multiple invocations, the proper thing to do should be:

Foo.expects(:a).twice.returns(1).then.returns(2)

Anyway, I am happy keeping this workaround not upstreamed, but if this seems like a reasonable patch to add to mocha, I'd be happy to make a pull request with tests around the new behavior. Let me know what you think, and thanks for bearing with my giant essays of comments =).

[floehopper]

Thanks for all your comments. This aspect of Mocha was modelled on the method dispatch behaviour of jMock v1. For some time I have wanted to change it to be modelled on the method dispatch behaviour of jMock v2, but this would be such a significant change in behaviour that it would require a change in major version and I just haven't managed to find the time to do it.

I do see that there is an apparent inconsistency in the existing behaviour and would be interested to see whether there is a similar inconsistency if we were to change to use the jMock v2 approach. I'm not sure whether your approach would be more or less intuitive to most people. I need to do some serious thinking about this when I can find the time! It'd be great if you could read the two jMock links above and let me know what you think.

In response to the actual problem you are trying to solve, this problem has come up quite a few times over the years. One option is to use Mocha's state machine mechanism. I've dug out this bit of code (which may be a bit out-of-date) as an example. I know it's a bit unwieldy, but it might give us more inspiration.

[floehopper]

I don't plan on incorporating this any time soon, so I'm going to mark this issue as closed. Thanks for your input. Cheers, James.

[floehopper]

A fix for this was released in v2.7.0 - better late than never?!