According to the unittest.mock documentation for side_effect:
If you pass in an iterable, it is used to retrieve an iterator which must yield a value on every call. This value can either be an exception instance to be raised, or a value to be returned from the call to the mock (
DEFAULThandling is identical to the function case).
Therefore, your socket.create_connection mock will return the function oddConnect for the first two calls, then return the Mock object for the last call. From what I understand, you want to mock create_connection object to actually call those functions as side effects rather than returning them.
I find this behavior rather odd, since you'd expect side_effect, to mean side_effect in every case, not return_value. I suppose the reason this is so lies in the fact that the value of the return_value property must be interpreted as-is. For instance, if your Mock had return_value=[1, 2, 3], would your Mock return [1, 2, 3] for every call, or would it return 1 for the first call?
Solution
Fortunately, there is a solution to this problem. According to the docs, if you pass a single function to side_effect, then that function will be called (not returned) every time the mock is called.
If you pass in a function it will be called with same arguments as the mock and unless the function returns the
DEFAULTsingleton the call to the mock will then return whatever the function returns. If the function returnsDEFAULTthen the mock will return its normal value (from thereturn_value).
Therefore, in order to achieve the desired effect, your side_effect function must do something different every time it is called. You can easily achieve this with a counter and some conditional logic in your function. Note that in order for this to work, your counter must exist outside the scope of the function, so the counter isn't reset when the function exits.
import mock
import socket
# You may wish to encapsulate times_called and oddConnect in a class
times_called = 0
def oddConnect():
times_called += 1
# We only do something special the first two times oddConnect is called
if times_called <= 2:
result = mock.MagicMock() # this is where the return value would go
raise socket.error # I want it assigned but also this raised
socket.create_connection = mock(spec=socket.create_connection,
side_effect=oddConnect)
# what I want: call my function twice, and on the third time return normally
# what I get: two function objects returned and then the normal return
for _ in xrange(3):
result = None
try:
# this is the context in which I want the oddConnect function call
# to be called (not above when creating the list)
result = socket.create_connection()
except socket.error:
if result is not None:
# I should get here twice
result.close()
result = None
if result is not None:
# happy days we have a connection
# I should get here the third time
pass
Answer from caleb531 on Stack OverflowAccording to the unittest.mock documentation for side_effect:
If you pass in an iterable, it is used to retrieve an iterator which must yield a value on every call. This value can either be an exception instance to be raised, or a value to be returned from the call to the mock (
DEFAULThandling is identical to the function case).
Therefore, your socket.create_connection mock will return the function oddConnect for the first two calls, then return the Mock object for the last call. From what I understand, you want to mock create_connection object to actually call those functions as side effects rather than returning them.
I find this behavior rather odd, since you'd expect side_effect, to mean side_effect in every case, not return_value. I suppose the reason this is so lies in the fact that the value of the return_value property must be interpreted as-is. For instance, if your Mock had return_value=[1, 2, 3], would your Mock return [1, 2, 3] for every call, or would it return 1 for the first call?
Solution
Fortunately, there is a solution to this problem. According to the docs, if you pass a single function to side_effect, then that function will be called (not returned) every time the mock is called.
If you pass in a function it will be called with same arguments as the mock and unless the function returns the
DEFAULTsingleton the call to the mock will then return whatever the function returns. If the function returnsDEFAULTthen the mock will return its normal value (from thereturn_value).
Therefore, in order to achieve the desired effect, your side_effect function must do something different every time it is called. You can easily achieve this with a counter and some conditional logic in your function. Note that in order for this to work, your counter must exist outside the scope of the function, so the counter isn't reset when the function exits.
import mock
import socket
# You may wish to encapsulate times_called and oddConnect in a class
times_called = 0
def oddConnect():
times_called += 1
# We only do something special the first two times oddConnect is called
if times_called <= 2:
result = mock.MagicMock() # this is where the return value would go
raise socket.error # I want it assigned but also this raised
socket.create_connection = mock(spec=socket.create_connection,
side_effect=oddConnect)
# what I want: call my function twice, and on the third time return normally
# what I get: two function objects returned and then the normal return
for _ in xrange(3):
result = None
try:
# this is the context in which I want the oddConnect function call
# to be called (not above when creating the list)
result = socket.create_connection()
except socket.error:
if result is not None:
# I should get here twice
result.close()
result = None
if result is not None:
# happy days we have a connection
# I should get here the third time
pass
I also encountered the problem of wanting a side effect to occur for only some items in a list of values.
In my case, I wanted to call a method from freezegun the third time my mocked method was called. These answers were really helpful for me; I ended up writing up a fairly general wrapper class, which I thought I'd share here:
class DelayedSideEffect:
"""
If DelayedSideEffect.side_effect is assigned to a mock.side_effect, allows you to
delay the first call of callback until after a certain number of iterations.
"""
def __init__(self, callback, delay_until_call_num: int, return_value=DEFAULT):
self.times_called = 0
self.delay_until_call_num = delay_until_call_num
self.callback = callback
self.return_value = return_value
def side_effect(self, *args, **kwargs):
self.times_called += 1
if self.times_called >= self.delay_until_call_num:
self.callback()
return self.return_value
To then return "my_default_return_value" without calling the lambda function on the first three calls:
with freeze_time(datetime.now()) as freezer:
se = DelayedSideEffect(callback=lambda: freezer.move_to(the_future), 3)
my_mock = MagicMock(return_value="my_default_return_value", side_effect=se)
Proposal to Modify MagicMock.side_effect for Better Function Mocking in Unit Tests
unit testing - Python - Mock requests with side_effect - Stack Overflow
python side_effect - mocking behavior of a method - Stack Overflow
Feat: unittest.Mock.side_effect fails on improper usage - Ideas - Discussions on Python.org
What is a side effect in Python?
Why should you avoid side effects in Python?
What is a pure function in Python?
Possible solution with module unittest (not pytest)
I have written this answer before @baguette added the code of its function func1(), so I have created a file called my_file_01.py which contains my production function func1():
import requests
def func1():
response1 = 'empty1'
response2 = 'empty2'
r = requests.get('http://www.someurl.com')
if r.status_code == 200:
response1 = r.json()
r = requests.get('http://www.some_other_url.com')
if r.status_code == 500:
response2 = r.json()
return [response1, response2]
As you can see func1() calls requests.get() two times and checks the status code of responses.
I have inserted the test code in a different file with the following content:
import unittest
from unittest import mock
from my_file_01 import func1
def request_resp1(url):
response_mock = mock.Mock()
response_mock.status_code = 200
response_mock.json.return_value = {'key1': 'value1'}
# the function return an instance of class Mock
return response_mock
def request_resp2(url):
response_mock = mock.Mock()
response_mock.status_code = 500
response_mock.json.return_value = "Failed"
# the function return an instance of class Mock
return response_mock
class TestFunc1(unittest.TestCase):
@mock.patch("my_file_01.requests")
def test_func1(self, mock_requests):
print("test func1()")
mock_requests.get.side_effect = [request_resp1(""), request_resp2("")]
[response1, response2] = func1()
print("response1 = " + str(response1))
print("response2 = " + str(response2))
if __name__ == "__main__":
unittest.main()
The test file defines the test class TestFunc1 which contains the test method test_func1().
Furthermore the file defines 2 functions called request_resp1() and request_resp2().
These functions are used to define different response values when it is called the method requests.get() by the code of func1(); to be more accurate:
- the first call to
requests.get()assigns to variablerthe Mock object withstatus_code = 200so it simulates a success; - the second call to
requests.get()assigns to variablerthe Mock object withstatus_code = 500so it simulates a failure.
If you try to execute the test code you can see that func1() return different values for the 2 different status_codes of the response. The output of the execution is composed by the 3 print() instructions present in test_func1() and is the followed:
test func1()
response1 = {'key1': 'value1'}
response2 = Failed
Useful links
For detail about
side_effectattribute see its documentation.This is an other example with
side_effectrequestis a Python module and this is a minimal documentation about it.
Based on the solution from @frankfalse, the two mocking functions can be replaced by a class.
class MockResponse:
def __init__(self, json_data, status_code=requests.codes.ok):
self.json_data = json_data
self.status_code = status_code
def json(self):
return self.json_data
With the previous class the file contained the test code of @frankfalse becomes:
import unittest
from unittest import mock
from my_file_01 import func1
import requests
class MockResponse:
def __init__(self, json_data, status_code=requests.codes.ok):
self.json_data = json_data
self.status_code = status_code
def json(self):
return self.json_data
class TestFunc1(unittest.TestCase):
@mock.patch("my_file_01.requests")
def test_func1(self, mock_requests):
print("test func1()")
mock_requests.get.side_effect = [MockResponse({'key1': 'value1'}), MockResponse('Failed', 500)]
[response1, response2] = func1()
print("response1 = " + str(response1))
print("response2 = " + str(response2))
The differences are:
- the mocking functions
request_resp1()andrequest_resp2()can be removed - it is necessary to add:
import requestsfor the presence of the assignmentstatus_code=requests.codes.okin the__init__()method - the instruction for the definition of
side_effectbecomes:
mock_requests.get.side_effect = [MockResponse({'key1': 'value1'}), MockResponse('error', 500)]
Because ith has been created the class MockResponse, the class Mock of the module unittest.mock is not used.
There are a few issues here.
The arguments you're passing to the
@patchdecorator aren't quite right. The target should be specified as a single argument:@patch('MyClass.foo')Your decorated test function will need to accept an additional argument for the patched target - e.g.
mock_foo:@patch('MyClass.foo') def test_foo(self, mock_foo):You can then set the side effect on
mock_foo:@patch('MyClass.foo') def test_foo(self, mock_foo): mock_foo.side_effect = self._side_effect_fooIn addition,
_side_effect_foo()should be made a proper instance method withselfas the first argument:def _side_effect_foo(self, var1): if condition: return new_valueLastly, you should instantiate
MyClasseither inside the test method itself, or inside thesetUp()method:def setUp(self): self.myClass = MyClass() @patch('MyClass.foo') def test_foo(self, mock_foo): mock_foo.side_effect = self._side_effect_foo self.myClass.bar()
An alternative to the accepted solution, more similar to the code in the question, is to use:
@patch.objectto patch an attribute of an objectnew=MagicMock(side_effect=...)to avoid passing the extra argument to the test function
Points 4. and 5. from Will Keeling's answer, about the side effect receiving self and instantiating MyClass in setUp, would still be important.
class TestClass(unittest.TestCase):
def setUp(self):
self.myClass = MyClass()
@patch.object(MyClass, "foo", new=MagicMock(side_effect=_side_effect_foo))
def test_foo(self):
self.myClass.bar()
def _side_effect_foo(self, var1):
if condition:
return new_value