Testing for Developers- Isolation Frameworks Fundamentals

In the series we will define the basic terms that every developer needs to know about testing. The purpose is to give all team members a shared understanding of the fundamental terminology of the quality assurance and all related processes. Later this will improve the communication and reviews quality. It will further increase the testing capabilities of each member. In this part, we will talk about the more advanced concepts and terminology in unit testing such as mocking and stubbing. In the previous part of the series- we talked about the basic concepts of unit testing.

As part of the professional services we provide at BELLATRIX, we consult companies and help them to improve their QA process and set up an automated testing infrastructure. After the initial review process and giving improvement recommendations for some companies we need to hire new talents that can help the company to scale-up the solutions we provided. This was part of training we did for a company we consulted so that we educate all of their developers.

Introduction

Flying people into space presents interesting challenges to engineers and astronauts, one of the more difficult being how to make sure the astronaut is ready to go into space and operate all the machinery during orbit. A full integration test for the space shuttle would have required being in space, and that’s obviously not a safe way to test astronauts. That’s why NASA built full simulators that mimicked the surroundings of a space shuttle’s control deck, which removed the external dependency of having to be in outer space.

An external dependency is an object in your system that your code under test interacts with and over which you have no control. (Common examples are file systems, threads, memory, time, and so on.)

A stub is a controllable replacement for an existing dependency (or collaborator) in the system. By using a stub, you can test your code without dealing with the dependency directly.

Break Dependency

You can’t test something? Add a layer that wraps up the calls to that something, and then mimic that layer in your tests.

Find the interface that the start of the unit of work under test works against. (In this case, “interface” isn’t used in the pure object-oriented sense; it refers to the defined method or class being collaborated with.)

If the interface is directly connected to your unit of work under test (as in this case—you’re calling directly into the file-system), make the code testable by adding a level of indirection hiding the interface.

Replace the underlying implementation of that interactive interface with something that you have control over. In this case, you’ll replace the instance of the class that your method calls (FileExtensionManager) with a stub class that you can control (StubExtensionManager), giving your test code control over external dependencies.

public List<string> GeneratePiesNamesByCurrentYear()
{
    var brandedPiesNames = new List<string>();
    foreach (var pie in _originalPiesNames)
    {
        brandedPiesNames.Add($"{DateTime.Now.Year} {pie}");
    }
    return brandedPiesNames;
}

DateTime.Now is our real dependency.

public interface IDateTimeFacade
{
    DateTime GetCurrentDateTime();
}

Now we replace the dependency. 

public List<string> GeneratePiesNamesByCurrentYear()
{
    var brandedPiesNames = new List<string>();
    foreach (var pie in _originalPiesNames)
    {
        brandedPiesNames.Add($"{_dateTimeFacade.GetCurrentDateTime().Year} {pie}");
    }
    return brandedPiesNames;
}

Refactoring is the act of changing code without changing the code’s functionality.

That is, it does exactly the same job as it did before. No more and no less. It just looks different. A refactoring example might be renaming a method and breaking a long method into several smaller methods.

Seams are places in your code where you can plug in different functionality.

Seams are places in your code where you can plug in different functionality, such as stub classes, adding a constructor parameter, adding a public settable property, making a method virtual so it can be overridden, or externalizing a delegate as a parameter or property so that it can be set from outside a class. Seams are what you get by implementing the Open-Closed Principle, where a class’s functionality is open for extenuation, but its source code is closed for direct modification.

There are two types of dependency-breaking refactorings, and one depends on the other. I call them Type A and Type B refactorings.

Type A Refactoring- abstracting concrete objects into interfaces or delegates.

Type B Refactoring- refactoring to allow injection of fake implementations of those delegates or interfaces.

public class AlwaysValidFakeExtensionManager : IExtensionManager
{
    public bool IsValid(string fileName)
    {
        return true;
    }
}

It’s not StubExtensionManager or MockExtensionManager. It’s FakeExtensionManager. A fake denotes an object that looks like another object but can be used as a mock or a stub.

This fake extension manager will always return true, so name the class AlwaysValidFakeExtensionManager, so that the reader of your future test will understand what will be the behavior of the fake object, without needing to read its source code.

classDiagram
    IExtensionManager <|.. AlwaysValidFakeExtensionManager
    LogAnalyzer --> IExtensionManager
    class IExtensionManager {
        <<interface>>
        +IsValid(string fileName) bool
    }
    class AlwaysValidFakeExtensionManager {
        +IsValid(string fileName) bool
    }
    class LogAnalyzer {
        -IExtensionManager manager
        +IsValidLogFileName(string fileName) bool
    }

Injection Types

Here are some of the most notable ways:

Receive an interface at the constructor level and save it in a field for later use.

Constructor Level

private readonly ILogger<TestCaseRunsController> _logger;
private readonly MeissaRepository _meissaRepository;
public TestCaseRunsController(ILogger<TestCaseRunsController> logger, MeissaRepository repository)
{
    _logger = logger;
    _meissaRepository = repository;
}

Property Level

public TestCaseRunsController(ILogger<TestCaseRunsController> logger, MeissaRepository repository)
{
    _logger = logger;
}
public MeissaRepository MeissaRepository { get; set; }

Method Parameter Level

public async Task SetAllActiveAgentsToVerifyTheirStatusAsync(IServiceClient<TestAgentDto> testAgentRepository, string tag)
{
    var testAgents = await GetAllActiveTestAgentsByTagAsync(tag);
    if (testAgents.Count > 0)
    {
        await UpdateAgentsStatusAsync(testAgents, TestAgentStatus.RequestActiveConfirmation);
    }
}

Receive an interface as a property get or set and save it in a field for later use.

Local Factory Method

public async Task SetAllActiveAgentsToVerifyTheirStatusAsync(string tag)
{
    var repo = CreateTestAgentRepo();
    var testAgents = await repo.GetAllActiveTestAgentsByTagAsync(tag);
    if (testAgents.Count > 0)
    {
        await UpdateAgentsStatusAsync(testAgents, TestAgentStatus.RequestActiveConfirmation);
    }
}
private IServiceClient<TestAgentDto> CreateTestAgentRepo() => new TestAgentServiceClient();

Factory Class

public class TestAgentRepoFactory : ITestAgentRepoFactory
{
    public IServiceClient<TestAgentDto> CreateTestAgentRepo()
    {
        return new TestAgentServiceClient();
    }
}

The constructor then sets a local field of the interface type in the class for later use by your method or any other. The fake extension manager is located in the same file as the test code because currently the fake is used only from within this test class.

classDiagram
    IExtensionManager <|.. FakeExtensionManager
    LogAnalyzer --> IExtensionManager
    class IExtensionManager {
        <<interface>>
        +IsValid(string fileName) bool
    }
    class FakeExtensionManager {
        +WillBeValid bool
        +IsValid(string fileName) bool
    }
    class LogAnalyzer {
        -IExtensionManager manager
        +LogAnalyzer(IExtensionManager mgr)
        +IsValidLogFileName(string fileName) bool
    }

internal class FakeExtensionManager : IExtensionManager
{
    public bool WillBeValid = false;
    public bool IsValid(string fileName)
    {
        return WillBeValid;
    }
}
public class LogAnalyzer
{
    private IExtensionManager manager;
    public LogAnalyzer(IExtensionManager mgr)
    {
        manager = mgr;
    }
    public bool IsValidLogFileName(string fileName)
    {
        return manager.IsValid(fileName);
    }
}
public interface IExtensionManager
{
    bool IsValid(string fileName);
}

public void IsValidFileName_NameSupportedExtension_ReturnsTrue()
{
    FakeExtensionManager myFakeManager = new FakeExtensionManager();
    myFakeManager.WillBeValid = true;
    LogAnalyzer log = new LogAnalyzer(myFakeManager);
    bool result = log.IsValidLogFileName("short.ext");
    Assert.True(result);
}

Constructor Injection Problems

Problems can arise from using constructors to inject implementations. If your code under test requires more than one stub to work correctly without dependencies, adding more and more constructors (or more and more constructor parameters) becomes a hassle, and it can even make the code less readable and less maintainable.

Solution: One solution is to create a special class that contains all the values needed to initialize a class and to have only one parameter to the method: that class type. That way, you only pass around one object with all the relevant dependencies. (This is also known as a parameter object refactoring.)

Property Injection

classDiagram
    IExtensionManager <|.. FileExtensionManager
    LogAnalyzer --> IExtensionManager
    class IExtensionManager {
        <<interface>>
        +IsValid(string fileName) bool
    }
    class FileExtensionManager {
        +IsValid(string fileName) bool
    }
    class LogAnalyzer {
        +ExtensionManager IExtensionManager
        +IsValidLogFileName(string fileName) bool
    }

Using properties to inject dependencies. This is much simpler than using a constructor because each test can set the properties that it needs to get the test underway.

public class LogAnalyzer
{
    private IExtensionManager manager;
    public LogAnalyzer()
    {
        manager = new FileExtensionManager();
    }
    public IExtensionManager ExtensionManager
    {
        get { return manager; }
        set { manager = value; }
    }
    public bool IsValidLogFileName(string fileName)
    {
        return manager.IsValid(fileName);
    }
}

public void IsValidFileName_SupportedExtension_ReturnsTrue()
{
    //set up the stub to use, make sure it returns true
    // ...
    //create analyzer and inject stub
    LogAnalyzer log = new LogAnalyzer();
    log.ExtensionManager = someFakeManagerCreatedEarlier;
    //Assert logic assuming extension is supported
    //...
}

When to Use?

Use this technique when you want to signify that a dependency of the class under test is optional or if the dependency has a default instance created that doesn’t create any problems during the test.

Factory Class

A test configures the factory class to return a stub object. The class under test uses the factory class to get that instance, which in production code would return an object that isn’t a stub.

classDiagram
    IExtensionManager <|.. FileExtensionManager
    IExtensionManager <|.. StubExtensionManager
    ExtensionManagerFactory --> IExtensionManager
    LogAnalyzer --> ExtensionManagerFactory
    class IExtensionManager {
        <<interface>>
        +IsValid(string fileName) bool
    }
    class ExtensionManagerFactory {
        +Create() IExtensionManager
    }
    class LogAnalyzer {
        +IsValidLogFileName(string fileName) bool
    }

The only thing you need to make sure of is that once you use these patterns, you add a seam to the factories you make so that they can return your stubs instead of the default implementations.

Extract and Override

In this scenario, you use a local virtual method in the class under test as a factory to get the instance of the extension manager. Because the method is marked as virtual, it can be overridden in a derived class, which creates your seam.

You inherit from the class under test so you can override its virtual factory method and return whatever you want, as long as it implements IExtensionManager. Then you perform your tests against the newly derived class.

public class DateTimeProvider : IDateTimeProvider
{
    public DateTime GetCurrentTime() => DateTime.Now;
}

But why stop there? What if you’re unable or unwilling to add a new interface every time you need control over some behavior in your code under test? In those cases, Extract and Override can help simplify things, because it doesn’t require writing and introducing new interfaces—just deriving from the class under test and overriding some behavior in the class.

Internal Instead of Public

public class LogAnalyzer
{
    //...
    internal LogAnalyzer(IExtensionManager extentionMgr)
    {
        manager = extentionMgr;
    }
    //...
}
using System.Runtime.CompilerServices;
[assembly:
InternalsVisibleTo("Meissa.Infrastructure")]

If the build flag is not present during the build, the callers to the annotated method won’t be included in the build. For example, this method will have all the callers to it removed during a release build, but the method itself will stay on.

[Conditional("DEBUG")]
public string GetRunningAssemblyPath()
{
    string codeBase = Assembly.GetExecutingAssembly().CodeBase;
    var uri = new UriBuilder(codeBase);
    string path = Uri.UnescapeDataString(uri.Path);
    return Path.GetDirectoryName(path);
}

It’s important to note that using conditional compilation constructs in your production code can reduce its readability and increase its “spaghetti-ness.” Beware!

Putting your methods or special test-only constructors between #if and #endif constructs will make sure they compile only when that build flag is set, as shown in the next listing.

#if DEBUG
public LogAnalyzer (IExtensionManager extensionMgr)
{
manager = extensionMgr;
}
#endif

This method is commonly used, but it can lead to code that looks messy.

Types of Testing

Value-based testing checks the value returned from a function. 

State-based testing is about checking for noticeable behavior changes in the system under test, after changing its state.

Interaction testing is testing how an object sends messages (calls methods) to other objects. You use interaction testing when calling another object is the end result of a specific unit of work.

You can also think of interaction testing as being action-driven testing. Action-driven testing means that you test a particular action an object takes (such as sending a message to another object). Always choose to use interaction testing only as the last option. But sometimes, as is the case of a third-party call to a logger, interactions between objects are the end result. That’s when you need to test the interaction itself.

Stubs vs Mocks

A mock object is a fake object in the system that decides whether the unit test has passed or failed. It does so by verifying whether the object under test called the fake object as expected.

But it’s a smarter breed of stub—a stub that records the calls made to it, and you use it to define if your test passed or not. That’s partly what a mock object is. There’s usually no more than one mock per test.

Fake- generic term that can be used to describe either a stub or a mock object.

A fake is a generic term that can be used to describe either a stub or a mock object (handwritten or otherwise), because they both look like the real object. Whether a fake is a stub or a mock depends on how it’s used in the current test. If it’s used to check an interaction (asserted against), it’s a mock object. Otherwise, it’s a stub.

When using a stub the assert is performed on the class under test. The stub aids in making sure the test runs smoothly.

The class under test communicates with the mock object, and all communication is recorded in the mock. The test uses the mock object to verify that the test passes.

This handwritten class implements an interface, as a stub does, but it saves some state for later, so that your test can then assert and verify that your mock was called correctly.

Spy- according to xUnit Test Patterns: Refactoring Test Code by Gerard Meszaros, this would be called a Test Spy.

Multiple Stubs

public async Task InsertCurrentTestAgent_When_TestAgentForCurrentMachineIsNotExistingInDatabase()
{
    // Arrange
    var testAgents = TestAgentFactory.CreateWithoutCurrentMachineName(TestAgentStatus.Inactive);
    var insertedTestAgent = default(Task<TestAgentDto>);
    _testAgentRepositoryMock.Setup(x => x.GetAllAsync()).Returns(Task.FromResult(testAgents));
    _testAgentRepositoryMock.Setup(x => x.CreateAsync(It.IsAny<TestAgentDto>())).
    Returns((Task<TestAgentDto> a) => insertedTestAgent = a);
    // Act
    await _testAgentStateSwitcher.SetTestAgentAsActiveAsync(testAgents.First().AgentTag);
    // Assert
    // ...
}

As you’ll see, it’s perfectly OK to have multiple stubs in a single test, but more than a single mock can mean trouble, because you’re testing more than one thing.

Multiple Asserts

Assert.That(insertedTestAgent.Result.TestAgentId, Is.Not.Null);
Assert.That(insertedTestAgent.Result.AgentTag, Is.EqualTo(testAgents.First().AgentTag));
Assert.That(insertedTestAgent.Result.MachineName, Is.EqualTo(Environment.MachineName));
Assert.That(insertedTestAgent.Status, Is.EqualTo(TestAgentStatus.Active));

Having several asserts can sometimes be a problem, because the first time an assert fails in your test, it actually throws a special type of exception that is caught by the test runner. That also means no other lines below the line that just failed will be executed. In this current case, it’s OK, because if one assert fails, you don’t care about the others because they’re all related to the same object, and they’re part of the same “feature.” If you cared about the other asserts being run even if the first one failed, it would be a good indication to you to break this test into multiple tests.

Multiple Mocks

_testRunRepositoryMock.Verify(x => x.UpdateAsync(It.IsAny<int>(),
It.Is<TestRunDto>(i => i.TestRunId == _testRunId && i.Status == status)), Times.Once);
_testRunRepositoryMock.Verify(x => x.UpdateAsync(It.IsAny<int>(),
It.IsAny<TestRunDto>()), Times.Once);

Having more than one mock per test usually means you’re testing more than one thing, and this can lead to complicated or brittle tests. Overspecification is the act of specifying too many things that should happen that your test shouldn’t care about; for example, that stubs were called. These extra specifications can make your test fail for all the wrong reasons.

Manual Mocks

There are several issues that crop up when using manual mocks and stubs:

It takes time to write the mocks and stubs.

It’s difficult to write stubs and mocks for classes and interfaces that have many methods, properties, and events.

To save state for multiple calls of a mock method, you need to write a lot of boilerplate code within the handwritten fakes.

Isolation Framework

An isolation framework is a set of programmable APIs that makes creating fake objects much simpler, faster, and shorter than hand-coding them.

A dynamic fake object is any stub or mock that’s created at runtime without needing to use a handwritten (hardcoded) implementation of that object.

Moq Isolation Framework

Install-Package Moq

You’ll use to generate fakes at runtime. Because Moq is a constrained framework, it works best with interfaces. For real classes, it will only work with nonsealed classes, and for those, it will only be able to fake virtual methods.

Moq Initialize

private Mock<IServiceClient<TestAgentDto>> _testAgentRepositoryMock;
private ITestAgentsService _testAgentsService;

public void TestInit()
{
    _testAgentRepositoryMock = new Mock<IServiceClient<TestAgentDto>>();
    _testAgentsService = new TestAgentsService(_testAgentRepositoryMock.Object);
}

Moq Stubs

_directoryProvider.Setup(x => x.DoesDirectoryExists(It.IsAny<string>())).Returns(true);
_dateTimeProvider.Setup(x => x.GetCurrentTime()).Returns(DateTime.Now);

Moq Async Stubs

var logs = LogFactory.CreateEmpty();
_logRepositoryMock.Setup(x => x.GetAllAsync()).Returns(Task.FromResult(logs));

Moq Stub Lambdas

_pathProvider.Setup(x => x.Combine(It.IsAny<string>(), It.IsAny<string>())).
Returns((string path1, string filePath) => Path.Combine(path1, filePath));
_fileProvider.Setup(x => x.WriteAllText(It.IsAny<string>(), It.IsAny<string>())).
Callback((string filePath, string content) => File.WriteAllText(filePath, content));

Moq Throw Exceptions

Mock<IFileConnection> fileConnection = new Mock<IFileConnection>();
fileConnection.Setup(item => item.Get(It.IsAny<string>, It.IsAny<string>))
.Throws(new IOException());

Raising Events from Mock Objects

var mockHeater = new Mock<IHeater>();
var mockThermostat = new Mock<IThermostat>();
mockThermostat.Setup(m => m.StartAsyncSelfCheck()).Raises(
m => m.HealthCheckComplete += null, new ThermoEventArgs { OK = false });
var controller = new Services.HeatingController(mockHeater.Object, mockThermostat.Object);
// Act
controller.PerformHealthChecks();
// Assert
mockHeater.Verify(m => m.SwitchOff());

Delegate Fired Manually

public void EventFiringManual()
{
    bool loadFired = false;
    SomeView view = new SomeView();
    view.Load += delegate
    {
        loadFired = true;
    };
    view.DoSomethingThatEventuallyFiresThisEvent();
    Assert.IsTrue(loadFired);
}

Raising Events in Response to Expectations

// Arrange
var mockHeater = new Mock<IHeater>();
var mockThermostat = new Mock<IThermostat>();
mockThermostat.Setup(m => m.StartAsyncSelfCheck()).Raises(
m => m.HealthCheckComplete += null, new ThermoEventArgs { OK = false });
var controller = new Services.HeatingController(mockHeater.Object, mockThermostat.Object);
// Act
controller.PerformHealthChecks();
// Assert
mockHeater.Verify(m => m.SwitchOff());
// Arrange
var mockHeater = new Mock<IHeater>();
var mockThermostat = new Mock<IThermostat>();
mockThermostat.Setup(m => m.StartAsyncSelfCheck()).Raises(
m => m.HealthCheckComplete += null, new ThermoEventArgs { OK = true });
var controller = new Services.HeatingController(mockHeater.Object, mockThermostat.Object);
// Act
controller.PerformHealthChecks();
// Assert
mockHeater.Verify(m => m.SwitchOff(), Times.Never());

References

The Art of Unit Testing, Second Edition - Manning

Refactoring: Improving the Design of Existing Code 2nd Edition

Working Effectively with Legacy Code (Michael C. Feathers)