In the new Build Enterprise Automation Framework Series, we will look into detailed explanations on creating enterprise test automation frameworks. Many people starting a new position have this particular assignment, so I think it is crucial to understand how to follow all high-quality standards and practices properly. I am not a fan of reinventing the wheel, so I will encourage you to leverage other proven open-source solutions if they can be used and fit your needs instead of writing your own. Leverage the knowledge and expertise of proven experts in the field. If you follow this advice, this series will be highly useful to you again because you will understand how the test automation frameworks are designed, written, and maintained + how to work internally. The information about the subject will be invaluable to you choosing the right tool for your tests. The series can be even more useful to you if you decide to do everything from scratch.
Article by article, we will discuss different aspects and features of the enterprise test automation frameworks. At the end of the course, we will have a full-fledged enterprise test automation framework. The first article from the series will talk not so much about code but rather on how to define what we need (our requirements), how to do research properly- finding the right tools that we will base our framework, and lastly, creating a detailed list of desired characteristics for our solution grouping all of the various features that we will build in the next articles. In the course, I will use as a demo/example our open-source BELLATRIX test automation framework, which I believe is one of the most feature richest open-source frameworks out there.
In the first article from the series “Enterprise Test Automation Framework: Define Requirements and Characteristics”, we defined our future framework’s requirements and qualities/characteristics. This publication will continue with defining more precisely the various features grouped by the previously described characteristics. They will help us to prioritize and plan our work on the framework. We already looked into a few features in part 1, part 2 and part3. Planning and designing framework modularity is essential for supporting more comfortable usage, maintainability, and extensibility. This is why we created such a plan in the Modularity Planning and Design publication. Next, we incorporated inversion of control containers required for plug-in architecture of the framework. In this article, we will design another crucial module of our framework responsible for retrieving data from configuration files depending on the build configuration, allowing us to have a separate config file for each execution environment - TEST, UAT, PROD, DEV.
Sample Usage and Examples
First, let’s discuss what we want to achieve and why. There are many components of any framework that it is quite useful to be configured on-demand based on its users’ contextual needs. For example, it is controlling how big are the timeouts before failing a particular test. Another useful scenario is to configure paths on the drive where specific files to be stored/read or whether a particular feature is enabled or disabled.
The accepted way to store such info in .NET Core is to use JSON configuration files. Here is a sample one:
{
"timeoutSettings": {
"waitForAjaxTimeout": 60,
"sleepInterval": 1,
"elementToBeVisibleTimeout": 60,
"elementToExistTimeout": 60,
"elementToNotExistTimeout": 60,
"elementToBeClickableTimeout": 60,
"elementNotToBeVisibleTimeout": 60,
"elementToHaveContentTimeout": 15
},
"videoRecordingSettings": {
"isEnabled": true,
"waitAfterFinishRecordingMilliseconds": 500,
"filePath": "ApplicationDataTroubleshootingVideos"
},
"screenshotsSettings": {
"isEnabled": true,
"filePath": "ApplicationDataTroubleshootingScreenshots"
},
"logging": {
"isEnabled": true,
"isConsoleLoggingEnabled": true,
"isDebugLoggingEnabled": true,
"isEventLoggingEnabled": false,
"isFileLoggingEnabled": true,
"outputTemplate": "[{Timestamp:HH:mm:ss}] {Message:lj}{NewLine}",
"addUrlToBddLogging": true
}
}.NET Core provides libraries that allow us to deserialize to C# object all of the JSON sections. We will have separate C# classes with properties having the same names as the JSON. .NET Core will map them automatically for us. Of course, we will create a more user-friendly API to access and use them. But here is a short example how we can read some of the above values.
bool shouldTakeVideos = ConfigurationService.Instance.GetVideoSettings().IsEnabled;As with the IoC container, we use the Singleton design pattern to provide a single instance of the configuration service responsible for providing various methods for accessing the configuration data.
Creating Configuration Infrastructure
As we discussed in previous parts we will put the configuration infrastructure code into a separate project called Bellatrix.Configuration. We will leverage the .NET Core native support of JSON configuration files. First, you need to install the following NuGet packages.
First, you need to install the following NuGet packages:
-
Microsoft.Extensions.Configuration.Json
-
Microsoft.Extensions.Configuration
-
Microsoft.Extensions.Configuration.Binder
Next, we need to create the JSON config file where we will store the test data. I want to be able to configure three different aspects of the framework as we previously discussed. For each of them, I will create a separate section in the JSON file. Create a new JSON file named testFrameworkSettings.json.
NOTE: This file will be placed not in the framework projects but rather in every .Tests project.
For each test environment we will have a separate copy of this file where you can change the data. For example, the initial testFrameworkSettings.json can hold the data for our DEV environment. The testFrameworkSettings.Debug.json for the LOCAL dev environment and so on. The structure of the files is based on the build configurations of your solution. So, for each test environment you will have a separate build configuration. When we change the build configuration the correct file will be copied to the bin folder and read by the code.
Next, we need to edit the MSBuild of the project by double clicking on it (VS 2019). You need to add the following piece.
<ItemGroup>
<None Update="testFrameworkSettings.$(Configuration).json">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</None>
<ItemGroup>Based on the configuration, the right JSON file will be copied to the output directory.
Next part is to create a class that allows us to access the values in the right config file.
ConfigurationService
public sealed class ConfigurationService
{
private static ConfigurationService _instance;
public ConfigurationService()
{
Root = InitializeConfiguration();
}
public static ConfigurationService Instance
{
get
{
if (_instance == null)
{
_instance = new ConfigurationService();
}
return _instance;
};
}
public IConfigurationRoot Root { get; }
private IConfigurationRoot InitializeConfiguration()
{
var builder = new ConfigurationBuilder();
if (string.IsNullOrEmpty(ExecutionContext.SettingsFileContent))
{
var executionDir = ExecutionDirectoryResolver.GetDriverExecutablePath();
var filesInExecutionDir = Directory.GetFiles(executionDir);
var settingsFile =
filesInExecutionDir.FirstOrDefault(x => x.Contains("testFrameworkSettings") && x.EndsWith(".json"));
if (settingsFile != null)
{
builder.AddJsonFile(settingsFile, optional: true, reloadOnChange: true);
}
}
else
{
builder.AddJsonStream(new MemoryStream(System.Text.Encoding.UTF8.GetBytes(ExecutionContext.SettingsFileContent)));
}
builder.AddEnvironmentVariables();
return builder.Build();
}
}Through the static variable _instance and public static property Instance, we implemented the singleton design pattern guaranteeing that only a single instance of the configuration service exists for the execution. Also, we expose the IConfigurationRoot through the public property Root, which we will later use in extension methods responsible for returning the specific objects that are a representation of the JSON sections. Because of the MSBuild code that we put in our project, there will be only a single testFrameworkSettings.json, which we load in the InitializeConfiguration method. After that, we use the .NET Core ConfigurationBuilder class and its method AddJsonStream to load the JSON configuration.
Configuration Extensions
I decided to create a utility class holding methods that can ease the configuration process. We will implement those as extension methods, and we will put them inside the class ConfigurationExtensions. The first method I added is the method NormalizeAppPath, which allows us to use the word “AssemblyFolder” instead to hard-code the actual path to where the assemblies are compiled. This is quite useful since many of the files that we will need are copied after compilation to this particular folder or its children.
public static class ConfigurationExtensions
{
public static string NormalizeAppPath(this string appPath)
{
if (string.IsNullOrEmpty(appPath))
{
return appPath;
}
else if (appPath.StartsWith("AssemblyFolder", StringComparison.Ordinal))
{
var executionFolder = ExecutionDirectoryResolver.GetDriverExecutablePath();
appPath = appPath.Replace("AssemblyFolder", executionFolder);
}
return appPath;
}
}ConfigurationService Usage in Projects
Let’s look into an example how we can use the classes from the new configuration module. For example, we need to expose such settings for the plug-in for video recording on test failure Bellatrix.TestExecutionExtensions.Video. Also, you need to install the NuGet package - Microsoft.Extensions.Logging.Abstractions. This project should reference Bellatrix.Configuration. You need to create a C# POCO class representation of the JSON configuration section you want to map.
public class VideoRecordingSettings
{
public int WaitAfterFinishRecordingMilliseconds { get; set; } = 500;
public string FilePath { get; set; }
public bool IsEnabled { get; set; }
}After that, we create a static class called ConfigurationServiceExtensions under the namespace Bellatrix. There we add an extension method to the ConfigurationService, which will retrieve the VideoRecordingSettings using the singleton of the ConfigurationService. Adding the extension method under the main namespace will allow us to access the method without explicitly writing the using statement.
namespace Bellatrix
{
public static class ConfigurationServiceExtensions
{
public static VideoRecordingSettings GetVideoSettings(this ConfigurationService service)
=> ConfigurationService.Instance
.Root
.GetSection("videoRecordingSettings")
.Get<VideoRecordingSettings>();
}
}After that we can access the video settings in the following manner.
bool shouldTakeVideos = ConfigurationService.Instance.GetVideoSettings().IsEnabled;Summary
In the article, we designed another crucial module of our framework responsible for retrieving data from configuration files depending on the build configuration, allowing us to have a separate config file for each execution environment - TEST, UAT, PROD, DEV. In the next post from the series, we will design a logging module, which is essential for troubleshooting and storing test execution data.
