In the previous article, we discussed what unit tests are, the purpose of unit tests, and the basic concepts of MSpec and xUnit tests. In this article, we will go through a simple .Net console application that we have created to demonstrate the implementation of MSpec and xUnit test cases. We will discuss how we usually write our code and what we should do to write code that is adaptable to unit testing. The code is available on GitHub. Please click here to view the application code.
Our application is a simple web scraping app that fetches the HTML content from a Wikipedia URL and outputs the content in the console. Our codebase contains two types of implementation of the web scraper. One is implemented in a simple way which we follow when writing code without considering unit tests. This code can be found in the CodeNotAdapatableToUnitTest folder of the GitHub link. The other implementation that can be found in the Code folder contains the code structure that we follow when are considering making our codebase adaptable to unit tests. The folder Domain contains the class which is shared between the two types of implementation. We can find the MSpec tests in MSpec folder and the xUnit tests in XUnit folder. The file Program.cs is the entry point to our application. It contains the required dependencies and manages the dependency injection in our application. It also instantiates the base WebScraper class and displays the final output in the console.
To keep the demonstration simple for this tutorial, I have not implemented unit tests for all the classes. I have just written the unit tests for the WebScraper.cs class now. However, in a later tutorial, I am planning to discuss the implementation of unit tests for other classes of this project as well.
You will notice that I have used threading-related Tasks to synchronize the async methods. I followed this approach so that I can demonstrate how we write unit tests for synchronous methods first. Once we acquire a strong grasp on unit tests, we can modify our code to write unit tests for async methods as well. Whenever you see the following type of code:
var task = Task.Run(() => {
try
{
return client.GetAsync(url);
}
catch (Exception e)
{
Console.WriteLine($"Http Get request error for url {url}.");
Console.WriteLine(e.Message);
Console.WriteLine(e.StackTrace);
return null;
}
});
task.Wait();
var response = task.Result;
ignore the complication and assume that we are just synchronously fetching responses from an asynchronous method.
Filename: WebScraper.cs
As discussed earlier, the WebScraper file makes a GET request to a specific URL and returns the result. To make the request, we use the HttpClientFactory object provided by C#.
First, we declare our constructor and inject the HttpClientFactory object.
readonly IHttpClientFactory _httpClientFactory;
public const string ScrapingUrl = "https://en.wikipedia.org/wiki/Unit_testing";
public WebScraper(IHttpClientFactory httpClientFactory)
{
_httpClientFactory = httpClientFactory;
}
We then use this object to create a HttpClient object. We can use that to make our HTTP requests using the GetAsync() method.
HttpResponse get(string url, HttpClient client)
{
var task = Task.Run(() => {
try
{
return client.GetAsync(url);
}
catch (Exception e)
{
Console.WriteLine($"Http Get request error for url {url}.");
Console.WriteLine(e.Message);
Console.WriteLine(e.StackTrace);
return null;
}
});
task.Wait();
var response = task.Result;
}
We obtain the result as HttpResponseMessage. If the status code of the HTTP response is 200, we then extract the content of the result as a string. Then we create our own HttpResponse object and return the data. Otherwise, we return null.
HttpResponse get(string url, HttpClient client)
{
// making a request
var response = task.Result;
if (response != null && response.IsSuccessStatusCode)
{
var responseGenerator = Task.Run(() => {
try
{
var content = response.Content.ReadAsStringAsync();
return content;
}
catch (Exception e)
{
Console.WriteLine($"Error creating response for the request sent to {url}");
}
return null;
});
responseGenerator.Wait();
var contentString = responseGenerator.Result;
return new HttpResponse()
{
StatusCode = response.StatusCode,
Content = contentString
};
}
else
{
if (response == null) Console.Write($"No response returned for url {url}");
else if (!response.IsSuccessStatusCode) Console.WriteLine($"Unsuccessful request sent to {url}. Status code: {((int)response.StatusCode)}");
}
return null;
}
The Scrape() method of the file, is the main public method that creates the client and returns the content from the web.
public string Scrape()
{
string result = null;
try
{
using (var client = _httpClientFactory.CreateClient())
{
var response = get(ScrapingUrl, client);
if (response.IsSuccessStatusCode)
{
Console.WriteLine("Fetched web content:");
Console.WriteLine(response.Content);
result = response.Content;
}
else
{
var message = "Client returned an error";
if (response != null) message = $"Client returned a status code of {response.StatusCode}";
Console.WriteLine($"Could not retrieve article of {ScrapingUrl}. {message}");
}
}
} catch(Exception e)
{
Console.WriteLine($"Received an exception: {e.Message}");
Console.WriteLine(e.StackTrace);
}
return result;
}
}
This approach seems fine when we are writing code without testing them. But when we start to write unit tests for this code, we may run into different types of issues.
One of the best practices for writing unit tests is that we should write our test in a completely isolated environment. When testing a method, we should make sure that other public methods are not executed. Instead, we should mock those methods and return a result to prevent them from executing. This makes the scope of our test smaller and makes it easier for us to cover different scenarios of the method.
In our case, we will be writing the test cases for our Scrape() method. This makes Scrape() our test subject. Since the get() method is a private method executed by our test subject we will bring it within our test scope as well.
However, we have two other public methods:
a. _httpClientFactory.CreateClient(): This method is executed by HttpClientFactory to return the HttpClient object.
b. client.GetAsync(url): This method is executed by HttpClient object to make a GET request.
Since these are public methods executed by different objects, we should mock these methods and not let them execute. They should have their separate test cases.
In the first case, we inject the HttpClientFactory as an interface (IHttpClientFactory) rather than an object. This enables us to mock the method of this interface easier.
However, in the second case, HttpClient is not an interface but an instance of an object. There are third-party libraries that enable us to mock the methods of this object but we will ignore them for this article. There are ways we can mock the internal functionality of the GetAsync() method, by mocking its MockHandler method but this process should also be avoided as it is not supported by many unit testing frameworks.
So, the approach that we should follow is to turn the HttpClient into an interface and then use the interface to mock its methods.
There can be different ways we can achieve the above target. The approach that I have followed is to wrap the HttpClient with a wrapper class that implements an interface. We then inject the wrapper instance into our WebScraper object. It enables us to mock the methods of the wrapper object without having to mock the HttpClient object directly.
The file HttpClientWrapper.cs is contains our wrapper class. It is an implementation of the IHttpClientWrapper interface that we have created. It receives an IHttpClientFactory instance in its constructor and creates a HttpClient object. We have implemented it as an IDisposable object so that we can dispose the HttpClient when the wrapper gets disposed. In this class we have implemented a Get method that receives a url, fetches its content and returns a HttpResponse.
public interface IHttpClientWrapper : IDisposable
{
HttpResponse Get(string url);
}
public class HttpClientWrapper : IHttpClientWrapper
{
readonly HttpClient _client;
public HttpClientWrapper(IHttpClientFactory httpClientFactory)
{
_client = httpClientFactory.CreateClient();
}
public HttpResponse Get(string url)
{
var task = Task.Run(() => {
try
{
return _client.GetAsync(url);
}
catch (Exception e)
{
Console.WriteLine($"Http Get request error for url {url}.");
Console.WriteLine(e.Message);
Console.WriteLine(e.StackTrace);
return null;
}
});
task.Wait();
var response = task.Result;
if (response != null && response.IsSuccessStatusCode)
{
var responseGenerator = Task.Run(() => {
try
{
var content = response.Content.ReadAsStringAsync();
return content;
} catch(Exception e)
{
Console.WriteLine($"Error creating response for the request sent to {url}");
}
return null;
});
responseGenerator.Wait();
var contentString = responseGenerator.Result;
return new HttpResponse() {
StatusCode = response.StatusCode,
Content = contentString
};
} else
{
if (response == null) Console.Write($"No response returned for url {url}");
else if (!response.IsSuccessStatusCode) Console.WriteLine($"Unsuccessful request sent to {url}. Status code: {((int)response.StatusCode)}");
}
return null;
}
public void Dispose()
{
if(_client != null) _client.Dispose();
}
}
We will use this wrapper to make the Http Request in our WebScraper class and mock the method in our unit test.
We then created a HttpRequestHandler.cs class that creates an HttpClientWrapper wrapper instance and returns the instance.
public interface IHttpRequestHandler {
IHttpClientWrapper CreateClient(IHttpClientFactory clientFactory);
}
public class HttpRequestHandler : IHttpRequestHandler
{
public HttpRequestHandler()
{
}
public IHttpClientWrapper CreateClient(IHttpClientFactory clientFactory)
{
return new HttpClientWrapper(clientFactory);
}
}
We finally refactored our WebScraper.cs class so that it receives the HttpRequestHandler interface during initialization. We can then use this object to create an implementation of HttpClientWrapper instance to make our requests.
public string Scrape()
{
string result = null;
try
{
using (var client = _requestHandler.CreateClient(_httpClientFactory))
{
var response = client.Get(ScrapingUrl);
if (response.IsSuccessStatusCode)
{
Console.WriteLine("Fetched web content:");
Console.WriteLine(response.Content);
result = response.Content;
}
else
{
var message = "Client returned an error";
if (response != null) message = $"Client returned a status code of {response.StatusCode}";
Console.WriteLine($"Could not retrieve article of {ScrapingUrl}. {message}");
}
}
}
catch (Exception e)
{
Console.WriteLine($"Received an exception: {e.Message}");
Console.WriteLine(e.StackTrace);
}
return result;
}
Since we are now dealing with only interfaces in our WebScraper.cs file we can easily write unit tests for our Scrape() method and mock other public methods that we have executed inside it. It will help us to keep the environment of our test isolated and the scope small.
In this article, we have seen how to write code that is adaptable to unit tests. We have seen what considerations we should keep in mind when writing our classes and methods. Finally, we have also seen the power of interfaces. It is true that in our naive approach we had just one class and now we have to deal with multiple classes. But this is a small tradeoff considering the advantages we will have when we finally write our test cases. In the next article, we will write our MSpec and xUnit test cases. We will finally compare both the testing framework and discuss some of its advantages and limitations. Please feel free to share your feedback and queries. I would love to hear your thoughts and suggestions. See you in the next article!