Introduction

We will be using configuration files in our projects so frequently that, without a configuration file, it becomes difficult to change the code each time.

Why do we need a configuration file?

Imagine I have created a project that runs on a server, with two environments: Development and Testing.

For development, I have a specific set of values for my user account, server host and port, log in details, database credentials, and data files read from storage. Similarly, for testing, I have a different set of values that need to be loaded when running the application or server.

Without a configuration file, we would have to make changes within the code in various files or places every time we switch environments. If we change database login details, user account login details, or storage file paths, we must find where these details are referenced and modify them accordingly in the code. Frequent code changes can lead to errors because you might forget where to make changes, which could cause issues when deploying the application.

Even if you are very careful and skilled at handling the code, remember that you are not the only person working on the project.

Suppose, you have a single configuration file that stores all necessary information, and your code is designed to read from this file at startup—loading the values into memory—you can easily update the configuration whenever needed without altering the code itself. This makes configuration files essential for managing your application settings efficiently and safely, allowing you to adjust settings without the risk of introducing errors into the codebase.

Configuration file sample

[General]
debug = True
log_level = info
log_file = /var/log/app.log

[Database]
db_name = example_db
db_host = localhost
db_port = 5432

[Server]
host_name = 0.0.0.0
port = 7896

The configuration file can be stored in various formats. Some common ones include:

config.ini
config.txt
config.cfg

Let us write a program config_reader.py to load the configuration file and access the values in it.

#config_reader.py

import configparser

# Create a ConfigParser object
config = configparser.ConfigParser()

# Read the configuration file
config.read('config.ini')

# Display available sections in the config file
def get_available_sections(config):
    return config.sections()

# Function to search for a key in the config file
def get_config_value(config, section, key):
    # Check if the section exists
    if section not in config:
        raise ValueError(f"Error: Section '{section}' not found in the configuration file.")

    # Check if the key exists in the section
    if key not in config[section]:
        raise KeyError(f"Error: Key '{key}' not found in section '{section}'.")

    # Retrieve and return the value if the section and key are valid
    return config.get(section, key)

# Example usage: search for a key
import configparser

# Create a ConfigParser object
config = configparser.ConfigParser()

# Read the configuration file
config.read('config.ini')

try:
    section = "Database"
    key = "db_name"
    sections = get_available_sections(config)
    print(sections)
    value = get_config_value(config, section, key)
    print(f"Value for '{key}' in section '{section}': {value}")
except (ValueError, KeyError) as e:
    print(e)

The first program reads the configuration file config.ini and displays its sections, which include General, Database, and Server.

The function get_available_sections accepts the config object of the ConfigParser class and prints the available sections of the config.ini file it read. [General, Database, Server] are the sections in the config file here.

The function get_config_value accepts the section name through the section variable and the key name of the value you want to search for via the key variable.

It first checks whether the specified section exists in the configuration file. If the section is not found, it raises a ValueError.
If the section matches, it then searches for the key within the keys of that section. If the key is not found, it raises a KeyError.

Testing a Configuration file

The program works. Let us write a few test cases for this program using Pytest and Unittest. To effectively test this program, we need to:

Mock the configuration file to avoid dependency on an actual file system.
Provide dummy configuration data for controlled testing.
Test various scenarios including successful retrievals and expected errors.

Let us first write a Fixture using Pytest.

What Are Pytest Fixtures?

Imagine you’re building a model rocket for a science project. Before you launch it, you need to prepare certain things: the rocket itself, the launch pad, and maybe some tools. Each time you want to test your rocket, you need these items set up in the same way.

In the world of programming and testing, fixtures are like those preparations. They are special pieces of code in a testing framework called pytest that help set up everything you need before running your tests.

How Do They Work?

Setup: A fixture sets up any resources you need for your test, like creating a database, opening a file, or initializing a web application.
Teardown: After your test runs, the fixture can clean up or remove anything it created, just like putting away your rocket and tools after the test flight.

Why Use Fixtures?

Reusability: You can use the same fixture for multiple tests, so you don’t have to write the setup code again and again.
Clarity: They make your tests easier to read and understand because all the setup happens in one place.

# test_config_reader_pytest.py

import pytest
from unittest.mock import patch
import configparser
from config_reader import get_config_value #importing the function from our config_reader file

@pytest.fixture
def mock_config():
    # Dummy configuration data
    dummy_config = """
    [Database]
    host = localhost
    port = 3306
    user = root
    password = secret

    [Settings]
    debug = true
    logfile = /var/log/app.log
    """
    config = configparser.ConfigParser()
    config.read_string(dummy_config)
    return config

@pytest.fixture Decorator: This decorator marks the function mock_config as a fixture, allowing you to reuse it across different test functions.
Dummy Configuration Data: The string dummy_config simulates a configuration file with two sections: Database and Settings. Each section contains some key-value pairs. This allows us to create a controlled environment for testing the behavior of configparser without needing to rely on an actual file. As we are simulating the data, we need not take the exact number of sections here. All we want to do is test the program with some sample data. Once the testing is successful that means our main program runs successfully with original data without any issues.
ConfigParser Object: The configparser.ConfigParser() object is used to parse the configuration string. Instead of reading from a file, the method config.read_string(dummy_config) reads from the string containing the configuration data.
Return the Config Object: The fixture returns the config object. This object can then be used in the test cases to simulate the behavior of reading an actual configuration file.

Let us add a test case now. The very first output in our program config_reader.py is displaying available sections of the configuration file. I want to write a test case to test this behavior now.

Testing Sections of a configuration file

# test_config_reader.py
###############################
PYTEST FIXTURE CODE HERE
##############################

def test_available_sections(mock_config):
    """Test that available sections are correctly listed."""
    sections = get_available_sections(mock_config)
    assert sections == ['Database', 'Settings']

def test_available_sections(mock_config):
- mock_config: This is a fixture that provides a mocked configuration object (an instance of ConfigParser). The mock simulates the behavior of a real configuration object, allowing for controlled testing.
sections = get_available_sections(mock_config)
- This line calls the function get_available_sections, passing the mock_config object as an argument.
- The function is expected to return a list of sections that are available in the mock configuration object. For this test, the expected sections are predefined in the mock configuration object.
assert sections == ['Database', 'Settings']
- This line checks whether the value returned from get_available_sections matches the expected list ['Database', 'Settings'].
- If the returned value equals the expected list, the assertion passes, indicating that the test was successful. If not, it fails, signaling that there is an issue with how sections are being retrieved.

If everything goes well, you will see the test case passed. We are comparing the sections returned by the mock_config object (mocked Configuration object to dummy data inside fixture) with [‘Database’, ‘Settings’]. If you change the dummy data sections to any different values or simply add an extra section then this test case fails.

Testing existing keys in a Configuration file

I want to test if the get_config_value() function of our program in config_reader.py returns correct key if it is existing in the sections. In this example, it checks for the key 'host' in the 'Database' section.

# test_config_reader.py

def test_get_existing_key(mock_config):
    """Test retrieving an existing key from a valid section."""
    value = get_config_value(mock_config, 'Database', 'host')
    assert value == 'localhost'

def test_get_existing_key(mock_config):
- mock_config: This is a fixture that provides a mocked configuration object, which simulates the behavior of a real configuration parser (like ConfigParser). This allows the test to run without needing actual configuration files.
value = get_config_value(mock_config, 'Database', 'host')
- This line calls the get_config_value function, passing three arguments:
  1. mock_config: The mocked configuration object.
  2. 'Database': The section from which we want to retrieve a value.
  3. 'host': The key for which we want to get the value.
- The function is expected to return the value associated with the key 'host' in the 'Database' section. In the context of this test, we expect that the mock configuration has been set up to return 'localhost' when this key is requested.
assert value == 'localhost'
- This line checks if the retrieved value matches the expected value, which is 'localhost'.
- If the value returned from get_config_value equals 'localhost', the assertion passes, meaning the test is successful. If not, the assertion fails, indicating a problem with how the value is being retrieved. Here, it passes because the value of host key in section Database is localhost.

Testing Exception behavior

We know that if a non-existing section is passed or an invalid section is passed to the get_config_value function then it raises a ValueError. Similarly, if a non-existing key is passed the function get_config_value function raises KeyError. Let us pass an invalid section and key to test whether the function is actually raising the exceptions or not.

#test_config_reader.py 

# Test case for invalid section
def test_get_nonexistent_section(mock_config):
    with pytest.raises(ValueError):
        get_config_value(mock_config, 'InvalidSection', 'host')

# Test case for invalid key
def test_get_nonexistent_key(mock_config):
    with pytest.raises(KeyError):
        get_config_value(mock_config, 'Database', 'nonexistent_key')

def test_get_nonexistent_section(mock_config):
- mock_config: This is a fixture that provides a mocked configuration object. This object simulates the behavior of a real configuration parser, allowing you to test how the function handles various scenarios without needing actual configuration files.
with pytest.raises(ValueError):
- This line sets up a context manager that expects a ValueError to be raised during the execution of the code within its block. As we are running get_config_value() inside this context manager, if the function raises ValueError, the test will pass; otherwise, the test will fail.
get_config_value(mock_config, 'InvalidSection', 'host'):
- Inside the with block, this line calls the get_config_value function with three arguments:
  - mock_config: The mocked configuration object.
  - 'InvalidSection': A section name that does not exist in the mock configuration.
  - 'host': The key for which we want to retrieve a value.
- Since 'InvalidSection' is not a valid section (not available) in the mocked configuration, the function is expected to raise a ValueError.
def test_get_nonexistent_key(mock_config):
- mock_config: This is a fixture that provides a mocked configuration object. This object simulates the behavior of a real configuration parser, allowing you to test how the function handles various scenarios without needing actual configuration files.
with pytest.raises(KeyError):
- Same as above. If the function running in the context (here get_config_value()) raises KeyError, the test will pass; otherwise, the test will fail.

Summary

We have implemented a simple program to read a configuration file. This program includes a function that displays the sections of the configuration file and another function that checks if the provided section is valid before verifying if the given key is also valid.

We have written 4 test cases by using a fixture mock_config:

A test case to evaluate the behavior of get_available_sections by printing the sections available in a mocked config object.
A test case to evaluate the behavior of get_config_value if there is an existing key and a section is passed to it.
A test case to test the exception behavior of get_config_value if there is a non-existent section is passed
A test case to test the exception behavior of get_config_value if there is a non-existent key is passed.

There are many ways to test configuration files, and I’ve shared one approach for testing your configuration files. I hope this gives you a clear idea of how to test config files accessed using the ConfigParser class.

Until then, your friend here, Hemachandra, is signing off...

For more courses, visit my website here.

Have a nice day!

How to test/mock Configuration files in Python

Table of contents