Comprehensive Guide to Foundry Fuzz Testing in Smart Contracts

Introduction

As a smart contract developer, you know how important it is to create different scenarios to test a contract since testing is crucial in ensuring its security. However, you cannot always find all the bugs in a contract or fully confirm its functionality with unit testing, the most common method of testing smart contracts. These limitations can be addressed with Property-Based Testing.

Property-based testing is a way of testing general behaviours as opposed to isolated scenarios, focusing on verifying that the software functions as expected. There are two main approaches to property-based testing: parametric testing and fuzzing.
This tutorial will guide you through running fuzz tests on a smart contract using Foundry, demonstrating how fuzz testing can help uncover hidden vulnerabilities.

Fuzzing and Foundry

Fuzz Testing is a method that involves providing the software with large amounts of random input data ("fuzz") to find vulnerabilities triggered by unexpected input.

Foundry offers built-in features for fuzz testing smart contracts. It leverages the Forge framework to generate random data inputs based on the expected data types stated in the contract. Specifically tailored for smart contract vulnerabilities, it focuses on areas like reentrancy attacks, access control issues, and integer overflows.

Comparing Unit Testing and Fuzz Testing: Practical Examples

To illustrate the differences between unit testing and fuzz testing, we will perform both tests on a simple arithmetic calculator smart contract. This will help you understand the advantages of fuzz testing over unit testing and learn how to use Foundry for fuzz testing.

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.13;

contract simpleArithmetic {
    uint addValue;
    uint subValue;
    uint divValue;
    uint mulValue;

    function addNum (uint a, uint b) external returns (uint) {
        addValue = a + b;
        return addValue;
    }

    function subNum (uint a, uint b) external returns (uint) {
        if (a >= b) {
            subValue = a - b;
        } else {
            subValue = 0;
        }
        return subValue;
    }

    function divNum (uint a, uint b) external returns (uint){
        if (a > 0 && a > b && b > 0) {
            divValue = a/b;
        } else {
            divValue = 0;
        }
        return divValue;
    }

    function mulNum (uint a, uint b) external returns (uint) {
        mulValue = a * b;
        return mulValue; 
    }
}

Unit Test

Let's start with a simple unit test for this contract:

// SPDX-License-Identifier: UNLICENSED
pragma solidity ^0.8.13;

import "forge-std/Test.sol";
import "../src/simpleArithmetic.sol";

contract arithmeticTest is Test {
    simpleArithmetic arithmetic;

    function setUp() public {
        arithmetic = new simpleArithmetic();
    }

    function testAdd () public {
        arithmetic.addNum(4, 0);
    }

    function testSub () public {
        arithmetic.subNum(7, 3);
    }

    function testMul () public {
        arithmetic.mulNum(11, 4);
    }

    function testDiv () public {
        arithmetic.divNum(22, 11);
    }
}

Running forge test yields the following output:

macbookair@Vicentia-Peace FoundryFuzzing % forge test
[⠊] Compiling...
[⠔] Compiling 2 files with 0.8.17
[⠑] Solc 0.8.17 finished in 3.48s
Compiler run successful

Running 4 tests for test/arithmetic.t.sol:arithmeticTest
[PASS] testAdd() (gas: 27697)
[PASS] testDiv() (gas: 27888)
[PASS] testMul() (gas: 27754)
[PASS] testSub() (gas: 27929)
Test result: ok. 4 passed; 0 failed; finished in 981.59µs

All tests have passed. However, these tests only confirm specific scenarios. How do we ensure that the contract works for any input? This is where fuzz testing becomes valuable.

Fuzz Test

Fuzz testing with Foundry is simple: test functions are named with the prefix testFuzz. This approach automatically generates a wide range of random inputs to test the contract's robustness.

// SPDX-License-Identifier: UNLICENSED
pragma solidity ^0.8.13;

import "forge-std/Test.sol";
import "../src/simpleArithmetic.sol";

contract arithmeticTest is Test {
    simpleArithmetic arithmetic;

    function setUp() public {
        arithmetic = new simpleArithmetic();
    }

    function testFuzz_Add (uint a, uint b) public {
        arithmetic.addNum(a, b);
    }

    function testFuzz_Sub (uint a, uint b) public {
        arithmetic.subNum(a, b);
    }

    function testFuzz_Mul (uint a, uint b) public {
        arithmetic.mulNum(a, b);
    }

    function testFuzz_Div (uint a, uint b) public {
        arithmetic.divNum(a, b);
    }
}

Now, let's run forge test again:

macbookair@Vicentia-Peace FoundryFuzzing % forge test
[⠔] Compiling...
[⠘] Compiling 1 files with 0.8.17
[⠃] Solc 0.8.17 finished in 3.28s
Compiler run successful

Running 4 tests for test/arithmetic.t.sol:arithmeticTest
[FAIL. Reason: Arithmetic over/underflow Counterexample: calldata=0x2db0ee450000000000000000000000000000000000000000000000000000000000000001ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff, args=[1, 115792089237316195423570985008687907853269984665640564039457584007913129639935]] testFuzz_Add(uint256,uint256) (runs: 268, μ: 27818, ~: 27818)
[PASS] testFuzz_Div(uint256,uint256) (runs: 256, μ: 17329, ~: 8043)
[FAIL. Reason: Arithmetic over/underflow Counterexample: calldata=0xf2cbce5b0000000000000000000000000004231514e56cb66159c47c753b9279add349cc0000000000000000000000000000000000003de146b367748c8943222342d5e4, args=[92259084507240287100640632255379180134549964, 1255075203225424273739937508152804]] testFuzz_Mul(uint256,uint256) (runs: 182, μ: 27899, ~: 27899)
[PASS] testFuzz_Sub(uint256,uint256) (runs: 256, μ: 19221, ~: 27963)
Test result: FAILED. 2 passed; 2 failed; finished in 46.43ms

Failing tests:
Encountered 2 failing tests in test/arithmetic.t.sol:arithmeticTest
[FAIL. Reason: Arithmetic over/underflow Counterexample: calldata=0x2db0ee450000000000000000000000000000000000000000000000000000000000000001ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff, args=[1, 115792089237316195423570985008687907853269984665640564039457584007913129639935]] testFuzz_Add(uint256,uint256) (runs: 268, μ: 27818, ~: 27818)
[FAIL. Reason: Arithmetic over/underflow Counterexample: calldata=0xf2cbce5b0000000000000000000000000004231514e56cb66159c47c753b9279add349cc0000000000000000000000000000000000003de146b367748c8943222342d5e4, args=[92259084507240287100640632255379180134549964, 1255075203225424273739937508152804]] testFuzz_Mul(uint256,uint256) (runs: 182, μ: 27899, ~: 27899)

Encountered a total of 2 failing tests, 2 tests succeeded

Of the four tests, two (testFuzz_Sub and testFuzz_Div) passed, while the others (testFuzz_Mul and testFuzz_Add) failed due to "Arithmetic overflow/underflow" errors. To solve this, we can downcast their input data type to uint128.

function testFuzz_Add (uint128 a, uint128 b) public {
        arithmetic

.addNum(a, b);
}

function testFuzz_Mul (uint128 a, uint128 b) public {
        arithmetic.mulNum(a, b);
}

Running forge test again shows all tests have passed successfully.

Conclusion

Fuzz testing in Foundry is a powerful tool that helps identify vulnerabilities missed by standard unit tests. By automatically generating a wide range of inputs, it exposes edge cases that might go unnoticed, enhancing your smart contract's security and robustness. Implementing fuzz tests in your workflow can lead to more reliable and secure smart contracts, essential in the fast evolving blockchain landscape.

References

Installing Foundry

Fuzz Testing in Foundry Documentation