Captured variables

Introduction

variable : a symbolic name associated with a value and whose associated value may be changed - Wikipedia

Variables are like placeholders tied to values. They are entities that can be captured, moved around, and tweaked. Taking about captured variable, they refer to the instances where a variable defined in one scope is accessed and utilized in a different scope, typically in a lambda expression. Before digging deep into captured variables, let's understand lambda expression.

Lambda Expression

Lambda expression were introduced in C# version 3.0. Lambda expression allows a compact way to quickly create a function without following the standard way of a full function declaration. Lambda expression are frequently called as anonymous function due to their ability to define inline and unnamed functions.

Lambda expression syntax: (parameter-list) => (expression-body)

You can read this as a function that takes the parameter-list as a parameter and returns the result of the expression-body. parameter-list is similar to regular parameter of a function/method. It can be any data type or object. The expression-body is the code that gets executed when the lambda expression is called. It is the logic body of function similar to method body. The difference is that the expression-body is inline. Lambda expressions are useful in scenarios where a short, scoped (one-time-use) function is needed. Here is a simple lambda expression which returns the sum of 2 numbers:

public Func<int, int, int> sumOf2Nums = (x, y) => x + y;

If we recall from my previous article on delegates, first two type parameters of above delegate Func<> sumOf2Nums takes 2 input parameter types(int, int), and the last int represents the return type. Then, we have a lambda expression (x, y) => x + y; which takes 2 int parameters which returns its sum. We can call this delegate like any other method/function:

sumOf2Nums(4,8);

When sumOf2Nums(4,8) is called, it invokes the anonymous function(lambda expression) with x = 4 and y = 8 which returns an int 12 which is the sum of x and y.

Now, let's write the similar method in standard way:

public int sumOf2NumsRegularWay(int x, int y)
{
    return x + y;
}

Both of our method (delegate and the standard way) returns same output but using lambda expression is slightly more concise. Lambda expression shines when you use extension method particularly working with sequences using LINQ. Here is an example:

// A delegate with lambda expression to get the total sum of a list.
public Func<List<int>, int> sumOfList = (listOfNums) => listOfNums.Sum();

List<int> someNumbers = new List<int> { 1, 5, 6, 9, 4, 2, 7, 11, 3, 16 };
int sum = eg.sumOfList(someNumbers); // invoke the delegate.
Console.WriteLine(sum);
// prints the sum of the list which is: 64

We can do a lot of computation in the expression body of lambda expression. What if we want to get the sum of odd numbers that are less than 6? Here's how we can utilize other extension method:

// A delegate with lambda expression to get the total sum of odd numbers that are less than 6 a list.
public Func<List<int>, int> sumOfOddNums = (listOfNums) => listOfNums.Where(x => x < 6 && x % 2 != 0).Sum();

sumOfOddNums expanded the usage of lambda expressions (x => x < 6 && x % 2 != 0) by implementing the LINQ extension Where method to filter odd numbers less than 6 before calculating their sum.

List<int> someNumbers = new List<int> { 1, 5, 6, 9, 4, 2, 7, 11, 3, 16 };
int sumOfOddNums = eg.sumOfOddNums(someNumbers); // invoke the delegate.
Console.WriteLine(sumOfOddNums);    
// prints the total sum of odd numbers which are less than 6 which is: {1,5,3} = 9

It demonstrate how lambda expressions make it easy to express complex conditions in a clean and compact manner. Now rolling back to captured variables.

Captured variables

When a variable defined in one scope is accessed and utilized in a different scope often within a lambda expression, it said to be captured variables. In simpler terms, any kind of variable that is declared outside of the lambda expression, but used within the lambda expression is said to be captured variable. It's important to note that the parameter of the lambda expression and variables declared within the lambda expression are not captured variables. Here's an example to demonstrate it:

public Action<int> IncrementCounterAction(int incrementBy)
{
    int defaultNumber = 2;
    Action<int> action = input =>
    {
        Console.WriteLine($"Default counter value: {defaultNumber}");
        int result = input + defaultNumber + incrementBy;
        Console.WriteLine($"Increment action by (method argument): {incrementBy}");
        Console.WriteLine($"Sum of method argument, lambda argument, and default local method value: {result}");
    };
    defaultNumber = 10; // modify local variable
    return action;
}

In above code, defaultNumber and incrementBy is captured variable. Both of them is beyond the scope of action() delegate. To call this method:

// eg is the instance of a class
Action<int> action = eg.IncrementCounterAction(5);
action(2);

What do you think the output would be?

Output:

Default counter value: 10
Increment action by (method argument): 5
Sum of method argument, lambda argument, and default local method value: 17

This might be surprising. The reason behind this is defaultNumber being captured by reference. Lambda expression captures variable by their references. When you use a variable within a lambda expression, it captures the reference to that variable rather than its value. This means any changes made to the captured variable outside the lambda expression will be reflected inside the lambda, and vice versa. Capturing variables using reference allows lambda expression to have access to current state of variable in the surrounding scope and avoids creating unwanted copies of variables especially when lambdas are passed as parameter or used in asynchronous tasks.

Being unaware of this behavior can lead to unexpected results when variables are modified after the creation of lambda expression.

Closures

Closure is a mechanism that allows a lambda expression to capture and remember the variables and the environment in its lexical scope { }, even if the lambda expression is executed outside that scope. Simply put, closures encapsulates variables and environment in which it was created, preserving both the variable and environment context. By environment, it is referring to the set of variables that are in scope and accessible at the time the closure is created. It ensures that the lambda expression keep a hold on its original surroundings, no matter when and where it is executed.

In our previous example, when the variable defaultNumber was captured in lambda expression, it formed a closure. This closure essentially locked onto the defaultNumber and its environment. So, when defaultNumber was modified outside the lambda expression, that change was also noticed and applied inside the lambda expression. Closures ensure that the lambda expressions have access to the current state of the variables in their surrounding scope.

Example:

Below, I'm creating a CreateCounterAction() method which returns an Action which essentially is a closure.

public Action CreateCounterAction()
{
    int incrementBy = 5;
    // here, action take empty parameter list.
    Action action = () => 
    {
        incrementBy++;
        Console.WriteLine($"Counter at: {incrementBy}");
    };
    return action;
}

The closure in action captures the variable incrementBy which is outside of action lambda expression. In the CreateCounterAction method, the environment consists of the variable incrementBy. When the closure is formed, it encapsulates not only the variable itself but also the entire environment in which it exists meaning it holds onto the context in which it was created.

// calling code
Action closureAction = eg.CreateCounterAction();
closureAction();
closureAction();
closureAction();
closureAction();

When the closureAction is invoked, it increments the captured incrementBy and prints the value.

Output:

Counter at: 6
Counter at: 7
Counter at: 8
Counter at: 9

The closure here is like cache, it holds the state of the incrementBy. Each time you call closureAction, it recalls the previous value and continue.

Closures might get confusing due to the mechanism of how it capture variables. The confusion arises from the fact that closures capture variables by reference, not by value. In a loop, when a closure captures a variable, it doesn't store the current value of that variable at the time of creation. Instead it maintains a reference to the variable. Simply put, when the closure is executed, it looks up the current value of the variable in its original scope.

Example:

Below, I created a method that returns an array of Action. Firstly, I created an array of four actions. Each action is a closure capturing the loop variable i. When invoked, all closures print the final value of i after the loop. This happens because all the closures captured the variable i by reference, and when the closure(lambda expression) is actually invoked and not when they are created, they see the final value of i after the loop has completed resulting in an output of "4" for each invocation.

public Action[] ClosureActionInLoop()
{
    Action[] actions = new Action[4];
    for(int i = 0; i < 4; i++)
    {
        actions[i] = () => Console.WriteLine(i);
    }
    return actions;
}

// calling code
Action[] closureLoopActions = eg.ClosureActionInLoop();
foreach(var loopAction in closureLoopActions)
{
    loopAction();
}

Here, they all reference the same variable i, which has been updated to 4 by the end of the loop. Consequently, they all print the current value of this shared reference, resulting in "4" for each invocation.

Output:

4
4
4
4

To fix this, we can introduce local variable inside the loop. Fixed code:

public Action[] FixedClosureActionInLoop()
{
    Action[] actions = new Action[4];
    for(int i = 0; i < 4; i++)
    {
        int localVar = i; // introduce local variable to let closure remember
        actions[i] = () => Console.WriteLine(localVar);
    }
    return actions;
}

By introducing localVar, each closure captures it own value, addressing the common issue of closures in loop.

// calling code
Action[] fixedClosureLoopActions = eg.FixedClosureActionInLoop();
foreach(var loopAction in fixedClosureLoopActions)
{
    loopAction();
}

Output:

0
1
2
3

Each closures assigned to actions[i] captures this local variable localVar. Unlike the previous example where the closure captured the loop variable i by reference, now it captures the localVar by value at the time of its creation.

Outside of a loop, variables are often captured by reference, while inside a loop, introducing a local variable allows for capturing by value, ensuring each closure remembers its value.

Closures impact on memory

It is crucial to understand the potential impact on memory while using closures. Yes, closures offer flexibility but when there is a long-lived object, you must consider its impact on memory. For instance, if a closure captures a reference to a large dataset, it keeps the data in memory even when it's no longer needed. This results in unnecessary memory consumption impacting the performance.

Some tips on overcoming this issue is to limit the scope of captured variable ensuring they only encapsulate necessary resource.

Conclusion

Captured variables are entities shared across scopes. Lambda expression or anonymous function serve as a concise function to create instant short-lived function. These lambda expressions often form closures, encapsulating both variables and their lexical environment { }(set of variables in a scope). Closures ensure that the lambda expression retain access to the external variable's state allowing flexibility and preservation.

Summary

1. Lambda Expressions: Short, inline functions; e.g., (x, y) => x + y sums two numbers.

2. Captured Variables: Extend scope by accessing external variables, crucial in lambda expressions for dynamic behavior. Illustrated in IncrementCounterAction method, showcasing capture of variables in a closure.

3. Closures: Preserve variable states and environment, ensuring access to original surroundings even outside the lexical scope { }. Shown in CreateCounterAction method, capturing variables to maintain context outside the method.

4. Memory Impact: Closures, while flexible, can lead to unnecessary memory consumption; mitigate by limiting the scope of captured variables. Addressed with examples, warning about unnecessary memory use in closures and recommending limited scope for efficient resource use.

Sourcecode.

More on closures:

• Martin Fowler's Closure article

• Jon Skeet's C# in depth