Looping defaultdict In Templates: A long-standing "Bug" in Django

I am new to Django and recently spent a good chunk of time trying to debug what I thought was quite trivial. Looping a defaultdict in templates.

This is a pretty common pattern when using Django, using a dict to pass and render data via templates, but there is something to be learned here. Let's jump into it!

The "Bug"

When you have a defaultdict whose default value is a list, you might run into this issue. Here is a simple example:

from collections import defaultdict

# Create a defaultdict with default type list
dd = defaultdict(list)

# Add some values to the defaultdict
dd['fruits'].append('apple')
dd['fruits'].append('banana')
dd['vegetables'].append('carrot')
dd['vegetables'].append('broccoli')

If you print(dd) the output will look something like

defaultdict(<class 'list'>, {'fruits': ['apple', 'banana'], 'vegetables': ['carrot', 'broccoli']})

All good till now. Now let's try to use this in a template like

{% for category, items in data.items %}
  <h1> {{ category}} </h1>
  <p> {{ items|join:", " }} </p>
{% endfor %}

#expected output
<h1> Fruits </h1>
<p> apple, banana </p>

<h1> Vegetables </h1>
<p> carrot, broccoli </p>

But in reality, it will print nothing! Let's see why this happens.

Explanation

Here is the discussion from the wayback machine which basically says

Indeed, it boils down to the fact that the template language uses the same syntax for dictionary and attribute lookups.

When Django resolves the template expression, it will first try to access data['items'] and since it's a defaultdict and there is no key called "items" it will set the value to an empty list and then try to iterate it and end up doing nothing!

The intended action would be to call the method items with no arguments of the instance data (in short: data.items()), but since data['items'] was a valid expression, Django stopped there and got the empty list just created.

Also since list is iterable, it doesn't raise any error. If you initialized your dict with something like defaultdict(int), it will throw an error "TypeError: 'int' object is not iterable".

Okay, now we know why this is happening, let's see how we can fix it.

Fix 1: Disabling the defaulting feature of defaultdict

You can disable the defaulting nature by setting

new_data.default_factory = None

Here, when you set the default value toNone, when Django tries to do data['items'] it will get None and it will move to try data.items() and hence act as intended.

Fix 2: Re-cast it to a Dict

You can simply cast the defaultdict to dict before passing it to the templates.

dict(data)

I like the 2nd fix more, but I would ideally suggest not using defaultdicts at all in Django templates since it might have unexpected behavior which is really easy to miss.