How null in Python works under the hood

The concept of null is fundamental across programming languages for representing the absence of a value. Null signifies that a variable exists but doesn’t currently hold any data. It’s used to represent a lack of value, a non-existent object, or an uninitialised state. Many languages automatically assign null to variables of reference types when they are declared but not explicitly initialised.

The Concept of 'None' in Python

In many other languages, null is just a synonym for 0, but null in Python is a full-blown object. None is distinct from 0, False and an empty string. It is a distinct data type (NoneType), and only None is capable of being None.

Here’s a breakdown of the internal implementation and behaviour of None in Python:

1. Type and Identity

   None is of type NoneType. You can check its type using type(None), which will return <class 'NoneType'>. The NoneType itself is a very simple type. It does not have any methods or properties beyond those inherited from the base object class.

    >>> type(None)
    <class 'NoneType'>
   

2. Singleton Pattern

   Python’s None is implemented as a singleton, which means there’s only one instance of None in a Python process. No matter how many times you use None, they all refer to the same single object in memory. Internally, this is achieved by creating the None object once and then returning a reference to it whenever None is used.

    >>> my_None = type(None)()  # Create a new instance
    >>> print(my_None)
    None
    >>> my_None is None
    True
   

3. Memory Allocation

   Since None is a singleton, it is allocated once in memory and persists throughout the runtime of the Python program. The reference to None is stored in a static variable in C code (when implementing Python in C). This also means that None does not require any garbage collection because it’s never deallocated.

    >>> id(None)
    4465912088
    >>> id(my_None)
    4465912088
   

4. C Implementation

   In CPython, the default Python implementation, None is defined in the C source code. It’s implemented in the object.c file of the CPython source code. Specifically, it’s created as a statically allocated object. The C struct representing the NoneType object is very minimal, with just the necessary fields to maintain its type identity.

    PyObject _Py_NoneStruct = {
        PyVarObject_HEAD_INIT(&PyNone_Type, 0)
    };
   
    PyObject *Py_None = &_Py_NoneStruct;
   

Difference between 'None' and other Python objects

None Object differs from other Python objects in several key ways:

1. Identity vs. Equality

   None : Do use the identity operators is and is not. Do not use the equality operators == and !=. In below example, the equality operators can be fooled when you’re comparing user-defined objects that override them so the equality operator == returns the wrong answer. The identity operator is, on the other hand, can’t be fooled because you can’t override it.

    >>> class Comparison:
    ...     def __eq__(self, other):
    ...         return True
    >>> b = Comparison()
    >>> b == None  # Equality operator
    True
    >>> b is None  # Identity operator
    False
   

   Other Objects: Other Python objects (like integers, strings, lists) are compared using the == operator, which checks for equality of value, not identity.

    >>> x = 5
    >>> y = 5
    >>> print(x == y)  # Output: True (they have the same value)
    >>> print(x is y)  # Output: True (small integers are cached, so they refer to the same object)
   
    >>> a = [1, 2, 3]
    >>> b = [1, 2, 3]
    >>> print(a == b)  # Output: True (they have the same contents)
    >>> print(a is b)  # Output: False (they are different objects in memory)
   

2. Type and Mutability

   None: None is of type NoneType, which is immutable, meaning it cannot be changed. There is no way to modify the None object or assign new attributes to it.

    >>> my_list = None
    >>> my_list
    <class 'NoneType'>
    >>> my_list.append('g')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    AttributeError: 'NoneType' object has no attribute 'append'
   

   Other Objects: Other Python objects can be mutable (e.g., lists, dictionaries) or immutable (e.g., integers, strings, tuples).

    >>> my_list = [1, 2, 3]
    >>> my_list.append(4)  # Mutable object (list) can be changed
    >>> s = "hello"
    >>> s = s.upper()  # Strings are immutable; a new string is created
   

3. Boolean Context

   None : None is falsy, which means not None is True. None is considered False in a boolean context, making it useful for checks in conditionals.

    >>> if not None:
    ...     print("None is falsy")  # This will be printed
   

   Other Objects: Other objects have their own truthy or falsy evaluations.

   For example:

   1. Empty sequences or collections (e.g., [], '', {}) are False.

   2. Non-empty sequences or collections are True.

   3. Numbers: 0 is False, and any non-zero number is True.

   4.  >>> if []:
       ...     print("Empty list is truthy")
       ... else:
       ...     print("Empty list is falsy")  # This will be printed
      

4. Assign value

   If you try to assign to None, then you’ll get a SyntaxError

    >>> None = 5
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    SyntaxError: can”t assign to keyword
   
    >>> None.age = 5
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    AttributeError: 'NoneType' object has no attribute 'age'
   
    >>> setattr(None, 'age', 5)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    AttributeError: 'NoneType' object has no attribute 'age'
   
    >>> setattr(type(None), 'age', 5)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: can't set attributes of built-in/extension type 'NoneType'
   

5. Inheritance

   You can’t subclass NoneType, either

    >>> class MyNoneType(type(None)):
    ...     pass
    ...
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: type 'NoneType' is not an acceptable base type
   

6. __builtins__

   Here, you can see None in the list of __builtins__ which is the dictionary the interpreter keeps for the builtins module. None is a keyword, just like True and False. But because of this, you can’t reach None directly from __builtins__ as you could, for instance, ArithmeticError. However, you can get it with a getattr() trick.

    >>> dir(__builtins__)
    ['ArithmeticError', ..., 'None', ..., 'zip']
   
    >>> __builtins__.ArithmeticError
    <class 'ArithmeticError'>
   
    >>> __builtins__.None
      File "<stdin>", line 1
        __builtins__.None
                        ^
    SyntaxError: invalid syntax
    >>> print(getattr(__builtins__, 'None'))
    None
   

Summary :

The concept of `null` represents the absence of a value in many programming languages. Python uses `None` instead of `null`, which is a singleton object of type `NoneType`. Unlike other objects, `None` is immutable and cannot be subclassed or assigned new attributes. It is evaluated as `False` in boolean contexts and should be compared using identity operators (`is` and `is not`) rather than equality operators (`==` and `!=`). Internally, `None` is implemented as a statically allocated object in CPython. It is part of Python's built-in keywords, accessible via `__builtins__` with special handling.