Inner Working Of Python

Understanding how Python code is internally interpreted is a fantastic method to enhance your programming abilities and understanding. Here is a brief overview of how Python, particularly CPython (the standard Python interpreter coded in C), handles code:

1. Lexical Analysis

• Tokenizer: Python code is initially handled by a tokenizer. This tool changes the characters in your code into tokens. Tokens are the fundamental syntax parts like names, numbers, and operators.

2. Parsing

• Parser: After the tokenizer, the tokens go into a parser. This checks if the expressions made from these tokens follow Python's rules. The parser uses a special grammar rule for Python to understand the tokens.

• Abstract Syntax Tree (AST): If the syntax is right, the parser creates an Abstract Syntax Tree (AST). This tree shows the structure of the code, with each node representing a part of the program.

3. Compilation

• Compiler: The AST is then compiled into bytecode. This is a lower-level, platform-independent representation of your source code. Bytecode is an intermediate state—it's more abstract than machine code but is designed to be executed by a virtual machine, not directly by hardware.

4. Bytecode Interpretation

• Python Virtual Machine (PVM): The compiled bytecode is then sent to the Python Virtual Machine (PVM), which is part of the Python interpreter. The PVM reads the bytecode instructions one by one and executes them by making appropriate calls to the Python/C API, manipulating data or performing operations as required.

5. Execution

• The execution process in the PVM is typically a loop that iterates over the bytecode instructions, decoding and executing them. This is often referred to as the "eval loop" or "CEval loop" (in CPython).

• Operations such as loops, function calls, and handling exceptions are handled here. The PVM takes care of memory allocation, garbage collection, and other technical details while running bytecode.

  You can Learn more about from Hitesh Sir from Youtube on his channel Chai aur Code.