Create a modern Python project

AndrewFox_DEVAndrewFox_DEV
11 min read

Let's look at the project tree!

The following project architecture is modifiable, however, it is already a good starting point!

my_python_project/
├── .env/
├── src/
│   ├── utils/
│   │   └── __init__.py
│   ├── index.py
│   └── __init__.py
├── logs/
├── docs/
├── LICENSE
├── pyproject.toml
├── requirements.txt
├── .gitignore
└── README.md

Let's take a look at virtual environments!

A virtual environment in Python (often called VENV) is an isolated space where you can install packages and dependencies without interfering with the rest of the system or other projects. It is used to avoid conflicts between libraries that require different versions and to keep each project independent. The virtual environment directory is commonly referred to as .venv, but you can name it whatever you like, for example, as in my case, .env. But let's take it slow.

Create the VENV.

To create a virtual environment in a directory called .env:

python -m venv .env

Activate the VENV.

To activate the virtual environment we must first make a distinction, because on Windows, and Linux or macOS the procedure changes!

  • On Windows we use CMD:
.env\Scripts\activate
  • On Linux or MacOS, then Unix-based operating systems, we will use the terminal.
source .env/bin/activate

Once activated, the terminal will show the name of the environment as a prefix.

Deactivate the VENV.

To deactivate the virtual environment, on the other hand, there is no need to distinguish between OSs.

deactivate

Let's take a look at dependency management!

A dependency is an external package, usually a library, tool, or framework, that a project needs to function properly. Dependencies are managed through a package manager (such as pip in the case of Python) and provide ready-made functionality, such as for routing, data management, authentication, or user interface. Using them allows you to avoid reinventing the wheel, speed up development, and keep your code more consistent, reliable, and maintainable.

Install a package.

In this case since we are doing a project, and since we want to have the dependencies isolated only to the project itself, we will have to activate the virtual environment. It would work even without the activation but the dependencies would be handled globally.

pip install <package-name>

Install a specific package.

pip install <package-name>==<version>

Install packages from requirements.txt.

The requirements.txt file lists the Python dependencies needed for a project, facilitating automatic installation.

pip install -r requirements.txt

Freeze current dependencies.

Exports all installed packages with versions to requirements.txt.

pip freeze > requirements.txt

Upgrade a package.

pip install --upgrade <package-name>

Uninstall a package.

pip uninstall <package-name>

Let's take a look at the .gitignore file.

The .gitignore file is a configuration file used by Git to exclude specific files or directories from version control. Basically, it tells Git which files should not be tracked or sent to the remote repository. This is useful to avoid including temporary, system, or automatically generated files that are not needed by the shared project. The dot in front in the file name indicates that it is a hidden file in Unix-like systems (Linux/ MacOS). It is a convention to mark configuration or system files that you normally do not want to see in the ordinary list of files. So it is good practice to do this even if you were on a Windows operating system.

What basic configurations to include in this file?

  1. Banally the virtual environment directory, which contains all the installed dependencies of the project.

     # Virtual Environment
     .env/
    
  2. Python cache directory or files, the asterisk instead of the filename says he means all files with that extension.

     # Python Cache
     __pycache__/
     *.pyc
    
  3. Temporary directories generated by the IDE, or by the editor.

     # Temporary IDEs or Editors
    
     # That is if you were using the Visual Studio Code editor.
     .vscode/
    
     # That is if you were using a Jet Brains IDE.
     .idea/
    
  4. The logs directory or files.

     # Logs
     *.log
    
     # You can also add the logs directory if you need it.
     logs/
    
  5. System temporaries (such as on macOS).

     # System files, as for macOS.
     .DS_Store
    
  6. And much more, at your discretion and decision!

Now we are going to configure the Python project, let's take a look!

We will use the pyproject.toml file, it is a text file where you put the configuration of the Python project, such as the name, version, author, necessary dependencies, settings for tools such as formatter or tests, and so on. It is most useful, and most modernly used, because it organizes everything in one file, is supported by many modern tools, and helps to create and distribute the project easily.

Create a pyproject.toml file.

In the root directory of your project, which is the root directory of your project, we need to create the pyproject.toml file. Now I will do everything step by step, and show you the various fragments of the file, then paste the complete file at the end of this topic.

Let's configure basic information.

[project]
name = "my-project"
version = "0.1.0"
description = "This is the project description!"
authors = [
    {name = "John Doe", email = "johndoe@example.com"}
]

Let's configure dependency information.

Dependencies should be written to the list, it is similar to Python, and they should be written to string.

dependencies = [
    "requests"
]

Let's specify the LICENSE file.

The LICENSE file specifies the rights to use, distribute and modify the project, providing legal clarity.

license = {file = "LICENSE"}

Let's specify the README.md file.

If we only wanted a README.md then this configuration choice is fine, otherwise it should be done dynamically.

readme = "README.md"

Here is the complete configuration seen so far.

Now below I show you what the complete pyproject.toml would look like up to what we have seen so far.

If you wanted to take a look at the official documentation of Poetry, it's modern Python dependency and packaging manager, click here.

[project]

# Basic information.
name = "my-project"
version = "0.1.0"
description = "This is the project description!"
authors = [
    {name = "John Doe", email = "johndoe@example.com"}
]

# The dependencies.
dependencies = [
    "requests"
]

# The LICENSE.
license = {file = "LICENSE"}

# The README.
readme = "README.md"

The LICENSE file, let's give a quick overview.

The LICENSE file is a document included in a software project, contains the legal terms that define how others may use, modify, distribute or contribute to the project, and serves to protect the author and clarify the rights of users.

There are various kinds of licenses!

I explain the most common ones below.

  • MIT: It is very permissive, allowing almost anything with only copyright retention.

  • GPL: It is more restrictive, forcing changes to be open source.

  • Apache 2.0: It is permissive but with stipulations on patents.

If you would like to see all the approved licenses, click here.

The README.md file, let's give a quick overview.

The README.md, also known simply as README, is a simple documentation file, and serves to explain what the project is, how to install it, and how to use it, providing a quick guide to anyone reading it. To write a README you need to know MarkDown, which is a lightweight markup language that allows you to write plain text with formatting. I trust you are already familiar with MarkDown, alternatively I will leave the official documentation here. Below I show you a simple README, really basic, it is just to give a little overview to write it simply and in a technical and readable way. For the license section, I chose the MIT License, but you can choose whatever license you need as explained earlier.

# Project Name

A short paragraph to describe the project.

## Installation

To use the project you must have Python installed on your device, alternatively I recommend downloading it from the [official site](https://www.python.org/downloads/).

Now download the dependencies through the package manager from the dependencies file, it is `requirements.txt`.

```bash
pip install -r requirements.txt
```

## Usage

Through code snippets write down the various uses of your project, and make them well documented for others' use.

## License

Copyright <YEAR> <COPYRIGHT HOLDER>

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Whatever you think is needed in the documentation add it. What I am going to say is just a piece of advice, anyone should be able to understand what you are explaining, be brief but comprehensive, and in my opinion organize the documentation as structured and schematically as possible!

Now finally, let's start talking about Python!

We will use the src directory to contain the source code of the project. All libraries, APIs, or features you develop must be placed inside the src directory to keep the project structure clean, modular, and easy to maintain. As for the __init__.py file, its presence in src/ may seem useless at first glance, but it formally declares the directory as a proper Python package. This makes it importable and compatible with tools and linters, even if the file itself is empty. The main source code will be contained in the index.py file, which will serve as the primary file of the project. This file can be executed directly as the main module (for example, by running python index.py), thus starting the application or program. In this way, index.py represents the central entry point from which the overall program flow is managed.

The utils directory.

The utils directory in the Python project is used to collect functions, classes, or utility modules that support the main operation of the application but are not a direct part of the core business logic. These components can include generic helpers, support functions, parsing tools, file management, recurring operations, or other reusable utilities that simplify and make the main code more modular. Organizing these functions in a separate direcotry keeps the project cleaner, promoting maintainability and readability. To make package imports cleaner and more readable, a __init__.py file in the utils directory is used to export and group features in a modular way.

The following is an example, this is the sub tree of the project, in this case of the src directory.

src
└── utils
    ├── __init__.py
    └── useful_module.py
└── index.py

In the utils directory we first go to define a variable which we will call in the index.py. So the path to the following code snippet will be: dwadwa. So the path from the root of the project of the following code snippet will be: src/utils/useful_module.py.

number: int = 100

Now we will create the __init__.py file, for a single module it does not seem useful this is true, but besides being a convention it helps to organize the project architecture well. So there you go src/utils/__init__.py.

# The import is in the current directory, which is why there is no name in front of the dot.
from .useful_module import *

Now, let's see what happens in src/index.py.

import utils

print(utils.number)

# from utils import *

# print(number)

As you can see now project management has become a very simple thing.

Always check the types! What is mypy?

The mypy tool, is a static type checking tool for Python, which is a type checker that analyzes your code before execution for type errors. This helps prevent bugs and makes your code more readable and maintainable. Python is a dynamically typed language, so type checking happens at runtime; although I strongly recommend always marking types, even if you were not using a type checker. But with mypy you can add type annotations, they are the type hints, and have them checked in advance, during development.

What is mypy used for?

  • Checks variables, functions, and classes are used consistently with the declared types.

  • Helps catch errors such as passing a str when an int is expected, or forgetting to handle None values.

  • Improves code documentation: readers immediately understand the expected types.

  • Eases refactoring by detecting type errors introduced by changes.

  • Integrates well with editors and IDEs, showing errors during coding.

Why is mypy useful?

  • It reduces hard-to-find bugs, especially in medium to large projects.

  • It helps maintain clearer and more self-explanatory code.

  • It supports team development with explicit type contracts.

  • It integrates with development tools (CI, pre-commit) for automatic checks.

How to use mypy?

Installation of mypy.

First of all you have to install it, remember the virtual environment must be active. Installation is as simple as any other Python package, via its package manager.

pip install mypy

Let's annotate the code with type hints.

For simplicity, let's use src/index.py.

def my_sum(number_1: int, number_2: int) -> int:
  return number_1 + number_2

if __name__ == '__main__':
  print(my_sum(3, 5))

Let's execute the static check.

In the root of the project, via terminal run the following command, if there are any errors, mypy will report them with file, line and type of problem.

mypy src/

If successful, you will see the following log message in the terminal.

Success: no issues found in 1 source file!

As mentioned above it is not mandatory, but besides being a good rule it is also quite logical to think that if the language does not type you should be the one to do it, it is simply called common sense to avoid stupid mistakes!

0
Subscribe to my newsletter

Read articles from AndrewFox_DEV directly inside your inbox. Subscribe to the newsletter, and don't miss out.

Written by

AndrewFox_DEV
AndrewFox_DEV