Virtual Environment and Python package manager
A virtual environment in Python is a self-contained directory that allows you to isolate a project's dependencies from the global Python environment. This ensures that each project can use its versions of libraries without affecting other projects, solving issues like dependency conflicts or version mismatches.
The diagram shows a Python Virtual Environment concept using three different systems:
System 1: Inventory Management
System 2: Accounting System
System 3: HR System
Each system uses its own Virtual Environment. Let me explain how this works in simple terms:
System 1 (Inventory Management) creates its own virtual environment. This is like creating a small isolated space where all the packages and tools needed for the inventory system are installed.
System 2 (Accounting System) also has its own virtual environment. This environment will contain only the tools required for the accounting system, preventing any interference with other systems.
System 3 (HR System) has a separate virtual environment, where only the HR system's tools and libraries exist.
virtual environments help to keep things organized by giving each project its own separate workspace, preventing problems caused by mixing different tools or versions between projects(Python Land)(Deepgram)(Software Engineering Blog)
Why virtual environments are important:
Dependency Management: Each project can have its own set of dependencies, ensuring that the right versions of libraries are used for that specific project.
Isolation: By isolating projects, you avoid conflicts between packages, especially if you’re working on multiple projects that require different versions of the same library.
Cleaner Development Workflow: Virtual environments help keep your global Python installation clean by only installing project-specific dependencies in a controlled environment.
How to Use Virtual Environments:
Create a Virtual Environment: You can use the built-in
venv
module to create a virtual environment. Run the following command in your terminal:python -m venv myenv
This creates a folder (
myenv
) that contains a Python interpreter and a separate package directory.Activate the Virtual Environment:
On Windows:
.\myenv\Scripts\activate
On Mac/Linux:
source myenv/bin/activate
After activation, the command prompt will show the name of your virtual environment, indicating that the environment is active.
Install Dependencies: Once the environment is active, any package you install using pip (
pip install package_name
) will only be available within that environment.Deactivate the Environment: After you finish working, you can deactivate the virtual environment by simply typing
deactivate
.
Managing Virtual Environments with Tools like Pipenv and Poetry:
Pipenv: Combines virtual environment management with package management, making it easier to handle dependencies in collaborative projects.
Poetry: Offers more advanced dependency management and packaging capabilities while also managing virtual environments, making it ideal for projects that require complex packaging(Python Land)(Deepgram)(DEV Community).
Incorporating virtual environments into your workflow can greatly simplify project management and help maintain a stable, conflict-free development environment.
Python package manager
A Python package manager is a tool that helps developers install, manage, and update external libraries and dependencies for their projects. The most widely known package manager is pip (which stands for "Pip Installs Packages"). Pip is the default and standard tool for managing packages from the Python Package Index (PyPI), which is a repository containing thousands of Python libraries. Pip simplifies the installation of packages and handles dependencies by resolving and downloading any needed packages automatically(Dhruv Awasthi)(Inedo Blog).
While pip is essential and widely used, there are other popular package managers that provide additional features:
Pipenv: This combines package management and virtual environment management. It is preferred for collaborative projects because it simplifies managing dependencies and creating isolated environments. It’s great for projects needing both development and production environments(DEV Community)(Inedo Blog).
Poetry: A more modern package manager that not only handles dependencies but also helps build and distribute Python applications. It uses a
pyproject.toml
file for configuration and is popular because it simplifies packaging and dependency management for both small and large projects(Jumping Rivers)(Slant).Conda: Unlike pip, Conda is a more versatile package manager designed not only for Python but also for other languages. It is popular in data science environments and is known for managing libraries and packages for scientific computing. Conda can also manage virtual environments similar to pip and Pipenv(Inedo Blog).
Each of these package managers has its strengths. For example, Pipenv and Poetry focus more on ensuring smooth workflows for development environments, while Conda is highly favored in scientific computing environments due to its ability to manage binary packages more effectively(Jumping Rivers)(Slant).
For any Python project, choosing the right package manager depends on your specific needs, such as collaboration, environment isolation, or performance requirements.
Ensure you don’t upload your virtual environment (venv
)
to GitHub or any cloud
How to manage your dependencies correctly:
Steps:
Use
.gitignore
to ignore the virtual environment: Create or modify a.gitignore
file in the root of your project and add the following line to ensure your virtual environment folder is not pushed to GitHub:# Ignore the virtual environment folder venv/
This will prevent Git from tracking your
venv
folder, ensuring it’s not included in your commits or pushed to your repository.Generate a
requirements.txt
file: When you're ready to push your code, save all the current dependencies (packages you’ve installed in the virtual environment) in arequirements.txt
file. This allows anyone who clones your repository to install the necessary dependencies.Run this command:
pip freeze > requirements.txt
- This will create a
requirements.txt
file with all the packages and their versions.
- This will create a
Clone or update the repository: When someone (or you) clones or pulls updates from the repository, instead of including the
venv
, they can recreate the same environment using therequirements.txt
file.After cloning or pulling the repository, run:
pip install -r requirements.txt
This installs all the dependencies listed in the
requirements.txt
file in your new virtual environment.
AGAIN
Use
.gitignore
to exclude thevenv
folder.Use
pip freeze > requirements.txt
to save all the packages you’ve installed.Use
pip install -r requirements.txt
to install dependencies when cloning or updating the project.
This is a best practice to avoid burdening your repository with unnecessary files while ensuring a smooth setup for others who work on your project.
Subscribe to my newsletter
Read articles from Thirdy Gayares directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by
Thirdy Gayares
Thirdy Gayares
I am a dedicated and skilled Software Engineer specializing in mobile app development, backend systems, and creating secure APIs. With extensive experience in both SQL and NoSQL databases, I have a proven track record of delivering robust and scalable solutions. Key Expertise: Mobile App Development: I make high-quality apps for Android and iOS, ensuring they are easy to use and work well. Backend Development: Skilled in designing and implementing backend systems using various frameworks and languages to support web and mobile applications. Secure API Creation: Expertise in creating secure APIs, ensuring data integrity and protection across platforms. Database Management: Experienced with SQL databases such as MySQL, and NoSQL databases like Firebase, managing data effectively and efficiently. Technical Skills: Programming Languages: Java, Dart, Python, JavaScript, Kotlin, PHP Frameworks: Angular, CodeIgniter, Flutter, Flask, Django Database Systems: MySQL, Firebase Cloud Platforms: AWS, Google Cloud Console I love learning new things and taking on new challenges. I am always eager to work on projects that make a difference.