⚈
Deep Tech Point
first stop in your tech adventure

Why Do We Need Virtual Environments in Python?

April 2, 2024 | AI

Virtual environments are one of the best practices in Python development, because they provide a way to manage project dependencies effectively while ensuring consistency and portability across different environments. Virtual environments in Python are used to create isolated environments for Python projects. Here’s why they are important and why we need to work with them.

Virtual Environments Isolate Dependency

Different projects often require different versions of libraries or packages. Virtual environments allow you to install dependencies for each project separately, preventing conflicts between different versions required by different projects.

Imagine you’re working on two separate Python projects: Project A and Project B.

Project A requires a library called “requests” version 2.25.0, which is the latest stable version at the time of development. Project B, however, was built a while ago and relies on an older version of “requests,” specifically version 2.22.0, due to compatibility reasons with other libraries it uses.

Now, if you were to install both versions of the “requests” library globally on your system, you’d encounter a conflict. Since Python looks for libraries in its system paths, installing both versions globally would mean that whichever version is installed last would overwrite the previous one. This creates a problem because Project A might not work with the older version, while Project B might break with the newer version.

This is where virtual environments come in handy:

1. Creating Virtual Environments

First, you create separate virtual environments for each project using a tool like ‘virtualenv’ or ‘venv’:

# Create virtual environment for Project A
$ python3 -m venv projectA_env

# Create virtual environment for Project B
$ python3 -m venv projectB_env

2. Activating Virtual Environments

Then, you activate the virtual environment for the project you’re working on:

# Activate Project A virtual environment
$ source projectA_env/bin/activate

# Activate Project B virtual environment
$ source projectB_env/bin/activate

3. Installing Dependencies

Within each activated virtual environment, you can install the required dependencies separately.
For Project A:

(projectA_env) $ pip install requests==2.25.0

For Project B:

(projectB_env) $ pip install requests==2.22.0

4. Working within Isolated Environments

Now, when you’re working on Project A, Python will use the “requests” version 2.25.0 installed in its virtual environment. Similarly, when you switch to working on Project B, Python will use the “requests” version 2.22.0 from its isolated environment.

As already said at the beginning and elaborated later on, by using virtual environments, you prevent conflicts between different versions of dependencies required by different projects. Each project operates within its own isolated environment, ensuring that the dependencies it needs are available without interfering with other projects or the system-wide Python installation. Neat, isn’t it?

Virtual Environment Helps You Avoid System-wide Changes

Installing packages globally (without virtual environments) can lead to unintended changes to system-wide Python installations. Virtual environments keep project dependencies isolated from the system Python installation, reducing the risk of unintentional changes.

Imagine you’re developing a new Python project, Project C, and you need to install a specific version of the NumPy library for it. However, you already have several other Python projects on your system that rely on different versions of NumPy or even other libraries that might conflict with the version required by Project C.

If you were to install the required version of NumPy globally, it could potentially affect other projects on your system that rely on a different version. This situation could lead to unintended consequences such as:

To avoid these issues, virtual environments indeed provide a solution:

Let’s illustrate this with an example: 

# Create a virtual environment for Project C
$ python3 -m venv projectC_env

# Activate Project C virtual environment
$ source projectC_env/bin/activate

# Install the required version of numpy for Project C
(projectC_env) $ pip install numpy==1.21.0

Voila! Now, the NumPy version 1.21.0 is installed only within the virtual environment for Project C. Other projects and the system-wide Python installation remain unaffected. This ensures that Project C can run with its required dependencies without causing conflicts or unintended changes elsewhere.

Portability – Deploying the Project Across Different Environments

Virtual environments encapsulate all the dependencies required for a project, making it easier to share or deploy the project across different environments. This ensures that the project will run consistently regardless of the environment it’s deployed to.

Imagine you’ve developed a Python web application called “MyApp” that utilizes various libraries and web template engines such as Flask, SQLAlchemy, and Jinja2. You want to share this application with a colleague who will run it on their own machine. However, your colleague may have a different system configuration or may not have the exact versions of the required libraries installed.

Without virtual environments, your colleague would need to manually install all the necessary dependencies and ensure that they match the versions specified by your application. This process can be cumbersome and error-prone, especially if there are compatibility issues between different versions of libraries.

Here’s where virtual environments come to the rescue:

1. Create a Virtual Environment

You create a virtual environment specifically for your “MyApp” project:

$ python3 -m venv myapp_env

2. Activate the Virtual Environment

You activate the virtual environment to work within its isolated environment:

$ source myapp_env/bin/activate

3. Install Dependencies

Within the activated virtual environment, you install all the dependencies required by your application using a requirements.txt file:

$ pip install -r requirements.txt

4. Share the Project

Now, you can share your “MyApp” project along with the virtual environment. Your colleague can simply activate the virtual environment on their machine and run the application without worrying about compatibility issues or missing dependencies:

# On your colleague's machine
$ source myapp_env/bin/activate
$ python app.py

By encapsulating all the dependencies within the virtual environment, you ensure that your “MyApp” project is fully self-contained and can be easily shared or deployed across different environments. Whether it’s running on your colleague’s machine, a development server, or a production server, the project will run consistently regardless of the environment it’s deployed to. This promotes portability and simplifies the process of sharing and deploying Python projects.

Testing and Development Phases

Virtual environments are especially useful during development and testing phases. They allow developers to experiment with different packages and versions without affecting other projects or the system environment.

Imagine you’re a developer working on a new feature for your Python web application. As part of your development process, you need to experiment with different packages or library versions to find the best solution. However, installing these packages globally could potentially disrupt other projects or the stability of your system environment.

Here’s where virtual environments come into play:

1. Creating a Virtual Environment for Development

You create a virtual environment specifically for your development work on the new feature:

$ python3 -m venv myproject_dev_env

2. Activating the Virtual Environment

You activate the virtual environment to work within its isolated environment:

$ source myproject_dev_env/bin/activate

3. Installing Packages for Development

Within the activated virtual environment, you install the necessary packages or libraries for your development work. For example, you might want to experiment with a different version of a package

(myproject_dev_env) $ pip install package==x.y.z

4. Development and Testing

You can now work on your new feature, experiment with different packages or versions, and run tests within the isolated environment of the virtual environment. Any changes or installations you make are confined to this environment and do not affect other projects or the system environment.

#this is written in python, but all other code examples in this article are written in bash
#Example: Testing a new feature
# Within the activated virtual environment
(myproject_dev_env) $ python test_feature.py

5. Safe Environment for Experimentation

If your experiments lead to unexpected results or issues, you can easily revert or modify the virtual environment without impacting other projects or your system. Virtual environments provide a safe space for experimentation and development, allowing you to iterate quickly and efficiently.

By using virtual environments during development and testing, developers can experiment with different packages, library versions, or configurations without the risk of affecting other projects or the stability of the system environment. This promotes flexibility, productivity, and confidence in the development process.

Cleanup and Management

Virtual environments can be easily created, activated, deactivated, and deleted as needed. This makes it simple to manage dependencies for different projects and helps keep the system clean.

Imagine you’re working on multiple Python projects simultaneously, each with its own set of dependencies. Over time, these projects may accumulate virtual environments, and managing them effectively becomes crucial to avoid clutter and confusion.

Here’s how virtual environments facilitate cleanup and management:

1. Creating Virtual Environments

Whenever you start a new project, you create a dedicated virtual environment for it:

$ python3 -m venv project1_env
$ python3 -m venv project2_env
$ python3 -m venv project3_env

2. Activating and Deactivating Virtual Environments

You can easily switch between different virtual environments as needed:

# Activate project1_env
$ source project1_env/bin/activate

# Deactivate project1_env
$ deactivate

3. Installing and Managing Dependencies

Within each activated virtual environment, you install project-specific dependencies. This ensures that dependencies are isolated and do not interfere with other projects or the system environment:

# Within project1_env
(project1_env) $ pip install -r requirements.txt

4. Deleting Virtual Environments

Once a project is completed or no longer needed, you can delete its virtual environment, like so:

$ rm -rf project3_env

5. Automated Management

Tools like pipenv or poetry provide higher-level management of virtual environments and dependencies, simplifying tasks such as creating, activating, deactivating, and deleting environments.

# Install pipenv
$ pip install pipenv

# Create and activate a virtual environment with pipenv
$ pipenv shell

# Install dependencies
$ pipenv install requests

# Exit the virtual environment
$ exit

6. Keeping the System Clean

By utilizing virtual environments and managing them effectively, you keep your system clean and organized. Unnecessary dependencies and environments are removed, reducing clutter and potential conflicts.

By leveraging virtual environments and proper management practices, developers can streamline dependency management, ensure project isolation, and maintain a clean and organized development environment. This approach enhances productivity, reduces errors, and fosters better collaboration among team members.

In conclusion

In conclusion, virtual environments are a cornerstone of best practices in Python development, providing developers with powerful tools to manage project dependencies effectively. They ensure consistency, portability, and a clean development environment, thereby enhancing productivity and reducing the risk of errors. Let’s recap the key points highlighted in the article:

In essence, virtual environments empower developers to work efficiently, collaborate seamlessly, and ensure the reliability and maintainability of Python projects. By incorporating virtual environments into their workflow, developers can navigate the complexities of dependency management with confidence, ultimately delivering high-quality software solutions.