1. Python virtual environment: Pipenv
   1. Dependency and environment management
      1. Idea of virtual environments
      2. Virtual Environments
   2. Installing Pipenv & Installing libraries with Pipenv
   3. Running things with Pipenv

Python virtual environment: Pipenv

Dependency and environment management

When you run ‘pip install scikit-learn,’ it searches in the directories listed in the $PATH variable, such as ~/anaconda3/bin/ in this case. Inside this folder, you have ‘pip,’ ‘python,’ and other packages. The ‘pip’ from this folder is used, and it connects to the Python Package Index (PyPI) at pypi.org. It then installs the latest version of the package into the directory specified in the $PATH variable.

Now, let’s consider a scenario where you have two applications:

1. Churn service, which relies on scikit-learn==0.24.2.
2. Lead scoring service, which depends on scikit-learn==1.0.

In this case, we have two different versions of scikit-learn in use, and they are isolated from each other. To manage this situation effectively, you can use different virtual environments.

Idea of virtual environments

We have two separate environments, each with its own Python installation for the services:

1. Churn service (Python resides in ~/venv/churn/bin/python)
2. Lead Scoring Service (Python resides in ~/venv/lead/bin/python)

Now, when you execute ‘pip install scikit-learn,’ it installs the package into its respective Python location. This approach ensures that you won’t encounter conflicts when using different versions of scikit-learn for the two services.

Virtual Environments

1. Virtual Environments: Virtual environments, often referred to as “venv” or “virtualenv,” allow you to create isolated Python environments for your projects. This separation helps manage dependencies and avoids conflicts between packages used in different projects.
2. Conda: Conda is an open-source package management and environment management system that can handle both Python and non-Python packages. It’s commonly used in data science and scientific computing for managing complex environments.
3. Poetry: Poetry is a dependency management and packaging tool for Python. It simplifies the process of managing project dependencies, packaging, and publishing Python packages. Poetry aims to provide a consistent and user-friendly approach to Python development.
4. Pipenv: Pipenv is another tool for managing dependencies and virtual environments. It combines pip (Python’s package installer) and virtualenv into one tool, making it easier to create and manage virtual environments and package dependencies for your projects.

Each of these tools has its strengths and may be more suitable for specific use cases. The choice of which one to use depends on your project’s requirements and your personal preferences.

Installing Pipenv & Installing libraries with Pipenv

To install Pipenv and libraries using Pipenv, follow these steps:

1. Install Pipenv by running the command:
   pip install pipenv
2. Install the desired libraries and specify their versions using the pipenv install command. For example:
   pipenv install numpy scikit-learn==0.24.2 flask gunicorn
   This will create two files, Pipfile and Pipfile.lock, in the local directory where this command is executed.
3. To use this environment on a different computer, navigate to the folder containing the Pipfile and Pipfile.lock files and run the following command:
   pipenv install
   This will recreate the virtual environment and install the specified libraries along with their versions, ensuring consistency across different machines.

Running things with Pipenv

To run things using Pipenv, you can follow these steps:

1. To activate the virtual environment for your project, use the command:
   pipenv shell
   Running “pipenv shell” allows you to use all the packages within the environment until you exit the environment with Ctrl+C.
   Alternatively, you can run a specific command within the virtual environment using:
   pipenv run <your_command_here>
   When you launch pipenv shell, it will display the folder where the virtual environment is stored.
2. You can confirm the location of a specific command within the virtual environment by running:
   which gunicorn
   This will show you the path to the version of the command installed in the virtual environment.
3. To retrieve the path to the virtual environment itself, you can use:
   echo $PATH
4. For running a specific command within the environment, like starting your web service with Gunicorn, you can use:
   pipenv run gunicorn --bind 0.0.0.0:9696 predict:app
   This allows you to use packages and commands within the virtual environment.

While Pipenv helps manage different Python package versions, it doesn’t address differences in system library versions installed via apt-get. For that level of isolation, you might need additional tools or configurations.