Skip to main content
Version: 1.0

1. Install Python virtual environment

Python virtual environment

When working with Python, there may be cases where you want to use multiple versions of Python environments or manage package versions separately for different projects.

To easily manage Python environments or Python package environments in a virtualized manner, there are tools available such as pyenv, conda, virtualenv, and venv.

Among these, MLOps for ALL covers the installation of pyenv and pyenv-virtualenv.
pyenv helps manage Python versions, while pyenv-virtualenv is a plugin for pyenv that helps manage Python package environments.

Installing pyenv

Prerequisites

Prerequisites vary depending on the operating system. Please refer to the following page and install the required packages accordingly.

Installation - macOS

  1. Install pyenv, pyenv-virtualenv
brew update
brew install pyenv
brew install pyenv-virtualenv
  1. Set pyenv

For macOS, assuming the use of zsh since the default shell has changed to zsh in Catalina version and later, setting up pyenv.

echo 'eval "$(pyenv init -)"' >> ~/.zshrc
echo 'eval "$(pyenv virtualenv-init -)"' >> ~/.zshrc
source ~/.zshrc

Check if the pyenv command is executed properly.

pyenv --help
$ pyenv --help
Usage: pyenv <command> [<args>]

Some useful pyenv commands are:
--version Display the version of pyenv
activate Activate virtual environment
commands List all available pyenv commands
deactivate Deactivate virtual environment
exec Run an executable with the selected Python version
global Set or show the global Python version(s)
help Display help for a command
hooks List hook scripts for a given pyenv command
init Configure the shell environment for pyenv
install Install a Python version using python-build
local Set or show the local application-specific Python version(s)
prefix Display prefix for a Python version
rehash Rehash pyenv shims (run this after installing executables)
root Display the root directory where versions and shims are kept
shell Set or show the shell-specific Python version
shims List existing pyenv shims
uninstall Uninstall a specific Python version
version Show the current Python version(s) and its origin
version-file Detect the file that sets the current pyenv version
version-name Show the current Python version
version-origin Explain how the current Python version is set
versions List all Python versions available to pyenv
virtualenv Create a Python virtualenv using the pyenv-virtualenv plugin
virtualenv-delete Uninstall a specific Python virtualenv
virtualenv-init Configure the shell environment for pyenv-virtualenv
virtualenv-prefix Display real_prefix for a Python virtualenv version
virtualenvs List all Python virtualenvs found in `$PYENV_ROOT/versions/*'.
whence List all Python versions that contain the given executable
which Display the full path to an executable

See `pyenv help <command>' for information on a specific command.
For full documentation, see: https://github.com/pyenv/pyenv#readme

Installation - Ubuntu

  1. Install pyenv and pyenv-virtualenv
curl https://pyenv.run | bash

If the following content is output, it means that the installation is successful.

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- 0 0 0 0 0 0 0 0 --:--:-- --:--:-- 100 270 100 270 0 0 239 0 0:00:01 0:00:01 --:--:-- 239
Cloning into '/home/mlops/.pyenv'...
r
...
Skip...
...
remote: Enumerating objects: 10, done.
remote: Counting objects: 100% (10/10), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 10 (delta 1), reused 6 (delta 0), pack-reused 0
Unpacking objects: 100% (10/10), 2.92 KiB | 2.92 MiB/s, done.

WARNING: seems you still have not added 'pyenv' to the load path.


# See the README for instructions on how to set up
# your shell environment for Pyenv.

# Load pyenv-virtualenv automatically by adding
# the following to ~/.bashrc:

eval "$(pyenv virtualenv-init -)"

  1. Set pyenv

Assuming the use of bash shell as the default shell, configure pyenv and pyenv-virtualenv to be used in bash.

sudo vi ~/.bashrc

Enter the following string and save it.

export PATH="$HOME/.pyenv/bin:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"

Restart the shell.

exec $SHELL

Check if the pyenv command is executed properly.

pyenv --help

If the following message is displayed, it means that the settings have been configured correctly.

$ pyenv
pyenv 2.2.2
Usage: pyenv <command> [<args>]

Some useful pyenv commands are:
--version Display the version of pyenv
activate Activate virtual environment
commands List all available pyenv commands
deactivate Deactivate virtual environment
doctor Verify pyenv installation and development tools to build pythons.
exec Run an executable with the selected Python version
global Set or show the global Python version(s)
help Display help for a command
hooks List hook scripts for a given pyenv command
init Configure the shell environment for pyenv
install Install a Python version using python-build
local Set or show the local application-specific Python version(s)
prefix Display prefix for a Python version
rehash Rehash pyenv shims (run this after installing executables)
root Display the root directory where versions and shims are kept
shell Set or show the shell-specific Python version
shims List existing pyenv shims
uninstall Uninstall a specific Python version
version Show the current Python version(s) and its origin
version-file Detect the file that sets the current pyenv version
version-name Show the current Python version
version-origin Explain how the current Python version is set
versions List all Python versions available to pyenv
virtualenv Create a Python virtualenv using the pyenv-virtualenv plugin
virtualenv-delete Uninstall a specific Python virtualenv
virtualenv-init Configure the shell environment for pyenv-virtualenv
virtualenv-prefix Display real_prefix for a Python virtualenv version
virtualenvs List all Python virtualenvs found in `$PYENV_ROOT/versions/*'.
whence List all Python versions that contain the given executable
which Display the full path to an executable

See `pyenv help <command>' for information on a specific command.
For full documentation, see: https://github.com/pyenv/pyenv#readme

Using pyenv

Install python version

Using the pyenv install <Python-Version> command, you can install the desired Python version.
In this page, we will install the Python 3.7.12 version that is used by Kubeflow by default as an example.

pyenv install 3.7.12

If installed normally, the following message will be printed.

$ pyenv install 3.7.12
Downloading Python-3.7.12.tar.xz...
-> https://www.python.org/ftp/python/3.7.12/Python-3.7.12.tar.xz
Installing Python-3.7.12...
patching file Doc/library/ctypes.rst
patching file Lib/test/test_unicode.py
patching file Modules/_ctypes/_ctypes.c
patching file Modules/_ctypes/callproc.c
patching file Modules/_ctypes/ctypes.h
patching file setup.py
patching file 'Misc/NEWS.d/next/Core and Builtins/2020-06-30-04-44-29.bpo-41100.PJwA6F.rst'
patching file Modules/_decimal/libmpdec/mpdecimal.h
Installed Python-3.7.12 to /home/mlops/.pyenv/versions/3.7.12

Create python virtual environment

Create a Python virtual environment with the pyenv virtualenv <Installed-Python-Version> <Virtual-Environment-Name> command to create a Python virtual environment with the desired Python version.

For example, let's create a Python virtual environment called demo with Python 3.7.12 version.

pyenv virtualenv 3.7.12 demo
$ pyenv virtualenv 3.7.12 demo
Looking in links: /tmp/tmpffqys0gv
Requirement already satisfied: setuptools in /home/mlops/.pyenv/versions/3.7.12/envs/demo/lib/python3.7/site-packages (47.1.0)
Requirement already satisfied: pip in /home/mlops/.pyenv/versions/3.7.12/envs/demo/lib/python3.7/site-packages (20.1.1)

Activating python virtual environment

Use the pyenv activate <environment name> command to use the virtual environment created in this way.

For example, we will use a Python virtual environment called demo.

pyenv activate demo

You can see that the information of the current virtual environment is printed at the front of the shell.

Before

mlops@ubuntu:~$ pyenv activate demo

After

pyenv-virtualenv: prompt changing will be removed from future release. configure `export PYENV_VIRTUALENV_DISABLE_PROMPT=1' to simulate the behavior.
(demo) mlops@ubuntu:~$

Deactivating python virtual environment

You can deactivate the currently active virtualenv by using the command source deactivate.

source deactivate

Before

(demo) mlops@ubuntu:~$ source deactivate

After

mlops@ubuntu:~$