While we have installed a small number of highly-used packages to each version of Python, we encourage users to maintain their own Python packages. This ensures that the user has no discontinuity in their workflow waiting for packages to be installed and that packages work with the user's chosen, loaded Python version module. Python packages are built specifically to a version of Python and may not work properly if a different version of Python is loaded. For example, there is still a great disparity between Python 2 and Python 3. O2's "module" feature allows users to quickly and easily switch between Python versions.
module avail python
|shows the versions of Python installed on O2 (requires |
module load python/version
|loads an individual module (substitute version for an actual version)|
module unload python/version
|unloads an individual module|
|shows the current version of Python loaded|
|shows what packages are installed for currently loaded Python module|
Note that in order for
module avail python to give any output, you must have
gcc/6.2.0 loaded into your environment. Alternately, you can use
module spider python to circumvent this, but you will still need
gcc/6.2.0 loaded in order to load any python modules that are installed.
Setting Up a Virtual Environment
These commands create a copy of our Python distribution in whichever directory you ran the command, placing it in a folder named after whatever you specified. You can name your environment however you'd like. The above examples generate a directories located at
If you'd like to use the packages that are pre-installed in the Python module, include the
--system-site-packages flag when creating the virtual environment:
--system-site-packages flag will allow your virtual environment to inherit all existing installed packages in that Python distribution (which means you don't need to install your own numpy or scipy). On O2, we have purposely kept the list of installed python modules as light as possible so that the user is afforded the maximum amount of flexibility to install the versions they need.
You can create as many virtual environments as you need.
To uninstall a virtual environment, simply
rm -rf the folder containing the directory:
To begin using the virtual environment, it needs to be activated. For a virtual environment located at
Your prompt will now look (something like) this:
From here you an confirm that you're using the python associated with your virtual environment:
If it points instead to something like
/usr/bin/python, that means your environment is not active. Once you have confirmed that you are indeed inside your virtual environment, you may do whatever you'd like, such as testing code or installing packages.
To deactivate the environment, simply type
deactivate, to see the corresponding change to your terminal prompt:
The executable for Python 3.x is python3 and not python, the choice was made by the Developers to mark a clear difference between version 2.x and 3.x, as old python 2.x scripts might not be fully compatible across the version jump.
A symbolic link python → python3 is created within each python 3.x virtual environment, this link allows to execute python3 scripts using python instead of python3 when the virtual environment is active, however the link does not exist outside of the python virtual environment.
Installing Packages (via
PyPI (The Python Package Index) is a repository of software for the Python programming language, with packages available listed at https://pypi.python.org/pypi.
To install something, simply type:
Notice that your virtual environment needs to be active. The above command will search PyPI and install the package to your virtual environment. If it is not active, you will get a permissions error because you will not have write access to the original installation.
You can also install packages manually; please refer to the instructions provided by the corresponding package if this is required (e.g. a README/INSTALL file or a documentation page).
Using Python Packages
In general, you will access packages by importing them at the top of your Python scripts:
This will allow you to access that package (called a "module")'s functions. Some modules will have extended functionality; please refer to the appropriate usage instructions provided by the package for more information on how to access them.
A Full Example
Here is an example of loading the desired version of Python on O2, setting up a virtual environment and activating it, installing a package to this personal environment, running a Python script, and deactivating the virtual environment (2.x chosen here arbitrarily):