Modules (Lmod)

From CAC Documentation wiki
Revision as of 16:02, 31 May 2019 by Pzv2 (talk | contribs) (Finished Finding Modules section)
Jump to navigation Jump to search

Environment Modules are designed to provide a flexible development environment. Some of the CAC's Private Clusters are managed with OpenHPC, which includes the use of modules via Lmod. Lmod is an environment module system developed by TACC which provides you with a very easy way to configure your environment for compiling, linking, and running applications using a variety of different compilers and a variety of external libraries.

Some of the information on this page has been adapted from the Cornell Virtual Workshop topic on the Stampede2 Environment, specifically The Module Utility.

Basic Usage

Generally, the module utility allows you to configure your environment to prioritize the compiler you want, the libraries you need to link against, and any runtime dependencies you may have. It also helps you to run selected pre-built applications in the correct, fully-configured environment.

Some Commands

  • module with no options set will get you a listing of the available options for module.
  • module avail to find out what's available via the module system. This displays all available modules that you can load.
  • module list to list all of the current modules that you have loaded.
    • It's good to know what you already have loaded in your environment, as some of these modules may conflict with each other.
    • The caveat here is that this doesn't tell you anything about environment settings that have been changed outside of the module command. So if you have attempted to manually configure or add a different compiler, it won't show up in this list. The moral of this is to be aware of potential conflicts when you manually add paths or introduce environment settings in other shell configuration files.
  • module spider - list all modules, even those not available to load
  • module load <module> add/load a module
  • module unload <module> unload a module

Finding Modules

In many cases, finding the module you need might be easily accomplished by using module avail or module spider, but let's consider a more complicated example. Say you'd like to run an R code that requires snow, an R package. Naturally, you would try module avail snow and/or module spider snow first, but both would fail to find it. However, it could still be there, included in another module.

If you try looking at R with module avail R, you will see the output arranged in groups, and that each group is ordered alphabetically so that uppercase letters precede lowercase ones. For example:

  Rstats/3.4.0
  RstatsPackages/3.4.0

Since snow is an R package, you can learn more about the which R packages are available with:

$ module spider rstatspackages

----------------------------------------------------------------------------
  RstatsPackages:
----------------------------------------------------------------------------
     Versions:
        RstatsPackages/3.4.0
        RstatsPackages/3.5.1

----------------------------------------------------------------------------

For detailed information about a specific "RstatsPackages" module (including how to load the modules) use the module's full name. For example:

$ module spider RstatsPackages/3.4.0

  ----------------------------------------------------------------------------
    RstatsPackages: RstatsPackages/3.4.0
  ----------------------------------------------------------------------------
  
      You will need to load all module(s) on any one of the lines below before the "RstatsPackages/3.4.0" module is available to load.
  
        intel/17.0.4  impi/17.0.3
   
      Help:
        This is the R statistics (RstatsPackages) package built on September 28, 2018.
        
        It includes the following accessory packages including but not limited to:
        Rmpi, snow, snowfall
        pdbMPI, pbdSLAP, pbdBASE,

And you can see in the above list that in this example we've found at least one module to load that includes snow, along with additional modules that are required. The additional modules turn out to be the defaults for the Intel compiler and Intel MPI.

Running the command module spider RstatsPackages/3.5.1 shows that it too contains snow. The reason we didn't see the 3.5.1 version initially, with module avail R, is because it depends on later versions of the Intel compiler and MPI. Once the appropriate modules are loaded, the 3.5.1 version of RstatsPackages becomes available. Therefore, prior to loading RstatsPackages/3.5.1, we would want to use module spider to find a valid pair of modules for the compiler and MPI, then make sure those modules are loaded.

Module Information

Loading Modules

Personal Modulefiles

Reference


References