# Installing and Running on Ubuntu¶

Users interested to work with PFLOTRAN-OGS at code level, can proceed with the installation and build from source code.

Ubuntu verions 16.04 and 18.04 are supported but not currently 20.04.

Below instructions are provided to install the software on the Ubuntu Desktop system. Support for clients working with their own builds is arranged on a case-by-case basis. Please get in touch with OpenGoSim to know more.

## Requirements¶

We assume an up to date Ubuntu installation, as well as:

• The software managment tool git, which can be installed with sudo apt install git

• Compilers for Fortran, C, C++, and other necesasary compilation tools and library. sudo apt install build-essential should fetch is required.

The other major requirment for PFLOTRAN-OGS is the library PETSc, which we will install in the next section. PETSc has a number of other dependencies we will need, but fortunately it can be configured to download and install all of these automatically, see the next section.

Use git to clone the PFLOTRAN-OGS 1.4 repo:

git clone https://bitbucket.org/opengosim/pflotran_ogs_1.4.git


This will place PFLOTRAN-OGS in a folder called pflotran_ogs_1.4.

## Installing PETSc¶

Create a directory for PETSc, for example:

mkdir petsc


Clone the PETSc repository from git into that directory using the following command:

git clone https://gitlab.com/petsc/petsc petsc


For compiling PETSc and PFLOTRAN-OGS we have to set some simple environmental variables. This is further explained in PETSc environmental variables but is not required reading.

First we must set PETSC_DIR to the path of the directory we cloned PETSc into, so for example:

export PETSC_DIR=/home/myusername/petsc


We must also set a variable called PETSC_ARCH. This is used for managing multiple installs/configurations of PETSc, so we could use anything vaguely descriptive. Following some standard convention let’s use:

export PETSC_ARCH=ubuntu-opt


because we’re installing on Ubuntu, and we’re going to configure for a fast running (optimised ‘opt’, as opposed to debugging, ‘dbg’) version of PETSc.

We’re going to need these variables defined every time we do any compilation of PETSc or PFLOTRAN, so it might be convinient to also paste these two lines into the .bashrc file so that they are defined every time a new terminal window is opened.

PFLOTRAN-OGS needs a specific snapshot of PETSc. We can check what this is in the file petsc-git-version.txt, included in the copy of PFLOTRAN-OGS we downloaded. For example:

nano pflotran_ogs_1.4/tools/buildbot/petsc/petsc-git-version.txt


For example the version at time of writing is v3.12.2, which we will use in this tutorial.

Next we go into the PETSc directory:

cd petsc


And checkout the specific snapshot which PFLOTRAN-OGS needs, for example:

git checkout v3.12.2


Now we configure PETSc. Among other things, this is the part where PETSc will download and install any dependent libraries if we ask it to. So we will ask it to:

./configure --download-mpich=yes --download-hdf5=yes --download-fblaslapack=yes --download-metis=yes  --download-cmake=yes  --download-ptscotch=yes download-hypre=yes --with-debugging=0 COPTFLAGS=-O3 CXXOPTFLAGS=-O3 FOPTFLAGS=-O3


For advanced user information about compiling PETSc against existing installs on any of these libraries, for example mpich or replacing it with openmpi, see this section.

This process will take a litle time. Eventually the configuration script will finish, and even tell us how to proceed. We should see and output like this at the end:

xxx=========================================================================xxx
Configure stage complete. Now build PETSc libraries with:
xxx=========================================================================xxx


So we copy paste this into the terminal:

make PETSC_DIR=/home/myusername/petsc PETSC_ARCH=ubuntu-opt all


Then we wait again for the build to complete. Eventually we will see more advice from the scripts:

Now to check if the libraries are working do:
=========================================


We again oblige by copy pasting:

make PETSC_DIR=/home/myusername/petsc PETSC_ARCH=ubuntu-opt check


Giving an output something like:

ubuntu-opt check
Running test examples to verify correct installation
Using PETSC_DIR=/home/daniel/new_petsc and PETSC_ARCH=ubuntu-opt
C/C++ example src/snes/examples/tutorials/ex19 run successfully with 1 MPI process
C/C++ example src/snes/examples/tutorials/ex19 run successfully with 2 MPI processes
C/C++ example src/snes/examples/tutorials/ex19 run successfully with hypre
C/C++ example src/vec/vec/examples/tutorials/ex47 run successfully with hdf5
Fortran example src/snes/examples/tutorials/ex5f run successfully with 1 MPI process
Completed test examples


## Installing PFLOTRAN-OGS¶

Now PETSc should be installed correctly and we are ready to move on to PFLOTRAN-OGS.

Navigate into the following directory:

cd pflotran_ogs_1.4/src/pflotran


Now we compile (in this case with four cores, to speed things up) by entering:

make -j4 pflotran


This should take a few minutes, after which, PFLOTRAN-OGS will be ready to use.

## Running Pflotran From the Command Line¶

This section will help you understand the options available for running PFLOTRAN-OGS from the command line.

### Through Scripts¶

In practice it is much more convinient to run PFLOTRAN-OGS through simple scripts. We provide an example here. Please open the file and read the short instructions before running. In particualar note that it must be updated with the path of your pflotran install directory (e.g., /home/myusername/pflotran), and needs the PETSC_DIR and PETSC_ARCH variables to be defined (see above).

### Directly Through the Command Line¶

A typical call to run PFLOTRAN-OGS from the command line will look like

/home/myusername/petsc/ubuntu-opt/bin/mpirun -np 4 /home/myusername/pflotran/src/pflotran -pflotranin spe10.in -output_prefix test_spe10_run


We now explain each part of this:

• /home/myusername/petsc/ubuntu-opt/bin/mpirun : we call mpirun to start a parallel program. Note that this is the mpirun binary that was installed as part of the PETSc configuration process above. We need to make sure we use the same MPI installation for compiling and running PFLOTRAN-OGS.

• -np 4 : an argument to mpirun, specifying how many proceseses to use. In this case, four.

• /home/myusername/pflotran/src/pflotran : this is the PFLOTRAN-OGS binary we compiled earlier.

• -pflotranin spe10.in : this is an argument to PFLOTRAN-OGS, specifying an input file to use, in this case called ‘spe10.in’.

• -output_prefix test_spe10_run : specify an output prefix to be applied to all output files generated by this run.

### Configuring against existing Open MPI Install¶

Here is an example:

CONFIGURE_OPTIONS = --with-debugging=0 --download-fblaslapack=1 --with-fc=/usr/lib64/openmpi/bin/mpif90 --with-cc=/usr/lib64/openmpi/bin/mpicc --with-cxx=/usr/lib64/openmpi/bin/mpicxx --with-mpi-include=/usr/include/openmpi-x86_64 --with-mpi-lib=/usr/lib64/openmpi/lib/libmpi.so --download-cmake=1 --download-ptscotch=1 -download-hypre=1 --download-hdf5=1 --with-c2html=0 COPTFLAGS=-O3 CXXOPTFLAGS=-O3 FOPTFLAGS=-O3 --with-shared-libraries=0


Note the lack of the --download-mpich=yes or --download-openmpi=yes options, as well as the presence of the following:

• A path to an MPI fortran compiler: --with-fc=/usr/lib64/openmpi/bin/mpif90

• A path to an MPI c compiler: --with-cc=/usr/lib64/openmpi/bin/mpicc

• A path to an MPI c++ compiler: --with-cxx=/usr/lib64/openmpi/bin/mpicxx

• A path to to the MPI include directory: --with-mpi-include=/usr/include/openmpi-x86_64

• A path to the MPI libraries: --with-mpi-lib=/usr/lib64/openmpi/lib/libmpi.so

Note that it might be necessary to ensure certain libraries are in path, e.g.

export LD_LIBRARY_PATH=/usr/lib64/openmpi/lib:\$LD_LIBRARY_PATH


Recall that you must use the mpirun binary associted with the Open MPI install for running PFLOTRAN-OGS, so the above example of running on the command line will generalizes to:

/location/of/correct/mpirun -np 4 /home/myusername/pflotran/src/pflotran -pflotranin spe10.in -output_prefix test_spe10_run