Introduction

This series of exercises will demonstrate the use of Make and CMake to compile C and C++ code using OpenMP. First, let's check that we have an appropriate GPU to work on using the rocm-smi command. Then, we'll enter the source directory and make sure that our working environment is clean.

The code

The code we will be running is contained in the files openmp_code_c.c for C and openmp_code_cxx.cc for C++. The source code is the same as that used in the OpenMP section of the notebook of the previous section 1b, i.e. moving the assignment and summation of an array to the GPU.

We can see the OpenMP instructions in the code by looking at the pragma omp statements in the code:

We will learn what these are doing in greater detail later, but for now it is enough to know that they are providing OpenMP instructions to the compiler.

Let's look at building these codes using Make.

Building with Make

The Makefile contained within this directory contains the instructions to build both the C and C++ versions of the code. Firstly, let's look at the Makefile:

Along with the usual CCFLAGS and LDFLAGS variables that we would expect to see in a standard Makefile, we can see an additional OPENMP_FLAGS_CC variable. This contains the flags necessary for the compiler to properly implement the OpenMP commands in our code. In particular, the -fopenmp is the flag that must be specified for both C and C++ to enable OpenMP directives.

ROCM_GPU is a string that denotes the GPUs architecture and can be a comma-separated list if targetting multiple GPUs in the same compilation. The series of commands $(strip $(shell rocminfo |grep -m 1 -E gfx[^0]{1} | sed -e 's/ *Name: *//')) checks our current system for ROCm compatible GPUs and finds this string. For the MI200s that we are using on this Cosma node, this string is gfx90a, and for the MI300 series, it is gfx942.

We can compile the C example using the openmp_code_c option within the Makefile. Let's build it now:

And now we can run the compiled code:

The compilation process is the same for C++, but using the openmp_code_cxx make command. Let's compile it now:

In order to execute the C++ version of this code, we need to set the environment variable HSA_XNACK to 1. This allows the operating system to automatically assign and move memory and control between the host and device. We will discuss this option further in later sections. For now, let's set it and run the code:

Congratulations, you've successfully built the OpenMP code using the Makefile! Now let's see how we can use CMake to build OpenMP code.

Building with CMake

We can also use CMake to build code with OpenMP directives. To accomodate our interactive notebooks, CMake has already been loaded to the environment. However, in some systems, you might need to load it with the command module load CMake.

CMake instructions are contained in the CMakeLists.txt file, which we will look at below:

The same -fopenmp and --offload-arch=${ROCM_GPU} flags that we used previously to enable OpenMP compilation are now included directly in the CMAKE_C_FLAGS variable.

We can now use CMake to set up a build directory within which we can compile the code. We do this by using the -B option, pointing it to the location of the desired directory where we wish it to build - commonly named build. We can then call CMake again with --build and the directory name we previously used to generate the build configuration, i.e. build. Let's do those now:

This will have compiled both the C and C++ versions of the code in the build directory.

We can now run the C version of the code in the usual way:

And finally the C++ version:

Now that we're comfortable enabling OpenMP directives in various build systems, let's start looking at single line pragma constructs in more detail.

This series of exercises will demonstrate the use of Make and CMake to compile Fortran code using OpenMP. First, let's check that we have an appropriate GPU to work on using the rocm-smi command. Then, we'll enter the source directory and make sure that our working environment is clean.

The code

The code we will be running is contained in the file openmp_code.F90. The source code is a version of that used in the OpenMP section of the notebook of the previous section 1b - i.e. moving the assignment and summation of an array to the GPU - ported to Fortran.

We can see the OpenMP instructions in the code by looking at the omp statements in the code:

We will learn what these are doing in greater detail later, but for now it is enough to know that they are providing OpenMP instructions to the compiler.

Let's look at building these codes using Make.

Building with Make

Firstly, let's look at the Makefile in this directory that contains the instructions to build the code:

Along with the usual FFLAGS and LDFLAGS variables that we would expect to see in a standard Makefile, we can see an additional OPENMP_FLAGS variable. This contains the flags necessary for the compiler to properly implement the OpenMP commands in our code. In particular, the -fopenmp is the flag that must be specified to enable OpenMP directives.

ROCM_GPU is a string that denotes the GPUs architecture and can be a comma-separated list if targetting multiple GPUs in the same compilation. The series of commands $(strip $(shell rocminfo |grep -m 1 -E gfx[^0]{1} | sed -e 's/ *Name: *//')) checks our current system for ROCm compatible GPUs and finds this string. For the MI200s that we are using on this Cosma node, this string is gfx90a, and for the MI300 series, it is gfx942.

As there is only one set of instructions within this Makefile, we can build the code by simply calling Make. We can also use the openmp_code option to build it directly.

We can compile the code:

And then run it:

Congratulations, you've successfully built the OpenMP code using the Makefile! Now let's see how we can use CMake to build OpenMP code.

Building with CMake

We can also use CMake to build code with OpenMP directives. To accomodate our interactive notebooks, CMake has already been loaded to the environment. However, in some systems, you might need to load it with the command module load cmake.

CMake instructions are contained in the CMakeLists.txt file, which we will look at below:

The same -fopenmp and --offload-arch=${ROCM_GPU} flags that we used previously to enable OpenMP compilation are now included directly in the CMAKE_Fortran_FLAGS variable.

We can now use CMake to set up a build directory within which we can compile the code. We do this by using the -B option, pointing it to the location of the desired directory where we wish it to build - commonly named build. We can then call CMake again with --build and the directory name we previously used to generate the build configuration, i.e. build. Let's do those now:

This will have compiled the code in the build directory.

We can now run it in the usual way:

Now that we're comfortable enabling OpenMP directives in various build systems, let's start looking at single line pragma constructs in more detail.

Q1:

Q2:

Q3: