This lesson introduces the basics of parallelism. It will explain the importance of parallel programming with reference to high performance computing. It will cover the different types of parallelism and cover the common challenges associated with parallel programming.

Learning Objectives

• Understand the concept of parallelisation and its significance in improving performance.
• Know the different types of parallelism, including shared memory parallelism.
• Gain familiarity with the basics of OpenMP and how it enables shared memory parallelism.

In this lesson we will introduce what OpenMP programming is, its basic principles, and concepts. We will illustrate how to use the language to add parallelism directive into you code. How then to build and run this code on a HPC system. Finally we will discuss how to achieve a hybrid approach to parallelism using both OpenMP and  MPI.

Learning Objectives

• To learn what OpenMP is.
• To understand how to use the OpenMP API.
• To learn how to compile and run OpenMP applications.
• To understand the difference between OpenMP and low level threading APIs.

In this lesson, we will show how parallelism works in a typical C/C++ program. We will cover variable types and their scoping, and associated potential race conditions. We will show how to parallelise a simple loop and explain thread numbering. We will cover what a scheduler is with relation to data chunking and how an auto scheduler works. Finally, we will look at different scheduler options.

Learning Objectives

• To learn how to parallelise work in a program using OpenMP.
• To be able to describe the two major OpenMP pragma directives.
• To define and use a parallel region in our code.
• To use OpenMP library functions to obtain the number of available threads and the current thread identifier.
• To be able to describe the classes of visibility (or scoping) of variables between threads.
• To parallelise a for loop using OpenMP.
• To be able to describe the different schedulers available for how OpenMP assigns loop iterations to threads.
• To change the scheduling behaviour for an example program.

In this lesson we will help you understand how to write effective and correct code. To do this we will cover what thread synchronisation is and how it can cause race conditions. This is particularly important for algorithms which are iterative. We will show how you can identify possible race conditions and how to use barriers to help with synchronisation. We will cover synchronisation regions and some some preventative measure to stop race conditions. Then, finally, we will look at Atomics.

Learning Objectives

• To understand what thread synchronisation is.
• To understand what a race condition is.
• To learn how to control thread synchronisation.
• To learn how to avoid errors caused by race conditions.

In this lesson we go over what hybrid parallelism is, how it relates to heterogeneous computing, look at its advantages and disadvantages, and what we need to use hybrid parallelism. 

We will demonstrate how to use MPI and OpenMP application, covering ranks and threads, as well as how to submit your application to a workload manager, such as Slurm.

Learning Objectives

• Learn what hybrid parallelism is.
• Understand the advantages and disadvantages of hybrid parallelism.
• Learn how to use OpenMP and MPI together.