Section: The Course | Directive-Based Programming for GPUs | DiRAC Training

GPU Developer

Section: The Course | Directive-Based Programming for GPUs | DiRAC Training

The Course
- Select activity Introduction
  
  Introduction Page
  
  Students must
  
  Mark as done
  
  This lesson introduces OpenMP, a pragma-based Application Program Interface (API) for shared-memory parallelism, in which directives are used to instruct the compiler on how to parallelise and offload code to the device.
  Throughout the lesson, its utility and how to begin implementing its pragma-based paradigm into code will be explained.
  It will discuss the compilers that can implement OpenMP directives and how to enable them when compiling.
  Further to this, it will explain how to incorporate these actions into make and cmake build systems.
  There will then be a discussion on the types of code that are best suited to GPU offloading, and how to know when it is appropriate to do so.
  The practical exercises will demonstrate how to compile a simple OpenMP-enabled code and run it on the GPU device.
  
  Learning objectives
  
  By the end of this lesson, you will be able to:
  
  Understand the basis of the OpenMP API and standard with respect to GPUs
  
  Know which compilers are available for OpenMP offload to GPUs and how to enable them
  
  Determine whether your code is appropriate for offloading to a GPU device
  
  Check that your code is running as expected on the GPU device
  
  Compile code incorporating OpenMP using make and cmake, and run it on a GPU device.
- Select activity Compute Constructs Single Line
  
  Compute Constructs Single Line Page
  
  This lesson introduces single line OpenMP constructs that offload simple calculations to the device.
  We will learn how to identify the areas of code that might benefit from such parallelisation and discuss the memory movement problems that can hold back potential performance improvements.
  We will introduce the most important pragmas that are used for this offloading and see some simple examples of their use.
  To check that everything is working as expected, we will learn ways to verify that the code is running correctly on the device, along with how to properly calculate the speed-up gained from implementing the offloading.
  
  Learning objectives
  
  By the end of this lesson, you will be able to:
  
  Know how to offload simple computational loops using one-line compute pragmas
  
  Determine that code is running correctly on the GPU
  
  Understand how to evaluate computational speed-up
  
  Understand the memory movement issues that might be encountered in GPU porting.
  
  Not available unless: You belong to any group
- Select activity Complex Compute Constructs
  
  Complex Compute Constructs Page
  
  This lesson will go into more detail regarding the OpenMP parallel pragma introduced in the previous section.
  It will break down the different clauses that make up the command, explaining their relevance in parallelisation and the specific role they play in it.
  It will then discuss how different combinations of these clauses impact parallel running on the device, and when these alternative scenarios might be useful.
  This lesson will also begin our journey into kernel optimisations (covered in more detail in the Optimisation section - by introducing additional tuning clauses, and giving us further command-line monitoring options that show how effectively we are using the device.
  
  Learning Objectives
  
  By the end of this lesson, you will be able to:
  
  Identify and understand the different compute clauses that make up the standard OpenMP parallel offload pragma
  
  Understand how different combinations of these clauses impacts the parallelisation of the offloaded code
  
  Use environment variables to check information on launched kernels and how they are being run on the device
  
  Understand some preliminary steps toward optimising offloaded code.
  
  Not available unless: You belong to any group
- Select activity Explicit Memory Directives
  
  Explicit Memory Directives Page
  
  In the previous lesson we've relied on OpenMP's automatic memory management to transfer data between host and device.
  This lesson introduces explicit memory management and movement, which affords us better control of how and when memory operations occur.
  This control ultimately allows us to optimise memory transfers in our programs.
  In this lesson we focus on the map and update clauses, two ways of moving data between host and device.
  We also discuss some common pitfalls associated with explicit memory management, particularly in complex cases, and introduce some tools available to help debug issues.
  
  Learning Objectives
  
  By the end of this lesson, you will be able to:
  
  Understand the concept of OpenMP regions and know the difference between structured and unstructured data regions
  
  Implement the map clause and update directive into your code to manage and move memory between the host and device
  
  Understand some of the pitfalls in explicit memory management on host and device
  
  Know how to output information on data transfers at runtime for all or parts of your code.
  
  Not available unless: You belong to any group
- Select activity Managed Memory
  
  Managed Memory Page
  
  In this lesson, we add to our growing list of memory management techniques in OpenMP by introducing managed memory: an approach that lets the OS and runtime coordinate memory operations in a more sophisticated way than OpenMP's traditional implicit memory management.
  This allows developers to write performant GPU code without explicitly managing memory and can enable offloading of traditionally tricky, dynamic data structures like std::vector.
  Specifically, we will learn about AMD's various implementations of managed memory systems, the hardware that supports them, and how we can write software that remains compatible with unsupported hardware.
  
  Learning Objectives
  
  By the end of this lesson, you will be able to:
  
  Understand the terminology for managed memory and similar capabilities
  
  Utilise managed memory on a system that supports shared unified memory
  
  Understand how to write code that supports unified shared memory, but can also run on a system without it
  
  Offload std::vector and valarray to a device with shared unified memory.
  
  Not available unless: You belong to any group
- Select activity Reduction Atomics Mutexes
  
  Reduction Atomics Mutexes Page
  
  This lesson will introduce the concepts of reductions, scans, atomics and mutexes in OpenMP: all tools that allow developers to write performant, parallel code while avoiding race conditions.
  After introducing these tools, we will explore their appropriate use through various examples, the performance implications of each tool, and potential optimisations if we're willing to part from using "safe" floating point operations.
  
  Learning Objectives
  
  By the end of this lesson, you will be able to:
  
  Understand what atomics and mutexes are, and how they can prevent race conditions
  
  Choose an appropriate approach to preventing memory conflicts in your code
  
  Understand how to use reduction operations, and when they are appropriate
  
  Use fast or safe atomic operations in your code
  
  Understand what a scan operation is.
  
  Not available unless: You belong to any group
- Select activity Device Subroutines
  
  Device Subroutines Page
  
  So far, we've used OpenMP to offload loops or regions of code that run simple operations without function or subroutine calls.
  This is because any functions called from the device must be explicitly annotated with particular OpenMP clauses to enable their compilation for the target architecture, the host architecture, or both.
  These clauses are introduced in this lesson.
  We will additionally discuss practical concerns about portability, compiler compatibility, and future updates.
  The practical exercises for this section will bring together everything learned through earlier lessons in a hands-on programming activity.
  
  Learning Objectives
  
  By the end of this lesson, you will be able to:
  
  Call a subroutine or function from a device target region
  
  Understand that a function called in a target region must be compiled for the specific device architecture on which it is running
  
  Compile versions of subroutines exclusively for the device, the host, or a combination of the two
  
  Declare data on the target device
  
  Understand portability concerns and the rapidly changing nature of modern OpenMP support in compilers.
  
  Not available unless: You belong to any group
- Select activity Optimisation
  
  Optimisation Page
  
  Previous lessons have introduced us to many features of OpenMP -such as reductions and explicit memory transfers- that allow us to write performant parallel code for our accelerating device.
  This lesson will instead focus solely on optimisation options available in OpenMP-enabled code.
  It will discuss important factors that impact performance, and some some techniques and ideas for improving it.
  In particular, we discuss the significant cost of data transfers between host and device, the impact of memory allocations and deallocations in performance-critical regions, and the impact of varying page size and memory alignment.
  We will learn various methods for mitigating these factors, either reducing their impact or removing them altogether.
  
  Learning Objectives
  
  By the end of this lesson, you will be able to:
  
  Understand that device/host data transfer is one of the largest impacts on performance for discrete GPUs, and identify ways to reduce this
  
  Allocate and free memory efficiently
  
  Implement a memory pool to reduce unavoidable allocation times
  
  Understand how page size impacts performance, and how to approach changing it on your system
  
  Change memory alignment in your code to improve performance
  
  Use the LIBOMPTARGET_KERNEL_TRACE environment variable to output kernel resource usage, and know the available OpenMP clauses that can influence these.
  
  Not available unless: You belong to any group
- Select activity Interoperability
  
  Interoperability Page
  
  So far, we've exclusively used OpenMP to offload kernels to a device, but in this lesson we will discuss the interoperability of OpenMP and HIP, AMD's lower level GPU programming language.
  We'll explore the benefits of including multiple programming models in a single application and the logistics involved.
  This will include how how HIP application code can call OpenMP kernels and vice-versa, how we can use compiler hints to signal whether data is stored on the host or device and how we can compile such codes with an example Makefile.
  We will even see how we can combine sources designed for different compilers into a single source file.
  
  Learning Objectives
  
  By the end of this lesson, you will be able to:
  
  Understand why multiple programming models may be beneficial for performant and portable code
  
  Call a HIP kernel from an OpenMP application and vice versa
  
  Use multiple compilers to correctly compile your interoperable code and understand how they can be combined into the same executable
  
  Understand how APUs can simplify interoperable programming.
  
  Not available unless: You belong to any group