CIS 4205 - Archive Page

Cornell University
Computing and Information Science 4205:
Effective Use of High Performance Computing

Steve Lantz, Instructor for Spring 2009

* ARCHIVE WEBSITE * Featuring the outline, examples, and assignments from previous years - for reference only - most links do not work
Course Location and Time: Upson 205, MWF 10:10–11:00	Duration: 8 weeks, Jan. 19–Mar. 13
Credits: 2 Credit Hours, student option Successful/Unsuccessful or Letter Grade	Prerequisites: Proficiency in C, C++, Fortran, or Fortran 90/95
Recommended Text: Parallel Programming in C with MPI and OpenMP, by Michael J. Quinn, on reserve at the Engineering Library	Office Hours: 533 Rhodes Hall, by appointment Send email to slantz@cac.cornell.edu
Synopsis: A hands-on introduction to high-performance computing (HPC) for graduate students or advanced undergraduate students who will use HPC as a tool in their work. Various HPC architectural platforms are described with a focus on computational clusters. Students learn how to identify and exploit parallelism in algorithms and legacy applications; how to measure parallel speedup and efficiency; and how to diagnose bottlenecks affecting performance. Parallel programming with MPI, OpenMP, and task-farming techniques (for Web services and grid computing) is covered in depth. Examples and assignments are taken from typical application areas such as matrix and Monte Carlo computations. The goal of the class is for students to gain practical HPC experience for use in their specific fields.

Course Outline and Lecture Notes:
(*** UNDER CONSTRUCTION ***)
The outline is tentative and may be modified—students will be given a voice in what is ultimately presented.

(Note: Lectures may contain concepts and examples from the textbook and other sources listed below)

Week 1: Introduction to High Performance Computing
Lecture Notes ( ppt, pdf )

Week 2: Parallel Application Design
Lecture Notes ( ppt, pdf )

Week 3 & 4: Distributed Memory Programming—Using Message Passing Interface, MPI
Lecture Notes ( ppt, pdf )

Week 5: Shared Memory Programming—Using OpenMP
Lecture Notes ( ppt, pdf )

Week 6: Parallel Code Debugging, Performance Tuning, & Scalability
Lecture Notes ( ppt, pdf )

Week 7: Wide Area Distributed Computing
Lecture Notes ( ppt, pdf )

Week 8: New HPC Strategies—Data Driven Science
Lecture Notes ( ppt, pdf )

Examples:
Week 1: Introduction to High Performance Computing
helloworld.c
pi.c
RoundTrip.c
sample.bat & helloworld.xml job file ( sample.zip )

Week 2: Parallel Application Design
Simple Data Parallelism example ( dp1.c )

Simple Functional Parallelism example ( fp1.c )

Simple Task Parallelism example ( tp1.c )

Simple Pipeline Parallelism example ( pp1.c )

Simple example of using malloc for a 2 dimensional array ( 2d.c )

Serial 2D Heat Transfer Problem using malloc ( heat2d.c )

Parallel 2D Heat Transfer Problem using malloc ( pheat2d.c )

2D Heat Transfer Movie ( heat.wmv ) Irfanview — A nice image conversion program

Week 3 & 4: Distributed Memory Programming—Using Message Passing Interface, MPI

MPI API example (api_examples.c )

MPI All to All example (alltoall.c )

MPI Gather examples (gather.c & gather2.c )

MPI BagBoy example ( BagBoy.c )

Serial Bubble Sort example ( bubble.c )

MPI BagBoy Homework solution ( bb2.c )

MPI Parallel Bubble Sort solution ( pbub.c )

Week 5: Shared Memory Programming—Using OpenMP
OpenMP Batch script omp.bat (omp.zip )
OpenMP HelloWorld ( omp_helloworld.c )
OpenMP Data Parallelism example (omp_dp1.c )
OpenMP Pi example ( omp_pi.c )
OpenMP heat2d ( omp_heat2d.c )
OpenMP bubble-sort (odd-even approach) ( omp_bubble.c )
OpenMP firstprivate example ( omp_1stpriv.c )
OpenMP lastprivate example ( omp_lastpriv.c )
OpenMP parallel code block example ( omp_parallel.c )
OpenMP single clause example ( omp_single.c )
OpenMP nowait clause example ( omp_nowait.c )
OpenMP functional parallelism example ( omp_fp1.c )

Week 6: Parallel Code Debugging, Performance Tuning, & Scalability

Week 7: Wide Area Distributed Computing

Week 8: New HPC Strategies—Data Driven Science

Assignments:
(Note: Scroll Down for the class FAQ)
Assignment 1 - Due Thursday 2/7
1. Read Chapters 1 & 2 in the text.
2. Try logging in to a CAC Linux login node

http://www.cac.cornell.edu/Documentation/Linux

3. Identify an HPC problem relevant to your area of research.
* Obtain source code, batch scripts, and sample data for this problem.
4. Email me three examples where HPC is used in Cornell research.
5. Email me three examples where HPC is used outside of Cornell.
6. Email me your name, email address, your field of study, and your goals for CIS 405.
* Note: The text provides examples of several HPC problems.
7. Build and Run helloworld example from class in batch on the CAC development nodes.
8. Run the helloworld example from class interactively on the CAC development nodes.

Assignment 2—Due Thursday 2/21

1. Build the RoundTrip.c example using mpicc

- Try to understand how this code works

- Run RoundTrip.exe in batch using a machines file and 4 development nodes. Run with 4 and 8 processes

- Change the number of megabytes sent in the code, rebuild & rerun

- Email me the output from 3 different runs of this code

2. Read Chapter 3 in the text book

- Pay particular attention to section 3.3 — Foster’s Design Methodology

- Review the text examples that apply Foster’s Methodology

3. Read Chapters 4-6

- Pay attention to the various MPI calls and how they are used

4. Design a parallel version of Bubble Sort.

- Download serial version (bubble.c )

- Apply Foster’s Methodology to this problem

- Write and MPI Implementation for your parallel bubble sort

- use rand() function to generate a large array of random values to sort

- Email me:

- Describe each of the 4 steps of Foster’s Methodology for your solution

- MPI source code for your solution

- Batch script for running your program

- Output from your batch run

5. Modify BagBoy.c example ( BagBoy.c ) so that:

- each customer is helped in order

- once you’ve received all the items for each customer you sort them so that the heaviest
items get printed out first (bottom of the bag)

- Note: groceries array is in order such that smaller indices are “lighter” than higher indices.

- Email me:

- Source code of your BagBoy solution

- Batch script to run your program

- Output from your Batch run

Assignment 3 — Due Tuesday 3/25

write a serial version of the code “Sieve of Eratosthenes” Use the following algorithm and perl script as a guide
- sieve.pl
- “ Sieve of Eratosthenes Algorithm”
Optimized your serial “Sieve of Eratosthenes” code with OpenMP
Write an MPI version of the “Sieve of Eratosthenes”
Hints:
- Use MPI_Allreduce to determine the next value of “k”
- Use MPI_Gather to collect the sub-arrays at the end for the master process to print out.
- gather_int_arrays.c is an example you can use as an example.
Please fill out Course Survey

Assignment 3 Solutions

OpenMP solution

MPI solution

Cornell University Computing and Information Science 4205: Effective Use of High Performance Computing

Steve Lantz, Instructor for Spring 2009

Cornell University
Computing and Information Science 4205:
Effective Use of High Performance Computing