Description
Install PnetCDF (https://github.com/Parallel-NetCDF/PnetCDF.git) and
S3D-IO (https://github.com/wkliao/S3D-IO)
3.1. Modify one or more (as many as you can) MPI-IO hints to improve the default I/O (both
read and write) performance of S3D-IO. Report the hints (and their values) that gave the best
performance. You may modify the hints in the source code or during runtime. (30 marks)
3.2. Study and understand the source code for non-blocking I/O (pnetcdf_m.f90). Modify the I/O
algorithm used in the code (mainly pnetcdf_m.f90) to improve read and write performance. You
may modify any other file you wish. (50 marks)
3.3. Execute your new improved algorithm of 3.2. on HPC2010. (Optional, 25 marks)
——————————————————————————————————————————
Execution Instructions
The default code uses nsc/n_spec (number of species) = 11. Add nsc as an input argument (have
it as the last argument). Test your code with 11, 52 and 100 species. You may alter the codebase
as you wish. Plot the read and write bandwidths for all configurations (given below). The y-axis
should have bandwidth (in MB/s). The x-axis should have the 9 configuration points (see below).
Repeat each experiment (configuration point) 10 times and plot the boxplot for each point. Use
CSE cluster for 3.1 and 3.2.
3.1. Run configurations (each with nsc=11,52,100) [Total 9]. The boxplot for each point should
correspond to the best set of hints for that configuration point (in case you found any difference
in performance depending on the hints for each point).
• ./s3d_io.x 100 100 200 2 2 4 1 F output/ (2 nodes x 8)
• ./s3d_io.x 100 200 200 2 4 4 1 F output/ (4 nodes x 8)
• ./s3d_io.x 200 200 200 4 4 4 1 F output/ (8 nodes x 8)
Directory structure (Necessary files)
– S3D-IO-3.1 (must contain code modifications that gave the best overall performance, if any)
– run.sh or run.py (should run all your options)
– output.txt (should contain the combined output of S3D from all your runs)
– report.pdf (what options did you try, and why, what options made the most significant impact,
and why, other observations, plots etc.)
– plot script
– plot.pdf
3.2. Run configurations (each with nsc=11,52,100) [Total 9] You may additionally alter the MPIIO hints, if you wish. However, the codebase should be uniform for all configurations.
• ./s3d_io.x 100 100 200 2 2 4 1 F output/ (2 nodes x 8)
• ./s3d_io.x 100 200 200 2 4 4 1 F output/ (4 nodes x 8)
• ./s3d_io.x 200 200 200 4 4 4 1 F output/ (8 nodes x 8)
Directory structure (Necessary files)
– S3D-IO-3.2 (should contain all source code changes)
– run.sh or run.py
– output.txt (should contain the combined output of S3D from all your runs)
– report.pdf (what changes were made and why did you make those changes, was there any
improvement, how much was the improvement, other observations, plots etc.)
– plot script
– plot.pdf
3.3. Almost same as 3.2, on HPC2010 and with the following run configurations (each with
nsc=11,52,100) [Total 9]
• ./s3d_io.x 200 400 400 2 4 4 1 F output/ nsc (4 nodes x 8)
• ./s3d_io.x 400 400 400 4 4 4 1 F output/ nsc (8 nodes x 8)
• ./s3d_io.x 400 400 800 4 4 8 1 F output/ nsc (16 nodes x 8)
General submission instructions
Each submission subfolder (named 3.1, 3.2, 3.3) should contain the modified codebase of S3D-IO
(S3D-IO-3.x), ‘readme’, job script (‘run.py’ or ‘run.sh’), ‘output.txt’, ‘report.pdf’, plot script and
‘plot.pdf’. The job script should run all the configurations. I should be able to run the job script
to execute your code. Optionally, you may automate plot generations. The ‘report.pdf’ should
contain your observations regarding performance/speedup reasons etc. as mentioned above.
You may document other issues and execution instructions in ‘readme’. Plot read and write
bandwidth for all configurations in a single file ‘plot.pdf’. You may have other auxiliary files, if
required.
Other components (20 marks)
– Use Assignment3 as the main folder name
– Follow the above instructions carefully
– Use git.cse.iitk.ac.in as a real git repo (i.e. marks will be awarded on progress)
– Neat coding, documented code
– Fully automated execution
Due date: 27-03-2019 (There will be NO extensions)
Penalty of -15 if I cannot run your code upfront, e.g., you do not follow the above instructions,
especially regarding naming conventions / missing files / hardcoded directory names ….
