Exercise 1 : ioping

ioping is a tool to monitor I/O latency in real time. It shows disk latency in the same way as ping shows network latency. It creates temporary files in the indicated directory and then tests the latency of read operations on them.

In your /short/c37 area on Raijin create a suitably named sub-directory for this session, and in there check out the following Git repos:

• FIO: https://github.com/axboe/fio.git
  
• ioping: https://github.com/koct9i/ioping.git

man ioping/ioping.1
man fio/fio.1

Note: Raijin has three filesystems accessible to end-users: /home, /jobfs and /short.

ioping : Measuring latency for single threaded sequential workloads

1. Using ioping, measure the IO latency for /short. Construct PBS jobs or use the interactive queue to use 1 CPU and run the ioping executable and record latency (in milliseconds), IOPS and B/W for block sizes of 4KB, 128KB and 1MB and working set sizes of 10MB, 100MB, 1024MB. Note: the `working set size' is the size of the temporary file that ioping creates.
   Hint: use shell commands in your batch file for running multiple experiments, and to extra results from the job output files. An example run, with 20 requests, 8KB block size and for working set of 1GB on a home directory, using cached I/O:

   % /short/z00/jxa900/ioping -c 20 -s 8KB -S 1024MB -C /home/900/jxa900
   8 KiB from /home/900/jxa900 (lustre
   10.9.103.3@o2ib3:10.9.103.4@o2ib3:/homsys): request=20 time=29 us
   
   --- /home/900/jxa900 (lustre --
   --10.9.103.3@o2ib3:10.9.103.4@o2ib3:/homsys) ioping statistics ---
   20 requests completed in 2.79 ms, 160 KiB read, 7.16 k iops, 55.9MiB/s
   min/avg/max/mdev = 28 us / 139 us / 289 us / 103 us
   

   From the above experiment, the corresponding data values are:
   

   Working Set (MB)	Block Size (KB)	Time Taken (ms)	Data Read (KB)	IOPS	B/W (MB/sec)
   1024	8	2.79	160	7160	55.9

   (Later, if you have time/interest): repeat the above without the -C and/or with -D, to see more effects of caching)

2. Using your results from the above questions, complete the following table.

Working Set (MB)	Block Size (KB)	Time Taken (ms)	Data Read (KB)	IOPS	B/W (MB/sec)
10	4
10	128
10	1024
100	4
100	128
100	1024
1024	4
1024	128
1024	1024

3. Explain the observed trends for IOPS and B/W for the three working set sizes, on changing the request block size.
4. (Optional) Run this exercise on a Raijin compute nodes’ /jobfs, produce a similar table and compare/contrast the results. From your batch script, you can access your jobs directory in this filesystem via /jobfs/local/$PBS_JOBID, and you can ensure sufficient space by adding the PBS detective #PBS -l jobfs=5GB

Exercise 2 : fio

fio: Measuring IOPS for single threaded sequential workloads

5. In your /short/c37 directory for this session, create an empty sub-directory called testIO, for fio to create temporary files in. Using fio, measure read and write IOPS for two, four and eight threads of IO running on /short for a block size of 1MB and a file size of 1GB.
   These can be set when running fio using the --bs=1M and --size=1G flags. For a sequential write workload, add -–readwrite=write. The flags --ioengine=libaio --gtod_reduce=1 are recommended for good and repeatable performance, respectively. For example, the following creates 2 threads (--thread --numjobs=2):

   fio/fio --ioengine=libaio --gtod_reduce=1 --bs=1024K  --size=1G \
     --readwrite=write --thread --numjobs=2 --name=test --directory=./testIO
   

   and will produce output like:

   test: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=1
   ...
   Starting 2 threads
   Jobs: 2 (f=2)
   test: (groupid=0, jobs=1): err= 0: pid=22754: Mon May 15 18:25:09 2017
     write: IOPS=627, BW=628MiB/s (658MB/s)(1024MiB/1631msec)
     ...
   test: (groupid=0, jobs=1): err= 0: pid=22755: Mon May 15 18:25:09 2017
     write: IOPS=652, BW=653MiB/s (684MB/s)(1024MiB/1569msec)
     ...
   Run status group 0 (all jobs):
     WRITE: bw=1256MiB/s (1317MB/s), 628MiB/s-653MiB/s (658MB/s-684MB/s), io=2048MiB (2147MB), run=1569-1631msec
   

   The command ls ./testIO will display the temporary files fio generated.

   Using a suitable batch script, run experiments on fio and complete the following table for sequential write IO performance (in the output, run=... is `Time Taken', and io=... is `Data Written'):

   No of Threads	File Size (MB)	Block Size (KB)	Time Taken (ms)	Data Written (MB)	IOPS	B/W (MB/sec)
   2	1024	1024
   4	1024	1024
   8	1024	1024