Exercise 1 : Lustre Striping

Setting striping on a Lustre filesystem

• the stripe count controls how many OSTs the file is striped over (a value of -1 means to stripe over all OSTs);
• the stripe size controls the number of bytes in each stripe; and
• the start index chooses where in the list of OSTs to start the round-robin assignment.
  (the default value -1 allows the system to choose the offset in order to load balance the file system).

% lfs setstripe [file,dir] [-c count] [-S size] [-i index]

% lfs getstripe [file,dir]

A file automatically inherits the striping parameters of the directory it is created in, so changing the parameters of a directory is a convenient way to set the parameters for a collection of files you are about to create. For instance, if your application creates output files in a sub-directory called output/, you can set the striping parameters on that directory once before your application runs, and all of your output files will inherit those parameters.

1. In your /short/c37 directory for this session, create a directory called testIO, run lfs getstripe ./testIO. Explain the output of the command, use the man pages if required.
2. Using the sub-directory called testIO, for fio to create temporary files in, use fio, to measure parameters for 4 threads of IO running on /short. The following options creates 2 threads (--thread --numjobs=2):

   fio/fio --ioengine=libaio --gtod_reduce=1 --bs=1024K  --size=1G \
     --readwrite=write --thread --numjobs=2 --name=test --directory=./testIO
   

   and will produce output like:

   test: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=1
   ...
   Starting 2 threads
   Jobs: 2 (f=2)
   test: (groupid=0, jobs=1): err= 0: pid=22754: Mon May 15 18:25:09 2017
     write: IOPS=627, BW=628MiB/s (658MB/s)(1024MiB/1631msec)
     ...
   test: (groupid=0, jobs=1): err= 0: pid=22755: Mon May 15 18:25:09 2017
     write: IOPS=652, BW=653MiB/s (684MB/s)(1024MiB/1569msec)
     ...
   Run status group 0 (all jobs):
     WRITE: bw=1256MiB/s (1317MB/s), 628MiB/s-653MiB/s (658MB/s-684MB/s), io=2048MiB (2147MB), run=1569-1631msec
   

   The command ls ./testIO will display the temporary files fio generated.

3. Fill in the below table, using FIO for the following Lustre stripe sizes and Lustre stripe counts for a 1GB file, utilizing a sequential write workload. Complete the following table, and comment on the trends:

Stripe Size (MB)	Stripe Count	No of Threads	Working Set (MB)	Block Size (KB)	Time Taken (ms)	Data Written (MB)	IOPS	B/W (MB/sec)
1	-1	4	1024	1024
1	2	4	1024	1024
1	4	4	1024	1024
4	-1	4	1024	1024
4	2	4	1024	1024
4	4	4	1024	1024

4. (Optional) Does stripe width and stripe count make sense on a /jobfs filesystem? If not, what is analogous to these for jobfs?
5. (Optional) Re-run the above with direct-IO (-D) and explain your results.