The plan was only to do two quick posts on RAID Performance on the X-25e, but this was compelling enough to post on it’s own. So in part I Mark Callaghan asked hey what gives with the SLC Intel’s single drive random write performance, It’s lower then the MLC drive. To be completely honest with you I had overlooked it, after all I was focusing on RAID performance. This was my mistake because this is actually caused by one of the Achilles heals of most flash on the market today, crappy performance when you fill more of the drive. I don’t really know what the official title for it is but I will call it “Drive Overeating”.
Let me try and put this simply: a quick trick most vendors use to push better random write #’s and help wear leveling is to not fully erase blocks and use free space on the drive as a place where they can basically “write random blocks sequentially”. They can then free the deleted blocks later on. When the drive becomes overly full ( Overeats), performance starts to degrade as blocks must be fully erased. So as you get closer to capacity drive speeds will slow. I had heard that intel reserves some flash that is not accessible to help boost performance ( for example 32GB is available, but an additional XGB is hidden for these operations ) but I have not seen this in print anywhere. If someone has a link on this I would appreciate it.
Anyways, what you want to see consistent #’s from various size datasets in fileio when using flash. We maybe adjusting the size, but the threads + the # of requests is going to stay the same… so theoretically a 32thread test with 2 million requests on a 20GB should produce similar #’s to a 40GB test with the same setup ( unlike regular disk we do not need to move around to find the random block ). Now their is always special circumstances, i.e. cache, external processes, etc that can influence the numbers slightly, but the goal is consistancy. Take a look at the earlier Violin SSD benchmarks the 100% writes ( 0% on the graph ) 20Gb is very close to the 360GB in terms of performance. Now lets take a closer look at the X-25e:
|Run #1 – 12G||3887.52|
|Run #2 – 12G||3837.52|
|Run #3 – 12G||3894.7|
|Run #1 – 20G||2943.68|
|Run #2 – 20G||2749.56|
|Run #3 – 20G||2745.14|
|Run #1 – 28G||2149.56|
|Run #2 – 28G||1966.99|
|Run #3 – 28G||1956.95|
You can see here that running the fileio test with a 12G dataset actually is nearly 2x faster then running the same test over 28G. Further let’s see what happens if you run a 12G test directly after a 28G test finishes:
|12G – After a 28G||1960.67|
|12G – After a 12G||3894.7|
Once again we double the performance of the test. So heavy write loads are going to suffer a substantial performance dip. This is going to also slowly degrade over time as the background process struggles to keep up. I put several 28G tests and ran them back to back to try and see if their was some point where I stabilized:
Even after 12 test’s we are still dropping, although its a much slower drop then when we started out. Bottom line here, I think a optimal performance best practice for using these drives is to reserve some amount of space when creating the partition to prevent the Overeating effect from hampering performance. Depending on your wrokload this maybe as high as 50% of the total drive, which then makes you really recalculate the $/gb & $/io costs.