Waffle Grid: 300% better then before

Maybe I should make the sub-tag line, make your MySQL database run up to 300% faster by using Waffle Grid…   Ahhh its all about marketing.

Had an interesting holiday, snuck away several times to benchmark and test out the new Waffle Grid release… nothing says turkey day like Waffles and a really fast database  ( Hey! Yes my wife knew she married a complete and utter nerd, she just might not have understood the depths of my nerdom ).  As I mentioned before with the release of MySQL 5.1.30 I switched over all my testing over to this version, and in my tests this combination is running really well.  Additionally I had a hardware issue which compelled me to retest some of the tests I did last week. So who loves benchmarks!  I DO I DO!!!  I like to go fast, so lets get right down to it.

I ran a series of 20W DBT2 benchmarks on MySQL 5.1.30 with and without waffle grid ( and with a new patched memcached ), take a look:

DBT2 – Memcached Patch TPM
768M Local 3218.54
768M Local/768 Remote No MC patch 6575.8
768M Local/768 Remote MC patch 9371.54
1500M Local 16508.54

As you see the baseline test used a innodb_buffer_pool of 768M ( all these tests ran 16 threads hitting 20Wby the way )  with which we achieved a baseline New Order TPM of 3228.  By Enabling Waffle Grid and allocating 768M  for a remote buffer pool (basically doubling the total memory for the innodb buffer pool ) we were able to achieve 6575 TPM, a 200% increase over the baseline.  While this is good,  when we introduced the patched memcached we discussed previously we raised the TPM to 9371, nearly 300% over the baseline, which is just awesome.  While this still is less then the 16508 TPM ( 500% ) a 1.5 GB all local buffer pool can achieve, Waffle grid is really a solution for those who have run out of local memory space.  What I want to see next is large database tests, i.e.  128GB tests, with 64GB local, and 64GB remote.  While the smaller sets of data should replicate larger datasets with similar %’s ( i.e.  the local buffer pool was sized to about 25% of the total data size, with another 25% allocated remotely using Waffle Grid, which should yeild similar #’s to 100GB system with 25GB local and 25GB remote), I still want to test this.

How did the patched Memcached get an extra 3K TPM?  Its all about the hit ratio.  Take a look at the difference:

Memcached Hit Ratio Hit Ratio
Memcached 5.1.25 Unpatched 68.80%
Memcached 5.1.25 Patched 94.90%

The new patched memcached enabled us to improve our hit ratio by just over 26%.  More cache hits equates to less disk activity, which is the hole goal of Waffle Grid.

I know their may still be some neigh sayers out their…  after all the only tests I have done up to now are DBT2 test.   Well here is a world exclusive!  Sysbench numbers, yes we can no longer be accused of being a one trick pony we do well on both synthetic benchmarks!

What’s interesting about these numbers are we almost equaled the performance of a completely local buffer with a Waffle remote buffer!

And finally, I talked about the performance benefit of having the fastest interconnect when you deploy a Waffle grid, I decided to put together a little test comparing localhost memcached to memcached connected over 1Gbe to memcached over a 100Mb connection.  Let the grudge match begin:

What was really surprising here is that the 1Gb connection almost matched memcached running on the localhost,  I expected more of a drop off in performance.  Some idea’s?  Because memcached and the database were on the same server for the localhost test its possible that they were fighting for CPU resources…  or thats the best guess I can come up with.  What’s telling though is the 100Mb connection test.  Using a 100Mb interconnect you might as well not bother deploying Waffle Grid, the extra few transactions hardly seems worth the complexity.

I would love to test this with faster interconnects and see what happens.

This entry was posted in mysql, performance, Waffle Grid. Bookmark the permalink.

8 Responses to Waffle Grid: 300% better then before

  1. Running DBT2 with 20 warehouses and 16 concurrent clients I get
    – 24634 NOTPM on a two-socket DualCore Intel machine, and
    – 42980 NOTPM on a quad-socket HexaCore Intel machine

  2. matt says:


    Is this with waffle grid or without? And how big is your innodb buffer pool? I am testing on an older 4 core machine, with small buffer pools. I would expect to see 25K TPM with all the data in memory. If you want to test for us, run dbt2 with a 768M buffer pool, O_Direct on. Then use waffle and create a 768M memcached buffer pool and run the same tests. It would be interesting to see the performance on faster cpu’s.

  3. I used a fairly new 4 core machine with

    I have to admit, that’s quite a difference to the 768MB you used.

    — Hakan

  4. Hi I’m just wondering if memcachedb might also be a decent stand in? It’s a slight performance decrease, but would allow you to increase your memcache L2 using your local file system, while removing the network out of the picture.

    I’d love to hear your thoughts on this


  5. Pingback: Idea: waffles & mdb (memcachedb)= waffle grid alt? | funyuns n pudding

  6. matt says:

    I have not looked at memcachedb, but I think the performance would make it a no go, but it maybe worth testing out to see if I am right:) We are caching the low level blocks, so we are replacing disk access with memory over network access… if I understand memcachedb correctly we would be replacing disk access with… more disk access. Plus their is no need for persistence, the data is already in the innodb datafiles.

  7. @Matt, I’ll be running a few benchmarks this week and will let you what the results are.

  8. Pingback: Big DBA Head! - Database Brain Power! » Waffle Grid: Remote Buffer Cache -VS- SSD Grudge Match

Comments are closed.