Just before Christmas, I was working with a client that needed to insert more than a billion rows into a NDB table. The cluster was big,
14 nodes, and after a few hundred millions rows, we got the following error:
2008-12-09 18:28:52 [ndbd] INFO -- dbacc/DbaccMain.cpp
2008-12-09 18:28:52 [ndbd] INFO -- DBACC (Line: 5274) 0x0000000e
2008-12-09 18:28:52 [ndbd] INFO -- Error handler shutting down system
2008-12-09 18:28:53 [ndbd] INFO -- Error handler shutdown completed - exiting
2008-12-09 18:28:56 [ndbd] ALERT -- Node 10: Forced node shutdown completed. Caused by error 2304: 'Array index out of range(Internal error, programming error or missing error message,
which looks like a bug in the NDB kernel. In fact, it is not really a bug. This error is the equivalent of a “table too large” error
with MyISAM when the row pointer size is too small. With MyISAM, you can use the “max_rows” specifier in the create table statement
to hint MySQL about the size of the pointer to use. The same concept apply with NDB, we created the table with a “max_rows” like:
CREATE TABLE TableName (
TablePK bigint(20) NOT NULL DEFAULT '0',
TableData` varbinary(440) DEFAULT NULL,
PRIMARY KEY (`TablePK`) USING HASH
) ENGINE=ndbcluster DEFAULT CHARSET=latin1 max_rows=3000000000;
and it solved the issue.
-
Recent Posts
Recent Comments
- Harrison on Helping to Organize a MySQL Bootcamp @ IOUG’s COLLABORATE 11
- Mark Callaghan on More Debate, More Flame, More Choosing the correct tool for the job
- Matthew Yonkovit on New Benchmark I am working on that tests MYSQL -vs- NOSQL
- Sam on New Benchmark I am working on that tests MYSQL -vs- NOSQL
- Matt Montgomery on New Benchmark I am working on that tests MYSQL -vs- NOSQL
Archives
- October 2011
- May 2011
- October 2010
- September 2010
- April 2010
- March 2010
- July 2009
- June 2009
- May 2009
- April 2009
- March 2009
- February 2009
- January 2009
- December 2008
- November 2008
- October 2008
- August 2008
- July 2008
- June 2008
- April 2008
- March 2008
- February 2008
- January 2008
- December 2007
- November 2007
- October 2007
- September 2007
Categories
Meta
ehh…i actually think it’s 2 bugs
1) it crashes the ndbd instead of reporting
that it’s full
2) the parameter is needed in the first place
but it definitely has a good work-around
does it do the same thing if you set a max_rows of a lot smaller?
we should also probably fix that bug….
I have not tried but I believe so.