{"id":3291,"date":"2013-04-18T11:43:37","date_gmt":"2013-04-18T01:43:37","guid":{"rendered":"http:\/\/www.flamingspork.com\/blog\/?p=3291"},"modified":"2013-04-18T11:45:44","modified_gmt":"2013-04-18T01:45:44","slug":"where-are-they-now-mysql-storage-engines","status":"publish","type":"post","link":"https:\/\/www.flamingspork.com\/blog\/2013\/04\/18\/where-are-they-now-mysql-storage-engines\/","title":{"rendered":"Where are they now: MySQL Storage Engines"},"content":{"rendered":"<p>There was once a big hooplah about the MySQL Storage Engine Architecture and how it was easy to just slot in some other method of storage instead of the provided ones. <a href=\"http:\/\/www.flamingspork.com\/blog\/2009\/02\/20\/improving-the-storage-engine-api\/\">Over<\/a> <a href=\"http:\/\/www.flamingspork.com\/blog\/2010\/04\/02\/the-drizzle-and-mysql-key-tuple-format\/\">the years<\/a> <a href=\"http:\/\/www.flamingspork.com\/blog\/2010\/04\/21\/storage-engine-api-write_row-create-select-and-ddl\/\">I&#8217;ve repeatedly<\/a> <a href=\"http:\/\/www.flamingspork.com\/blog\/2010\/05\/24\/using-the-row-buffer-in-drizzle-and-mysql\/\">mentioned<\/a> <a href=\"http:\/\/www.flamingspork.com\/blog\/2010\/05\/26\/blobs-in-the-drizzlemysql-storage-engine-api\/\">how this<\/a> <a href=\"http:\/\/www.flamingspork.com\/blog\/2010\/10\/25\/storage-engine-api-state-graph\/\">wasn&#8217;t really<\/a> <a href=\"http:\/\/www.flamingspork.com\/blog\/2010\/10\/26\/cursor-states\/\">the case<\/a> <a href=\"http:\/\/www.flamingspork.com\/blog\/2010\/11\/29\/a-more-complete-look-at-storage-engine-api\/\">and that<\/a> <a href=\"http:\/\/www.flamingspork.com\/blog\/2011\/01\/05\/is-your-storage-engine-buggy-or-the-database-server\/\">it was<\/a> <a href=\"https:\/\/bugs.launchpad.net\/drizzle\/+bug\/588251\">remarkably<\/a> <a href=\"http:\/\/www.flamingspork.com\/blog\/2010\/06\/29\/enum-now-works-properly-in-drizzle\/\">non trivial<\/a>.<\/p>\n<p>Over the years there have been many storage engines crop up and then disappear. So&#8230; <strong>where are they now?<\/strong><\/p>\n<ul>\n<li><strong>ISAM<\/strong><br \/>\nThis became MyISAM&#8230;. you know you&#8217;ve been around MySQL a long time if you&#8217;ve ever had to deal with an ISAM table.<\/li>\n<li><strong><span style=\"line-height: 15px;\">Gemini<br \/>\n<\/span><\/strong><span style=\"line-height: 15px;\">This was the first big test of the GPL in court. Basically, you have to obey the GPL (see <a href=\"https:\/\/en.wikipedia.org\/wiki\/MySQL\">wikipedia<\/a> for more info). The code was released as GPL and development stopped. This has been dead since ca 2002.<\/span><\/li>\n<li><strong>Amira &#8211; <a href=\"http:\/\/launchpad.net\/amira\">http:\/\/launchpad.net\/amira<\/a><\/strong><br \/>\n<a href=\"http:\/\/antbits.blogspot.com.au\/2008\/10\/amira-has-landed.html\">Antony first mentioned this in 2008 on his blog<\/a>. This was a continuation of the Gemini engine, you can actually <a href=\"https:\/\/launchpad.net\/amira\">go over to launchpad and get the code<\/a>. This was one of the projects to have a transactional storage engine not owned by Oracle after Innobase Oy was acquired by them. It went nowhere special as <a href=\"http:\/\/en.wikipedia.org\/wiki\/Netfrastructure\">Netfrastructure<\/a> was acquired which became <a href=\"https:\/\/en.wikipedia.org\/wiki\/Falcon_%28storage_engine%29\">Falcon<\/a>.<\/li>\n<li><strong>BDB<\/strong><br \/>\notherwise known as the BerkeleyDB engine. It was seldom used and never gained much of a userbase. It was <a href=\"http:\/\/archive09.linux.com\/articles\/56835\">unceremoniously dropped back in 2006<\/a>\u00c2\u00a0and both users didn&#8217;t really exist.<\/li>\n<li><strong>PBXT -\u00c2\u00a0<a href=\"http:\/\/pbxt.blogspot.com\/\">http:\/\/pbxt.blogspot.com\/<\/a><br \/>\n<\/strong>I think we can credit PBXT with at least half of the features and performance improvements to InnoDB since it first emerged back in 2006. <a href=\"http:\/\/pbxt.blogspot.ca\/2006\/04\/pbxt-mentioned-by-ceo-mrten-mickos-at.html\">It got attention very quickly<\/a>. Why? Because it was different. It had the\u00c2\u00a0<strong>very<\/strong> rare ability to outperform InnoDB in some places. You can still find PBXT in MariaDB, but sadly it can be hard to fund development of a MySQL storage engine, especially one as tied to MySQL as PBXT is, and it&#8217;s no longer under active development. Closely related was the Blob Streaming project which was way ahead of its time as an AlsoSQL access method. The good news is that the code was released under a BSD license in 2012 (was previously GPL). We even had PBXT in Drizzle for a while.<\/li>\n<li><strong>Blob Streaming (PBMS) -\u00c2\u00a0<a href=\"http:\/\/bpbdev.blogspot.com\/\">http:\/\/bpbdev.blogspot.com\/<\/a><br \/>\n<\/strong>This project was closely related to (but not depending exclusively on) PBXT. It embedded a HTTP server inside the database and could use it to read and write BLOBs. This was not only fairly cool but way ahead of its time. We owe the existence of both HandlerSocket and the memcached interface to InnoDB to PBMS (it was also an inspiration for the JSON server plugin for Drizzle, to address some of the use cases of the PBMS plugin).<\/li>\n<li><strong>Federated<\/strong><br \/>\nIt&#8217;s still there&#8230; but is effectively unmaintained and dead. There&#8217;s even FederatedX in MariaDB which is an improvement, but still, the MySQL server really doesn&#8217;t lend itself kindly to this type of engine&#8230; it&#8217;s always been an oddity only suitable for very specific tasks.<\/li>\n<li><strong>Archive<br \/>\n<\/strong>Although useful, effectively unmaintained. I kinda don&#8217;t want to say dead&#8230; but if it went away, I wouldn&#8217;t exactly be surprised.<\/li>\n<li><strong>CSV<\/strong><br \/>\nCurrently used to access the log tables in MySQL&#8230; and hardly used otherwise. It&#8217;s odd that the same code doesn&#8217;t deal with SELECT INTO OUTFILE and LOAD DATA INFILE, and I doubt this will ever change. I&#8217;d say effectively niche\/dead.<\/li>\n<li><strong>SolidDB<\/strong><br \/>\n<a href=\"http:\/\/blogs.the451group.com\/opensource\/2008\/03\/05\/ibm-abandons-soliddb-for-mysql\/\">Purchased by IBM, abandoned<\/a>.<\/li>\n<li><strong>DB2<\/strong><br \/>\nOnly ever on System i. Useful for very very few people&#8230; but you can still find it around if you&#8217;re one of them.<\/li>\n<li><strong>Infobright<\/strong><br \/>\n<a href=\"http:\/\/www.infobright.com\/\">OMG it exists!<\/a> This is probably because they&#8217;re largely just using the MySQL server as a way to implement the MySQL network protocol and all of the heavy lifting is done by their own code.<\/li>\n<li><strong>Xeround<br \/>\n<\/strong>I&#8217;m quite surprised these guys are still around, as they&#8217;re a proprietary storage engine as a service, and <a href=\"http:\/\/www.wc220.com\/?p=221\">initial testing wasn&#8217;t entirely promising<\/a>.<\/li>\n<li><strong>TokuDB<br \/>\n<\/strong>I cannot\u00c2\u00a0emphasize\u00c2\u00a0how much more interesting TokuDB would be if it were open source. It actually holds some promise&#8230; and with their recent work with mongo, perhaps this is a good way forward for them&#8230;<\/li>\n<li><strong>Maria\/Aria<\/strong><br \/>\nAnother &#8220;OMG Oracle just bought Innobase Oy&#8221; engine. This was a project to take MyISAM and turn it into a lean, mean, transactional storage engine machine. It&#8217;s still not there and I don&#8217;t think it ever will be.<\/li>\n<li><strong>Falcon<\/strong><br \/>\nThis was the hot new thing. It came out of Netfrastructure, which MySQL AB acquired in order to help get a transactional storage engine after Innobase Oy was acquired by Oracle. If you&#8217;re keeping count, that&#8217;s\u00c2\u00a0<strong>three<\/strong> projects for a transactional storage engine. Falcon was the star though, receiving all the press and publicity (well before it was ready). There are many reasons why Falcon isn&#8217;t around today &#8211; the chief one probably being that Oracle bought Sun who had bought MySQL and thus a need for an &#8220;InnoDB replacement&#8221; instantly vanished. There was also immense management pressure for performance to be greater than InnoDB, without any allowance for or focus on correctness&#8230;. and this showed. This was quite disappointing as Falcon had a lot of good architectural things going for it.<\/li>\n<li><strong>BlitzDB<\/strong> -\u00c2\u00a0<a href=\"https:\/\/launchpad.net\/blitzdb\">https:\/\/launchpad.net\/blitzdb<br \/>\n<\/a>I\u00c2\u00a0had hoped we&#8217;d replace MyISAM with BlitzDB in Drizzle. It was a wrapper around Tokyo Cabinet to the storage engine API in Drizzle. Unfortunately, the ties to MyISAM are incredibly deep (see <a href=\"http:\/\/www.flamingspork.com\/blog\/2013\/04\/04\/refactoring-internal-temporary-tables-another-stab-at-it\/\">my recent post on internal temporary tables<\/a>) and we never quite got there.<\/li>\n<\/ul>\n<p>I think this is all the notable engines that were aimed at widespread adoption&#8230; what ones have I forgotten?<\/p>\n<p>It&#8217;s interesting to note that only Archive, CSV, Xeround, TokuDB and Infobright can be gotten anywhere, and the latter two only in their own distribution (one proprietary) and Xeround only as a service.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>There was once a big hooplah about the MySQL Storage Engine Architecture and how it was easy to just slot in some other method of storage instead of the provided ones. Over the years I&#8217;ve repeatedly mentioned how this wasn&#8217;t &hellip; <a href=\"https:\/\/www.flamingspork.com\/blog\/2013\/04\/18\/where-are-they-now-mysql-storage-engines\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[76,75,14],"tags":[509,202,514,520,512,522,524,70,511,521,510,517,519,513,628,330,518,523,80,515,516],"class_list":["post-3291","post","type-post","status-publish","format-standard","hentry","category-code","category-drizzle-work-et-al","category-mysql","tag-amira","tag-archive","tag-aria","tag-bdb","tag-blitzdb","tag-csv","tag-db2","tag-drizzle","tag-falcon","tag-federated","tag-gemini","tag-infobright","tag-isam","tag-maria","tag-mysql","tag-pbms","tag-pbxt","tag-soliddb","tag-storage-engine-api","tag-tokudb","tag-xeround"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p5a6n8-R5","jetpack-related-posts":[{"id":1409,"url":"https:\/\/www.flamingspork.com\/blog\/2009\/02\/20\/improving-the-storage-engine-api\/","url_meta":{"origin":3291,"position":0},"title":"Improving the Storage Engine &#8220;API&#8221;","author":"Stewart Smith","date":"2009-02-20","format":false,"excerpt":"I increasingly enclose the API part of \"Storage Engine API\" in quotes as it does score a rather large number on the API Design Rusty levels (Coined by Rusty Russell). I give it a 15 (out of 18. lower is better) in this case \"The obvious use is wrong\". The\u2026","rel":"","context":"In &quot;code&quot;","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":2253,"url":"https:\/\/www.flamingspork.com\/blog\/2011\/01\/05\/is-your-storage-engine-buggy-or-the-database-server\/","url_meta":{"origin":3291,"position":1},"title":"Is your Storage Engine buggy or the database server?","author":"Stewart Smith","date":"2011-01-05","format":false,"excerpt":"If your storage engine returns an error from rnd_init (or doStartTableScan as it's named in Drizzle) and does not save this error and return it in any subsequent calls to rnd_next, your engine is buggy. Namely it is buggy in that a) an error may not be reported back to\u2026","rel":"","context":"In &quot;code&quot;","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3237,"url":"https:\/\/www.flamingspork.com\/blog\/2013\/03\/08\/mysql-modularity-are-we-there-yet\/","url_meta":{"origin":3291,"position":2},"title":"MySQL modularity, are we there yet?","author":"Stewart Smith","date":"2013-03-08","format":false,"excerpt":"MySQL is now over four times the size than it was with MySQL 3.23. This has not come in the shape of plugins. Have we improved modularity over time? I decided to take LoC count for plugins and storage engines (in the case of Drizzle, memory, myisam and innobase are\u2026","rel":"","context":"In &quot;code&quot;","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3345,"url":"https:\/\/www.flamingspork.com\/blog\/2013\/05\/23\/mysql-vs-drizzle-plugin-api\/","url_meta":{"origin":3291,"position":3},"title":"MySQL vs Drizzle plugin APIs","author":"Stewart Smith","date":"2013-05-23","format":false,"excerpt":"There's a big difference in how plugins are treated in MySQL and how they are treated in Drizzle. The MySQL way has been to create a C API in front of the C++-like (I call it C- as it manages to take the worst of both worlds) internal \"API\". The\u2026","rel":"","context":"In &quot;code&quot;","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3302,"url":"https:\/\/www.flamingspork.com\/blog\/2013\/04\/20\/the-memory-storage-engine\/","url_meta":{"origin":3291,"position":4},"title":"The MEMORY storage engine","author":"Stewart Smith","date":"2013-04-20","format":false,"excerpt":"I recently wrote about\u00c2\u00a0Where are they now: MySQL Storage Engines\u00c2\u00a0and\u00c2\u00a0The MERGE storage engine: not dead, just resting\u00e2\u20ac\u00a6. or forgotten.\u00c2\u00a0Today, it's the turn of the MEMORY storage engine - otherwise known as HEAP. This is yet another piece of the MySQL server that sits largely unmaintained and unloved. The MySQL Manual\u2026","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3295,"url":"https:\/\/www.flamingspork.com\/blog\/2013\/04\/19\/the-merge-storage-engine-not-dead-just-resting-or-forgotten\/","url_meta":{"origin":3291,"position":5},"title":"The MERGE storage engine: not dead, just resting&#8230;. or forgotten.","author":"Stewart Smith","date":"2013-04-19","format":false,"excerpt":"Following on from my fun post on Where are they now: MySQL Storage Engines, I thought I'd cover the few storage engines that are really just interfaces to a collection of things. In this post, I'm talking about MERGE. The MERGE engine was basically a multiplexer down to a number\u2026","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/3291","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/comments?post=3291"}],"version-history":[{"count":3,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/3291\/revisions"}],"predecessor-version":[{"id":3294,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/3291\/revisions\/3294"}],"wp:attachment":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/media?parent=3291"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/categories?post=3291"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/tags?post=3291"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}