{"id":3683,"date":"2014-02-16T12:32:28","date_gmt":"2014-02-16T02:32:28","guid":{"rendered":"https:\/\/www.flamingspork.com\/blog\/?p=3683"},"modified":"2014-03-10T17:40:38","modified_gmt":"2014-03-10T07:40:38","slug":"ghosts-of-mysql-past-part-6-the-engine-revs","status":"publish","type":"post","link":"https:\/\/www.flamingspork.com\/blog\/2014\/02\/16\/ghosts-of-mysql-past-part-6-the-engine-revs\/","title":{"rendered":"Ghosts of MySQL Past, Part 6: The engine revs"},"content":{"rendered":"

This week I\u00e2\u20ac\u2122ve been writing based on my linux.conf.au 2014 talk, which you can watch the recording of<\/a>. Also see Part 1<\/a>, Part 2<\/a>, Part 3,<\/a> Part 4<\/a> and Part 5<\/a>. My feed feel off Planet MySQL for a bit so you may have missed those posts.<\/p>\n

Netfrastructure <\/a>was many other things apart from just a database engine, but the one part that was interesting to MySQL was the storage engine. One interesting thing about Netfrastructure was the inbuilt Java VM, which meant you could do all sorts of neat tricks right near the data and with Jim and Ann coming from much more feature complete databases, some parts of MySQL were going to be a little bit of a shock.<\/p>\n

There was still work to be done on the Falcon engine itself, it wasn’t completely finished and ready to just drop in, even though you may have heard things like “to be completed later this year”.<\/p>\n

There was also another experiment to see if MySQL could get a storage engine that wasn’t owned by somebody else: resurrect Gemini. Remember, it was GPL licensed now, and it could conceivably be updated to the latest MySQL versions. You can go and see the code for the Amira project on launchpad<\/a>. Ultimately, this went nowhere, but it could have been an interesting story if this project got more resources. Ultimately, it was Falcon that got all the attention and effort.<\/p>\n

This was going to be the ultimate triumph of the Pluggable Storage Engine Architecture, something that had been touted in presentations and marketing material for years. What really happened? Well, it took NDB (now MySQL Cluster) somewhere between two and four years before you could really say it was stable and free of many “obvious” bugs\/unintended deviations in behavior. Would Falcon be any different?<\/p>\n

In reality, the Falcon project unearthed all of the warts of the storage engine interface. There were many exchanges of “Is it really<\/strong> like that?”, “Yes, yes it is, sorry.”<\/p>\n

The APIs (and I use the term loosely) between a storage engine and the database server were originally used to plug in ISAM and then MyISAM, and the first transactional store (InnoDB) required a bunch of hacks to work and the API was never cleaned up. When the third transactional engine came along (MySQL Cluster) – which was also a distributed engine – there were again no real fixes in the API, just more of the same work-arounds and oddness. So by the time Falcon came along, if you were lucky<\/strong> the API was merely just not what you’d expect. There were many, many situations where not only was the official documentation inaccurate, but you’d have to commit several gross layering violations just to get something rather fundamental to a transactional database working.<\/p>\n

Arguably the worst thing was foreign keys, a requirement for the Falcon project. You had to implement your own SQL parser to get any information on foreign keys. Let’s not even get started on auto_increment – something which the correct behavior is pretty much completely undocumented.<\/p>\n

Next, we’ll look at PBXT.<\/p>\n","protected":false},"excerpt":{"rendered":"

This week I\u00e2\u20ac\u2122ve been writing based on my linux.conf.au 2014 talk, which you can watch the recording of. Also see Part 1, Part 2, Part 3, Part 4 and Part 5. My feed feel off Planet MySQL for a bit … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[14],"tags":[628,279],"class_list":["post-3683","post","type-post","status-publish","format-standard","hentry","category-mysql","tag-mysql","tag-storage-engine"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p5a6n8-Xp","jetpack-related-posts":[{"id":1642,"url":"https:\/\/www.flamingspork.com\/blog\/2009\/05\/15\/mysql-storage-engine-sloccount-over-releases\/","url_meta":{"origin":3683,"position":0},"title":"MySQL Storage Engine SLOCCount over releases","author":"Stewart Smith","date":"2009-05-15","format":false,"excerpt":"For a bit more info, what about various storage engines over MySQL releases. Have they changed much? Here we're looking at the storage\/X\/ directory for code, so for some engines this excludes the handler that interfaces with the MySQL Server. You can view the data on the spreadsheet.","rel":"","context":"In "mysql"","block_context":{"text":"mysql","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/mysql\/"},"img":{"alt_text":"","src":"http:\/\/spreadsheets.google.com\/pub?key=rD5FOoJk29j9RK7sm0abSsg&oid=1&output=image","width":350,"height":200,"srcset":"http:\/\/spreadsheets.google.com\/pub?key=rD5FOoJk29j9RK7sm0abSsg&oid=1&output=image 1x, http:\/\/spreadsheets.google.com\/pub?key=rD5FOoJk29j9RK7sm0abSsg&oid=1&output=image 1.5x"},"classes":[]},{"id":3347,"url":"https:\/\/www.flamingspork.com\/blog\/2013\/05\/24\/an-old-note-on-the-storage-engine-api\/","url_meta":{"origin":3683,"position":1},"title":"An old note on the Storage Engine API","author":"Stewart Smith","date":"2013-05-24","format":false,"excerpt":"Whenever I stick my head into the MySQL storage engine API, I'm reminded of a MySQL User Conference from several years ago now. Specifically, I'm reminded of a slide from an early talk at the MySQL User Conference by\u00c2\u00a0Paul McCullagh describing developing PBXT. For \"How to write a Storage Engine\u2026","rel":"","context":"In "code"","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":904,"url":"https:\/\/www.flamingspork.com\/blog\/2007\/10\/16\/mysql-5122ish-stew1\/","url_meta":{"origin":3683,"position":2},"title":"MySQL 5.1.22(ish)-stew1","author":"Stewart Smith","date":"2007-10-16","format":false,"excerpt":"I've decided to publish my patch series. The goal of the -stew patches is to collect things I find interesting and that at some point could (should) make it into the main MySQL tree (even if others don't think so). It's not designed for use in production.. I don't really\u2026","rel":"","context":"In "mysql"","block_context":{"text":"mysql","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/mysql\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3304,"url":"https:\/\/www.flamingspork.com\/blog\/2013\/05\/13\/the-mysql-cluster-storage-engine\/","url_meta":{"origin":3683,"position":3},"title":"The MySQL Cluster storage engine","author":"Stewart Smith","date":"2013-05-13","format":false,"excerpt":"This is one close to my heart. I've recently written on other storage engines:\u00c2\u00a0Where are they now: MySQL Storage Engines,\u00c2\u00a0The MERGE storage engine: not dead, just resting\u00e2\u20ac\u00a6. or forgotten\u00c2\u00a0and The MEMORY storage engine. Today, it's the turn of MySQL Cluster. Like InnoDB, MySQL Cluster started outside of MySQL. Those of\u2026","rel":"","context":"In "code"","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1409,"url":"https:\/\/www.flamingspork.com\/blog\/2009\/02\/20\/improving-the-storage-engine-api\/","url_meta":{"origin":3683,"position":4},"title":"Improving the Storage Engine “API”","author":"Stewart Smith","date":"2009-02-20","format":false,"excerpt":"I increasingly enclose the API part of \"Storage Engine API\" in quotes as it does score a rather large number on the API Design Rusty levels (Coined by Rusty Russell). I give it a 15 (out of 18. lower is better) in this case \"The obvious use is wrong\". The\u2026","rel":"","context":"In "code"","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":829,"url":"https:\/\/www.flamingspork.com\/blog\/2007\/04\/27\/adding-a-pluggable-information-schema-table-to-a-pluggable-engine-in-mysql-51\/","url_meta":{"origin":3683,"position":5},"title":"adding a pluggable information schema table to a pluggable engine in mysql 5.1","author":"Stewart Smith","date":"2007-04-27","format":false,"excerpt":"Also now up is the patch series in my \"ndb-work\" tree which small patch for adding INFORMATION_SCHEMA.NDB_NODE_STATUS. It's nearly useful... I haven't brought in the nice \"id to string\" functions in the management client that make pretty printing nice... so not quite end user friendly :) But it's a nice\u2026","rel":"","context":"In "mysql"","block_context":{"text":"mysql","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/mysql\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/3683","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/comments?post=3683"}],"version-history":[{"count":1,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/3683\/revisions"}],"predecessor-version":[{"id":3684,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/3683\/revisions\/3684"}],"wp:attachment":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/media?parent=3683"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/categories?post=3683"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/tags?post=3683"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}