{"id":4276,"date":"2017-05-03T12:25:43","date_gmt":"2017-05-03T02:25:43","guid":{"rendered":"https:\/\/www.flamingspork.com\/blog\/?p=4276"},"modified":"2017-05-03T12:25:43","modified_gmt":"2017-05-03T02:25:43","slug":"api-abi-and-backwards-compatibility-are-a-hard-necessity","status":"publish","type":"post","link":"https:\/\/www.flamingspork.com\/blog\/2017\/05\/03\/api-abi-and-backwards-compatibility-are-a-hard-necessity\/","title":{"rendered":"API, ABI and backwards compatibility are a hard necessity"},"content":{"rendered":"<p>Recently, I was reading a <a href=\"https:\/\/lkml.org\/lkml\/2017\/3\/30\/767\">thread on LKML<\/a> on a proposal to change the behavior of the open system call when confronted with unknown flags. The thread is worth a read as the topic of augmenting things that exist probably by accident to be &#8220;better&#8221; is always interesting, as is the definition of &#8220;better&#8221;.<\/p>\n<p>Keeping API and\/or ABI compatibility is something that isn&#8217;t a new problem, and it&#8217;s one that people are pretty good at sometimes messing up.<\/p>\n<p>This problem does not go away just because &#8220;we have cloud now&#8221;. In any distributed system, in order to upgrade it (or &#8220;be agile&#8221; as the kids are calling it), you <strong>by definition<\/strong> are going to have either downtime or at least two versions running concurrently. Thus, you have to have your interfaces\/RPCs\/APIs\/ABIs\/protocols\/whatever cope with changes.<\/p>\n<p>You cannot instantly upgrade the world, it happens gradually. You also have to design for at least <strong>three<\/strong> concurrent versions running. One is the original, the second is your upgrade, your third is the urgent fix because the upgrade is quite broken in some new way you only discover in production.<\/p>\n<p>So, the way you do this? Never ever <strong>EVER<\/strong> design for N-1 compatibility only. Design for going back a long way, much longer than you officially support. You want to have a design and programming culture of backwards compatibility to ensure you can both do new and exciting things and experiment off to the side.<\/p>\n<p>It&#8217;s worth going and rereading Rusty&#8217;s API levels posts from 2008:<\/p>\n<ul>\n<li><a href=\"http:\/\/ozlabs.org\/~rusty\/index.cgi\/tech\/2008-03-30.html\">How Do I Make This Hard to Misuse?<\/a><\/li>\n<li><a href=\"http:\/\/ozlabs.org\/~rusty\/index.cgi\/tech\/2008-04-01.html\">What If I Don&#8217;t Actually Like My Users?<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Recently, I was reading a thread on LKML on a proposal to change the behavior of the open system call when confronted with unknown flags. The thread is worth a read as the topic of augmenting things that exist probably &hellip; <a href=\"https:\/\/www.flamingspork.com\/blog\/2017\/05\/03\/api-abi-and-backwards-compatibility-are-a-hard-necessity\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[76],"tags":[714,78,633,715],"class_list":["post-4276","post","type-post","status-publish","format-standard","hentry","category-code","tag-abi","tag-api","tag-code","tag-compatibility"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p5a6n8-16Y","jetpack-related-posts":[{"id":1008,"url":"https:\/\/www.flamingspork.com\/blog\/2008\/02\/11\/getting-a-file-size-on-windows\/","url_meta":{"origin":4276,"position":0},"title":"Getting a file size (on Windows)","author":"Stewart Smith","date":"2008-02-11","format":false,"excerpt":"The first point I'd like to make is that you're using a Microsoft Windows API, so you have already lost. You are just not quite aware of how much you have lost. A quick look around and you say \"Ahh... GetFileSize, that's what I want to do!\" Except, of course,\u2026","rel":"","context":"In &quot;grumble&quot;","block_context":{"text":"grumble","link":"https:\/\/www.flamingspork.com\/blog\/category\/grumble\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":2168,"url":"https:\/\/www.flamingspork.com\/blog\/2010\/10\/25\/storage-engine-api-state-graph\/","url_meta":{"origin":4276,"position":1},"title":"Storage Engine API state graph","author":"Stewart Smith","date":"2010-10-25","format":false,"excerpt":"Drizzle still has a number of quirks inherited from the MySQL Storage Engine API (e.g. BLOBs, row buffer, CREATE SELECT and lack of DDL transaction boundaries, key tuple format). One of the things we fixed a long time ago was to have proper methods for StorageEngines to be called for:\u2026","rel":"","context":"In &quot;code&quot;","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"State transitions for a transaction. Transaction can be empty OR have one or more statements","src":"https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2010\/10\/edge30-300x163.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":2182,"url":"https:\/\/www.flamingspork.com\/blog\/2010\/10\/26\/cursor-states\/","url_meta":{"origin":4276,"position":2},"title":"Cursor states","author":"Stewart Smith","date":"2010-10-26","format":false,"excerpt":"Following on from my post yesterday on the various states of a Storage Engine, I said I'd have a go with the Cursor object too. A Cursor is used by the Drizzle kernel to get and set data in a table. There can be more than one cursor open at\u2026","rel":"","context":"In &quot;code&quot;","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2010\/10\/graph1-284x300.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":3345,"url":"https:\/\/www.flamingspork.com\/blog\/2013\/05\/23\/mysql-vs-drizzle-plugin-api\/","url_meta":{"origin":4276,"position":3},"title":"MySQL vs Drizzle plugin APIs","author":"Stewart Smith","date":"2013-05-23","format":false,"excerpt":"There's a big difference in how plugins are treated in MySQL and how they are treated in Drizzle. The MySQL way has been to create a C API in front of the C++-like (I call it C- as it manages to take the worst of both worlds) internal \"API\". The\u2026","rel":"","context":"In &quot;code&quot;","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1745,"url":"https:\/\/www.flamingspork.com\/blog\/2009\/12\/09\/drizzle-frm-replacement-the-table-proto\/","url_meta":{"origin":4276,"position":4},"title":"Drizzle FRM replacement: the table proto","author":"Stewart Smith","date":"2009-12-09","format":false,"excerpt":"Drizzle originally inherited the FRM file from MySQL (which inherited it from UNIREG). The FRM file stores metadata about a table; what columns it has, what type those columns are, what indexes, any default values, comments etc are all stored in the FRM. In the days of MyISAM, this worked\u2026","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1409,"url":"https:\/\/www.flamingspork.com\/blog\/2009\/02\/20\/improving-the-storage-engine-api\/","url_meta":{"origin":4276,"position":5},"title":"Improving the Storage Engine &#8220;API&#8221;","author":"Stewart Smith","date":"2009-02-20","format":false,"excerpt":"I increasingly enclose the API part of \"Storage Engine API\" in quotes as it does score a rather large number on the API Design Rusty levels (Coined by Rusty Russell). I give it a 15 (out of 18. lower is better) in this case \"The obvious use is wrong\". The\u2026","rel":"","context":"In &quot;code&quot;","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/4276","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/comments?post=4276"}],"version-history":[{"count":1,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/4276\/revisions"}],"predecessor-version":[{"id":4277,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/4276\/revisions\/4277"}],"wp:attachment":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/media?parent=4276"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/categories?post=4276"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/tags?post=4276"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}