{"id":2362,"date":"2011-04-21T17:09:59","date_gmt":"2011-04-21T07:09:59","guid":{"rendered":"http:\/\/www.flamingspork.com\/blog\/?p=2362"},"modified":"2011-04-21T17:09:59","modified_gmt":"2011-04-21T07:09:59","slug":"http-json-alsosql-interface-to-drizzle","status":"publish","type":"post","link":"https:\/\/www.flamingspork.com\/blog\/2011\/04\/21\/http-json-alsosql-interface-to-drizzle\/","title":{"rendered":"HTTP JSON AlsoSQL interface to Drizzle"},"content":{"rendered":"<p>So&#8230; I had another one of those &#8220;hrrm&#8230; this shouldn&#8217;t be hard to hack a proof-of-concept&#8221; moments. Web apps are increasingly speaking JSON all around the place. Why can&#8217;t we speak JSON to\/from the database? Why? Seriously, why not?<\/p>\n<p>One reason why MongoDB has found users is that JSON is very familiar to people. It has gained popularity in spite of having pure disregard for the integrity and safety of your data.<\/p>\n<p>So I started with a really simple idea: http server in the database server. Thanks to the simple code to do that with libevent, I got that going fairly quickly. Finding a rather nice C++ library to create and parse JSON was the next challenge. I found <a href=\"http:\/\/jsoncpp.sourceforge.net\/index.html\">JSONcpp<\/a>, a public domain library with a nice API and proceeded to bring it into the tree (it&#8217;s not much code). I then\u00c2\u00a0created a simple way to find out the version of the Drizzle server you were speaking to:<\/p>\n<pre>$ curl http:\/\/localhost:8765\/0.1\/version\r\n{\r\n   \"version\" : \"2011.04.15.2285\"\r\n}<\/pre>\n<p>But that wasn&#8217;t nearly enough&#8230; I also wanted to be able to issue arbitrary queries. Thanks to the supporting code we have in the Drizzle server for EXECUTE() (also used by the replication slave), this was also pretty easy. I created a way to execute the content of a HTTP POST request as if you had done so with EXECUTE() &#8211; all nicely wrapped in a transaction.<\/p>\n<p>I created a simple table using the drizzle client, connecting over a normal TCP socket speaking the MySQL protocol and inserted a row in it:<\/p>\n<pre>$ ..\/client\/drizzle --port 9306 test\r\nWelcome to the Drizzle client..  Commands end with ; or \\g.\r\nYour Drizzle connection id is 4\r\nConnection protocol: mysql\r\nServer version: 2011.04.15.2285 Source distribution (json-interface)\r\n\r\nType 'help;' or '\\h' for help. Type '\\c' to clear the buffer.\r\n\r\ndrizzle&gt; show create table t1\\G\r\n*************************** 1. row ***************************\r\n       Table: t1\r\nCreate Table: CREATE TABLE `t1` (\r\n  `a` INT NOT NULL AUTO_INCREMENT,\r\n  `b` VARCHAR(100) COLLATE utf8_general_ci DEFAULT NULL,\r\n  PRIMARY KEY (`a`)\r\n) ENGINE=InnoDB COLLATE = utf8_general_ci\r\n1 row in set (0.001209 sec)\r\n\r\ndrizzle&gt; insert into t1 (b) values (\"from mysql protocol\");\r\nQuery OK, 1 row affected (0.00207 sec)<\/pre>\n<p>Now to select rows from it via HTTP and get a JSON object back with the result set:<\/p>\n<pre>$ curl http:\/\/localhost:8765\/0.1\/sql --data 'select * from t1;'\r\n{\r\n   \"query\" : \"select * from t1;\",\r\n   \"result_set\" : [\r\n      [ \"1\", \"from mysql protocol\" ],\r\n      [ \"\", \"\" ]\r\n   ],\r\n   \"sqlstate\" : \"00000\"\r\n}\r\n<\/pre>\n<p>I can also insert more rows using the HTTP interface and then select them from the MySQL protocol interface:<\/p>\n<pre>$ curl http:\/\/localhost:8765\/0.1\/sql --data 'insert into t1 values (NULL, \\\"from HTTP!\\\");'\r\n{\r\n   \"query\" : \"insert into t1 values (NULL, \\\\\\\"from HTTP!\\\\\\\");\",\r\n   \"sqlstate\" : \"00000\"\r\n}\r\n\r\ndrizzle&gt; select * from t1;\r\n+---+---------------------+\r\n| a | b                   |\r\n+---+---------------------+\r\n| 1 | from mysql protocol | \r\n| 2 | from HTTP!          | \r\n+---+---------------------+\r\n2 rows in set (0.000907 sec)<\/pre>\n<p>So what does this get us? With the addition of proper authentication, you could start doing some really quite neat and nifty things. I imagine we could add interfaces to avoid SQL and directly do key lookups, table scans and index range scans, giving really quite sweet performance. We could start building web tools to manage and manipulate the database speaking the native language of the web.<\/p>\n<p>But&#8230; there&#8217;s more!<\/p>\n<p>Since we have a web server and a way to execute queries via HTTP along with getting the result set as JSON, why can&#8217;t we have a simple Web UI for monitoring the database server and running queries <strong>built into the database server<\/strong>?<\/p>\n<p>Yes we can.<\/p>\n<p><a href=\"https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2011\/04\/Screenshot-11.png\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" data-attachment-id=\"2363\" data-permalink=\"https:\/\/www.flamingspork.com\/blog\/2011\/04\/21\/http-json-alsosql-interface-to-drizzle\/screenshot-11\/\" data-orig-file=\"https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2011\/04\/Screenshot-11.png?fit=512%2C394&amp;ssl=1\" data-orig-size=\"512,394\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;}\" data-image-title=\"Screenshot-11\" data-image-description=\"\" data-image-caption=\"\" data-large-file=\"https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2011\/04\/Screenshot-11.png?fit=512%2C394&amp;ssl=1\" class=\"aligncenter size-medium wp-image-2363\" title=\"Screenshot-11\" src=\"https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2011\/04\/Screenshot-11-300x230.png?resize=300%2C230\" alt=\"\" width=\"300\" height=\"230\" srcset=\"https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2011\/04\/Screenshot-11.png?resize=300%2C230&amp;ssl=1 300w, https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2011\/04\/Screenshot-11.png?w=512&amp;ssl=1 512w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>If you wanted a WHERE condition or anything else, easy. Change the query, hit execute:<\/p>\n<p><a href=\"https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2011\/04\/Screenshot-12.png\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" data-attachment-id=\"2364\" data-permalink=\"https:\/\/www.flamingspork.com\/blog\/2011\/04\/21\/http-json-alsosql-interface-to-drizzle\/screenshot-12\/\" data-orig-file=\"https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2011\/04\/Screenshot-12.png?fit=460%2C318&amp;ssl=1\" data-orig-size=\"460,318\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;}\" data-image-title=\"Screenshot-12\" data-image-description=\"\" data-image-caption=\"\" data-large-file=\"https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2011\/04\/Screenshot-12.png?fit=460%2C318&amp;ssl=1\" class=\"aligncenter size-full wp-image-2364\" title=\"Screenshot-12\" src=\"https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2011\/04\/Screenshot-12.png?resize=460%2C318\" alt=\"\" width=\"460\" height=\"318\" srcset=\"https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2011\/04\/Screenshot-12.png?w=460&amp;ssl=1 460w, https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2011\/04\/Screenshot-12.png?resize=300%2C207&amp;ssl=1 300w\" sizes=\"auto, (max-width: 460px) 100vw, 460px\" \/><\/a>No TCP connection or parsing the MySQL protocol or anything. Just HTTP requests straight to the database server from the browser with a bit of client side javascript producing the HTML for the table.<\/p>\n<p>Proof of concept code is up on launchpad in\u00c2\u00a0<a href=\"https:\/\/code.launchpad.net\/~stewart\/drizzle\/json-interface\">lp:~stewart\/drizzle\/json-interface<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>So&#8230; I had another one of those &#8220;hrrm&#8230; this shouldn&#8217;t be hard to hack a proof-of-concept&#8221; moments. Web apps are increasingly speaking JSON all around the place. Why can&#8217;t we speak JSON to\/from the database? Why? Seriously, why not? One &hellip; <a href=\"https:\/\/www.flamingspork.com\/blog\/2011\/04\/21\/http-json-alsosql-interface-to-drizzle\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[75],"tags":[],"class_list":["post-2362","post","type-post","status-publish","format-standard","hentry","category-drizzle-work-et-al"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p5a6n8-C6","jetpack-related-posts":[{"id":2374,"url":"https:\/\/www.flamingspork.com\/blog\/2011\/05\/13\/drizzle-json-interface-merged\/","url_meta":{"origin":2362,"position":0},"title":"Drizzle JSON interface merged","author":"Stewart Smith","date":"2011-05-13","format":false,"excerpt":"https:\/\/code.launchpad.net\/~stewart\/drizzle\/json-interface\/+merge\/59859 Currently a very early version of course, but it's there in trunk if you want to play with it. Just have libcurl and libevent installed and you can submit queries via HTTP and JSON. Of course, the next steps are getting a true non-sql interface going and seeing how\u2026","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3137,"url":"https:\/\/www.flamingspork.com\/blog\/2012\/07\/28\/finding-out-whats-next-at-barcampmel-2012-with-drizzle-sql-javascript-and-a-web-browser\/","url_meta":{"origin":2362,"position":1},"title":"Finding out What&#8217;s Next at BarCampMel 2012 with Drizzle, SQL, JavaScript and a web browser","author":"Stewart Smith","date":"2012-07-28","format":false,"excerpt":"Just for the pure insane fun of it, I accepted the challenge of \"what can you do with the text format of the schedule?\" for BarCampMel. I'm a database guy, so I wanted to load it into a database (which would be Drizzle), and I wanted it to be easy\u2026","rel":"","context":"In &quot;code&quot;","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3929,"url":"https:\/\/www.flamingspork.com\/blog\/2015\/02\/07\/mysql-next-drizzle-5-years-ago\/","url_meta":{"origin":2362,"position":2},"title":"MySQL-next = Drizzle 5 years ago?","author":"Stewart Smith","date":"2015-02-07","format":false,"excerpt":"With JSON functionality, alternate protocols (HTTP, memcache), a move towards saner defaults and crash safety, pluggable logging etc it really looks like MySQL is following what we did in Drizzle years ago, which is great!","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":4180,"url":"https:\/\/www.flamingspork.com\/blog\/2016\/09\/27\/mysql-removes-the-frm-7-years-after-drizzle-did\/","url_meta":{"origin":2362,"position":3},"title":"MySQL removes the FRM (7 years after Drizzle did)","author":"Stewart Smith","date":"2016-09-27","format":false,"excerpt":"The new MySQL 8.0.0 milestone release that was recently announced brings something that has been a looooong time coming: the removal of the FRM file. I was the one who implemented this in Drizzle way back in 2009 (July 28th 2009 according to Brian)- and I may have had a\u2026","rel":"","context":"In &quot;code&quot;","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3291,"url":"https:\/\/www.flamingspork.com\/blog\/2013\/04\/18\/where-are-they-now-mysql-storage-engines\/","url_meta":{"origin":2362,"position":4},"title":"Where are they now: MySQL Storage Engines","author":"Stewart Smith","date":"2013-04-18","format":false,"excerpt":"There was once a big hooplah about the MySQL Storage Engine Architecture and how it was easy to just slot in some other method of storage instead of the provided ones. Over the years I've repeatedly mentioned how this wasn't really the case and that it was remarkably non trivial.\u2026","rel":"","context":"In &quot;code&quot;","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":2149,"url":"https:\/\/www.flamingspork.com\/blog\/2010\/09\/30\/drizzle7-beta\/","url_meta":{"origin":2362,"position":5},"title":"Drizzle7 Beta!","author":"Stewart Smith","date":"2010-09-30","format":false,"excerpt":"Just in case you missed it, I'm rather thrilled that our latest tarball of Drizzle is named Beta. Specifically, we're calling it Drizzle7. Seven is a very nice number, and it seems rather appropriate. This release is for a stand alone database server. A lot of the infrastructure for replication\u2026","rel":"","context":"In &quot;code&quot;","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/2362","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/comments?post=2362"}],"version-history":[{"count":1,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/2362\/revisions"}],"predecessor-version":[{"id":2365,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/2362\/revisions\/2365"}],"wp:attachment":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/media?parent=2362"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/categories?post=2362"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/tags?post=2362"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}