{"id":3137,"date":"2012-07-28T18:20:20","date_gmt":"2012-07-28T08:20:20","guid":{"rendered":"http:\/\/www.flamingspork.com\/blog\/?p=3137"},"modified":"2012-07-28T18:20:20","modified_gmt":"2012-07-28T08:20:20","slug":"finding-out-whats-next-at-barcampmel-2012-with-drizzle-sql-javascript-and-a-web-browser","status":"publish","type":"post","link":"https:\/\/www.flamingspork.com\/blog\/2012\/07\/28\/finding-out-whats-next-at-barcampmel-2012-with-drizzle-sql-javascript-and-a-web-browser\/","title":{"rendered":"Finding out What&#8217;s Next at BarCampMel 2012 with Drizzle, SQL, JavaScript and a web browser"},"content":{"rendered":"<p>Just for the pure insane fun of it, I accepted the challenge of &#8220;what can you do with the text format of the schedule?&#8221; for BarCampMel. I&#8217;m a database guy, so I wanted to load it into a database (which would be Drizzle), and I wanted it to be easy to keep it up to date (this is an unconference after all).<\/p>\n<p>So&#8230; the text file itself isn&#8217;t in any standard format, so I&#8217;d have to parse it. I&#8217;m lazy and didn&#8217;t want to leave the comfort of the database. Luckily, inside Drizzle, we have a js plugin that lets you execute arbitrary JavaScript. Parsing solved. I needed to get the program and luckily we have the http_functions plugin that uses libcurl to allow us to perform HTTP GET requests. I also wanted it in a table so I could query it when not online, so I needed to load the data. Luckily, in Drizzle we have the built in EXECUTE functionality, so I could just use the JavaScript to parse the response from the HTTP GET request and construct SQL to load the data into a table to then query.<\/p>\n<p>So, grab your Drizzle server with &#8220;plugin-add=js&#8221; and &#8220;plugin-add=http_functions&#8221; in the config file or as options to drizzled (prefixed with &#8211;) and&#8230;.<\/p>\n<p>This simple one liner pulls the current schedule and puts it into a table called &#8216;schedule&#8217;:<\/p>\n<p><code>SELECT EXECUTE(JS(\"function sql_quote(s) {return s ? '\\\"'+ s.replace('\\\"', '\\\\\\\"') + '\\\"' : 'NULL'} function DrizzleDateString(d) { function pad(n) { return n&lt;10 ? '0'+n : n } return d.getFullYear()+'-'+pad(d.getMonth()+1)+'-'+pad(d.getDate())+' '+pad(d.getHours())+':'+pad(d.getMinutes())+':'+pad(d.getSeconds()) } var sql = 'COMMIT;CREATE TABLE IF NOT EXISTS schedule (start_time datetime, stage varchar(1000), mr2 varchar(1000), mr1 varchar(1000), duration int); begin; delete from schedule;' ; var time= new Date;var input= arguments[0].split(\\\"\\\\n\\\"); var entry = new Array(); var stage, mr2, mr1; for(var i=0; i &lt; input.length; i++) { var p= input[i].match('^(.*?) (.*)$'); if(p) {if(p[1]=='Time') { time=new Date(Date.parse(p[2]));} if(p[1]=='Duration') { sql+='INSERT INTO schedule (start_time,stage,mr2,mr1,duration) VALUES (\\\"' + DrizzleDateString(time) + '\\\", ' + sql_quote(stage) + ', ' + sql_quote(mr2) + ',' + sql_quote(mr1) + ',' + p[2] + '); '; time= new Date(time.getTime()+p[2]*60*1000); stage= mr2= mr1= ''; } if(p[1]=='stage') {stage=p[2]} if (p[1]=='mr2') {mr2=p[2]} if (p[1]=='mr1') {mr1=p[2]} }}; sql+='COMMIT;'; sql\", (select http_get('https:\/\/dl.dropbox.com\/s\/01yh7ji7pswjwwk\/live-schedule.txt?dl=1'))));<\/code><\/p>\n<p><code><\/code><span style=\"font-family: Monaco, Consolas, 'Andale Mono', 'DejaVu Sans Mono', monospace; font-size: small;\"><span style=\"line-height: normal;\">Which you can then find out &#8220;what&#8217;s on now and coming up&#8221; with this query:<\/span><\/span><\/p>\n<div><code>select * from schedule where start_time &gt; DATE_ADD(now(), INTERVAL 9 HOUR) ORDER BY start_time limit 2\\G<\/code><\/div>\n<div><\/div>\n<div>But it&#8217;s totally not fun having to jump to the command line all the time, and you may want it in JSON format for consuming with some web thing&#8230;. so you can load the json_server plugin and browse to the port that it&#8217;s running on (default 8086) and type the SQL in there and get a JSON response, or just look at the pretty table there.<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Just for the pure insane fun of it, I accepted the challenge of &#8220;what can you do with the text format of the schedule?&#8221; for BarCampMel. I&#8217;m a database guy, so I wanted to load it into a database (which &hellip; <a href=\"https:\/\/www.flamingspork.com\/blog\/2012\/07\/28\/finding-out-whats-next-at-barcampmel-2012-with-drizzle-sql-javascript-and-a-web-browser\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[76,75],"tags":[492,70,494,431,109],"class_list":["post-3137","post","type-post","status-publish","format-standard","hentry","category-code","category-drizzle-work-et-al","tag-barcampmel","tag-drizzle","tag-insanity","tag-javascript","tag-sql"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p5a6n8-OB","jetpack-related-posts":[{"id":2362,"url":"https:\/\/www.flamingspork.com\/blog\/2011\/04\/21\/http-json-alsosql-interface-to-drizzle\/","url_meta":{"origin":3137,"position":0},"title":"HTTP JSON AlsoSQL interface to Drizzle","author":"Stewart Smith","date":"2011-04-21","format":false,"excerpt":"So... I had another one of those \"hrrm... this shouldn't be hard to hack a proof-of-concept\" moments. Web apps are increasingly speaking JSON all around the place. Why can't we speak JSON to\/from the database? Why? Seriously, why not? One reason why MongoDB has found users is that JSON is\u2026","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2011\/04\/Screenshot-11-300x230.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":1894,"url":"https:\/\/www.flamingspork.com\/blog\/2010\/04\/16\/announcing-haildb\/","url_meta":{"origin":3137,"position":1},"title":"Announcing HailDB","author":"Stewart Smith","date":"2010-04-16","format":false,"excerpt":"I just announced our continuation of the Embedded InnoDB project under the name of HailDB. Check out the announcement over at http:\/\/www.haildb.com\/. HailDB is a relational database that is embeddable within applications. You embed\u00c2\u00a0HailDB by linking to a shared library and calling a clean and simple API.\u00c2\u00a0HailDB is a continuation\u2026","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1847,"url":"https:\/\/www.flamingspork.com\/blog\/2010\/03\/15\/thoughts-on-thoughts-on-drizzle\/","url_meta":{"origin":3137,"position":2},"title":"Thoughts on Thoughts on Drizzle :)","author":"Stewart Smith","date":"2010-03-15","format":false,"excerpt":"Mark has some good thoughts on drizzle. I think they're all valid... and have some extra thoughts too: \"I have problems to solve today\". This is (of course) an active concern in my brain... If we don't have something out that solves some set of problems with reasonable stability and\u2026","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1840,"url":"https:\/\/www.flamingspork.com\/blog\/2010\/03\/14\/drizzle-developer-day-2010\/","url_meta":{"origin":3137,"position":3},"title":"Drizzle Developer Day 2010","author":"Stewart Smith","date":"2010-03-14","format":false,"excerpt":"Hi one and all! Interested in database systems? Interested because you use them? Because you manage them? Write SQL that goes to them? Or are you one of the people of questionable sanity like myself who develops them? Well... do we have the offer for you. Friday, April 16th. Right\u2026","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1285,"url":"https:\/\/www.flamingspork.com\/blog\/2008\/12\/04\/what-constitutes-an-identifier-for-a-table\/","url_meta":{"origin":3137,"position":4},"title":"What constitutes an identifier for a table?","author":"Stewart Smith","date":"2008-12-04","format":false,"excerpt":"Well... there's: database table name (both of these are quite obvious). But then you have: temporary tables Well... two types of temporary tables: those created in the course of query execution, typically in \/tmp\/ those created during ALTER TABLE, typically in the database directory You may have seen these \"#sql-foo.frm\"\u2026","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1719,"url":"https:\/\/www.flamingspork.com\/blog\/2009\/10\/10\/how-many-cpu-cycles-does-a-sql-query-take-or-pagefaults-caused-or-l2-cache-misses-or-cpu-migrations\/","url_meta":{"origin":3137,"position":5},"title":"How many CPU cycles does a SQL query take? (or pagefaults caused&#8230; or L2 cache misses&#8230; or CPU migrations&#8230;)","author":"Stewart Smith","date":"2009-10-10","format":false,"excerpt":"I like profilers. I use them when trying to make software (such as Drizzle) faster. Many profilers suck - and pretty much all of them are impossible to attach to a running system. Two notable exceptions are oprofile and dtrace (for Linux and Solaris respectively). The downside of oprofile is\u2026","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/3137","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/comments?post=3137"}],"version-history":[{"count":1,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/3137\/revisions"}],"predecessor-version":[{"id":3138,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/3137\/revisions\/3138"}],"wp:attachment":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/media?parent=3137"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/categories?post=3137"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/tags?post=3137"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}