We are pretty close to having the table protobuf message format being final (there’s a few bits left to clean up, but expect them done Real Soon Now (TM)). You can see the definition (which is pretty simple to follow) in drizzled/message/table.proto. Also check out my series of blog posts on the table message (more posts coming, I promise!).

Drizzle allows either your StorageEngine or the Drizzle kernel to take care of storage of table metadata. You tell the Drizzle kernel that your engine will take care of metadata itself by specifying HTON_HAS_DATA_DICTIONARY to the StorageEngine constructor. If you don’t specify HTON_HAS_DATA_DICTIONARY, the Drizzle kernel stores the serialized Table protobuf message in a “table_name.dfe” file in a directory named after the database. If you have specified that you have a data dictionary, you’ll also have to implement some other methods in your StorageEngine. We’ll cover these in a later post.

If you ever dealt with creating a table in MySQL, you may recognize this method:

virtual int create(const char *name, TABLE *form, HA_CREATE_INFO *info)=0;

This is not how we do things in Drizzle. We now have this function in StorageEngine that you have to implement:

int doCreateTable(Session* session, const char *path,
                  Table& table_obj,
                  drizzled::message::Table& table_message)

The existence of the Table parameter is largely historic and at some point will go away. In the Embedded InnoDB engine, we don’t use the Table parameter at all. Shortly we’ll also get rid of the path parameter, instead having the table schema in the Table message and helper functions to construct path names.

Methods name “doFoo” (such as doCreateTable) mean that there is a method named foo() (such as createTable()) in the base class. It does some base work (such as making sure the table_message is filled out and handling any errors) while the “real” work is done by your StorageEngine in the doCreateTable() method.

The Embedded InnoDB engine goes through the table message and constructs a data structure for the Embedded InnoDB library to create a table. The ARCHIVE storage engine is much simpler, and it pretty much just creates the header of the ARZ file, mostly ignoring the format of the table. The best bet is to look at the code from one of these engines, depending on what type of engine you’re working on. This code, along with the table message definition should be more than enough

Up until November 2008 (that’s 3 years and 11 months for those playing at home) I worked on MySQL Cluster. Still love the product and love how much better we’re making Drizzle so it’ll be the best SQL interface to NDB :)

The ideas behind Drizzle had been talked about for a while… and with my experience with internals of the MySQL server, I thought that some change and dramatic improvement was sorely needed.

Then, in 2008, Brian created a tree. I was soon sending in patches at nights, we announced to the whole world at OSCON and it captured a lot of attention.

Since November 2008 I’ve been working on Drizzle full time. It was absolutely awesome that I had the opportunity to spend all my days hacking on Drizzle – both directly with fantastic people and for fantastic people.

But… the Sun set… which was exciting and sad at the same time.

Never to fear! There were plenty of places wanting Drizzle hackers (and MySQL hackers). For me, it came down to this: “real artists ship”. While there were other places where I would no doubt be happy and work on something really cool, the only way I could end up working out where I should really be was: what is the best way to have Drizzle make a stable release that we’d see be suitable for deployment? So, Where Am I Now?

Rackspace.

Where I’ll again be spending all my time hacking Drizzle.

It’s currently scheduled for 7pm on April 13th.

Come along, it will be awesome.

but then… just a bit down the way…. it hit:

It was “a bit wet”. Big blocks of ice falling from the sky (that hurt).

Anyway, on the way back we found a storm water drain:

Yes, behind Michael is just all water (and I’m not talking about the Bay).

Still managed to get a 36.5km ride out of it, so not all bad.

The first step is to create a skeleton StorageEngine plugin.

You can see my skeleton embedded_innodb StorageEngine plugin in its merge request.

The important steps are:

1. Create the plugin directory

e.g. mkdir plugin/embedded_innodb

2. Create the plugin.ini file describing the plugin

create the plugin.ini file in the plugin directory (so it’s plugin/plugin_name/plugin.ini)
An example plugin.ini for embedded_innodb is.

[plugin]
title=InnoDB Storage Engine using the Embedded InnoDB library
description=Work in progress engine using libinnodb instead of including it in tree.
sources=embedded_innodb_engine.cc
headers=embedded_innodb_engine.h

This gives us a title and description, along with telling the build system what sources to compile and what headers to make sure to include in any source distribution.

3. Add plugin dependencies

Your plugin may require extra libraries on the system. For example, the embedded_innodb plugin uses the Embedded InnoDB library (libinnodb).

Other examples include the MD5 function requiring either openssl or gnutls, the gearman related plugins requiring gearman libraries, the UUID() function requiring libuuid and BlitzDB requiring Tokyo Cabinet libraries.

For embedded_innodb, pandora-build has a macro for finding libinnodb on the system. We want to run this configure check, so we create a plugin.ac file in the plugin directory (i.e. plugin/plugin_name/plugin.ac) and add the check to it.

For embedded_innodb, the plugin.ac file just contains this one line:

PANDORA_HAVE_LIBINNODB

We also want to add two things to plugin.ini; one to tell the build system only to build our plugin if libinnodb was found and the other to link our plugin with libinnodb. For embedded_innodb, it’s these two lines:

build_conditional="x${ac_cv_libinnodb}" = "xyes"
ldflags=${LTLIBINNODB}

Some plugins (such as the md5 function) have a bit more custom auto-foo in plugin.ini and plugin.ac (as one of two libraries can be used). You can do pretty much anything with the plugin system, but you’re a lot more likely to keep it simple like we have here.

4. Add skeleton source code for your StorageEngine

While this will change a little bit over time (and is a little long to just paste into here), you can see what I did for embedded_innodb in the skeleton-embedded-innodb-engine tree.

5. Build!

You will need to re-run ./config/autorun.sh so the build system picks up your new plugin. When you run ./configure --help afterwards, you should see options for building with/without your new plugin.

6. Add a test

You will probably want to add a test to see that your plugin loads successfully. When your plugin is built, the test suite automatically picks up any tests you have in the plugin/plugin_name/tests directory. This is in the same format as general MySQL and Drizzle tests: tests go in a t/ directory, expected results in a r/ directory.

Since we are loading a plugin, we will also need some server options to make sure that plugin is loaded. These are stored in the rather inappropriately named test-master.opt file (that’s the test name with “-master.opt” appended to the end instead of “.test“). For the embedded_innodb plugin_load test, we have a plugin/embedded_innodb/tests/t/plugin_load-master.opt file with the following content:

--plugin_add=embedded_innodb

You can have pretty much anything in the plugin_load.test file… if you’re fancy, you’ll have a SELECT query on data_dictionary.plugins to check that the plugin really is there. Be sure to also add a r/plugin_load.result file (My preferred method is to just create an empty result file, run the test suite and examine the rejected output before renaming the .reject file to .result)

Once you’ve added your test, you can run it either by just typing “make test” (which will run the whole test suite), or you can go into the main tests/ directory and run ./test-run.pl --suite=plugin_name (which will just run the tests for your plugin).

7. Check the code in, feel good about self

and you’re done. Well… the start of a Storage Engine plugin is done :)

Leah playing guitar hero, me sitting across the room only slightly distracting her with a camera.

Maybe I will end up getting a Holga one of these days… being restricted can be fun.

My laptop came with a Windows Vista license. An anti-feature in itself – I didn’t want it, have never used it (I run Ubuntu and love freedom).

However, if you try and read the license key off this sticker, it’s increasingly difficult to do so. It’s being worn away. Why? Because it’s on the bottom of the laptop and I’m using it on my lap (so friction rubs it away).

Luckily I don’t run Windows Vista and need to re-install it any time soon.

This is totally not confined to at-work presentations.

The number of sessions I have sat through that could have taken 5 minutes instead of 20,30,40 or even 60 is amazing. Remember: I have not flown half way around the globe to see you read. I have come to hear a story, to see how conclusions were formed and interact.

Often, the tools are deficient. Powerpoint encourages bad habits (you can use PowerPoint for excellent slide decks too, but ignore the temptations of boring templates, bad effects and dot lists). The dot point list is more often than not your enemy. I (and anybody else in the audience who has learnt to read) can read your dot points faster than you can. While I’m reading, I’m not listening to you. If you spoke a cure for all forms of cancer just after having put a slide up filled with dot points… 90% of people will miss it.

Now, dot points are an excellent way to remind you what the heck you’re meant to be talking about (and in what order). Use presenter notes! They are really useful.

If your laptop/presentation software doesn’t support a “presenter” mode that lets you view presenter notes but not the whole room, simply write them down, print them out, or anything like that. One simple practice run through will make you be able to do this seamlessly.

The last couple of presentations I did were completely assembled using 280slides.com. An excellent web app for doing presentations. It will import and export ODF (and other formats) so you’re not tied to a (unfortunately) non open source web app. That being said, it ran fine in my browser and unlike OpenOffice.org, did not make me want to stab people repeatedly every time I used it.

So, Stewart’s quick tips:

• Tell a story. How did you get to your conclusions?
• Don’t just read. Use visuals to accompany the talk. Visuals aren’t the talk.
• Practice. Just once or twice through will make things a lot smoother.

Equipment:

• Make sure your equipment works beforehand. Nobody wants to see you fiddle around with your Windows/OSX laptop only to find out you didn’t bring the dongle or can’t operate the Displays control panel. (Interestingly enough, I see Linux “just work” more than Windows or OSX these days).
• If there is a microphone, use it. I don’t want to struggle to hear you.
• If you are constantly using a laser pointer you either have too much on your slides or the slide does not highlight the important information. (laser pointers are useful when people ask questions though)

One blog I love on the subject is Presentation Zen. I’ll also recommend the book, but you can get so much just from the web site.

Some excellent recent presentations:

• Simplicity Through Optimization – Paul McKenney
  Paul is able to explain RCU clearly and concisely through visuals. You are left with no doubt that this does really work. The visuals are not everything, they assist in the telling of the RCU story
• Teach every child about food – Jamie Oliver
  I watched this online. Note how not everything was smooth the whole way. Also note how this was still effective. Passion is an awesome tool. Check out the simple graph showing lead causes of death: simple and effective.
• Bill Gates on energy: Innovating to Zero!
  Historically, Bill Gates has not been the most engaging speaker. We can all forget the horrible PowerPoint slides with four hundred dot points about some release of something that nobody cared about. This is different. Clear, concise, engaging and simple visuals to make the point.

So, I got it and thought “hrrm… I better run a roll of film through it to make sure it works”. Off to the fridge i went to find the cheapest, shittiest roll of film possible… I found “Walgreens” brand film. Manufactured by one of many, bought for cheap, and run through the F80.

Some shots turned out pretty good. I have the full set (most of the roll) up on flickr. A few choice ones are:

Which due to some nice accident of lighting, turned out pretty good. IIRC this was pretty late at night and I was editing photos as Michael came over (bringing much needed beer).

Slides and beer, do you need anything else? I just like this because it’s a snapshot of what I was working on (well, kinda, I was mostly just manipulating digital images).

Leah and I went bushwalking… so had to snap a shot of her. I do like the Nikon 50mm as a portrait lens. The film… well… it was cheap, but not too bad actually.

A shallow depth of field can be a lot of fun. Although not entirely sure how I feel about the bokeh….

Which has some odd colours. Nice, but odd.

I like my “new” body. It’ll be fun.

The worst is renting… the amount of times you have to press skip (or the damn disc doesn’t work) you do start to wonder if you would have had a much better user experience if you just downloaded it instead.