I have recently been constructing a 100 page document going over a whole bunch of the details for the Monorail we’re building at Burning Man this year.

Apart from randomly freezing, and then suddenly not displaying images until I had restarted it – it’s also really slow.

The last straw was when leafing through the document before getting it printed. I had inserted a bunch of pages before this last section. But now, there was this empty page in the last section of the document.  The part that I hadn’t touched for days. If I tried to remove the blank page, all the images on nearby pages moved so that they were on top of each other.

I ended up just printing it. There is a blank page that I can’t get rid of.

It is a piece of software that worries me. Is this really meant to be an alternative? It has NEVER worked well for me. Basic tasks sure, but I continually find myself pining for Word 5.1a on the Mac (System 7 that is) or Nisus Writer or even ClarisWorks.

If opening Microsoft Word documents fairly accurately is your only good feature, how do you expect to survive in the free (software) world?

So, while my twitter stream may suggest desires for punning the developers in the face or their early demise through painful methods….. I really just wish that sometime in the past 10 years you had made it not shit me to tears.

Certainly another failure of Sun Microsystems and I don’t expect Oracle to do any better at all (especially considering recent actions).

So, for a very long time now, Drizzle has been using a protobuf based structure to describe the structure of tables. The idea was to be able to have engines rather painlessly generate this structure themselves (which several now do). A secondary idea was to use this structure itself for CREATE TABLE (in progress, and embedded_innodb does in fact does only use the table message for its CREATE TABLE codepath). The third idea was to just put the table protobuf message into the replication stream instead of the CREATE TABLE statement (i.e. a SQL string). This means that you could (easily) write a replicator to a DBMS with different SQL syntax, or to a system that doesn’t speak SQL at all.

The final step, to reduce duplicated code functionality, would be to have the code that does SHOW CREATE TABLE to use a common bit of code for turning a table protobuf message back into a SQL string.

We now have that.

Just now, my branch to replace the old SHOW CREATE TABLE code (that was using TableShare and friends) with the statement_transform code (that we also use in converting a replication log to SQL commands) hit trunk.

Yay.

Sure, you could start running the whole test suite against your engine…. but where it wasn’t specifically MyISAM, you’d certainly hit things that were.. well… as simple as having to change the result file so that SHOW CREATE TABLE masked out the ENGINE= part.

Also, if you were just starting out and were trying to incrementally write your engine, instead of just hacking away for 6 months on “everything” and then testing “everything” and hoping that anything at all worked – you were out of luck.

So when working on the embedded_innodb engine for Drizzle I wanted to finally fix this problem. I was not going to fix it perfectly, or completely. What I wanted was a set of simple tests, that were very short and that tested distinct bits of the engine.

So… if you look in the Drizzle tree, in plugin/embedded_innodb/tests/ you’ll find a bunch of test cases beginning with basic_ (they’re also now in plugin/pbxt/tests). These check very basic functionality and should be useful when starting to write your own storage engine.

They should also port easily to MySQL too (possibly without changes), but I haven’t tried it. Porting to PBXT was simple.. a regex to replace InnoDB with PBXT.

Enjoy.

So, what made it in? A lot of bug fixes (especially around error reporting and tables without an explicit primary key), SELECT FOR UPDATE, support for temporary tables and support of prefix indexes.

We also now, if it’s available, link to HailDB instead of Embedded InnoDB. At some point “soon” we shall just require HailDB.

Here are some of the shots I got:

(granted I didn’t press the shutter release, but I like it)

Is the one that sealed it for me. This was the “ahh… I can use this for all sorts” shot.

All of these were developed and scanned at the Walgreens down the street. I could probably do better scans of some… but this was awfully less work for me.

I am really liking the skin tones from it. The vividness of colours also comes through while retaining excellent skin tones (certainly not always the case). I may even end up shooting some at Burning Man (did buy more rolls yesterday!)

There is (of course) more being added to my Kodak Portra 160VC set on flickr.

Yazz has been making an epic over engineered shower for camp.

Migrating your software to use HailDB is really simple. In fact, for this release, it shouldn’t take more than 5 minutes.

Highlights of this release:

• A lot of compiler warnings have been fixed.
• The build system is now pandora-build.
• some small bugs have been fixed
• Header file is now haildb.h instead of innodb.h
• We display “HailDB” instead of “Embedded InnoDB”
• Library name is libhaildb instead of libinnodb
• It is probably binary compatible with the last Embedded InnoDB release, but we don’t have explicit tests for that, so YMMV.

Check out the Launchpad page on 2.0.0 and you can download the tarball either from there or right here:

• haildb-2.0.0.tar.gz
  MD5:  183b81bfe2303aed435cdc8babf11d2b
  SHA1:  065e6a2f2cb2949efd7b8f3ed664bc1ac655cd75

Testing how much power the rope will pull off the generators

Simon Phipps this past week at OSCON talked about Open Source Continuity in practice – specifically mentioning some open source software projects that were at Sun but have since been abandoned by Oracle and different strategies you can put in place to ensure your software survives, and check lists for software you use to see if it will survive.

So what can you do to not destroy your community, but ensure you never get one to begin with?

Similar to destroying your community, you can just make it hard: “#1 is to make the project depend as much as possible on difficult tools.”

#1 A Contributor License Agreement and Copyright Assignment.

If you happen to be in the unfortunate situation of being employed, this means you get to talk to lawyers. While your employer may well have an excellent Open Source Contribution Policy that lets you hack on GPL software on nights and weekends without a problem – if you’re handing over all the rights to another company – there gets to be lawyer time.

Your 1hr of contribution has now just ballooned. You’re going to use up resources of your employer (hey, lawyers are not cheap), it’s going to suck up your work time talking to them, and if you can get away from this in under several hours over a few weeks, you’re doing amazingly well – especially if you work for a large company.

If you are the kind of person with strong moral convictions, this is a non-starter. It is completely valid to not want to waste your employers’ time and money for a weekend project.

People scratching their own itch, however small is how free software gets to be so awesome.

I think we got this almost right with OpenStack. If you compare the agreement to the Apache License, there’s so much common wording it ends up pretty much saying that you agree you are able to submit things to the project under the Apache license.  This (of course) makes the entire thing pretty redundant as if people are going to be dishonest about submitting things under the Apache licnese there’s no reason they’re not going to be dishonest and sign this too.

You could also never make it about people – just make it about your company.

#2 Make it all about the company, and never about the project

People are not going to show up, do free work for you to make your company big, huge and yourself rich.

People are self serving. They see software they want only a few patches away, they see software that serves their company only a few patches away. They see software that is an excellent starting point for something totally different.

I’m not sure why this is down at number three… it’s possibly the biggest one for danger signs that you’re going to destroy something that doesn’t even yet exist…

#3 Open Core

This pretty much automatically means that you’re not going to accept certain patches for reasons of increasing your own company’s short term profit. i.e. software is no longer judged on technical merits, but rather political ones.

There is enough politics in free software as it is, creating more is not a feature.

So when people ask me about how I think the OpenStack launch went, I really want people to know how amazing it can be to just not fuck it up to begin with. Initial damage is very, very hard to ever undo. The number of Open Source software projects originally coming out of a company that are long running, have a wide variety of contributors and survive the original company are much smaller than you think.

PostgreSQL has survived many companies coming and going around it, and is stronger than ever. MySQL only has a developer community around it almost in spite of the companies that have shepherded the project. With Drizzle I think we’ve been doing okay – I think we need to work on some things, but they’re more generic to teams of people working on software in general rather than anything to do with a company.