Being clear in licensing is a great benefit to users. I’m quite proud that with Drizzle we got to a really simple situation:
We had the complete okay from Sun to have libdrizzle be BSD licensed and for us to work on that.
There was a point a few years ago where Sun could have had the next generation UNIX filesystem. It was in Solaris (and people were excited), there was a port to MacOS X (that was quite exciting for people) and there was a couple of ways to run it on linux (and people were excited). So… instead of the fractured landscape of ext3, HFS+ and (the various variations of) UFS we could have had one file system that was common between all of the commonly used UNIX-like variants. Think of being able to use a file system on a removable drive that isn’t FAT and being able to take it from machine to machine (well… Windows would be a problem, but it always is).
There was some really great work done in OpenSolaris with integration between the file manager and ZFS snapshots (a slider bar to browse the history of a directory, an idea I’ve championed for over a decade now, although the Sun implementation was likely completely independently developed). The integration with the package manager was also completely awesome, crash safe upgrades!
However, all this is pretty much moot. Solaris is used by fewer people than ever, it’s out of OS X and BTRFS is going to take the place that ZFS could have held in the Linux world. So, unfortunately, ZFS is essentially dead. This is a shame…. it could have been something huge.
I’d meant to finish writing this way back in July… but I failed at that. Now is a good time to talk about Rookie-O as my again new colleague Andrew Hutchings (Buy his and Sergei’s book on MySQL 5.1 Plugin Development!) just went through the same thing (but in London instead of Hong Kong) given by the same trainer (Hi Eddie!).
Rackspace is the second employer I’ve had that has some kind of new hire training (the first being Sun). I am, of course, not quite counting Salmiakki as new-hire training for MySQL (although I probably should). To quote from the Wikipedia article: “Although the rumor of the heart attack was a hoax, the drink may still cause harm. The strong flavor almost completely masks the presence of ethanol, and the drinker may not realize he is consuming a drink almost 40% alcohol by volume (80-proof), leading to possible alcohol poisoning.” A promising introduction to the company.
Monty, Mårten and Kaj with Salmiakki singing Helan Går at the MySQL User Conference Japan in 2007
I could possibly say something about the Sun New-Hire training… but I’m just trying to find something positive to say – and I can’t. I got a bit of hacking done? Seriously.
Actually coordinating a time to attend a Rookie-O (Rookie Orientation, the Rackspace name for new hire training) was rather tricky. There was one right before the MySQL User Conference back in April (not the best of timing), one during an upcoming team meeting (again, not ideal) and one that got organised in the middle of everything for the office in Hong Kong. So, I headed to Hong Kong.
The Hong Kong office is relatively new (late 2008) and there were people there who hadn’t gone through the standard Rackspace Rookie-O (Orientation).
It was rather cool to hang out with other people who worked for the company – and in totally different areas than I do. I did get a better understanding for how the rest of the company operates and the people involved. The training itself was useful and substantially less geared towards not-my-job than Sun’s was.
The good news is that Andrew thought it was useful too. Pretty impressed so far.
So I sometimes get asked if we funnel back bug reports or patches back to MySQL from Drizzle. Also, MariaDB adds some interest here as they are a lot closer (and indeed compatible with) to MySQL. With Drizzle, we have deviated really quite heavily from the MySQL codebase. There are still some common areas, but they’re getting rarer (especially to just directly apply a patch).
Back in June 2009, while working on Drizzle at Sun, I found a bug that I knew would affect both. The patch would even directly apply (well… close, but I made one anyway).
So the typical process of me filing a MySQL bug these days is:
This happened within a really short time.
Unfortunately, what happens next isn’t nearly as awesome.
Namely, nothing. For a year.
So a year later, I filed it in launchpad for MariaDB.
So, MariaDB is gearing up for a release, it’s a relatively low priority bug (but it does have a working, correct and obvious patch), within 2 months, Monty applied it and improved the error checking around it.
So MariaDB bug 588599 is Fix Committed (June 2nd 2010 – July 20th 2010), MySQL Bug 45377 is still Verified (July 20th 2009 – ….).
(and yes, this tends to be a general pattern I find)
But Mark says he gets things through… so yay for him.2
A couple of months ago (December 1st for those playing along at home) it marked five years to the day that I started at MySQL AB (now Sun, now Oracle). A good part of me is really surprised it was for that long and other parts surprised it wasn’t longer. Through MySQL and Sun, I met some pretty amazing people, worked with some really smart ones and formed really solid and awesome friendships. Of course, not everything was perfect (sometimes not even close), but we did have some fun.
Up until November 2008 (that’s 3 years and 11 months for those playing at home) I worked on MySQL Cluster. Still love the product and love how much better we’re making Drizzle so it’ll be the best SQL interface to NDB :)
The ideas behind Drizzle had been talked about for a while… and with my experience with internals of the MySQL server, I thought that some change and dramatic improvement was sorely needed.
Then, in 2008, Brian created a tree. I was soon sending in patches at nights, we announced to the whole world at OSCON and it captured a lot of attention.
Since November 2008 I’ve been working on Drizzle full time. It was absolutely awesome that I had the opportunity to spend all my days hacking on Drizzle – both directly with fantastic people and for fantastic people.
But… the Sun set… which was exciting and sad at the same time.
Never to fear! There were plenty of places wanting Drizzle hackers (and MySQL hackers). For me, it came down to this: “real artists ship”. While there were other places where I would no doubt be happy and work on something really cool, the only way I could end up working out where I should really be was: what is the best way to have Drizzle make a stable release that we’d see be suitable for deployment? So, Where Am I Now?
Where I’ll again be spending all my time hacking Drizzle.
So i got the T1000 working again (finally, after much screwing about trying to get the part). I then hit the ever annoying “no console” problem, where the console didn’t work – kind of problematic.
After a firmware upgrade, and passing “console=/dev/ttyS0″ to the kernel, things work.
So the T1000 firmware 6.3 doesn’t work with modern debian kernels. Thing swork with 6.7 though.
We’re having a Drizzle Developer Day just after the MySQL Conference and Expo next week. You don’t have to be attending the conference to come to the Drizzle Developer Day. Just bring your enthusiasm for free databases, Drizzle and good software. Spaces are limited, so head on over to the signup page and fill in your name if you haven’t already
If coming from the Hyatt Regency Santa Clara (where the MySQL Conference and Expo is), at least I will be driving from there, so let me know if you want a lift.
Stopping All Servers
All 221 tests were successful.
The servers were restarted 14 times
Spent 1424.921 of 1521 seconds executing testcases
(All tests have passed on OpenSolaris on x86 for a while now).
As usual, you can likely ignore the “Menlo Park, CA” part (I haven’t checked though… but am 99% sure).
Upgraded VirtualBox and booted up my OpenSolaris VM. VirtualBox 2.1.0 finally fixes the bug where if 127.0.0.1 was in resolv.conf on the host – no DNS for you in the guest (unless in the guest you were running a DNS server).
Haven’t tried it yet… but OpenGL Accelleration makes at least a checkbox appearance in VirtualBox 2.1…. so that could be rather awesome.
Going a lot better with OpenSolaris 2008.11 than previous releases.. It looks like it might be quite easy to get to the stage of building Drizzle on it.
Just figured out how to change to Dvorak! Yay, I can type again! (Go to Input Methods preference panel and add US/DVORAK as a language, move it to the top, and enable the input method application and do it that way).
Currently installing sunstudioexpress. Why not gcc? I’m pretty sure the version in OpenSolaris is still ancient (so won’t build drizzle) and Sun Studio does produce different warnings (which indicate real bugs in a bunch of cases).
Things I wish were packaged: latest protobufs, latest bzr, gcc 4.x
After what many hours trying to netboot the T1000 to install Solaris Express, I wondered “how hard is it for debian?”.
Easy. get the sparc64 boot.img, put it on TFTP server, add “filename “boot.img”;” or similar to dhcp, boot the T1000 from the service console something like this “bootmode bootscript=”boot net:dhcp”\n restart -c” and install away!
As for Solaris?
Well… dhcpd.conf:
option space SUNW;
option SUNW.root-mount-options code 1 = text;
option SUNW.root-server-ip-address code 2 = ip-address;
option SUNW.root-server-hostname code 3 = text;
option SUNW.root-path-name code 4 = text;
option SUNW.swap-server-ip-address code 5 = ip-address;
option SUNW.swap-file-path code 6 = text;
option SUNW.boot-file-path code 7 = text;
option SUNW.posix-timezone-string code 8 = text;
option SUNW.boot-read-size code 9 = unsigned integer 16;
option SUNW.install-server-ip-address code 10 = ip-address;
option SUNW.install-server-hostname code 11 = text;
option SUNW.install-path code 12 = text;
option SUNW.sysid-config-file-server code 13 = text;
option SUNW.JumpStart-server code 14 = text;
option SUNW.terminal-name code 15 = text;
option SUNW.SbootURI code 16 = text;
host hurricane {
hardware ethernet 0:14:4f:1e:28:e;
fixed-address 192.168.1.19;
option host-name “hurricane”;
filename “sparc64-etch-boot.img”;
# filename “sol-nv-b103-sparc”;
# option SUNW.install-server-ip-address 192.168.1.1;
# option SUNW.install-server-hostname “saturn”;
# option SUNW.install-path “/mnt/sol-nv-b103-sparc/”;
# option SUNW.root-server-ip-address 192.168.1.1;
# option SUNW.root-server-hostname “saturn”;
# option SUNW.root-path-name “/mnt/sol-nv-b103-sparc/Solaris_11/Tools/Boot”;
}
(obviously changing the comments around) and having the Solaris Express DVD mounted and NFS exported…. it *still* doesn’t work. It goes “unable to mount filesystem” with no further hints (even when tcpdumping the network).
Documentation for doing the simple thing of using $dhcp_server and $nfs_server to network boot a Solaris install on a Sparc box is *COMPLETELY* missing.
Now, I’m a smart guy (and if you don’t believe that, at least believe I’m not stupid). If I can’t get it to boot the installer, what chance do others have?
I’ll try OpenSolaris out when it’s on SPARC (and please oh please oh please just have an easy way to net boot the installer using a linux host). Please take the debian way (just a single file on tftp).
So now it’s goodbye Solaris (I’m not going to have something I can’t re-install, upgrade or security patch) and it’s hello Debian (and sanity).
Yes, this does mean I’ll care about Drizzle on Linux Sparc.
How?
Seriously, how do I do this?
Following constraints: T1000 has no local DVD drive. No other Solaris hosts on network.
(Note that it takes about 15 seconds to find the Debian install instructions. “Put this file on tftp server, boot.”)
Using the wonderful bittorrent, I got the CD image in next to no time (in contrast to the Solaris Express image I’m currently downloading via HTTP that’s taking forever).
Boot time in VirtualBox (off the ISO image) was rather quick, usual questions on keyboard layout and desired language (it’d be neater if these were GUI questions… but anyway). The GNOME desktop loaded up, popup window informed me that it had connected to the network. Awesome.
Package manager: opend quickly, using the opensolaris.org repository and it does seem to have a lot of packages… even MySQL 5.0.67 (and 4.0.24). Not 5.1 though, but it is early days (and it was just released as GA the other day).
At least one unusual thing was SUNWgrub and SUNWgrubS (where the S is for source). I assume this is some packaging oddity as I don’t ese other packages like this.
SUNWii wins the odd package name award.
The Time Slider seems like possibly the most awesome thing ever. It periodically takes ZFS snapshots of your disk and presents you with a time slider in nautilus so you can just view your data how it was in the past (at previous snapshots).
I can’t see how to change the keyboard layout to DVORAK (at least while booted off the CD image).
The getting started guide also shows how to get a development environment going… this is quite promising. Will do proper install shortly and do a step by step “building drizzle on OpenSolaris 2008.11″ post.
Real noisy fucker. So loud, that if it’s in the garage but the back door is open, I still hear it.
Being used for drizzle dev on Solaris… although a switch to OpenSolaris or Linux is likely imminent. Straight Solaris 10 is just too annoying.
The past 3 years, 11 months I have worked full time on NDB (MySQL Cluster). It’s been awesome. Love the product and people. In the time I’ve been on the Cluster team, we’ve gone from a small group that would easily fit in the (old old) Stockholm office to one that requires large rooms to house us all in. It’s also been all about smart people (you have to be to work on a distributed database).
With MySQL Cluster 6.4 we’re getting in a bunch of features that have been on the “wide adoption” wishlist. With each release of NDB we’ve gained a wedge of applications that can be used with it – and 6.4 is no exception.
One of the biggest things that’s been worked on is multithreaded data nodes. If you check out Jonas‘ recent posts on 500,000 reads/sec and then a massive 700,000 reads/sec.
We’ve also got a Microsoft Windows port coming up, which a number of people have asked for over the years. Mostly I think this is a “I want to try it out” thing and not a deployment thing. (can any sane person deploy a HA app on Win32?)
I’ve used “NDB$INFO” as the ultimate answer to any problem for a while now. It’s been the much-wanted monitoring interface. We have a lot of info inside NDB that currently isn’t easily user accessible (or only accessible through the magic DUMP interface or by gathering up many events in the cluster log). We have the start of NDB$INFO in 6.4 now and Martin will be continuing my work in making it truly awesome.
So go and grab the 6.4 tree and have a look – things are looking sweet.
What next for me?
Well… a while ago I started hacking on Drizzle. Why? Well… I thought we could move the database server in a new direction and make it more modular, leaner, meaner query machine.
And now, I’m starting to work on it full time.
It’s exciting, and I’ll be blogging on the first TODO which is remove the FRM file and switch to a full discovery method shortly.
UPDATE: Yes, I’m working full time on Drizzle for Sun Microsystems (in the CTO group). While not spending work time on NDB anymore, no doubt you’ll still see fun-time patches.
It’s like removing the brakes from your car. yes, it will go faster (slightly less weight) but, dude, you just removed the brakes.
So after seeing Paul Fenwick rave about the presenter screen for OO 3, I decided to grab the debs and give it a go.
It still is very slow opening large presentations (i.e. mine), but it does look nicer at least… well… at least some of the widgets do.
Will report back when I’ve had a bit more time to fiddle with it.
Thanks to Glynn for pointing me to the right thread on opensolaris.org (in a comment on my Good adventures with OpenSolaris post). The package verification thingy (pkg verify -v -f SUNWfirefox) did actually throw an error (indicating some sort of problem). So that’s pretty neat. The fact that it got into trouble in the first place isn’t good, but corruption detection is the next best thing.
I still occationally hit the bug in VirtualBox where if you have 127.0.0.1 in your resolv.conf on your host (e.g. running a local caching nameserver), VirtualBox passes this through to the guest, so the guest tries to use the guest 127.0.0.1 as a nameserver – this usually doesn’t work so well.
The good news is, Firefox now works in my OpenSolaris VM.
The bad news is that even though I’ve gone and set my keyboard layout as DVORAK (with the Input Method Switcher applet), whath should be ctrl-l (for location bar) in Firefox, actually brings up the Print dialog (on DVORAK, L is where P is on QWERTY).
But, I’ve managed to download bazaar now, and the install was simple (just follow INSTALL in the bzr tarball). At some point I’ll badger someone to make an OpenSolaris package for it so you could do “pkg install bzr”, but you can’t do that yet.
The next challenge will be to branch repositories from the host onto a temp drive, build and test.
First of all, thanks to everyone who commented on my previous OpenSolaris entry (which wasn’t really positive at all).
I recently tried again – this time starting with an ISO of build 93. I’d recommend completely ignoring the 2008.05 release and going straight for the build 93 image.
Installed easily in VirtualBox, adding the VirtualBox extensions was easy. Select “Devices -> Install Guest Additions” in the VirtualBox menu, then when logged into the OpenSolaris install, do the following:
su
pkgadd -d /media/VBOXADDITIONS_1.6.0_30421/VBoxSolarisAdditions.pkg
(you then say yes, i really do want to install it. rather obvious. I had to do this step again after the “pkg image-update” below though). Just logging out and then back in again gets you all the awesomeness you’d expect from running other guests (such as that system released by a large corporation in Redmond).
The “pkg image-update” went as expected, and I’m now running build 94.
I installed SunStudio Express (compilers) pretty easily – “pkg install sunstudio”. Unfortunately, this is all in /opt/SunStudioExpress and not in $PATH, which would have been much more useful. I guess there’s still a bit to go before usability nirvana. Also, no .desktop entries, so have to explicitly run /opt/SunStudioExpress/bin/sunstudio to get the NetBeans gui. Presumably if i add /opt/SunStudioExpress/bin to PATH, building random software packages will be nicer.
So, I then want bzr so i can pull source repositories. Monty Taylor informs me that the magic packages you want are: SUNWgcc, gcc-dv and SUNWtoo. Then you can build bzr as downloaded from the website. Installed these easily.
However, now trying to get the bzr source:
$ firefox
ld.so.1: firefox-bin: fatal: /usr/lib/firefox/libxul.so: corrupt or truncated file
and then symbol kPStaticModules: referenced symbol not found.
So maybe I shouldn’t have upgraded to build 94…..
But certainly in much better shape than the may release, but be warned, it’s still a work-in-progress and some things may sporadically not work from time to time (e.g. like firefox and now).
Hopefully, some time soon I’ll get a MySQL build (well… really I want MySQL Cluster, and later drizzle) going and will really be able to hammer these things with dtrace.
Arrived okay – long travel, but in one piece. Staying at the doubletree.