getarg calls srand() ???

storage/ndb/test/src/getarg.c

Guess what? It calls srand(time(NULL)) in getarg(). Why you ask? well.. what you want to be able to when specifying a flag is have it be true, false or it could “maybe” be set.

That’s right kids… maybe.

I’m sure it’s used somewhere in our test suite to get coverage on different things.. but umm.. yeah, interesting discovery for today.

Best use of the word ‘fuck’ for today

A Web OS? Are You Dense? – Ted Dziuba

All this crap about moving everything into another 5 layers of abstraction down just means we’re never going to have a computer that’s visibly any faster than an Apple // to any user sitting in front of it… Sure, it can recalculate your spreadsheet faster (as long as it’s not an online spreadsheet running in a browser), but menus take longer to appear.

SetFileValidData Function (Windows) – Now with added FAIL

SetFileValidData Function (Windows)

There seems to be two options on Win32 for preallocating disk space to files.

Basically, I want a equivilent to posix_fallocate or the ever wonderful xfsctl XFS_IOC_RESVSP64 call.

The idea being to (quickly) create a large file on disk that is stored efficiently (i.e. isn’t fragmented).

From SQL, you’d do something like “CREATE LOGFILE GROUP lg1 ADD UNDOFILE ‘uf1’ INITIAL_SIZE 1G;” and expect a 1GB file on disk. One way of getting this is calling write() (or WriteFile() on Win32) repeatedly until you’ve written a 1GB file full of zeros. This means you’re generating approximately 1GB of IO.

Except it’s worse than that: every time you extend the file, you’re going to be changing the metadata (file and free space information). If you’re lucky, you won’t be using a file system that writes a new transaction to the journal for each time you do this.

If your file system allocator doesn’t like you today (even more likely when you’ve got more than one process doing IO), you may end up with rather fragmented files as well – especially if you’re doing synchronous IO. So you want some method of saying “this file will be size X, please allocate disk space to it in the most efficient way for a file size of X” as it’s not possible to infer this from everyday IO calls (I guess the Win32 CopyFile and CopyFileEx calls could though).

It probably doesn’t do it, but having a CopyFile call would be neat for copy on write file systems and saving space… although I wonder how many Win32 apps would cope with ENOSPC on a write to an existing part of a file.

On IRIX we used the magic xfsctl() with the XFS_IOC_RESVSP64 argument. On Linux (with XFS), we use the same. On ext2/ext3 the only way to get the same has been to (with the file system unmounted), parse the file system and implement it yourself. Although (and this just in) the brand new fallocate() call should help with this. The posix_fallocate() call in GNU libc has just been a wrapper around the simple method of writing 0 to a file from start to end (albeit rather efficiently).

XFS implements something called “unwritten extents”. An unwritten extent says “this range of blocks is allocated to this file. If reading from this range, return a zero page. If writing, split the unwritten extent into 3 parts: before, the newly written extent (which isn’t unwritten: i.e. now valid data), and the after extent.” Simple, rather efficient and gets really good allocation as XFS gets to search the free space btrees based on size.

So what to do on Win32 (apart from drink heavily to try and make it all go away)?

There’s SetFileValidData, but that needs special permissions and may expose previously deleted data from other users. i.e. massive security hole. FAIL

There’s SetEndOfFile which, quoting the MS docs: “If the file is extended, the contents of the file between the old end of the file and the new end of the file are not defined.” Not exactly reassuring… but introduced in W2k, so rather safe to use today. Doesn’t save you from having to fill the file with zeros as part of initialisation though.

There’s SetFileInformationByHandle, which looks like it may do exactly what I want… if you read between the lines of the documentation. But it’s only supported starting with Vista. Which you all use of course, so that’s not a problem.

Building MySQL on Windows – MySQL Forge Wiki

Building MySQL on Windows – MySQL Forge Wiki

This one covers running mysqld in the VisualStudio debugger, which can be useful.

I have no special ndb_mgmd.exe or ndbd.exe in debugger instructions or wisdom (running them from mysql-test-run.pl at least). I’ve attached debugger to already running (started by mysql-test-run.pl) ndb processes, but haven’t made any changes to mtr to make it like the mysqld of “go and enter this”.

Some thoughts on VirtualBox 2.0….

I like it.

UI looks more polished (maybe QT4 renders nicer on my Ubuntu GNOME based desktop… but it certainly looks more like the rest of my apps).

Possibly a bit faster?

64bit guests – WIN!

Shared Folders still cause the XP guest to panic if i try to do anything non-trivial (e.g. build, run cmake). I still use Samba on host to share with guest – which works okay, but isn’t super-happy-fun-time fast.

Seemless mode is a bit nicer, the Windows task bar sits above the GNOME task bar, making it a lot easier to interact. It’s not perfect yet, you can still get some real visual oddities, but it’s pretty sweet.

When just using normal, in window, it can cause X to use a bit much CPU, which goes away when the VirtualBox window is minimised – leading to faster compiles in guest.

Having a guest running (in this case Windows XP) does cause a lot more CPU wakeups – draining battery life.

I wish the snapshots showed how large they were, it’s currently kinda guesswork to work out how much disk space a snapshot is using.

MySQL Conference & Expo 2009 – CFP open

Is it that time already? MySQL Conference & Expo 2009 has opened the CFP.

Submit (well) early and often. It’s always an exciting (and exhausting) conf. Good technical, relevant content is what makes it good. Getting to talk to people who do amazing things, people who use your software, people looking to use it, people who want to chat about how you can learn off each other.

Any suggestions for what you’d like to hear from me (Cluster, Drizzle et al) are welcome – either via private mail or comments here.

OpenOffice.org 3 dev release

So after seeing Paul Fenwick rave about the presenter screen for OO 3, I decided to grab the debs and give it a go.

It still is very slow opening large presentations (i.e. mine), but it does look nicer at least… well… at least some of the widgets do.

Will report back when I’ve had a bit more time to fiddle with it.

the power dongles come to ruin our way of life

I recently moved.

I found enough of the power supplies for USB disk enclosures to power on the most crucial disks (read: the ones attached to mythtv).

However, I have a disk I use for offsite backups. I need to find the dongle for it. Can I find it? No, of course not. I’m user it’s one of these 5 boxes of misc office crap (including *lots* of cables).

Why can’t we just have a standard USB disk enclosure power cable?

Please, pretty please.

Building MySQL Cluster on Windows (for Windows)

You will need:

  • CMake (at least 2.4.7)
  • Bazaar (the newer the better – 1.6 was just released – at least use that)
  • Gnu Bison
  • Visual Studio (Express works, but I’m talking about 2005 here)
  • … and all this installed on a Microsoft Windows machine.
  • … and to hate yourself, you are going to be using Windows after all.

Then, get and build it:

  1. Get the source:
    bzr branch lp:~mysql/mysql-server/mysql-5.1-telco-6.4-win
  2. Run CMake. the CMake GUI can now be used to select compile options! You’ll have to set the path “where is the source code” to where you put the source code in step 1.
  3. Hit “Configure” in CMake
  4. Select the target (i.e. the version of Visual Studio you’re going to use)
  5. Select the build options. HINT: WITH_NDBCLUSTER_STORAGE_ENGINE may be a useful one to enable
  6. Hit Configure again
  7. Hit Ok.
  8. CMAKE now generates the Visual Studio project. Use this time to drink some good scotch.
  9. Open Mysql.sln (which should launch Visual Studio)
  10. Go Build -> Build Solution (or hit F7)

Now you can go and have much whisky as this will take a few minutes. You should now have a set of built binaries for MySQL Cluster on Windows. Scary.

rather appropriate xkcd (internet connectivity)

xkcd

yeah, one about internet in new apartment. really feeling that at the moment… except it’s a house, not apartment.

Bought an Unwired prepaid thing today… just to tide me over for the 5-7 working days (at least) before they (being Telstra) actually get my phone line working (and then DSL can get connected).

gah, gah gah.

ndb_mgm.exe builds (and works) in mysql-5.1-telco-6.4-win

“MySQL Cluster 6.4 Windows tree” branch in Launchpad

(which really should have the -fail suffix… but anyway)

In what will (soon) be mirrored to launchpad, all but 17 targets (yeah, working on that… but it’s out of 130 or something) build.

Not only that, I’ve used the management client (ndb_mgm.exe) to monitor the cluster running my Bugzilla instance (which is now a rather old 6.3 build).

Getting closer to NDB on Windows.

Be afraid. Be very, very afraid.

“MySQL Cluster 6.4 Windows tree” branch in Launchpad

“MySQL Cluster 6.4 Windows tree” branch in Launchpad

That’s right folks, I’m pushing up patches for MySQL Cluster on Windows. This tree is incomplete, and no promises on when enough will be pushed for it to even compile on Windows.

Tree is updated when launchpad pulls from our internal tree.

linux.conf.au paper review

<sarcasm>Because I had nothing else on this month.</sarcasm> I’m currently reviewing linux.conf.au papers. This is fun, brutal and hard.

For those of you who submitted: never be disheartened by not having it accepted: there are so many good papers for linux.conf.au we could probably hold two conferences and they’d both be excellent.

We do, however, only have one conference – so good papers get left out.

P.S. since I just bought a house the only forms of bribes currently being accepted are large contributions towards my mortgage.

P.P.S. smaller contributions probably accepted too.

P.P.P.S. I’m not the only reviewer you need to bribe… but if you’ve got a spare few hundred thousand dollars, you probably have enough to bribe enough.