{"id":1229,"date":"2008-10-14T14:51:25","date_gmt":"2008-10-14T04:51:25","guid":{"rendered":"http:\/\/www.flamingspork.com\/blog\/?p=1229"},"modified":"2008-10-14T14:51:25","modified_gmt":"2008-10-14T04:51:25","slug":"mysql-cluster-ndb-on-win32-progress","status":"publish","type":"post","link":"https:\/\/www.flamingspork.com\/blog\/2008\/10\/14\/mysql-cluster-ndb-on-win32-progress\/","title":{"rendered":"MySQL Cluster (NDB) on Win32 progress"},"content":{"rendered":"<p>Many things have been happenning in the land of NDB on Win32 as of late.<\/p>\n<p>I&#8217;ve fixed about 700 compiler warnings (some of which were real bugs) leaving about 161 to go on Win32 (VS2003). We&#8217;re getting a few more warnings on Win64 (some of which look merely semantic, while others could be real bugs), but the main focus now is getting 32bit going really well.<\/p>\n<p>I fixed a number of bugs that were around preventing lots of things from working properly:<\/p>\n<p>Disk Data (i.e. CREATE TABLESPACE, CREATE LOGFILE GROUP, and CREATE TABLE&#8230; TABLESPACE ts1 STORAGE DISK) now works. The main problem here was that our filesystem abstraction layer for the NDB kernel (ndbd) once had a Win32 port&#8230; which has sorely bitrotted over the years. As new features were introduced to the file IO interface, they (of course) weren&#8217;t also added to the Win32 abstraction. In the disk data case, the OM_INIT feature, which on FSOPENREQ (open a file) allows data to be passed in for initialising the file. Previously, I fixed this to allocate the file on disk and create a file of the same size, but i didn&#8217;t add the feature that writes initial data to the file. This caused bugs as soon as you tried to use the disk data tables (the files weren&#8217;t initialised, so you hit asserts on corrupt disk data files).<\/p>\n<p>Paths in the server: for whatever bizarre and stupid reason, the MySQL server can end up having paths to a table as .\/database\/table OR .\\database\\table. The latter *never* shows up on non-Win32 platforms but can *sometimes* show up on Win32. Ick ick ick ick. Anyway, we (in the NDB handler) weren&#8217;t dealing with this properly, causing problems around some metadata ops.<\/p>\n<p>Our pushbuild system takes each push to a source tree, builds it on a variety of platfroms and runs the mysql-test-run.pl test suite. The Win32 hosts are actually running on vmware. In order to make tests run faster, on Linux we use \/dev\/shm for the data files. Microsoft Windows doesn&#8217;t have a good ram disk, so we create a file on \/dev\/shm on the host and map that as a drive inside Windows (and format it as NTFS). This drive is only 1GB. This is not enough disk space for running all the clusters (yes, plural) started by the test suite (and everything would die with ENOSPC). The workaround I&#8217;ve come up with is that for debug builds, we simply enable NTFS file compression on files ndbd creates.<\/p>\n<p>Win64 is also working! Pushbuild builds and runs on 64bit, and the Win64 host is building with NDB and passing about the same amount of tests as the Win32 hosts!<\/p>\n<p>The bad news is that the NDB with replication tests are pretty much all failing&#8230; so I&#8217;m fairly confident that cluster replication is very broken on Win32 (and 64) at the moment.<\/p>\n<p>I&#8217;ve had to do a fair amount of fixing on a bunch of the test cases (mainly to do with finding where various NDB utilities are). They&#8217;ve also prompted fixes in NDB (automatically converting \/ to \\ in ndbd on Win32 for CREATE DATAFILE\/UNDOFILE).<\/p>\n<p>If you want to give it a go &#8211; you can get the source from launchpad. Either in the mysql-5.1-telco-6.4 tree, or if you want a few more things fixed, always have a look at the mysql-5.1-telco-6.4-win tree. Hopefully both are synced with the latest internal trees (i.e. plain 6.4 is working on win32) by the time you read this.<\/p>\n<p>Iggy and I discussed installers for NDB on Windows in Riga, and we should have something soon-ish for those of you who don&#8217;t build from source.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Many things have been happenning in the land of NDB on Win32 as of late. I&#8217;ve fixed about 700 compiler warnings (some of which were real bugs) leaving about 161 to go on Win32 (VS2003). We&#8217;re getting a few more &hellip; <a href=\"https:\/\/www.flamingspork.com\/blog\/2008\/10\/14\/mysql-cluster-ndb-on-win32-progress\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[1],"tags":[628,54,67,66,68,69,36,64,65],"class_list":["post-1229","post","type-post","status-publish","format-standard","hentry","category-general","tag-mysql","tag-ndb","tag-pushbuild","tag-replication","tag-vs2003","tag-vs2005","tag-win32","tag-win64","tag-windows"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p5a6n8-jP","jetpack-related-posts":[{"id":1215,"url":"https:\/\/www.flamingspork.com\/blog\/2008\/09\/18\/ndb-windows-port-shaping-up\/","url_meta":{"origin":1229,"position":0},"title":"NDB Windows port shaping up&#8230;","author":"Stewart Smith","date":"2008-09-18","format":false,"excerpt":"It's getting there. The tree should now pretty much always compile, and (at least mostly) doesn't break anything on other platforms. It even works on win32... at least basic functionality. There will be monsters (bugs.. but scarier, becuase it's win32).","rel":"","context":"In &quot;mysql&quot;","block_context":{"text":"mysql","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/mysql\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":894,"url":"https:\/\/www.flamingspork.com\/blog\/2007\/09\/20\/ndb_mgmd-on-win32-an-alpha\/","url_meta":{"origin":1229,"position":1},"title":"ndb_mgmd on Win32 (an Alpha)","author":"Stewart Smith","date":"2007-09-20","format":false,"excerpt":"So, here is an Alpha quality port of the MySQL Cluster management server to Win32 based on the current MySQL 5.0 tree. This isn't going into 5.0, so don't expect to ever have that. This isn't going into 5.1 either, so don't expect it there. It'll go into some future\u2026","rel":"","context":"In &quot;mysql&quot;","block_context":{"text":"mysql","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/mysql\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1217,"url":"https:\/\/www.flamingspork.com\/blog\/2008\/09\/19\/visual-studio-2008-unreferenced-local-variable-bug\/","url_meta":{"origin":1229,"position":2},"title":"Visual Studio 2008 unreferenced local variable bug","author":"Stewart Smith","date":"2008-09-19","format":false,"excerpt":"screenshot 'cause typing is for wusses UPDATE: not actually VS bug. Nasty macro defining strtok_r to strtok on Win32. ouch.","rel":"","context":"In &quot;mysql&quot;","block_context":{"text":"mysql","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/mysql\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2008\/09\/vs_bug.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":891,"url":"https:\/\/www.flamingspork.com\/blog\/2007\/09\/19\/mysql-cluster-ndb-on-microsoft-windows\/","url_meta":{"origin":1229,"position":3},"title":"MySQL Cluster (NDB) on Microsoft Windows","author":"Stewart Smith","date":"2007-09-19","format":false,"excerpt":"Well... there's been some work. Even some in-progress patches. Being involved with this has just perfectly refreshed my memory of why I left the platform. Oh my it's a horrible, horrible platform. Everything from UI to API... ick. Expect something around soon....","rel":"","context":"In &quot;General&quot;","block_context":{"text":"General","link":"https:\/\/www.flamingspork.com\/blog\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":578,"url":"https:\/\/www.flamingspork.com\/blog\/2006\/02\/15\/information_schemafiles-querying-disk-usage-from-sql\/","url_meta":{"origin":1229,"position":4},"title":"INFORMATION_SCHEMA.FILES (querying disk usage from SQL)","author":"Stewart Smith","date":"2006-02-15","format":false,"excerpt":"In MySQL 5.1.6 there's a new INFORMATION_SCHEMA table. Currently, it only has information on files for NDB but we're hoping to change that in a future release (read: I think it would be neat). This table is a table generated by the MySQL server listing all the different files that\u2026","rel":"","context":"In &quot;mysql&quot;","block_context":{"text":"mysql","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/mysql\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":478,"url":"https:\/\/www.flamingspork.com\/blog\/2005\/10\/03\/a-funky-thing-done-last-week\/","url_meta":{"origin":1229,"position":5},"title":"a funky thing done last week&#8230;","author":"Stewart Smith","date":"2005-10-03","format":false,"excerpt":"still have to talk to people about standards for this sort of thing and all that. But as a first checkin - funkyness++! mysql> select * from INFORMATION_SCHEMA.DATAFILES; select * from INFORMATION_SCHEMA.TABLESPACES; Empty set (0.03 sec) Empty set (0.00 sec) mysql> CREATE TABLESPACE ts1 ADD DATAFILE 'datafile.dat' USE LOGFILE GROUP\u2026","rel":"","context":"In &quot;mysql&quot;","block_context":{"text":"mysql","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/mysql\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/1229","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/comments?post=1229"}],"version-history":[{"count":1,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/1229\/revisions"}],"predecessor-version":[{"id":1234,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/1229\/revisions\/1234"}],"wp:attachment":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/media?parent=1229"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/categories?post=1229"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/tags?post=1229"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}