{"id":83,"date":"2003-03-31T21:21:46","date_gmt":"2003-04-01T02:21:46","guid":{"rendered":"http:\/\/www.flamingspork.com\/blog\/?p=83"},"modified":"2003-03-31T21:21:46","modified_gmt":"2003-04-01T02:21:46","slug":"block-allocation","status":"publish","type":"post","link":"https:\/\/www.flamingspork.com\/blog\/2003\/03\/31\/block-allocation\/","title":{"rendered":"block allocation"},"content":{"rendered":"<p>B+Trees sorted by size and location (a-la XFS) provides:<br \/>\n&#8211; ability to allocate large\/small objects efficiently (size)<br \/>\n&#8211; ability to allocate blocks near existing objects (e.g. for object expansion) by using the location B+Tree<\/p>\n<p>B+Trees are good, therefor use them.<\/p>\n<p>Split up into allocation groups (a-la XFS and BFS). Allows more parallel operation as different threads can work on different allocation groups (during updating).<\/p>\n<p>Idea for fine grained B+Tree locking:<br \/>\n&#8211; each node gets an ID (which should be pretty unique&#8230; but not essential to correct operation).<br \/>\n&#8211; when process needs to update node, places a lock on it&#8217;s ID<br \/>\n&#8211; when a process accesses a node, before reading, it checks to see if a lock has been placed on that ID. IF so, we spin, waiting for the lock to be released<br \/>\n&#8211; when a process has finished updating a node, it releases the lock on it&#8217;s ID.<br \/>\n&#8211; the process waiting to read it will stop spinning and be able to read the new copy.<\/p>\n<p>IF IDs overlap, there is no problem, we&#8217;ll just be spinning when a different node is being updated.<\/p>\n<p>However, if we are updating a tree of nodes, then some careful locking wil have to be employed, from the BOTTOM UP! that is, update from the bottom up, locking each individual one as we go as if we only lock the root of the tree being updated, another process could have already passed it and screw us up royally.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>B+Trees sorted by size and location (a-la XFS) provides: &#8211; ability to allocate large\/small objects efficiently (size) &#8211; ability to allocate blocks near existing objects (e.g. for object expansion) by using the location B+Tree B+Trees are good, therefor use them. &hellip; <a href=\"https:\/\/www.flamingspork.com\/blog\/2003\/03\/31\/block-allocation\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[4],"tags":[],"class_list":["post-83","post","type-post","status-publish","format-standard","hentry","category-hons-project"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p5a6n8-1l","jetpack-related-posts":[{"id":759,"url":"https:\/\/www.flamingspork.com\/blog\/2006\/11\/13\/disk-allocation-xfs-ndb-disk-data-and-more\/","url_meta":{"origin":83,"position":0},"title":"Disk allocation, XFS, NDB Disk Data and more&#8230;","author":"Stewart Smith","date":"2006-11-13","format":false,"excerpt":"I've talked about disk space allocation previously, mainly revolving around XFS (namely because it's what I use, a sensible choice for large file systems and large files and has a nice suite of tools for digging into what's going on).Most people write software that just calls write(2) (or libc things\u2026","rel":"","context":"In &quot;linux-kernel&quot;","block_context":{"text":"linux-kernel","link":"https:\/\/www.flamingspork.com\/blog\/category\/linux-kernel\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":515,"url":"https:\/\/www.flamingspork.com\/blog\/2005\/11\/29\/disk-space-allocation-part-4-allocating-an-extent\/","url_meta":{"origin":83,"position":1},"title":"disk space allocation (part 4: allocating an extent)","author":"Stewart Smith","date":"2005-11-29","format":false,"excerpt":"For XFS, in normal operation, an extent is only allocated when data has to be written to disk. This is called delayed allocation. If we are extending a file by 50MB - that space is deducted from the total free space on the filesystem, but no decision on where to\u2026","rel":"","context":"In &quot;linux-kernel&quot;","block_context":{"text":"linux-kernel","link":"https:\/\/www.flamingspork.com\/blog\/category\/linux-kernel\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":925,"url":"https:\/\/www.flamingspork.com\/blog\/2007\/11\/14\/better-disk-allocation-with-mythtv-and-xfs\/","url_meta":{"origin":83,"position":2},"title":"Better disk allocation with MythTV and XFS","author":"Stewart Smith","date":"2007-11-14","format":false,"excerpt":"Running MythTV on XFS? Noticed that all your recordings end up rather fragmented? (use xfs_bmap to find out) Well, the culprit is MythTV not being too nice to the file system. Good news is, it's rather fixable. From the MythTV source code, edit libs\/libmythtv\/ThreadedFileWrite.cpp and look for the following: void\u2026","rel":"","context":"In &quot;mythtv&quot;","block_context":{"text":"mythtv","link":"https:\/\/www.flamingspork.com\/blog\/category\/mythtv\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1201,"url":"https:\/\/www.flamingspork.com\/blog\/2008\/09\/08\/setfilevaliddata-function-windows-now-with-added-fail\/","url_meta":{"origin":83,"position":3},"title":"SetFileValidData Function (Windows) &#8211; Now with added FAIL","author":"Stewart Smith","date":"2008-09-08","format":false,"excerpt":"SetFileValidData Function (Windows) There seems to be two options on Win32 for preallocating disk space to files. Basically, I want a equivilent to posix_fallocate or the ever wonderful xfsctl XFS_IOC_RESVSP64 call. The idea being to (quickly) create a large file on disk that is stored efficiently (i.e. isn't fragmented). From\u2026","rel":"","context":"In &quot;mysql&quot;","block_context":{"text":"mysql","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/mysql\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":511,"url":"https:\/\/www.flamingspork.com\/blog\/2005\/11\/23\/disk-space-allocation-part-1-seeing-whats-happenned\/","url_meta":{"origin":83,"position":4},"title":"disk space allocation (part 1: seeing what&#8217;s happenned)","author":"Stewart Smith","date":"2005-11-23","format":false,"excerpt":"(a little while ago I was writing a really long entry on everything possible. I realised that this would be a long read for people and that less people would look at it, so I've split it up). This sprung out of doing work on the NDB disk data tree.\u2026","rel":"","context":"In &quot;linux-kernel&quot;","block_context":{"text":"linux-kernel","link":"https:\/\/www.flamingspork.com\/blog\/category\/linux-kernel\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":89,"url":"https:\/\/www.flamingspork.com\/blog\/2003\/04\/23\/xfs-and-other-cool-things\/","url_meta":{"origin":83,"position":5},"title":"XFS and other cool things","author":"Stewart Smith","date":"2003-04-23","format":false,"excerpt":"Been re-reading a lot of the XFS papers that are on the SGI website (http:\/\/oss.sgi.com\/projects\/xfs\/) and thinking more about what I want out of an object store. There are a lot of similar design goals (I think) yet some very different ways of implementing things. Having a large B+Tree full\u2026","rel":"","context":"In &quot;hons-project&quot;","block_context":{"text":"hons-project","link":"https:\/\/www.flamingspork.com\/blog\/category\/hons-project\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/83","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/comments?post=83"}],"version-history":[{"count":1,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/83\/revisions"}],"predecessor-version":[{"id":2620,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/83\/revisions\/2620"}],"wp:attachment":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/media?parent=83"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/categories?post=83"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/tags?post=83"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}