{"id":515,"date":"2005-11-29T02:15:25","date_gmt":"2005-11-28T16:15:25","guid":{"rendered":"http:\/\/www.flamingspork.com\/blog\/?p=515"},"modified":"2005-11-29T02:15:25","modified_gmt":"2005-11-28T16:15:25","slug":"disk-space-allocation-part-4-allocating-an-extent","status":"publish","type":"post","link":"https:\/\/www.flamingspork.com\/blog\/2005\/11\/29\/disk-space-allocation-part-4-allocating-an-extent\/","title":{"rendered":"disk space allocation (part 4: allocating an extent)"},"content":{"rendered":"<p>For XFS, in normal operation, an extent is only allocated when data has to be written to disk. This is called delayed allocation. If we are extending a file by 50MB &#8211; that space is deducted from the total free space on the filesystem, but no decision on where to place that data is made until we start writing it out &#8211; due to memory pressure or the kernel automatically starts writing the dirty pages out (the sync once every 5 seconds on linux).<\/p>\n<p>When an extent needs to be allocated, XFS looks it up in one of two b+trees it has of free space. There is one sorted by starting block number (so you can search for &#8220;an extent near here&#8221;) and one by size (so you can search for &#8220;an extent of x size&#8221;).<\/p>\n<p>The ideal situation being that you want as large an extent as possible as close to the tail end of the file as possible (i.e. just making the current extent bigger).<\/p>\n<p>The worst-case scenario is having to allocate extents to multiple files at once with all of them being written out synchronously (O_SYNC or memory pressure) as this will cause lots of small extents to be created.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>For XFS, in normal operation, an extent is only allocated when data has to be written to disk. This is called delayed allocation. If we are extending a file by 50MB &#8211; that space is deducted from the total free &hellip; <a href=\"https:\/\/www.flamingspork.com\/blog\/2005\/11\/29\/disk-space-allocation-part-4-allocating-an-extent\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[8,14],"tags":[],"class_list":["post-515","post","type-post","status-publish","format-standard","hentry","category-linux-kernel","category-mysql"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p5a6n8-8j","jetpack-related-posts":[{"id":511,"url":"https:\/\/www.flamingspork.com\/blog\/2005\/11\/23\/disk-space-allocation-part-1-seeing-whats-happenned\/","url_meta":{"origin":515,"position":0},"title":"disk space allocation (part 1: seeing what&#8217;s happenned)","author":"Stewart Smith","date":"2005-11-23","format":false,"excerpt":"(a little while ago I was writing a really long entry on everything possible. I realised that this would be a long read for people and that less people would look at it, so I've split it up). This sprung out of doing work on the NDB disk data tree.\u2026","rel":"","context":"In &quot;linux-kernel&quot;","block_context":{"text":"linux-kernel","link":"https:\/\/www.flamingspork.com\/blog\/category\/linux-kernel\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":514,"url":"https:\/\/www.flamingspork.com\/blog\/2005\/11\/29\/disk-space-allocation-part-3-storing-extents-on-disk\/","url_meta":{"origin":515,"position":1},"title":"disk space allocation (part 3: storing extents on disk)","author":"Stewart Smith","date":"2005-11-29","format":false,"excerpt":"Here I'm going to talk about how file systems store what part of the disk a part of the file occupies. If your database files are very fragmented, performance will suffer. How much depends on a number of things however. XFS can store some extents directly in the inode (see\u2026","rel":"","context":"In &quot;linux-kernel&quot;","block_context":{"text":"linux-kernel","link":"https:\/\/www.flamingspork.com\/blog\/category\/linux-kernel\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1201,"url":"https:\/\/www.flamingspork.com\/blog\/2008\/09\/08\/setfilevaliddata-function-windows-now-with-added-fail\/","url_meta":{"origin":515,"position":2},"title":"SetFileValidData Function (Windows) &#8211; Now with added FAIL","author":"Stewart Smith","date":"2008-09-08","format":false,"excerpt":"SetFileValidData Function (Windows) There seems to be two options on Win32 for preallocating disk space to files. Basically, I want a equivilent to posix_fallocate or the ever wonderful xfsctl XFS_IOC_RESVSP64 call. The idea being to (quickly) create a large file on disk that is stored efficiently (i.e. isn't fragmented). From\u2026","rel":"","context":"In &quot;mysql&quot;","block_context":{"text":"mysql","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/mysql\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":759,"url":"https:\/\/www.flamingspork.com\/blog\/2006\/11\/13\/disk-allocation-xfs-ndb-disk-data-and-more\/","url_meta":{"origin":515,"position":3},"title":"Disk allocation, XFS, NDB Disk Data and more&#8230;","author":"Stewart Smith","date":"2006-11-13","format":false,"excerpt":"I've talked about disk space allocation previously, mainly revolving around XFS (namely because it's what I use, a sensible choice for large file systems and large files and has a nice suite of tools for digging into what's going on).Most people write software that just calls write(2) (or libc things\u2026","rel":"","context":"In &quot;linux-kernel&quot;","block_context":{"text":"linux-kernel","link":"https:\/\/www.flamingspork.com\/blog\/category\/linux-kernel\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":512,"url":"https:\/\/www.flamingspork.com\/blog\/2005\/11\/23\/disk-space-allocation-part-2-examining-your-database-files\/","url_meta":{"origin":515,"position":4},"title":"disk space allocation (part 2: examining your database files)","author":"Stewart Smith","date":"2005-11-23","format":false,"excerpt":"memberdb\/log.MYD: EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL 0: [0..943]: 5898248..5899191 3 (36536..37479) 944 1: [944..1023]: 6071640..6071719 3 (209928..210007) 80 2: [1024..1127]: 6093664..6093767 3 (231952..232055) 104 3: [1128..1279]: 6074800..6074951 3 (213088..213239) 152 4: [1280..1407]: 6074672..6074799 3 (212960..213087) 128 5: [1408..1423]: 6074264..6074279 3 (212552..212567) 16 memberdb\/log.MYI: EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL\u2026","rel":"","context":"In &quot;linux-kernel&quot;","block_context":{"text":"linux-kernel","link":"https:\/\/www.flamingspork.com\/blog\/category\/linux-kernel\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":761,"url":"https:\/\/www.flamingspork.com\/blog\/2006\/11\/22\/create-insert-select-drop-benchmark\/","url_meta":{"origin":515,"position":5},"title":"CREATE, INSERT, SELECT, DROP benchmark","author":"Stewart Smith","date":"2006-11-22","format":false,"excerpt":"Inspired by PeterZ's Opening Tables scalability post, I decided to try a little benchmark. This benchmark involved the following: Create 50,000 tables CREATE TABLE t{$i} (i int primary key) Insert one row into each table select * from each table drop each table I wanted to test file system impact\u2026","rel":"","context":"In &quot;General&quot;","block_context":{"text":"General","link":"https:\/\/www.flamingspork.com\/blog\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/515","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/comments?post=515"}],"version-history":[{"count":0,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/515\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/media?parent=515"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/categories?post=515"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/tags?post=515"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}