{"id":84,"date":"2003-03-31T21:36:22","date_gmt":"2003-04-01T02:36:22","guid":{"rendered":"http:\/\/www.flamingspork.com\/blog\/?p=84"},"modified":"2003-03-31T21:36:22","modified_gmt":"2003-04-01T02:36:22","slug":"variety-in-allocation-block-sizes","status":"publish","type":"post","link":"https:\/\/www.flamingspork.com\/blog\/2003\/03\/31\/variety-in-allocation-block-sizes\/","title":{"rendered":"variety in allocation block sizes?"},"content":{"rendered":"<p>some studies have shown that for multimedia applications, a larger block size improves throughput (e.g. 256kb blocks). For large media files, the waste of an average 128kb per file is insignificant (over several megabytes to many hundred mb or indeed GB). But, for smaller files (typically occupied by configuration files or small system binaries) a smaller block size saves more space (4k block for 100byte file).<\/p>\n<p>ReiserFS goes for something slightly different allowing several files to share a single block. I am unsure if the extra effort involved in implementing this is worth it, or the slower access times that reiser reports for allowing this.<\/p>\n<p>Maybe different allocation groups could have different block sizes? maybe there could be some kind of block-size migration system? or would the overhead not be worth it? Could it be one of those &#8220;maintenance&#8221; tasks that you run every month\/year? How often does the average usage of a disk change that we&#8217;d need something like this?<\/p>\n","protected":false},"excerpt":{"rendered":"<p>some studies have shown that for multimedia applications, a larger block size improves throughput (e.g. 256kb blocks). For large media files, the waste of an average 128kb per file is insignificant (over several megabytes to many hundred mb or indeed &hellip; <a href=\"https:\/\/www.flamingspork.com\/blog\/2003\/03\/31\/variety-in-allocation-block-sizes\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[4],"tags":[],"class_list":["post-84","post","type-post","status-publish","format-standard","hentry","category-hons-project"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p5a6n8-1m","jetpack-related-posts":[{"id":512,"url":"https:\/\/www.flamingspork.com\/blog\/2005\/11\/23\/disk-space-allocation-part-2-examining-your-database-files\/","url_meta":{"origin":84,"position":0},"title":"disk space allocation (part 2: examining your database files)","author":"Stewart Smith","date":"2005-11-23","format":false,"excerpt":"memberdb\/log.MYD: EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL 0: [0..943]: 5898248..5899191 3 (36536..37479) 944 1: [944..1023]: 6071640..6071719 3 (209928..210007) 80 2: [1024..1127]: 6093664..6093767 3 (231952..232055) 104 3: [1128..1279]: 6074800..6074951 3 (213088..213239) 152 4: [1280..1407]: 6074672..6074799 3 (212960..213087) 128 5: [1408..1423]: 6074264..6074279 3 (212552..212567) 16 memberdb\/log.MYI: EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL\u2026","rel":"","context":"In &quot;linux-kernel&quot;","block_context":{"text":"linux-kernel","link":"https:\/\/www.flamingspork.com\/blog\/category\/linux-kernel\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":514,"url":"https:\/\/www.flamingspork.com\/blog\/2005\/11\/29\/disk-space-allocation-part-3-storing-extents-on-disk\/","url_meta":{"origin":84,"position":1},"title":"disk space allocation (part 3: storing extents on disk)","author":"Stewart Smith","date":"2005-11-29","format":false,"excerpt":"Here I'm going to talk about how file systems store what part of the disk a part of the file occupies. If your database files are very fragmented, performance will suffer. How much depends on a number of things however. XFS can store some extents directly in the inode (see\u2026","rel":"","context":"In &quot;linux-kernel&quot;","block_context":{"text":"linux-kernel","link":"https:\/\/www.flamingspork.com\/blog\/category\/linux-kernel\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":511,"url":"https:\/\/www.flamingspork.com\/blog\/2005\/11\/23\/disk-space-allocation-part-1-seeing-whats-happenned\/","url_meta":{"origin":84,"position":2},"title":"disk space allocation (part 1: seeing what&#8217;s happenned)","author":"Stewart Smith","date":"2005-11-23","format":false,"excerpt":"(a little while ago I was writing a really long entry on everything possible. I realised that this would be a long read for people and that less people would look at it, so I've split it up). This sprung out of doing work on the NDB disk data tree.\u2026","rel":"","context":"In &quot;linux-kernel&quot;","block_context":{"text":"linux-kernel","link":"https:\/\/www.flamingspork.com\/blog\/category\/linux-kernel\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":515,"url":"https:\/\/www.flamingspork.com\/blog\/2005\/11\/29\/disk-space-allocation-part-4-allocating-an-extent\/","url_meta":{"origin":84,"position":3},"title":"disk space allocation (part 4: allocating an extent)","author":"Stewart Smith","date":"2005-11-29","format":false,"excerpt":"For XFS, in normal operation, an extent is only allocated when data has to be written to disk. This is called delayed allocation. If we are extending a file by 50MB - that space is deducted from the total free space on the filesystem, but no decision on where to\u2026","rel":"","context":"In &quot;linux-kernel&quot;","block_context":{"text":"linux-kernel","link":"https:\/\/www.flamingspork.com\/blog\/category\/linux-kernel\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":228,"url":"https:\/\/www.flamingspork.com\/blog\/2004\/04\/10\/tightly-packing-onodes\/","url_meta":{"origin":84,"position":4},"title":"Tightly packing onodes","author":"Stewart Smith","date":"2004-04-10","format":false,"excerpt":"The current problem is that an onode, however much we can pack forks into a block, still takes up a minimum of one disk block. A disk block typically being 4kb, and a tendancy to want to be bigger (think large media files), and also being the unit of atomicity\u2026","rel":"","context":"In &quot;fcfs&quot;","block_context":{"text":"fcfs","link":"https:\/\/www.flamingspork.com\/blog\/category\/fcfs\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":83,"url":"https:\/\/www.flamingspork.com\/blog\/2003\/03\/31\/block-allocation\/","url_meta":{"origin":84,"position":5},"title":"block allocation","author":"Stewart Smith","date":"2003-03-31","format":false,"excerpt":"B+Trees sorted by size and location (a-la XFS) provides: - ability to allocate large\/small objects efficiently (size) - ability to allocate blocks near existing objects (e.g. for object expansion) by using the location B+Tree B+Trees are good, therefor use them. Split up into allocation groups (a-la XFS and BFS). Allows\u2026","rel":"","context":"In &quot;hons-project&quot;","block_context":{"text":"hons-project","link":"https:\/\/www.flamingspork.com\/blog\/category\/hons-project\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/84","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/comments?post=84"}],"version-history":[{"count":1,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/84\/revisions"}],"predecessor-version":[{"id":2621,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/84\/revisions\/2621"}],"wp:attachment":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/media?parent=84"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/categories?post=84"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/tags?post=84"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}