{"id":1627,"date":"2009-05-09T00:55:38","date_gmt":"2009-05-08T14:55:38","guid":{"rendered":"http:\/\/www.flamingspork.com\/blog\/?p=1627"},"modified":"2009-05-09T00:55:38","modified_gmt":"2009-05-08T14:55:38","slug":"does-linux-fallocate-zero-fill","status":"publish","type":"post","link":"https:\/\/www.flamingspork.com\/blog\/2009\/05\/09\/does-linux-fallocate-zero-fill\/","title":{"rendered":"Does linux fallocate() zero-fill?"},"content":{"rendered":"<p>In an email disscussion for pre-allocating binlogs for MySQL (something we&#8217;ll likely have to do for Drizzle and replication), Yoshinori brought up the excellent point of that in some situations you don&#8217;t want to be doing zero-fill as getting up and running quickly is the most important thing.<\/p>\n<p>So what does Linux do? Does it zero-fill, or behave sensibly and pre-allocate quickly?<\/p>\n<p>Let&#8217;s look at hte kernel:<\/p>\n<p>Inside the fallocate implementation (fs\/open.c):<\/p>\n<p>if (inode-&gt;i_op-&gt;fallocate)<br \/>\nret = inode-&gt;i_op-&gt;fallocate(inode, mode, offset, len);<br \/>\nelse<br \/>\nret = -EOPNOTSUPP;<\/p>\n<p>and for ext4:<br \/>\n\/*<br \/>\n* currently supporting (pre)allocate mode for extent-based<br \/>\n* files _only_<br \/>\n*\/<br \/>\nif (!(EXT4_I(inode)-&gt;i_flags &amp; EXT4_EXTENTS_FL))<br \/>\nreturn -EOPNOTSUPP;<\/p>\n<p>XFS has always done fast pre-allocate, so it&#8217;s not a problem there and the only other filesystems to currently support fallocate are btrfs and ocfs2 &#8211; which we don&#8217;t even have to start worrying too much about yet :)<\/p>\n<p>But this is just kernel behaviour &#8211; i *think* libc ends up wrapping it<br \/>\nwith a ENOTSUPP from kernel being &#8220;let me zero-fill&#8221; (which might be<br \/>\nuseful to check). Anybody want to check libc for me?<\/p>\n<p>This was all on slightly post 2.6.30-rc3 (from git: 8c9ed899b44c19e81859fbb0e9d659fe2f8630fc)<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In an email disscussion for pre-allocating binlogs for MySQL (something we&#8217;ll likely have to do for Drizzle and replication), Yoshinori brought up the excellent point of that in some situations you don&#8217;t want to be doing zero-fill as getting up &hellip; <a href=\"https:\/\/www.flamingspork.com\/blog\/2009\/05\/09\/does-linux-fallocate-zero-fill\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[76,75,8,14],"tags":[],"class_list":["post-1627","post","type-post","status-publish","format-standard","hentry","category-code","category-drizzle-work-et-al","category-linux-kernel","category-mysql"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p5a6n8-qf","jetpack-related-posts":[{"id":785,"url":"https:\/\/www.flamingspork.com\/blog\/2007\/01\/29\/larger-inodes-make-for-some-happy-apps\/","url_meta":{"origin":1627,"position":0},"title":"Larger inodes make for (some) happy apps","author":"Stewart Smith","date":"2007-01-29","format":false,"excerpt":"Mikal talks about Ted talking about Tridge talking about how larger inodes can improve samba4 performance. Well, not just Samba4. Beagle and SELinux are also common heaver users of extended attributes which can often be stored inside the inode (e.g. on XFS). There used to be the case where the\u2026","rel":"","context":"In &quot;linux-kernel&quot;","block_context":{"text":"linux-kernel","link":"https:\/\/www.flamingspork.com\/blog\/category\/linux-kernel\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":514,"url":"https:\/\/www.flamingspork.com\/blog\/2005\/11\/29\/disk-space-allocation-part-3-storing-extents-on-disk\/","url_meta":{"origin":1627,"position":1},"title":"disk space allocation (part 3: storing extents on disk)","author":"Stewart Smith","date":"2005-11-29","format":false,"excerpt":"Here I'm going to talk about how file systems store what part of the disk a part of the file occupies. If your database files are very fragmented, performance will suffer. How much depends on a number of things however. XFS can store some extents directly in the inode (see\u2026","rel":"","context":"In &quot;linux-kernel&quot;","block_context":{"text":"linux-kernel","link":"https:\/\/www.flamingspork.com\/blog\/category\/linux-kernel\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":759,"url":"https:\/\/www.flamingspork.com\/blog\/2006\/11\/13\/disk-allocation-xfs-ndb-disk-data-and-more\/","url_meta":{"origin":1627,"position":2},"title":"Disk allocation, XFS, NDB Disk Data and more&#8230;","author":"Stewart Smith","date":"2006-11-13","format":false,"excerpt":"I've talked about disk space allocation previously, mainly revolving around XFS (namely because it's what I use, a sensible choice for large file systems and large files and has a nice suite of tools for digging into what's going on).Most people write software that just calls write(2) (or libc things\u2026","rel":"","context":"In &quot;linux-kernel&quot;","block_context":{"text":"linux-kernel","link":"https:\/\/www.flamingspork.com\/blog\/category\/linux-kernel\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":861,"url":"https:\/\/www.flamingspork.com\/blog\/2007\/07\/12\/reading-maildirs-fast\/","url_meta":{"origin":1627,"position":3},"title":"reading maildirs&#8230;. fast&#8230;","author":"Stewart Smith","date":"2007-07-12","format":false,"excerpt":"So, for a side project i'm hacking on, i'm wanting to read in Maildirs really fast (and then pump them into something else... for current purposes I'm just putting everything in one file.. getting the read speed up is of current importance). I've done a bit of experimenting and my\u2026","rel":"","context":"In &quot;General&quot;","block_context":{"text":"General","link":"https:\/\/www.flamingspork.com\/blog\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":715,"url":"https:\/\/www.flamingspork.com\/blog\/2006\/06\/14\/arjens-mysql-community-journal-hyperthreading-not-on-a-mysql-server\/","url_meta":{"origin":1627,"position":4},"title":"Arjen&#8217;s MySQL Community Journal &#8211; HyperThreading? Not on a MySQL server&#8230;","author":"Stewart Smith","date":"2006-06-14","format":false,"excerpt":"Arjen's MySQL Community Journal - HyperThreading? Not on a MySQL server... I blame the Linux Process Scheduler. At least it's better than the earlier 2.6 days where things would get shunted a lot from one \"cpu\" to the other \"cpu\" for no real reason. Newer kernel verisons are probably better...\u2026","rel":"","context":"In &quot;linux-kernel&quot;","block_context":{"text":"linux-kernel","link":"https:\/\/www.flamingspork.com\/blog\/category\/linux-kernel\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":391,"url":"https:\/\/www.flamingspork.com\/blog\/2005\/04\/14\/binstat\/","url_meta":{"origin":1627,"position":5},"title":"~\/bin\/stat","author":"Stewart Smith","date":"2005-04-14","format":false,"excerpt":"What have you always wanted on the command line? stat? i: hear; you cry! (insert more [silly] {punctuation} around 'ere!) I present, to you, my ~\/bin\/stat #!\/usr\/bin\/perl use POSIX qw(strftime); my $FORMAT=\"%H:%M %a, %e %b %y\"; foreach(@ARGV) { my ($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size, $atime,$mtime,$ctime,$blksize,$blocks) = stat($_); print $_.\"\\n\"; print \"Dev: $dev\\n\"; print \"inode:\u2026","rel":"","context":"In &quot;General&quot;","block_context":{"text":"General","link":"https:\/\/www.flamingspork.com\/blog\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/1627","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/comments?post=1627"}],"version-history":[{"count":2,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/1627\/revisions"}],"predecessor-version":[{"id":1629,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/1627\/revisions\/1629"}],"wp:attachment":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/media?parent=1627"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/categories?post=1627"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/tags?post=1627"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}