{"id":4361,"date":"2018-04-18T16:32:13","date_gmt":"2018-04-18T06:32:13","guid":{"rendered":"https:\/\/www.flamingspork.com\/blog\/?p=4361"},"modified":"2018-04-18T16:32:13","modified_gmt":"2018-04-18T06:32:13","slug":"ccache-and-op-build","status":"publish","type":"post","link":"https:\/\/www.flamingspork.com\/blog\/2018\/04\/18\/ccache-and-op-build\/","title":{"rendered":"ccache and op-build"},"content":{"rendered":"

You may have heard of ccache (Compiler Cache) which saves you heaps<\/strong> of real world time when rebuilding a source tree that is rather similar to one you’ve recently built before. It’s really<\/strong> useful in buildroot based projects where you’re building similar trees, or have done a minor bump of some components.<\/p>\n

In trying to find a commit which introduced a bug in op-build (OpenPOWER firmware), I noticed that hostboot wasn’t being built using ccache and we were always<\/strong> doing a full build. So, I started digging into it.<\/p>\n

It turns out that a bunch of the perl scripts for parsing the Machine Readable Workbook XML in hostboot did a bunch of things like foreach $key (%hash)<\/code> – which means that the code iterates over the items in hash<\/strong> order rather than an order that would produce predictable output such as “attribute name” or something. So… much messing with that later, I had hostboot generating the same output for the same input on every build.<\/p>\n

Next step was to work out why I was still getting a lot of CCACHE misses. It turns out the default ccache size is 5GB. A full hostboot build uses around 7.1GB of that.<\/p>\n

So, if building op-build with CCACHE, be sure to set both BR2_CCACHE=y<\/code> in your config as well as something like BR2_CCACHE_INITIAL_SETUP=\"--max-size 20G\"<\/code><\/p>\n

Hopefully my patches hit hostboot and op-build soon.<\/p>\n","protected":false},"excerpt":{"rendered":"

You may have heard of ccache (Compiler Cache) which saves you heaps of real world time when rebuilding a source tree that is rather similar to one you’ve recently built before. It’s really useful in buildroot based projects where you’re … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[1],"tags":[732,731,587],"class_list":["post-4361","post","type-post","status-publish","format-standard","hentry","category-general","tag-ccache","tag-op-build","tag-openpower"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p5a6n8-18l","jetpack-related-posts":[{"id":323,"url":"https:\/\/www.flamingspork.com\/blog\/2004\/12\/21\/distcc-make-build-go-quicker\/","url_meta":{"origin":4361,"position":0},"title":"distcc make build go quicker","author":"Stewart Smith","date":"2004-12-21","format":false,"excerpt":"ahh... ccache, distcc and cross compilers make builds go quicker. two 350mhz PIIs, a 2.8Ghz P4 and a 1Ghz G4 are better than just the G4. although it's annoying having to wait on the autotools stuff for each clone of a tree.","rel":"","context":"In "General"","block_context":{"text":"General","link":"https:\/\/www.flamingspork.com\/blog\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3775,"url":"https:\/\/www.flamingspork.com\/blog\/2014\/07\/17\/openpower-firmware-up-on-github\/","url_meta":{"origin":4361,"position":1},"title":"OpenPower firmware up on github!","author":"Stewart Smith","date":"2014-07-17","format":false,"excerpt":"With the whole OpenPower thing, a lot of low level firmware is being open sourced, which is really exciting for the platform - the less proprietary code sitting in memory the better in my books. If you go to https:\/\/github.com\/open-power you'll see code for a bunch of the low level\u2026","rel":"","context":"In "code"","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":4599,"url":"https:\/\/www.flamingspork.com\/blog\/2020\/05\/13\/a-op-build-v2-5-rc1-based-raptor-blackbird-build\/","url_meta":{"origin":4361,"position":2},"title":"A op-build v2.5-rc1 based Raptor Blackbird Build","author":"Stewart Smith","date":"2020-05-13","format":false,"excerpt":"I have done a few builds of firmware for the Raptor Blackbird since I got mine, each of them based on upstream op-build plus a few patches. The previous one was Yet another near-upstream Raptor Blackbird firmware build that I built a couple of months ago. This new build is\u2026","rel":"","context":"In "cool gadgets"","block_context":{"text":"cool gadgets","link":"https:\/\/www.flamingspork.com\/blog\/category\/cool-gadgets\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":463,"url":"https:\/\/www.flamingspork.com\/blog\/2005\/09\/04\/memberdb-election-results-performance-on-new-laptop\/","url_meta":{"origin":4361,"position":3},"title":"MemberDB election-results performance on new laptop","author":"Stewart Smith","date":"2005-09-04","format":false,"excerpt":"So I picked up my new laptop on friday. It's an ASUS V6V - nice and fast, light, good resolution screen and lots of disk and RAM (it came with 1GB, I've got 2GB). Anyway, the transfer of data from my PowerBook went fine. I waited for xfsdump to dump\u2026","rel":"","context":"In "cool gadgets"","block_context":{"text":"cool gadgets","link":"https:\/\/www.flamingspork.com\/blog\/category\/cool-gadgets\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":4345,"url":"https:\/\/www.flamingspork.com\/blog\/2017\/12\/11\/a-simplified-view-of-openpower-firmware-development\/","url_meta":{"origin":4361,"position":4},"title":"A (simplified) view of OpenPOWER Firmware Development","author":"Stewart Smith","date":"2017-12-11","format":false,"excerpt":"I've been working on trying to better document the whole flow of code that goes into a build of firmware for an OpenPOWER machine. This is partially to help those not familiar with it get a better grasp of the sheer scale of what goes into that 32\/64MB of flash.\u2026","rel":"","context":"In "code"","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2017\/12\/devflow-legend.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2017\/12\/devflow-legend.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2017\/12\/devflow-legend.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2017\/12\/devflow-legend.png?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":4200,"url":"https:\/\/www.flamingspork.com\/blog\/2016\/10\/20\/workaround-for-opal-prd-using-100-cpu\/","url_meta":{"origin":4361,"position":5},"title":"Workaround for opal-prd using 100% CPU","author":"Stewart Smith","date":"2016-10-20","format":false,"excerpt":"opal-prd is the Processor RunTime Diagnostics daemon, the userspace process that on OpenPower systems is responsible for some of the runtime diagnostics. Although a userspace process, it memory maps (as in mmap) in some code loaded by early firmware (Hostboot) called the HostBoot RunTime (HBRT) and runs it, using calls\u2026","rel":"","context":"In "IBM"","block_context":{"text":"IBM","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/ibm-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/4361","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/comments?post=4361"}],"version-history":[{"count":1,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/4361\/revisions"}],"predecessor-version":[{"id":4362,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/4361\/revisions\/4362"}],"wp:attachment":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/media?parent=4361"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/categories?post=4361"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/tags?post=4361"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}