{"id":3965,"date":"2015-06-12T17:01:19","date_gmt":"2015-06-12T07:01:19","guid":{"rendered":"https:\/\/www.flamingspork.com\/blog\/?p=3965"},"modified":"2015-06-12T17:37:53","modified_gmt":"2015-06-12T07:37:53","slug":"gcov-code-coverage-for-openpower-firmware","status":"publish","type":"post","link":"https:\/\/www.flamingspork.com\/blog\/2015\/06\/12\/gcov-code-coverage-for-openpower-firmware\/","title":{"rendered":"gcov code coverage for OpenPower firmware"},"content":{"rendered":"

For skiboot<\/a> (which provides the OPAL boot and runtime firmware for OpenPower machines), I’ve been pretty interested at getting some automated code coverage data for booting on real hardware (as well as in a simulator). Why? Well, it’s useful to see that various test suites are actually testing what you think they are, and it helps you be able to define more tests to increase what you’re covering.<\/p>\n

The typical way to do code coverage is to make GCC build your program with GCOV, which is pretty simple if you’re a userspace program. You build with gcov, run program, and at the end you’re left with files on disk that contain all the coverage information for a tool such as lcov to consume. For the Linux kernel, you can also do this, and then extract the GCOV data out of debugfs and get code coverage for all\/part of your kernel. It’s a little bit more involved for the kernel, but not too<\/strong> much so.<\/p>\n

To achieve this, the kernel has to implement a bunch of stub functions itself rather than link to the gcov library as well as parse the GCOV data structures that GCC generates and emit the gcda files in debugfs when read. Basically, you replace the part of the GCC generated code that writes the files out. This works really nicely as Linux has fancy things like a VFS and debugfs.<\/p>\n

For skiboot, we have no such things. We are firmware, we don’t have a damn file system interface. So, what do we do? Write a userspace utility to parse a dump of the appropriate region of memory, easy! That’s exactly what I did, a (relatively) simple user space app to parse out the gcov gcda files from a skiboot memory image – something we can easily dump out of the simulator, relatively easily (albeit slower) from the FSP on an IBM POWER system and even just directly out of a running system (if you boot a linux kernel with the appropriate config).<\/p>\n

So, we can now get a (mostly automated) code coverage report simply for the act of booting to petitboot: https:\/\/open-power.github.io\/skiboot\/boot-coverage-report\/<\/a> along with our old coverage report which was just for the unit tests (https:\/\/open-power.github.io\/skiboot\/coverage-report\/<\/a>). My current boot-coverage-report is just on POWER7 and POWER8 IBM FSP based systems – but you can see that a decent amount of code both is (and isn’t) touched simply from the act of booting to the bootloader.<\/p>\n

The numbers we get are only approximate for any code run on more than one CPU as GCC just generates code that does a load\/add\/store rather than using an atomic increment.<\/p>\n

One interesting observation was that (at least on smaller systems, which are still quite large by many people’s standards), boot time was not really noticeably increased.<\/p>\n

For more information on running with gcov, see the in-tree documentation: https:\/\/github.com\/open-power\/skiboot\/blob\/master\/doc\/gcov.txt<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

For skiboot (which provides the OPAL boot and runtime firmware for OpenPower machines), I’ve been pretty interested at getting some automated code coverage data for booting on real hardware (as well as in a simulator). Why? Well, it’s useful to … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[76,570,8,588,615],"tags":[86,622,636,587,584],"class_list":["post-3965","post","type-post","status-publish","format-standard","hentry","category-code","category-ibm-work-et-al","category-linux-kernel","category-opal","category-powerpc","tag-gcc","tag-gcov","tag-opal","tag-openpower","tag-skiboot"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p5a6n8-11X","jetpack-related-posts":[{"id":3775,"url":"https:\/\/www.flamingspork.com\/blog\/2014\/07\/17\/openpower-firmware-up-on-github\/","url_meta":{"origin":3965,"position":0},"title":"OpenPower firmware up on github!","author":"Stewart Smith","date":"2014-07-17","format":false,"excerpt":"With the whole OpenPower thing, a lot of low level firmware is being open sourced, which is really exciting for the platform - the less proprietary code sitting in memory the better in my books. If you go to https:\/\/github.com\/open-power you'll see code for a bunch of the low level\u2026","rel":"","context":"In "code"","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3912,"url":"https:\/\/www.flamingspork.com\/blog\/2014\/12\/03\/running-skiboot-opal-on-the-power8-simulator\/","url_meta":{"origin":3965,"position":1},"title":"Running skiboot (OPAL) on the POWER8 Simulator","author":"Stewart Smith","date":"2014-12-03","format":false,"excerpt":"skiboot is open source boot and runtime firmware for OpenPOWER. On real POWER8 hardware, you will also need HostBoot to do this (basically, to make the chip work) but in a functional simulator (such as this one released by IBM) you don't need a bunch of hardware procedures to make\u2026","rel":"","context":"In "code"","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":4076,"url":"https:\/\/www.flamingspork.com\/blog\/2016\/07\/12\/using-smatch-static-analysis-on-openpower-opal-firmware\/","url_meta":{"origin":3965,"position":2},"title":"Using Smatch static analysis on OpenPOWER OPAL firmware","author":"Stewart Smith","date":"2016-07-12","format":false,"excerpt":"For Skiboot, I'm always looking at new automated systems to find bugs in the code. A little while ago, I read about the Smatch tool developed by some folks at Oracle (they also wrote about using it on the Linux kernel). I was eager to try it with skiboot to\u2026","rel":"","context":"In "code"","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3972,"url":"https:\/\/www.flamingspork.com\/blog\/2015\/06\/16\/opal-firmware-specification-conformance-and-documentation\/","url_meta":{"origin":3965,"position":3},"title":"OPAL firmware specification, conformance and documentation","author":"Stewart Smith","date":"2015-06-16","format":false,"excerpt":"Now that we have an increasing amount of things that run on top of OPAL: Linux hello_world (in skiboot tree) ppc64le_hello (as I wrote about yesterday) FreeBSD and that the OpenPower ecosystem is rapidly growing (especially around people building OpenPower machines), the need for more formal specification, conformance testing and\u2026","rel":"","context":"In "code"","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3940,"url":"https:\/\/www.flamingspork.com\/blog\/2015\/02\/03\/building-openpower-firmware-for-use-in-power8-simulator\/","url_meta":{"origin":3965,"position":4},"title":"Building OpenPower firmware for use in POWER8 Simulator","author":"Stewart Smith","date":"2015-02-03","format":false,"excerpt":"Previously, I blogged on how to Run skiboot (OPAL) on the POWER8 Simulator. If you want to build the full Open Power firmware environment, including the Petitboot bootloader and kernel, you can now do so! My pull request for an op-build target for the simulator has been merged, so you\u2026","rel":"","context":"In "code"","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":4345,"url":"https:\/\/www.flamingspork.com\/blog\/2017\/12\/11\/a-simplified-view-of-openpower-firmware-development\/","url_meta":{"origin":3965,"position":5},"title":"A (simplified) view of OpenPOWER Firmware Development","author":"Stewart Smith","date":"2017-12-11","format":false,"excerpt":"I've been working on trying to better document the whole flow of code that goes into a build of firmware for an OpenPOWER machine. This is partially to help those not familiar with it get a better grasp of the sheer scale of what goes into that 32\/64MB of flash.\u2026","rel":"","context":"In "code"","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2017\/12\/devflow-legend.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2017\/12\/devflow-legend.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2017\/12\/devflow-legend.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/www.flamingspork.com\/blog\/wp-content\/uploads\/2017\/12\/devflow-legend.png?resize=700%2C400&ssl=1 2x"},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/3965","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/comments?post=3965"}],"version-history":[{"count":1,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/3965\/revisions"}],"predecessor-version":[{"id":3966,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/3965\/revisions\/3966"}],"wp:attachment":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/media?parent=3965"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/categories?post=3965"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/tags?post=3965"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}