{"id":1395,"date":"2009-02-12T23:15:08","date_gmt":"2009-02-12T13:15:08","guid":{"rendered":"http:\/\/www.flamingspork.com\/blog\/?p=1395"},"modified":"2009-02-21T13:25:09","modified_gmt":"2009-02-21T03:25:09","slug":"performance-schema-show-me-the-code","status":"publish","type":"post","link":"https:\/\/www.flamingspork.com\/blog\/2009\/02\/12\/performance-schema-show-me-the-code\/","title":{"rendered":"Performance Schema: Show me the code"},"content":{"rendered":"<p>For such a long worked on feature, with such potential &#8211; I find the resistence to publishing a source tree curious (my comments on the topic have been moderated away but others have asked too). I could go and grep through the <a href=\"http:\/\/lists.mysql.com\/commits\">commits<\/a> list searching for things (hint: look for mysql-6.0-perf), and then start to re-construct a tree; but I have more important things to do (yes, Brian, like FRM patches :)<\/p>\n<p>Instead of re-inventing the wheel in Drizzle for a performance schema like interface, it&#8217;d be great to go with existing work. Evaluating the code as it&#8217;s coming along is important.<\/p>\n<p>I also have concerns about the code itself:<\/p>\n<ul>\n<li>Mutex instrumentation:\n<ul>\n<li>how expensive is this in the common case of not instrumenting.<\/li>\n<li>Is this yet-another wrapper around pthread_mutex_t?<\/li>\n<li>Could this be done in another, more generic way?<\/li>\n<li>How can engine devs use this? Do you have to completely be integrated with the MySQL server way of doing things (and give up being able to be a sep piece of software) if you&#8217;re going to use this. If there&#8217;s a null header, what license is it under?<\/li>\n<li>Can we use some of the code in (for example) the ndbd and then pass back mutex data from remote systems to the SQL server in a usable mannar? If we do this via NDB$INFO, could this show up in the performance schema?<\/li>\n<li>Is the code clean or littered with ugly #ifdef?<\/li>\n<li>Could this be done without having a special mutex type?<\/li>\n<\/ul>\n<\/li>\n<li>memory instrumentation\n<ul>\n<li>is it there at all?<\/li>\n<li>how are they doing it?<\/li>\n<li>MEM_ROOT based? what about new\/delete malloc\/free and various buffers and how this all ties to session etc?\n<ul>\n<li>In drizzle I&#8217;ve been seriously looking at talloc to help with instrumentation of memory usage.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<li>IO instrumentation\n<ul>\n<li>mmap?<\/li>\n<\/ul>\n<\/li>\n<li>how are the performance schema tables being generated (constructing table shares, running CREATE TABLE manually by the user, CREATE TABLE in a helper thread, similar to I_S or some black magic?)\n<ul>\n<li>for NDB$INFO, we generate SQL CREATE TABLE statements and run them to generate a FRM file (I architected and wrote the base kernel code, Martin wrote the MySQL code) as other approaches were considered too hairy and likely to produce bugs.<\/li>\n<li>For Drizzle, I&#8217;m completely removing the FRM files in favour of a discovery based interface that&#8217;ll let the engines be in charge of metadata. It&#8217;s all in protobufs, so a standard and easy to read data format. So I&#8217;m one of the few people on the planet that know about the related data structures. I would like our proto based code to work for performance schema as well.<\/li>\n<\/ul>\n<\/li>\n<li>Is the instrumentation always-in, or using d-trace style no-op funkyness?<\/li>\n<li>Is it better to hook into d-trace (or similar) on platforms that have it instead of providing custom code?<\/li>\n<li>Could performance be gained by using LD_PRELOAD or similar linker foo to only enable some instrumentation but allow others to be selectable on server startup?<\/li>\n<\/ul>\n<p>So stuck waiting for some code to look at to answer the questions (random commits on the commits list doesn&#8217;t really do it like the finished product &#8211; i really hate commit lists).<\/p>\n<p>So I can&#8217;t currently comment on the performance schema work much at all, nor if it&#8217;s useful to Drizzle. Hopefully will soon.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>For such a long worked on feature, with such potential &#8211; I find the resistence to publishing a source tree curious (my comments on the topic have been moderated away but others have asked too). I could go and grep &hellip; <a href=\"https:\/\/www.flamingspork.com\/blog\/2009\/02\/12\/performance-schema-show-me-the-code\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[75,14],"tags":[70,628,88],"class_list":["post-1395","post","type-post","status-publish","format-standard","hentry","category-drizzle-work-et-al","category-mysql","tag-drizzle","tag-mysql","tag-performance_schema"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p5a6n8-mv","jetpack-related-posts":[{"id":3160,"url":"https:\/\/www.flamingspork.com\/blog\/2012\/09\/20\/impact-of-mysql-slow-query-log\/","url_meta":{"origin":1395,"position":0},"title":"Impact of MySQL slow query log","author":"Stewart Smith","date":"2012-09-20","format":false,"excerpt":"So, what impact does enabling the slow query log have on MySQL? I decided to run some numbers. I'm using my laptop, as we all know the currently most-deployed database servers have mulitple cores, SSDs and many GB of RAM. For the curious:\u00c2\u00a0Intel(R) Core(TM) i7-2620M CPU @ 2.70GHz The benchmark\u2026","rel":"","context":"In &quot;code&quot;","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":2286,"url":"https:\/\/www.flamingspork.com\/blog\/2011\/02\/08\/drizzle-metadata-tables\/","url_meta":{"origin":1395,"position":1},"title":"Drizzle metadata tables","author":"Stewart Smith","date":"2011-02-08","format":false,"excerpt":"Giuseppe has a great post about the Evolution of MySQL metadata, and I thought I'd have a look at what we have in Drizzle. It's pretty easy to work out how many tables are in each schema, we just query the standard INFORMATION_SCHEMA.TABLES view: drizzle> select table_schema,count(table_name) -> from information_schema.tables\u2026","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":4180,"url":"https:\/\/www.flamingspork.com\/blog\/2016\/09\/27\/mysql-removes-the-frm-7-years-after-drizzle-did\/","url_meta":{"origin":1395,"position":2},"title":"MySQL removes the FRM (7 years after Drizzle did)","author":"Stewart Smith","date":"2016-09-27","format":false,"excerpt":"The new MySQL 8.0.0 milestone release that was recently announced brings something that has been a looooong time coming: the removal of the FRM file. I was the one who implemented this in Drizzle way back in 2009 (July 28th 2009 according to Brian)- and I may have had a\u2026","rel":"","context":"In &quot;code&quot;","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1813,"url":"https:\/\/www.flamingspork.com\/blog\/2010\/03\/12\/writing-a-storage-engine-for-drizzle-part-2-create-table\/","url_meta":{"origin":1395,"position":3},"title":"Writing A Storage Engine for Drizzle, Part 2: CREATE TABLE","author":"Stewart Smith","date":"2010-03-12","format":false,"excerpt":"The DDL code paths for Drizzle are increasingly different from MySQL. For example, the embedded_innodb StorageEngine CREATE TABLE code path is completely different than what it would have to be for MySQL. This is because of a number of reasons, the primary one being that Drizzle uses a protobuf message\u2026","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3770,"url":"https:\/\/www.flamingspork.com\/blog\/2014\/06\/05\/performance-impact-of-mysql-query-cache-on-modern-hardware\/","url_meta":{"origin":1395,"position":4},"title":"Performance impact of MySQL query cache on modern hardware","author":"Stewart Smith","date":"2014-06-05","format":false,"excerpt":"Recently, Morgan has been writing on deprecating some MySQL features and inspired by that while working on MySQL on POWER, I wondered \"What is the impact of the MySQL query cache on modern hardware?\" We've known for over six years (since before we started Drizzle) that the query cache hurt\u2026","rel":"","context":"In &quot;code&quot;","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3237,"url":"https:\/\/www.flamingspork.com\/blog\/2013\/03\/08\/mysql-modularity-are-we-there-yet\/","url_meta":{"origin":1395,"position":5},"title":"MySQL modularity, are we there yet?","author":"Stewart Smith","date":"2013-03-08","format":false,"excerpt":"MySQL is now over four times the size than it was with MySQL 3.23. This has not come in the shape of plugins. Have we improved modularity over time? I decided to take LoC count for plugins and storage engines (in the case of Drizzle, memory, myisam and innobase are\u2026","rel":"","context":"In &quot;code&quot;","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/1395","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/comments?post=1395"}],"version-history":[{"count":4,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/1395\/revisions"}],"predecessor-version":[{"id":1421,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/1395\/revisions\/1421"}],"wp:attachment":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/media?parent=1395"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/categories?post=1395"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/tags?post=1395"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}