{"id":1274,"date":"2008-11-26T04:54:43","date_gmt":"2008-11-25T18:54:43","guid":{"rendered":"http:\/\/www.flamingspork.com\/blog\/?p=1274"},"modified":"2009-02-21T13:56:50","modified_gmt":"2009-02-21T03:56:50","slug":"temporary-tables-in-any-engine","status":"publish","type":"post","link":"https:\/\/www.flamingspork.com\/blog\/2008\/11\/26\/temporary-tables-in-any-engine\/","title":{"rendered":"Temporary tables in any engine&#8230;."},"content":{"rendered":"<p>Well&#8230; nearly any engine.<\/p>\n<p>I have a plan forming in my head to add some hooks to engines to help with creating temporary tables (the ones created while executing a query, not ones created during ALTER TABLE).<\/p>\n<p>Currently, if you ALTER TABLE and we require a temporary table, it&#8217;s still database.table but we generate a table name that&#8217;s small, unique and begins with &#8220;#sql&#8221;.<\/p>\n<p>I&#8217;ve changed some of the handler interface to accept two strings (database name, table name) instead of one &#8220;path&#8221; that may (or may not) end in &#8220;.FRM&#8221; and may (or may not) begin with &#8220;.\/&#8221; and may (or may not) use the &#8220;\/&#8221; separator between database and table name (hint: on win32, it&#8217;s sometimes &#8220;&#8221;).<\/p>\n<p>So ha_delete_table is now: ha_delete_table(db, table_name). Sanity!<\/p>\n<p>(this has the downside of being a incompatible change that doesn&#8217;t break the build as there seems to be no way in C++ to say &#8220;derived classes cannot implement a function of this name&#8221;).<\/p>\n<p>This is annoying for temporary tables however.<\/p>\n<p>They don&#8217;t reside in a database&#8230; they&#8217;re off in la-la land (otherwise known as opt_drizzle_tmpdir).<\/p>\n<p>Now&#8230; in MySQL 5.1 the ability for multiple temporary directories was added, and the MySQL server will cycle through them. The clusterfuck part of this was that a mutex was added&#8230; so every time the code goes to get the name of a (or for 99.99999999% of cases, the) temporary directory, it has to grab a mutex. With modern systems being able to have *many* simultaneous IO operations (e.g. create files) this is just dumb. It&#8217;s gone.<\/p>\n<p>If you want to use multiple spindles for temporary tables it&#8217;s called RAID people (or buy an SSD, mkfs.ext2 it and just wipe it on reboot. simple.)<\/p>\n<p>Anyway&#8230; back to temporary tables:<\/p>\n<p>they&#8217;re in \/tmp or something. So an API that&#8217;s foo(const char *db, const char *table_name) doesn&#8217;t work as well.<\/p>\n<p>Also, for a bunch of engines, it&#8217;s good to know that you&#8217;re using a temporary table. You probably want to store these somewhere that never needs fsync() or anything like that. If the server goes away, these tables are *gone*. So optimise for that.<\/p>\n<p>Heck, it may even be good to store temporary tables in your distributed engine (if that engine is memory based) as it&#8217;s often faster to access remote memory than local disk (although with SSD this is a whole different ball game&#8230; in fact, I&#8217;m not sure if it&#8217;s even still a ball game.. it&#8217;s possibly frisbee)<\/p>\n<p>But currently there&#8217;s hardcoded a mi_create (MyISAM create) call in the server and in the 6.0-maria tree, an #ifdef around if it&#8217;s mi_create or maria_create.<\/p>\n<p>So not any engine yet&#8230; but one can dream. Yes, I dare to dream.<\/p>\n<p>(although why I was dreaming of a small board with a PowerPC 603, 8MB RAM and a mini DVI port the other night is quite beyond me)<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Well&#8230; nearly any engine. I have a plan forming in my head to add some hooks to engines to help with creating temporary tables (the ones created while executing a query, not ones created during ALTER TABLE). Currently, if you &hellip; <a href=\"https:\/\/www.flamingspork.com\/blog\/2008\/11\/26\/temporary-tables-in-any-engine\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[75,14],"tags":[70],"class_list":["post-1274","post","type-post","status-publish","format-standard","hentry","category-drizzle-work-et-al","category-mysql","tag-drizzle"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p5a6n8-ky","jetpack-related-posts":[{"id":1285,"url":"https:\/\/www.flamingspork.com\/blog\/2008\/12\/04\/what-constitutes-an-identifier-for-a-table\/","url_meta":{"origin":1274,"position":0},"title":"What constitutes an identifier for a table?","author":"Stewart Smith","date":"2008-12-04","format":false,"excerpt":"Well... there's: database table name (both of these are quite obvious). But then you have: temporary tables Well... two types of temporary tables: those created in the course of query execution, typically in \/tmp\/ those created during ALTER TABLE, typically in the database directory You may have seen these \"#sql-foo.frm\"\u2026","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3234,"url":"https:\/\/www.flamingspork.com\/blog\/2013\/04\/04\/refactoring-internal-temporary-tables-another-stab-at-it\/","url_meta":{"origin":1274,"position":1},"title":"Refactoring Internal temporary tables (another stab at it)","author":"Stewart Smith","date":"2013-04-04","format":false,"excerpt":"A few weekends ago, I started to again look at the code in Drizzle for producing internal temporary tables. Basically, we have a few type of tables: Standard Temporary (from CREATE TEMPORARY TABLE) Temporary (from ALTER TABLE) Internal temporary (to help with query execution) If you're lucky enough to be\u2026","rel":"","context":"In &quot;code&quot;","block_context":{"text":"code","link":"https:\/\/www.flamingspork.com\/blog\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1869,"url":"https:\/\/www.flamingspork.com\/blog\/2010\/03\/23\/on-tableidentifier-and-the-death-of-path-as-a-parameter-to-storageengines\/","url_meta":{"origin":1274,"position":2},"title":"on TableIdentifier (and the death of path as a parameter to StorageEngines)","author":"Stewart Smith","date":"2010-03-23","format":false,"excerpt":"As anybody who has ever implemented a Storage Engine for MySQL will know, a bunch of the DDL calls got passed a parameter named \"path\". This was a filesystem path. Depending on what platform you were running, it may contain \/ or \\ (and no, it's not consistent on each\u2026","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":2329,"url":"https:\/\/www.flamingspork.com\/blog\/2011\/03\/23\/multi-tenancy-drizzle\/","url_meta":{"origin":1274,"position":3},"title":"Multi-tenancy Drizzle","author":"Stewart Smith","date":"2011-03-23","format":false,"excerpt":"My previous post focused on some of the problems of doing multi-tenant MySQL. One of the reasons why I started hacking on Drizzle was that the multi-tenancy options for MySQL just weren't very good (this is also the reason why I run my blog in a VM and not a\u2026","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1743,"url":"https:\/\/www.flamingspork.com\/blog\/2009\/10\/21\/first-flesh-wound-in-create_tmp_table\/","url_meta":{"origin":1274,"position":4},"title":"First flesh wound in create_tmp_table()","author":"Stewart Smith","date":"2009-10-21","format":false,"excerpt":"If you have needed a good reason to drink heavily and forget, may I suggest taking a look at create_tmp_table() and those who call it. It's probably one of the best illustrations of rot and awful, incomprehensible APIs in the server (Drizzle inherited it from MySQL). In the normal paths\u2026","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1763,"url":"https:\/\/www.flamingspork.com\/blog\/2009\/12\/12\/the-table-protobuf-message-format\/","url_meta":{"origin":1274,"position":5},"title":"The Table protobuf message format","author":"Stewart Smith","date":"2009-12-12","format":false,"excerpt":"If you've ever opened up drizzled\/message\/table.proto in the Drizzle source tree you will have seen what's in the table message: the structure that describes a database table in Drizzle. Previously I've talked about the Table message more generally, giving a fair bit of history of the FRM file and how\u2026","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/1274","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/comments?post=1274"}],"version-history":[{"count":2,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/1274\/revisions"}],"predecessor-version":[{"id":1553,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/1274\/revisions\/1553"}],"wp:attachment":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/media?parent=1274"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/categories?post=1274"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/tags?post=1274"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}