{"id":1716,"date":"2009-10-01T14:47:50","date_gmt":"2009-10-01T04:47:50","guid":{"rendered":"http:\/\/www.flamingspork.com\/blog\/?p=1716"},"modified":"2009-10-01T14:47:50","modified_gmt":"2009-10-01T04:47:50","slug":"table_raw_reader-reading-the-table-proto-from-disk-and-examining-everything","status":"publish","type":"post","link":"https:\/\/www.flamingspork.com\/blog\/2009\/10\/01\/table_raw_reader-reading-the-table-proto-from-disk-and-examining-everything\/","title":{"rendered":"table_raw_reader &#8211; reading the table proto from disk and examining everything"},"content":{"rendered":"<p>Ever wondered exactly *what* was in a Drizzle Table proto? Well, wonder no more. A while back this little utility called table_raw_reader hit the drizzle codebase. It&#8217;s a simple command line utility that takes a .proto file as an argument, reads it off disk and then prints out a text representation using the <a href=\"http:\/\/code.google.com\/apis\/protocolbuffers\/docs\/reference\/cpp\/google.protobuf.text_format.html\">TextFormat<\/a> class of the <a href=\"http:\/\/code.google.com\/apis\/protocolbuffers\/\">protobuf<\/a> library.<\/p>\n<p>An example:<\/p>\n<pre>stewart@willster:~\/drizzle\/jay-and-stewart-remove-pack_flag$ .\/drizzled\/message\/table_raw_reader .\/tests\/var\/master-data\/test\/t1.dfe\r\nname: \"t1\"\r\nengine {\r\n  name: \"InnoDB\"\r\n}\r\nfield {\r\n  name: \"id\"\r\n  type: INTEGER\r\n  format: DefaultFormat\r\n  options {\r\n    length: 11\r\n  }\r\n}\r\nfield {\r\n  name: \"padding\"\r\n  type: VARCHAR\r\n  format: DefaultFormat\r\n  options {\r\n    length: 800\r\n  }\r\n  string_options {\r\n    length: 200\r\n    collation_id: 45\r\n    collation: \"utf8_general_ci\"\r\n  }\r\n}\r\nindexes {\r\n  name: \"PRIMARY\"\r\n  is_primary: true\r\n  is_unique: true\r\n  type: UNKNOWN_INDEX\r\n  key_length: 4\r\n  index_part {\r\n    fieldnr: 0\r\n    compare_length: 4\r\n    key_type: 8\r\n  }\r\n  options {\r\n  }\r\n}\r\ntype: STANDARD\r\noptions {\r\n  collation: \"utf8_general_ci\"\r\n  collation_id: 45\r\n  pack_record: true\r\n  row_type: ROW_TYPE_DEFAULT\r\n}<\/pre>\n<p>This can help a lot when debugging, it already has helped me a lot. It&#8217;s left as an exercise for the reader to create a utility that lets you modify the table proto in this text based format.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Ever wondered exactly *what* was in a Drizzle Table proto? Well, wonder no more. A while back this little utility called table_raw_reader hit the drizzle codebase. It&#8217;s a simple command line utility that takes a .proto file as an argument, &hellip; <a href=\"https:\/\/www.flamingspork.com\/blog\/2009\/10\/01\/table_raw_reader-reading-the-table-proto-from-disk-and-examining-everything\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[75],"tags":[70,185,186],"class_list":["post-1716","post","type-post","status-publish","format-standard","hentry","category-drizzle-work-et-al","tag-drizzle","tag-protobuf","tag-table_raw_reader"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p5a6n8-rG","jetpack-related-posts":[{"id":1655,"url":"https:\/\/www.flamingspork.com\/blog\/2009\/06\/09\/drizzle-pluggable-metadatastore-or-no-table-definition-file-on-disk\/","url_meta":{"origin":1716,"position":0},"title":"Drizzle pluggable MetadataStore (or: no table definition file on disk)","author":"Stewart Smith","date":"2009-06-09","format":false,"excerpt":"My code is shaping up rather nicely (see https:\/\/code.launchpad.net\/~stewart\/drizzle\/discovery) and I'm planning to submit a merge-request for it later today. I'm about to commit code that implements a MetadataStore for the ARCHIVE engine. This means that for ARCHIVE tables, you only have the .ARZ file on disk. The table definition\u2026","rel":"","context":"In &quot;General&quot;","block_context":{"text":"General","link":"https:\/\/www.flamingspork.com\/blog\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1745,"url":"https:\/\/www.flamingspork.com\/blog\/2009\/12\/09\/drizzle-frm-replacement-the-table-proto\/","url_meta":{"origin":1716,"position":1},"title":"Drizzle FRM replacement: the table proto","author":"Stewart Smith","date":"2009-12-09","format":false,"excerpt":"Drizzle originally inherited the FRM file from MySQL (which inherited it from UNIREG). The FRM file stores metadata about a table; what columns it has, what type those columns are, what indexes, any default values, comments etc are all stored in the FRM. In the days of MyISAM, this worked\u2026","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1650,"url":"https:\/\/www.flamingspork.com\/blog\/2009\/05\/27\/pluggable-metadata-stores-or-the-revenge-of-table-discovery\/","url_meta":{"origin":1716,"position":2},"title":"Pluggable Metadata stores (or&#8230; the revenge of table discovery)","author":"Stewart Smith","date":"2009-05-27","format":false,"excerpt":"Users of the ARCHIVE or NDB storage engines in MySQL may be aware of a MySQL feature known as \"table discovery\". For ARCHIVE, you can copy the archive data file around between servers and it magically works (you don't need to copy the FRM). For MySQL Cluster (NDB) it works\u2026","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1765,"url":"https:\/\/www.flamingspork.com\/blog\/2009\/12\/11\/the-schema-protobuf-message-drizzles-metadata-on-a-schema\/","url_meta":{"origin":1716,"position":3},"title":"The Schema protobuf message: Drizzle&#8217;s metadata on a schema","author":"Stewart Smith","date":"2009-12-11","format":false,"excerpt":"I've previously talked about table metadata in Drizzle and how we use the table protobuf message to describe a table (see Drizzle FRM Replacement and others). The model in Drizzle is that the engine is responsible for its metadata. For schemas (you may be thinking 'database' but we're moving to\u2026","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1763,"url":"https:\/\/www.flamingspork.com\/blog\/2009\/12\/12\/the-table-protobuf-message-format\/","url_meta":{"origin":1716,"position":4},"title":"The Table protobuf message format","author":"Stewart Smith","date":"2009-12-12","format":false,"excerpt":"If you've ever opened up drizzled\/message\/table.proto in the Drizzle source tree you will have seen what's in the table message: the structure that describes a database table in Drizzle. Previously I've talked about the Table message more generally, giving a fair bit of history of the FRM file and how\u2026","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1713,"url":"https:\/\/www.flamingspork.com\/blog\/2009\/10\/01\/pack_flag-is-dead\/","url_meta":{"origin":1716,"position":5},"title":"pack_flag is dead!","author":"Stewart Smith","date":"2009-10-01","format":false,"excerpt":"(At least in Drizzle) Brian merged my jay-and-stewart-remove-pack_flag branch and it's now in trunk. Removing pack_flag was a task for the bell milestone and in true collaborative effort, it took more than one person to unravel its dark mysteries. Hats go off to Jay who bravely ripped more of it\u2026","rel":"","context":"In &quot;drizzle&quot;","block_context":{"text":"drizzle","link":"https:\/\/www.flamingspork.com\/blog\/category\/work-et-al\/drizzle-work-et-al\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/1716","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/comments?post=1716"}],"version-history":[{"count":2,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/1716\/revisions"}],"predecessor-version":[{"id":1718,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/posts\/1716\/revisions\/1718"}],"wp:attachment":[{"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/media?parent=1716"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/categories?post=1716"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.flamingspork.com\/blog\/wp-json\/wp\/v2\/tags?post=1716"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}