On Wed, Jul 24, 2019 at 04:06:50PM -0700, Josh Steadmon wrote: > This is a proof of concept series that formalizes the structure of trace2 > event > output using JSON-Schema [1]. > > It provides a validator (written in Go) that verifies the events in a given > trace2 event output file match the schema. I am happy to rewrite this > validator > in some other language, provided that the language has a JSON-Schema library > supporting at least draft-04. > > It runs the validator as part of the CI suite (it increase the runtime > by about 15 minutes). It tests that the trace output of "make test" > conforms to the schema.
I don't like this approach. 15 minutes is a lot, and spending that much time on JSON schema validation in every CI build is overkill and (IMO unacceptably) wasteful. I mean, the test suite involves e.g. thousands of 'git (add|commit|checkout|reset|...)' executions to set up the test cases, but surely it's not necessary to validate the trace output of every single one of them. > I would appreciate any feedback on better ways to integrate the > validator into the CI suite. How about adding a test script dedicated to JSON schema validation, which runs only as many git commands as needed to cover all trace2 events.
