This is the mail archive of the libc-alpha@sourceware.org mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [PATCH] Valid bench.out against a JSON schema


On Tue, May 27, 2014 at 09:30:00AM +0100, Will Newton wrote:
> On 22 May 2014 11:59, Siddhesh Poyarekar <siddhesh@redhat.com> wrote:
> 
> Hi Siddesh,
> 
> > This patch adds a JSON schema for the benchmark output file and also
> > adds a script that validates the generated output against the schema.
> >
> > Siddhesh
> >
> >         * benchtests/scripts/validate_benchout.py: New script.
> >         * benchtests/Makefile (bench-func): Call it.
> >         * benchtests/scripts/benchout.schema.json: New file.
> 
> I like the idea of this patch, however I am not sure how we will
> handle other benchmark types (e.g. string, malloc) and if we will need
> multiple schema or if we will need to extend this one. I haven't gone
> far enough down that road yet to know the answer but I wonder if it is
> something you have considered?

I haven't given it a lot of thought either, but given that the current
string tests have a clearly different output format, we would need a
separate schema to JSONify it.  Of course, given that there have been
complaints in the past about the string tests being less than useful,
it might be a good idea to revisit them and maybe even come up with a
compatible output format for them.

The malloc microbenchmark you had posted seems to have a similar
format with some extensions*, so we should be good there.

> > +import jsonschema
> 
> I think last time I looked at this patch jsonschema was not a module
> Python installed by default so I suggested importing inside a
> try/catch.

Thanks, I'll fix this.

Siddhesh

* I'll get to reviewing the malloc benchmark soon-ish.

Attachment: pgpEJs01TNOBm.pgp
Description: PGP signature


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]