Add script to automatically fix up data.js
authorAryeh Gregor <AryehGregor+gitcommit@gmail.com>
Fri, 16 Sep 2011 13:33:19 -0600
changeset 585 12706b7f7471
parent 584 3cc579b879ac
child 586 3c450adef205
Add script to automatically fix up data.js

This removes a tedious manual step from regenerating the tests. Now my
procedure is:

1) Run gentest.html in Chrome and Firefox. This will take 15ish minutes
on this machine (sigh), with Chrome being the limiting factor because
it's well-behaved and nices the background tab. I notice when it's done
because of the CPU meter I have at the top of my screen.

2) Switch to some other branch on my server so I can keep working.

3) When (1) is done, copy results to /tmp/webkit and /tmp/gecko on
localhost. I do this locally because trying to paste 1MB of text into a
remote terminal seems not to work well.

4) diff -u /tmp/webkit /tmp/gecko | less on localhost. Check for any
unexpected differences. There are some known mismatches due to browser
bugs, which are mentioned in the spec. (I should automate this step
too, perhaps? It could be error-prone.)

5) Switch back to the branch I was working on on the server.

6) scp /tmp/webkit to the right place on my server, overwriting data.js.
I use WebKit instead of Gecko because it has fewer bogus results that
I've noticed.

7) conformancetest/fixdata on the server to fix it.

8) git diff to check that the test changes all look correct.

9) git commit -a --amend to add the test changes to the existing commit.

10) Rebase the branch onto master, delete the branch, and ./publish.

I'm going to have to think of more ways to simplify this process,
although with branches it's surprisingly painless.
conformancetest/fixdata
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/conformancetest/fixdata	Fri Sep 16 13:33:19 2011 -0600
@@ -0,0 +1,5 @@
+#!/bin/sh
+# Currently I cross-check data generation in Gecko and WebKit, then populate
+# using the WebKit copy because it has fewer bugs to fix.  This fixes the bugs
+# for me.  Must be run from the root of the spec repo.
+sed -i 's!"fo<i>\[o</i><span style=\\"font-style:oblique\\">b]ar</span>baz"!"fo<i>[o</i><span style=\\"font-style:oblique\\"><i>b]</i>ar</span>baz"!;s!"fo<span style=\\"font-style:italic\\">\[o</span><span style=\\"font-style:oblique\\">b]ar</span>baz"!"fo<span style=\\"font-style:italic\\">[o</span><span style=\\"font-style:oblique\\"><span style=\\"font-style:italic\\">b]</span>ar</span>baz"!' conformancetest/data.js