SVG benchmarks

For rasterizing SVG images we've used librsvg for some time. Another contender which we didn't choose originally is Batik. There are several respects in which librsvg is unsatisfactory and Batik might be preferable.

librsvg: In favour: fast; totally free/libre. Against: Lots of GNOME-ish dependencies, which makes tracking updates difficult; We have to maintain a security patch to disable the loading of external URL resources, since librsvg maintainers weren't interested in it; Lots of rendering bugs (but it's improved over time); Unclear fonts/text rendering situation.

Batik: In favour: more mature; higher quality rendering; built-in security mode. Against: slow!; doesn't (or at least didn't) run on free/libre Java (GCJ etc).

Benchmark

Brion did a quick test of SVG rendering (19 Dec 2006), with 50 images pulled off Commons. Test run on a MacBook, 2 GHz Intel Core 2 Duo, in Ubuntu running under Parallels (using one virtual processor). rsvg is Ubuntu's 2.16.0; batik is 1.6, running on Sun's J2SE 6 runtime.

rsvg: 18.07 seconds; 361.46ms per image

batik: 108.44 seconds; 2168.84ms per image

Batik is probably slowed down by having to start up the VM and JIT on every call. It likely can be sped up by using a daemon process and handing off render requests to that.

Batik also refused to render one file that rsvg did render.

ivar notes that to avoid JDK startup time you can use nailgun.

Additional notes

Batik command-line renderer appears to scale image height to be proportional with the given width. This could produce width/height inconsistencies between specified heights from MW and results.
I'm pretty sure Batik has a mode for increased safety (rejecting external links), which we have to maintain as a patch hack for librsvg. Not having to maintain a patch is attractive.
We frequently get requests about font support. It'd be nice to know how to answer such questions.
rsvg's resource usage can be limited with ulimit etc easily; what about a batik daemon? How to stop giiaaannttttt files from eating up RAM without taking down the server?
You'd have to look at jvm args... -- chris

Further testing

2008 (57 images)

David Gerard ran a test of ImageMagick 6.2.4, Inkscape 0.45.1, Batik 1.7.0 (on Sun Java 1.6.0_03-b05) and rsvg 2.18.2 on 57 Commons images, including some in the megabytes. Those which reliably gave an error from the renderers were left out. Test box was a Dell Latitude D610, Pentium M 1.6 GHz, 512MB memory, running Kubuntu 7.10. All test applications except Batik were Ubuntu repository versions.

All four programs were run through the 57 images 115 times and timed with /usr/bin/time. X running in Failsafe (konsole as only app), nothing else running.

	median real	mean real	median CPU	mean CPU
Batik	139.71	139.95	137.02	137.04
ImageMagick	112.43	112.67	111.7	111.73
Inkscape	51.43	51.52	51.18	51.2
rsvg	25.97	26.03	25.8	25.8

Worrying:

Batik's JVM crapped itself several times for no apparent reason - 39 times in the run, i.e. 39 in 4485 attempted renderings. Not on any particular SVG either.
Inkscape thrashed the disk. (Single physical disk in laptop.)

(Possible explanation: that rsvg is a lib whereas Inkscape is a full app - could the overhead of starting and stopping the app be significant?)

What this suggests: use rsvg if possible (anything Unix), use Inkscape if rsvg's dependencies are too troublesome (e.g. on Windows, where Inkscape is available as a standalone app) and you have a lot of memory.

See /test 20080112 for full data.

2009 (56 files)

Hk kng re-ran the tests in 2009 using a http server solution for Batik: batikd. Full test results are at /test 20090816. WikiMedia Commons presents a gallery of the test images.

A comparison of CPU times is not as easily done, since the real image conversion for Batik is done in the separate server process. The wallclock times look like this:

Program	Median time
Batik	57.76 s
ImageMagick	81.86 s
Inkscape	70.95 s
rsvg	41.48 s

2021 (337+1338+141 files, including correctness ratio)

JoKalliauer ran several test suites in 2021 and wrote detailed explanations about rendering issues; see c:User:JoKalliauer/SVG_test_suites for details.

	librsvg 2.50	resvg 0.14.0	Inkscape 1.0	Batik 1.13; 1.14^{[clarification needed]}
Correctness according to the W3C test suite	0,662	0,831	0,745	0,801
cpu-time taken in the W3C test suite (512 px)	1m 46,776s	1m 04,490s	7m 57,465s	6m 51,560s
Correctness according to resvg test suite	0.754	0.956	0.729	0.703
Time measured by resvg test suite	4min 05sek	2min 30sek	46min 22sek	61min 29sek
Correctness with Wikimedia featured pictures	0.92	1.00
Time with Wikimedia featured pictures	5m 17,701s	4m 46,639s	15m 28,202s	11m 30,768s