Progress on XQuery compilation

By Michael Kay on October 24, 2006 at 02:18p.m.

A progress report: I'm now within a whisker of being able to compile the whole of the XQuery Test Suite into Java code. One or two negative tests generate code that won't compile, but the problems are minor. So far, however, I've only actually executed a very small sample of the tests. The next stages, which will overlap to some extent, are (a) to get the generated code working correctly, (b) to make sure it performs (which includes generating code for some of the optimizations currently done dynamically by the interpreter), (c) general clean-up: there's a lot of repetitive code that was generated by cut-and-paste, and therefore many opportunities for refactoring, and I don't want to be fixing the same bug in a dozen places, (d) removing restrictions in the current implementation, for example there's no support currently for extension functions, (e) diagnostics, especially the quality of run-time error reporting and tracing, and (f) packaging, for example defining the interfaces to control the compilation process. That's a fair bit of work still to do, even though the bulk of the code may now have been written.

An interesting problem emerged in using the XQTS tests: a very large number of them contain expressions that are actually evaluated at compile time during the constant-folding process. To get reasonable coverage for the Java code geenrator I therefore had to suppress such constant-folding operations, which I basically achieved by wrapping string and numeric literals in an artificial function call. This in turn generated some very obscure expressions which in one or two cases broke the existing interpreter! It also turns out there are a few features missing from the test suite, some of which are quite tricky: not only things that are intrinsically non-portable like extension functions and dynamically-selected collation URIs, but also rarely-used but well-defined constructs like X/position() and X/last().