Best approach for simple transformations on very large files?

["Randall J. Parr" <RParr at TemporalArts dot COM>]

What is the best alternative and/or approach for performing relatively
simple transformations on very large files?

I am trying to export database files to XML files; use XSLT to transform
them into a new format (change element/column names, split files, add
new element/column which are required, etc.); import the XML files back
into the database. The XML files thus produced have a relatively simple
table/row structure and (generally) do NOT have a DTD/Schema.

I have been using Oracle's XSU utility but it there are several problems
using XSU
1) running out of memory (both import and export)
2) insufficient ability to customize and/or add elements (export)
3) inability on import to impliment "if insert fails bcs aleady exists,
then update instead"
4) inadequate documentation and group support (there may be ways to
overcome my problems but I can't figure it out on my own; especially
with no access to source)

To overcome these problems I have decide to convert my existing
import/export programs so as to read/write XML instead of delimited
files.

My understanding is the only way to avoid the memory problems is to use
the SAX parser or a package based on the SAX parser.

I tried using Saxon (because of its PreviewManager) but find the
PreviewManager doesn't seem to work unless I have a DTD.

My question is (yes, I do have a question):

What is the best approach and/or package to use?

Java/JDBC/SAX (raw and uncut, as it were)?

SAXON Java API?

SAXON with extensions?

something else?

Thanks
R.Parr
Temporal Arts
330 S. Yellowstone Drive, Suite 3
Madison, WI 53705, U.S.A.
608.833.4578
RParr@TemporalArts.COM




 XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list