Very large XML documents are generated and processed in several contexts, in particular in those involving scientific data and logs. In order to process such large documents we have designed and implemented techniques based on data partitioning for the evaluation of XQuery queries and updates on Map/Reduce clusters.

The proposed technique applies when queries and updates are iterative, i.e., they iterate the same query/update operations on a sequence of subtrees of the input document. We have developed schema-less, static analysis techniques to i) recognize iterative queries/updates, and ii) extract path information to be used for data partitioning purposes. Our system exploits both dynamic and static data partitioning to distribute the processing load among the machines of a Map/Reduce cluster. To boost the I/O performance across the distributed file system, our system uses EXI compression at each stage of the computation, from data partitioning to query/update execution.

After an introduction to the main techniques behind our system, a demonstration will show its abilities in dealing with complex workloads and large documents.