The SP²Bench SPARQL Performance Benchmark
Abstract
Recently, the SPARQL query language for RDF has reached the W3C
recommendation status. Addressing the needs of this new standard,
the database community is currently exploring efficient storage
techniques for RDF data and evaluation strategies for SPARQL queries.
Unfortunately, the evaluation of virtually all existing approaches
relies on customized, user-defined experiments. This makes it
particularly difficult to (1) compare optimization strategies,
(2) estimate their generality, and (3) predict their benefits for
real-world scenarios. In order to overcome these deficiencies we
have developed a comprehensive benchmark for SPARQL, called SP²Bench.
Our benchmark comprises a data-generator for arbitrarily large
documents, which builds upon the well-known DBLP scenario, and
thus comes close to a real-world application scenario. The
benchmark queries implement meaningful requests on top of this
data, thereby testing typical SPARQL operator constellations and
RDF access patterns. With this focus, our benchmark allows to
easiliy detect deficiencies in current SPARQL implementations
and can be used to tune existing engines.
Resources
On this site, we provide the following resources.
- Publications
- The SP²Bench Technical Report, including a detailed description of the SP²Bench framework and benchmark results for selected engines
- A short version of the Technical Report above has been accepted for publication at ICDE'09 [ .pdf ]
- An experimental study of RDF Data Management approaches carried out on top of the SP²Bench framework has been carried out in our ISWC'08 publication [ .pdf ]
- Benchmark specification, description, data generator, and other material
- A short description of the DBLP RDF data that is created by the SP²Bench data generator
- The SP²Bench benchmark queries
- A download section, where you find both the current version of the data generator and the benchmark queries as text files
- SQL-translations of the benchmark queries as used for the experiments of our ISWC publication [ .pdf ]
The ICDE paper and the Technical Report linked above highlight design decisions and provide detailed information on data generation, the DBLP characteristics that are mirrored by our data generator, an in-depth discussion of the specific challenges that the benchmark queries impose to SPARQL engines, and benchmark results for selected SPARQL engines.
Project Members
- Contact: Michael Schmidt (Freiburg University Database Group)
- Thomas Hornung (Freiburg University Database Group)
- Norbert Küchlin (Freiburg University Database Group)
- Georg Lausen (Freiburg University Database Group)
- Christoph Pinkel (MTC Infomedia OHG)
References and Related Projects
- The Berlin SPARQL Benchmark is another nice benchmark project for SPARQL, designed by Chris Bizer and Andreas Schultz [ home ]
- W3C RDF(S) Specifications [ RDF | RDF Semantics | RDF Schema ]
- W3C SPARQL Specification [ SPARQL ]
- Original DBLP Dataset [ home ]
- D2R Server Publishing the DBLP Bibliography Database in RDF [ home ]
- Ontologies that are used by our data generator [ FOAF | SWRC | Dublin Core ]