TripleStoreLoadTesting

There is a need for a standard load testing suite for accurately comparing different triple store configurations. This page attempts to gather some initial notes on how this suite might be constructed.

For pointers to reports on load testing, and pointers to users with large scale deployments of particular stores, see TripleStoreScalability.

Scalability will only be one factor in deciding on the right store to use. For example, the needs of a desktop/client application with low-cost install requirements will be different from that of a server application. Factors such as installation overhead, pricing, replication and stability are all important. There is a definite need in the community to have some reliable metrics for store scalability.

Sample Data Sets and Queries

For now see AnRdfHarvesterStartingPoint.

The best kind of data sets will be those available as native RDF (RDF/XML, N3 or similar). Database dumps aren't really suitable as they won't easily be migrated between different store configurations.

As the structure/graph for each data set will vary, the kinds of queries that are applied will need to be tailored, e.g. to explore how stores handle different kinds of data. E.g. handling of large literals.

• TAP has lots of large datasets from different domains.
• GrammaTech's static analysis software can dump annotated control-flow-graphs in RDF. We can easily generate 10s of millions of triples (a single line-of-code yields ~400 triples). Contact Michael Mcdougall (mcdougall -at-sign- grammatech.com) if you want sample data. We don't have SPARQL queries for the dataset yet.

Software Configurations

A list of different configurations to test out.

Test Harness Description

Descriptions of test harnesses and configurations. For example single vs. multi-threaded harnesses, description of transaction handling, etc.