It can be done using for e.g. http://labs.carrotsearch.com/junit-benchmarks.html
These tests should be run only in the performance-test profile.
In the first pass use the in-memory db, but ideally you should be able to specify a maven profile to run tests on a different db i.e. datasets must be db agnostic (in the dbunit format).
Write tests for concept searching so that we can compare the current search to
TRUNK-2999 and later on measure performance implications of changes we make to queries or caching.
Use the MVP dictionary as a dataset. It should not be stored in git rather installed as an artifact to the maven repository so that it is only downloaded when needed and is versioned.