As long time readers will know, I don’t hold any particular truck with benchmarks. They are too generic, too easily fiddled and lots of vendors don’t bother with them so they only represent a limited sample of the market. Proofs of concept with your own data and your own queries (in the case for data warehousing) are much more important.

Having said all that you’ve got to be impressed when a single vendor, Exasol, announces 5 TPC-H benchmarks (see www.tpc.org/tpch/results/tpch_perf_results.asp­) in a single hit. Running Exasol 4.0 on Dell PowerEdge 710, Exasol now holds records at 100Gb, 300Gb, 1,000Gb, 3,000Gb and 10,000Gb. In other words, everything up to 10Tb. Moreover, these aren’t just performance records, Exasol also now has the best price/performance in all of these categories also.

It is worth looking at the figures in some detail because some benchmarks are more popular than others. The 300Gb benchmark, for example, had previously stood since 2008 (actually to Exasol), while the previous 10,000Gb standard was set by IBM in 2007. So, in neither of these cases can you pay much attention to the figures.

Actually, the previous best for 3,000Gb was also in 2008 (again by Exasol) but Oracle published a benchmark at this level in March this year, which couldn’t beat Exasol’s previous record, let alone its latest one. In fact, Exasol latest figures are more than 14 times faster across the range of TPC-H tests than Oracle 11G R2 Enterprise Edition with partitioning. And it was more than 60 times better when it comes to price/performance.

The other company most likely to be miffed by these results is Ingres. It published a new 100Gb record in March this year, beating Exasol’s old record, but it has now been leapfrogged once again with better than 4 times the performance and 3 times the price/performance.

Of course, performance isn’t everything and neither is price but between them these do add up to a significant part of the whole. Moreover, by publishing five results at once, all running on the same hardware, the company has also demonstrated scalability, at least in terms of disk capacity. Again, that isn’t everything, but it is a lot.