Benchmark Overview
The OLTP Benchmark is an open, transparent, and reproducible benchmark for comparing database performance under transactional workloads. Inspired by projects like ClickBench (opens in a new tab), we aim to provide the community with reliable data for making database decisions.
Goals
- Transparency - All benchmark code, configurations, and raw results are publicly available for inspection and reproduction.
- Fairness - Each database is tested with both default and optimized configurations.
- Reproducibility - Anyone can run the same benchmarks on their own infrastructure and verify results.
- Real-World Relevance - Workloads reflect actual OLTP patterns found in production applications.
Configuration Modes
We test each database in two configuration modes:
- Defaults - Out-of-the-box configuration with minimal changes. Shows what you get without tuning.
- Optimized - Tuned configurations reviewed by database communities. (Coming Soon)
This approach helps answer two questions:
- Defaults: "What performance can I expect with minimal setup?"
- Optimized: "What's the ceiling with proper tuning?"
Benchmark Categories
We separate our benchmarks into two distinct categories:
- Database Benchmarks - Self-hosted database engines tested on standardized local hardware. Measures raw database performance.
- Service Provider Benchmarks - Managed database services tested as offered. Measures real-world cloud performance. (Coming Soon)
Current Benchmark: TPC-C
We currently run TPC-C style workloads, the industry standard for OLTP benchmarking.
What is TPC-C?
TPC-C simulates a wholesale supplier environment with a mix of transaction types:
| Transaction | Description | Mix |
|---|---|---|
| New Order | Create new customer orders | 45% |
| Payment | Process customer payments | 43% |
| Order Status | Query order status | 4% |
| Delivery | Process batch deliveries | 4% |
| Stock Level | Check warehouse inventory | 4% |
Why TPC-C?
- Industry Standard: Widely recognized and understood
- Mixed Workload: Combines reads, writes, and complex transactions
- Scalable: Dataset scales with warehouses for different test sizes
- Real-World Representative: Models actual business operations
Future Benchmarks
We're considering additional benchmarks from BenchBase (opens in a new tab), including YCSB and other OLTP workloads.
Databases
- PostgreSQL - The world's most advanced open source relational database
- MySQL - Popular open source relational database
Getting Started
- View Results - Explore the latest benchmark results and compare databases.
- Methodology - Learn about our testing methodology and how to reproduce results.