Theme speed reviews contradict each other because almost nobody publishes their method. A vendor tests a clean install; a reviewer tests with plugins; a forum post tests on $3 hosting. Since our comparisons — the 2026 benchmark report and every head-to-head we publish — drive real purchase decisions, here is the full protocol so you can replicate it or argue with it.
Hardware and software
- One dedicated staging VPS: 4 vCPU, 8 GB RAM, NVMe storage. Nothing else runs on it during a test week.
- nginx, PHP 8.3 with opcache, MariaDB 10.11, identical my.cnf across tests.
- WordPress current release; themes installed fresh per test run, never reused installs.
The content set
Five page types with realistic data: a homepage, a blog post with images and a table, a category archive with 50 entries, a WooCommerce shop page, and a 500-product catalog variant for scale tests. Demo content is identical across themes — we import the same fixture set rather than each theme’s own demo (whose assets would skew weight measurements).
Measurement passes
- Uncached pass. WebPageTest mobile 4G profile, three runs, median. This is the theme’s true floor — what crawlers and first-time visitors see.
- Cached pass. Full-page cache enabled with a standard configuration (documented in the caching guide), three runs, median.
- Query count. A query monitor plugin records total queries and total query time per page type.
- Asset breakdown. Total transferred bytes, theme-attributed bytes, and CSS/JS split — the numbers behind the CSS output-size study.
Why our numbers differ from vendor claims
Vendors benchmark with their Pro version, their demo content, their recommended optimization plugins, and sometimes their own CDN. Each choice shaves milliseconds. We deliberately test free versions, neutral content, and no optimization plugins — then add caching as a separate pass so both numbers exist.
Known limitations (stated honestly)
- One hosting environment. Absolute numbers shift on other stacks; relative rankings have been stable across three hardware refreshes.
- Laboratory metrics, not field data — real-user CWV varies with geography and devices.
- Themes update constantly; we re-run top candidates quarterly and note changes in comparison updates like the Astra vs GeneratePress re-test.
Use this protocol yourself
The full rig is a weekend of setup. The reduced version — one staging site, two themes, WebPageTest cached and uncached, query monitor — is an afternoon and answers most purchase questions. If you would rather skip the work entirely, the free audit applies the same measurement discipline to your live site.
Different content, hosting, plugin stacks, and cached-vs-uncached conditions. Always check whether a review publishes its method before trusting its numbers. Both: uncached is what crawlers and first-time visitors see; cached is what returning visitors see. A theme good only cached still loses crawler goodwill. Plugins mask theme overhead. We measure the theme's true floor first, then add caching as a separate pass so both numbers are visible. Yes — the reduced version (one staging site, two themes, WebPageTest + query monitor, cached and uncached) takes an afternoon and answers most questions.Why do theme speed benchmarks disagree?
Are uncached or cached numbers more important?
Why don't you test with optimization plugins?
Can I replicate your benchmark on my own server?
