Archive - Oxbow Research

Benchmark with AI (pt3/3): How I actually use AI for platform evaluation

AI agents are useful for platform evaluation when execution is delegated carefully and judgment stays with the human.

Mar 18 • Joe Harris

Benchmark with AI (pt2/3): Giving agents knowledge instead of freedom

The fix is not a smarter prompt, but tools that make invalid benchmark methodology impossible to run.

Mar 16 • Joe Harris

Benchmark with AI (pt1/3): When AI agents are confident and wrong

AI agents can produce benchmark-shaped output with total confidence, even when the methodology is invalid.

Mar 13 • Joe Harris

February 2026

How Much Faster is DuckDB 1.5 vs 1.0? A lot

I benchmarked every DuckDB minor release from 1.0.0 through the 1.5.0 dev build on TPC-H, TPC-DS, ClickBench, and SSB. The results tell a clear story of…

Feb 27 • Joe Harris

The Egress Tax: how cloud providers engineer data gravity

Free ingress, cheap storage, 18x egress markup. The asymmetry is intentional.

Feb 18 • Joe Harris

Why database benchmarks are broken

A 2018 study examined 16 published database benchmark papers. Not one reported enough information to reproduce the results.

Feb 16 • Joe Harris

Benchmarking is good, actually

Everyone knows benchmarks are flawed. But when you have two weeks to pick a million-dollar platform, flawed data beats no data.

Feb 11 • Joe Harris

Explore SQL vs DataFrame performance with the BenchBox MCP and Claude Code

Use the BenchBox MCP to execute benchmarks, compare results, and investigate performance outliers - without leaving Claude Code.

Feb 5 • Joe Harris

Does your database allow benchmarks? A 2026 DeWitt clause survey

Since 1983 many database vendors forbade independent benchmarks. That's started to change in 2021. Here's the current status of benchmark publication…

Feb 5 • Joe Harris

Introducing Oxbow Research

Independent data platform analysis using open-source methodology.

Feb 2 • Joe Harris

#nojs-banner { position: fixed; bottom: 0; left: 0; padding: 16px 16px 16px 32px; width: 100%; box-sizing: border-box; background: red; color: white; font-family: -apple-system, "Segoe UI", Roboto, Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; font-size: 13px; line-height: 13px; } #nojs-banner a { color: inherit; text-decoration: underline; } This site requires JavaScript to run correctly. Please turn on JavaScript or unblock scripts