Does your database allow benchmarks? A 2026 DeWitt clause survey

Since 1983 many database vendors forbade independent benchmarks. That's started to change in 2021. Here's the current status of benchmark publication rights in 2026.

TL;DR:

For 40 years, "DeWitt clauses" let vendors legally block benchmark publication. Since 2021, that's changed: open source databases have no restrictions, major cloud vendors (AWS, Azure, Google, Databricks, Snowflake) now allow benchmarks with methodology disclosure, and only legacy holdouts (Oracle, SQL Server) still require permission. The pattern is clear: vendors confident in their performance welcome scrutiny.

---

What is the "DeWitt clause"?

The DeWitt Clause is part of a software license agreement that prohibits users from publishing benchmark results without the vendor's approval. From Oracle's current license:

"You may not disclose results of any program benchmark tests without our prior consent."¹

Microsoft SQL Server's license contains a similar restriction:

"You must obtain Microsoft's prior written approval to disclose to a third party the results of any benchmark test of the software."²

These clauses create a chilling effect on database evaluation. Researchers can't publish comparative studies. Consultants can't share findings with the broader community. Customers have to make purchasing decisions based on vendor marketing rather than independent verification. The only databases that can be rigorously critiqued in public are open source, which puts proprietary vendors at an unfair advantage when their performance is actually competitive, and shields them when it isn't.

The clauses also have a corrosive, self-reinforcing quality. Once one vendor adopts a DeWitt clause, competitors feel disadvantaged without one. After all, if Vendor A can't be publicly critiqued but Vendor B can, Vendor B faces asymmetric scrutiny regardless of actual performance³.

The origin of the clause

The DeWitt clause story starts in 1983, when David DeWitt and his colleagues created the Wisconsin Benchmark to measure relational database performance⁴. When they published their findings, Oracle's performance stood out as particularly poor. According to DeWitt, Oracle CEO Larry Ellison was furious. He called the department chair and demanded: "You have to fire this guy."⁵

Oracle didn't succeed in getting DeWitt fired. Instead, they did something with longer-lasting consequences: they added a clause to their license agreement prohibiting customers from publishing benchmark results without Oracle's prior written consent⁶.

This provision became known as the "DeWitt Clause", somewhat ironic given that DeWitt championed benchmarking. The clause spread throughout the database industry like a virus, adopted by nearly every major commercial vendor. For 40 years, DeWitt's name has been synonymous with preventing the very transparency he fought for⁷.

But things are changing. Since 2021, there has been a shift in vendor attitudes toward benchmark publication. This post surveys the current landscape as of early 2026, documenting which vendors restrict benchmarks, which have opened up, and what it means for anyone trying to make informed database decisions.

---

The 2021 turning point

For nearly four decades, DeWitt clauses were just how the database industry worked. But cloud hyperscalers were the quiet exception. AWS and Microsoft Azure never adopted traditional DeWitt clauses. Instead, they used reciprocal terms from the start: you could publish benchmarks as long as you shared your methodology and granted them the same rights to benchmark your products. They didn't make a big deal about it and mostly used it as a shield when competing with each other.

Then Databricks turned benchmarking transparency into a competitive weapon. In November, 2021, Databricks announced that Databricks SQL had set a new world record on the 100TB TPC-DS benchmark, outperforming the previous record by 2.2x⁸. This was significant not just for the result itself, but because it was the first official TPC-audited benchmark from a cloud data warehouse vendor. The results were verified by the Transaction Processing Performance Council in a 37-page disclosure report⁹.

Six days later, Databricks announced they were eliminating the DeWitt Clause from their service terms entirely¹⁰. But they went further, introducing what they called a "DeWitt Embrace Clause":

"If a competitor or vendor benchmarks Databricks or instructs a third party to do so, this new provision invalidates the vendor's own DeWitt Clause to allow reciprocal benchmarking."

In other words: benchmark us, and we can benchmark you back, regardless of what your license says. The move signaled Databricks was confident in their performance and wanted to force competitors into the open. It worked. Within weeks, Snowflake responded with their own benchmarks and removed their DeWitt clause¹¹. What followed was a brief benchmark war between the two companies, with competing claims, counter-benchmarks, and accusations of unfair methodology.

The specifics of who "won" that battle matter less than the outcome: two major cloud data warehouse vendors had permanently abandoned benchmark restrictions. Others followed. By the end of 2023, SingleStore had eliminated their clause¹². AWS and Azure, which had long maintained reciprocal (rather than restrictive) benchmark terms, saw their approach validated.

The dam hasn't broken entirely. Oracle, Microsoft SQL Server, and several cloud-only services still maintain restrictions. But when I look at the industry now versus 2020, the shift is unmistakable.

---

Current status by vendor category

I surveyed over 25 database vendors and cloud services to document current benchmark publication policies as of January 2026. Here's what I found, and what the patterns tell us about vendor confidence.

Open source: no restrictions, of course

\(\begin{array}{|l|l|l|} \hline \textbf{Database} & \textbf{License} & \textbf{Benchmark Status} \\ \hline \text{PostgreSQL} & \text{PostgreSQL License (BSD-like)} & \text{Unrestricted} \\ \hline \text{DuckDB} & \text{MIT} & \text{Unrestricted} \\ \hline \text{SQLite} & \text{Public Domain} & \text{Unrestricted} \\ \hline \text{ClickHouse} & \text{Apache 2.0} & \text{Unrestricted} \\ \hline \text{Apache Spark} & \text{Apache 2.0} & \text{Unrestricted} \\ \hline \text{Trino} & \text{Apache 2.0} & \text{Unrestricted} \\ \hline \text{PrestoDB} & \text{Apache 2.0} & \text{Unrestricted} \\ \hline \text{Apache DataFusion} & \text{Apache 2.0} & \text{Unrestricted} \\ \hline \text{Polars} & \text{MIT} & \text{Unrestricted} \\ \hline \text{NVIDIA RAPIDS cuDF} & \text{Apache 2.0} & \text{Unrestricted} \\ \hline \end{array}\)

Open source licenses cannot contain DeWitt clauses by definition. The Apache 2.0 license grants users the right to "use, reproduce, and distribute" the software for any purpose¹³. The MIT license permits use "without restriction"¹⁴. The PostgreSQL license explicitly allows use "for any purpose, without fee, and without a written agreement"¹⁵.

This isn't a loophole, it's fundamental to what open source means. You can benchmark PostgreSQL, publish the results, and PostgreSQL Global Development Group has no legal recourse (nor would they want any).

DuckDB goes further, actively encouraging benchmarks. Their FAQ recommends using preview releases for fairness, references their academic paper "Fair Benchmarking Considered Difficult" on methodology pitfalls, and asks only that you report version numbers¹⁶.

If you're benchmarking open source databases, you have nothing to worry about legally. Publish away.

Cloud vendors with reciprocal rights (DeWitt embrace)

\(\begin{array}{|l|l|l|l|} \hline \textbf{Vendor} & \textbf{Services} & \textbf{Policy Summary} & \textbf{When Adopted} \\ \hline \text{AWS} & \text{Redshift, Athena, Aurora} & \text{May benchmark; must provide replication details; AWS may benchmark you} & \text{Pre-2021} \\ \hline \text{Microsoft Azure} & \text{Synapse Analytics} & \text{May benchmark; competitors must share methodology and waive own restrictions} & \text{Pre-2021} \\ \hline \text{Google Cloud} & \text{BigQuery, Spanner, AlloyDB} & \text{May benchmark; reciprocal rights; hyperscaler exclusion} & \text{Updated 2022-2024} \\ \hline \text{Databricks} & \text{Databricks SQL, Delta Lake} & \text{May benchmark; must share methodology; grants reciprocal rights} & \text{Nov 2021} \\ \hline \text{Snowflake} & \text{Snowflake} & \text{May benchmark; reciprocal rights apply} & \text{Nov 2021} \\ \hline \text{SingleStore} & \text{SingleStore} & \text{May benchmark; must be reproducible; reciprocal rights} & \text{Nov 2023} \\ \hline \text{Firebolt} & \text{Firebolt} & \text{May benchmark (no DeWitt clause per terms review)} & \text{Unknown} \\ \hline \end{array}\)

These vendors permit benchmark publication with reciprocal rights provisions, what Databricks branded the "DeWitt Embrace" approach. Notably, AWS and Azure had these terms in place before Databricks coined the phrase; the hyperscalers never adopted traditional restrictive clauses. The key elements are consistent across vendors:

1. You may benchmark and publish results

2. You must provide methodology sufficient for reproduction

3. By publishing, you grant the vendor reciprocal benchmarking rights

The AWS Service Terms¹⁷ state that you may benchmark and disclose results, but you must include "all information necessary to replicate such Benchmark," and AWS gains the right to benchmark your products in return. Microsoft's Online Services Terms¹⁸ use nearly identical language.

The reciprocal element is clever: if you're a competitor and you publish benchmarks of their service, you've just waived your own benchmark restrictions. This creates mutual assured transparency, at least among those who choose to engage.

You can benchmark these services freely, provided you document your methodology thoroughly. For most evaluators (who aren't competing database vendors), the reciprocal obligation is irrelevant.

Google Cloud: a recent convert

Google Cloud quietly removed their DeWitt clause. Sometime between 2022 and 2024, they updated their benchmark terms, a change that hasn't been widely commented upon.

The old Google Cloud terms (circa 2022) required customers to "obtain Google's prior written consent" before publishing any benchmark results¹⁹. This was a traditional restrictive DeWitt clause, similar to Oracle's.

The current terms use the same reciprocal approach as AWS and Azure:

"Customer may conduct benchmark tests of the Services (each a 'Test'). Customer may only publicly disclose the results of such Tests if (a) the public disclosure includes all necessary information to replicate the Tests, and (b) Customer allows Google to conduct benchmark tests of Customer's publicly available products or services and publicly disclose the results of such tests."²⁰

No prior approval required, just reciprocity.

The hyperscaler exclusion: Google does maintain one unique restriction:

"Customer may not do either of the following on behalf of a hyperscale public cloud provider without Google's prior written consent: (i) conduct (directly or through a third party) any Test or (ii) disclose the results of any such Test."²¹

This means AWS, Azure, or other hyperscale competitors can't commission benchmarks of Google Cloud services without permission. Independent researchers, enterprises, and consultants are unaffected.

Service-specific restrictions: A few Google Cloud services still have full benchmark prohibitions:

• Cloud NGFW Enterprise (firewall)

• Cloud IDS (intrusion detection)

For these security services, customers "will not disclose, publish, or otherwise make publicly-available any benchmark, or performance or comparison tests."

Note: Some surveys (including Cube.dev's DeWitt Clause list) still categorize Google Cloud as having a restrictive DeWitt clause. This appears to be based on the older terms or the hyperscaler exclusion. For independent evaluators using current terms, Google Cloud is functionally equivalent to AWS and Azure.

You can benchmark BigQuery, Spanner, AlloyDB, and other Google Cloud database services freely, same rules as AWS and Azure. The hyperscaler exclusion only matters if you're literally acting as an agent for a competing cloud provider.

Still restricted: the holdouts

\(\begin{array}{|l|l|} \hline \textbf{Vendor} & \textbf{Restriction} \\ \hline \text{Oracle Database} & \text{Prior written consent required} \\ \hline \text{Microsoft SQL Server (on-premises)} & \text{Prior written approval required} \\ \hline \text{MariaDB SkySQL/Xpand} & \text{Restrictions in cloud terms} \\ \hline \text{Elastic Cloud} & \text{DeWitt clause in DBaaS terms} \\ \hline \text{InfluxDB Cloud} & \text{Publication restrictions} \\ \hline \text{PlanetScale} & \text{Benchmark restrictions} \\ \hline \text{Couchbase Capella} & \text{Restrictions in cloud offering} \\ \hline \end{array}\)

Oracle maintains the original DeWitt clause that started it all. Microsoft SQL Server (the on-premises product, distinct from Azure SQL/Synapse) still requires written approval. Several cloud-only database services restrict benchmarks, particularly for their managed offerings even when the underlying technology is open source.

Why do these vendors hold out? I see a few patterns:

• Legal inertia: The clause has always been there; removing it requires someone to affirmatively decide to change it

• Performance concerns: If you're not confident in your performance, transparency is risky. The vendors who embrace open benchmarking tend to be the ones winning benchmarks.

• Customer lock-in: Existing customers can't easily compare alternatives when they can't see independent comparisons

• Different market dynamics: Enterprise sales happen behind closed doors; public benchmarks matter less when you're selling to procurement committees

The pattern is telling: vendors confident in their performance actively encourage benchmarks. Vendors who restrict them are telling you something about their confidence level.

---

Practical guidance

For database evaluators

Here's what I do before publishing any benchmark, and what I recommend you do too:

1. Check the license first. Before running any benchmark you plan to publish, read the terms of service. A few minutes of legal review can save significant headaches. I learned this the hard way.

2. Open source is always safe. PostgreSQL, DuckDB, ClickHouse, Spark: benchmark freely. This is one reason I favor open source for my work.

3. DeWitt embrace vendors require methodology. For AWS, Azure, Databricks, Snowflake, and similar services, document everything:

   • Hardware specifications (instance types, CPU, RAM, storage)

   • Software versions (database version, OS, drivers)

   • Configuration (all non-default settings)

   • Data generation process

   • Query execution methodology

   • Commands to reproduce

   Note: Using BenchBox makes it very simple to meet these requirements.

4. Restricted vendors require permission. For Oracle, SQL Server, and similar products, either get written approval, anonymize results ("Database A" vs "Database B"), or don't publish.

5. When in doubt, ask. Vendor legal teams can clarify what's permitted. Get it in writing.

For enterprises

Questions to ask during vendor evaluation:

• "Can we publish benchmark results comparing your product to alternatives?"

• "What restrictions apply to sharing performance data with our industry peers?"

• "Will you provide a benchmarking waiver as part of our contract?"

A vendor's answer tells you something about their confidence in their product.

For content creators

\(\begin{array}{|l|l|} \hline \textbf{Vendor Category} & \textbf{What You Can Publish} \\ \hline \text{Open source} & \text{Anything} \\ \hline \text{DeWitt Embrace} & \text{Anything, with full methodology} \\ \hline \text{Conditional (BigQuery)} & \text{Anything, unless working for a hyperscaler} \\ \hline \text{Restricted} & \text{Nothing without permission, or anonymize} \\ \hline \end{array}\)

The vector database exception

The rise of AI and vector search has created a new category of databases, and a new set of benchmark restrictions.

A 2024 survey of vector databases found that several cloud offerings restrict benchmarks even when their open source cores don't:

\(\begin{array}{|l|l|l|} \hline \textbf{Database} & \textbf{Open Source Status} & \textbf{Cloud Benchmark Policy} \\ \hline \text{Milvus/Zilliz} & \text{Apache 2.0} & \text{Unrestricted} \\ \hline \text{Qdrant} & \text{Apache 2.0} & \text{Unrestricted} \\ \hline \text{Weaviate} & \text{BSD-3} & \text{Unrestricted} \\ \hline \text{Pinecone} & \text{Proprietary} & \text{Removed restrictions May 2024} \\ \hline \text{Elasticsearch} & \text{SSPL} & \text{Elastic Cloud restricted} \\ \hline \text{Couchbase} & \text{Apache 2.0 (core)} & \text{Capella cloud restricted} \\ \hline \end{array}\)

The pattern is notable: vendors restrict benchmarks specifically for their managed cloud offerings, even when the underlying database engine is open source. This suggests the restriction is about protecting cloud margins rather than the technology itself.

---

What should change

DeWitt clauses are anti-consumer and anti-competitive. Here's what I think needs to happen:

• Academic exemptions should be universal. Researchers should be able to publish benchmark results without fear of legal action. Some licenses technically permit academic use; this should be explicit and standard across the industry.

• The market is moving toward transparency, and that's good. The vendors with the best performance actively encourage benchmarks. The correlation isn't coincidental. Transparency favors the winners. The holdouts should take note.

• TPC should continue expanding. The Transaction Processing Performance Council's acceptance of cloud-native benchmarks (starting with Databricks' 2021 TPC-DS submission) has helped legitimize comparative performance testing. More standardized, audited benchmarks across more workloads would benefit everyone.

• Enterprises should push back. During your next vendor evaluation, ask: "Can we publish benchmark results comparing your product to alternatives?" A vendor's answer tells you something about their confidence. Consider adding benchmarking rights to your contract negotiations.

---

Conclusion

The DeWitt Clause, born from Larry Ellison's fury at unflattering benchmark results in 1983, spread throughout the database industry for four decades. But the landscape has fundamentally shifted since 2021.

Today, the majority of databases can be benchmarked and published freely:

• All major open source databases have no restrictions

• All major cloud warehouses (AWS, Azure, Google Cloud, Databricks, Snowflake) permit benchmarks with methodology disclosure

• Traditional enterprise vendors (Oracle, SQL Server) maintain restrictions

For anyone evaluating databases in 2026, this is good news. You can conduct independent, reproducible performance testing of most modern data platforms and share your findings with the community.

Before your next database evaluation, check the vendor's benchmarking terms. If they restrict publication, ask yourself what they're hiding. And if you're negotiating an enterprise contract, push for benchmarking rights. The more customers demand transparency, the faster the holdouts will fold.

All of the benchmarking research from Oxbow Research is published with complete reproduction instructions via BenchBox.

The database industry's four-decade experiment with benchmark censorship is ending. Not with a legal ruling or regulatory mandate, but with competitive pressure from vendors confident enough in their performance to welcome scrutiny.

I hope David DeWitt approves.

---

Methodology

This survey was conducted in January 2026 by reviewing:

• Vendor terms of service and acceptable use policies

• License agreements and service-specific terms

• Historical vendor announcements and blog posts

• Third-party analyses from Cube.dev, benchANT, and others

Terms of service change frequently. I recommend verifying current terms before publishing any benchmark results. Links to primary sources are provided in the footnotes.

This post reflects my good-faith understanding of vendor policies and does not constitute legal advice.

---

Footnotes

1. Bitton, D., DeWitt, D. J., & Turbyfill, C. (1983). Benchmarking database systems: A systematic approach. VLDB Conference Proceedings

2. That time Oracle tried to have a professor fired for benchmarking their database - Dan Luu

3. The DeWitt Clause: Why You Rarely See Database Benchmarks - Brent Ozar, May 2018

4. DeWitt clause, or Can you benchmark a database and get away with it - Cube Blog

5. Oracle Technology Network License Agreement - Oracle

6. Microsoft SQL Server License Terms - Microsoft

7. The DeWitt clause's censorship should be illegal - David A. Wheeler

8. Databricks Sets Official Data Warehousing Performance Record - Databricks Blog, November 2021

9. TPC-DS Full Disclosure Report for Databricks - TPC, November 2021

10. Eliminating the DeWitt Clause for Database Benchmarking - Databricks Blog, November 2021

11. Snowflake vs Databricks: TPC-DS Benchmark Wars - LinkedIn, November 2021

12. Eliminating the DeWitt Clause for Greater Transparency in Benchmarking - SingleStore Blog, November 2023

13. Apache License 2.0 - Apache Software Foundation

14. MIT License - Open Source Initiative

15. PostgreSQL License - PostgreSQL Global Development Group

16. DuckDB FAQ - DuckDB

17. AWS Service Terms - Amazon Web Services. Section on Benchmarking.

18. Microsoft Online Services Terms - Microsoft

19. Google Cloud Service Specific Terms - Google Cloud. Section 7 (Benchmarking). Note: Terms were updated between 2022-2024 to remove prior written consent requirement.

20. To Benchmark Vector Databases or to Get Sued for breaching a DeWitt Clause? - benchANT, April 2024

21. DeWitt Clause discussion - Google Developer forums - August 2022. Discusses older Google Cloud terms that required prior written consent.