DataKitchen, Inc.

Home
United States
Lexington, MA
DataKitchen, Inc.

Say goodbye to the complexity of writing data quality validation tests.

TestGen takes care of that for you, using AI to automate data test creation, ex*****on, and profiling.

11/01/2026

Stop treating a data contract like a legal document. Treat it like a receipt.

A receipt is cheap proof that what you ordered showed up the way you expected. Every clause maps to a test, so the contract either passed or failed last night. That turns "your data seems off" into "row 47 promised a non-null ship_date, and 12% of rows are null this week."

📉 The vendor's file agreement is a spreadsheet nobody has opened in eight months.
🕳️ A forecasting model won't notice the revenue column started including refunds.
🔒 You can't defend an AI-driven decision to an auditor without proof of what went in.

So write one contract, on one dataset you already ship.

🎯 Run it before the Friday export fires. If it fails, the export doesn't ship.
🔌 Hand your supplier open source TestGen and the contract. Ten minutes, zero dollars.
📈 Pass and fail over time is the receipt for quality work nobody else sees.

Read the full story here:
https://hubs.ly/Q04CgVnf0

10/31/2026

Working at 70% today beats perfect someday.

That's the last of eight trade-offs in the DataOps Quality Manifesto. We've run the 12-month assessment that produced a document and fixed nothing. We've heard about bad data from the customer at 9am. These are what we value now instead.

🧪 Automated tests over hope and manual checks.
📉 Test coverage over lineage diagrams.
🔄 Continuous practice over one-time projects.

Five more, and 24 principles behind them.

🎯 Generated tests over hand-written rules.
🔦 Quality stations on the line over inspection at the end.
🔑 Measurement over opinion. Without test results, you're just another person with an opinion.

Read it and sign it:
https://hubs.ly/Q04CgTPW0

10/30/2026

A check fails at 6:14am. Finance opens the dashboard at 8.

You have about two hours to pick one of four moves: stop, quarantine, retry, or publish. If you haven't decided which in advance, the orchestrator's default decides for you.

🔄 A retry on a load that isn't idempotent turns 1,000 rows into 2,000.
🕳️ Fabric's ON MISMATCH DROP deletes failing rows and logs a number. The gold table looks clean because the evidence left.
⏳ An alert arrives after the bad data has already moved.

So decide per rule, with three questions: how bad is a wrong number, can you pull it back after someone uses it, and how old can the data be.

🎯 Halt when the data would be silently wrong. Warn when it would be obviously incomplete.
🔑 Quarantine keeps a count that reconciles: 61,948 source orders equal 61,465 migrated plus 483 quarantined.
⚡ Retry only timing failures. Bad source values come back identical on the second run.

Read the full answer here:
https://hubs.ly/Q04C2Tr30

10/30/2026

Your Fabric pipeline finished green. The Power BI report is wrong anyway.

A source system changed a column type on Tuesday and nothing in the chain was watching. TestGen 5.92.1 now connects to Microsoft OneLake, in open source and Enterprise, through the Fabric SQL analytics endpoint. Same tests and scorecards you run on Snowflake or SQL Server.

🕳️ Shortcuts and mirroring land bronze through a pipe you didn't build, so you get no logs.
⏳ Direct Lake reads gold straight from OneLake. A bad write hits the dashboard in seconds.
📉 100 tables per layer needs about 5,000 tests. By hand that's over a year of work.

So TestGen profiles every layer and generates the tests.

🎯 Bronze gets monitored for freshness, volume, and schema on every refresh.
🔌 Tripwires between bronze, silver, and gold keep gold on the last good data.
🔒 Read-only SQL, and only aggregates come back. No rows leave your tenant.

Read the full story here:
https://hubs.ly/Q04CgTTT0

10/29/2026

The first brand gets a platform. The second brand finds out what kind.

If the first was built as a set of reports, the second is a rebuild. One small pharma we run commercial data for found that at most 40% of its two-year-old warehouse could be reused for its second indication.

📉 Patients on therapy go from fewer than a hundred into the thousands.
🕳️ Claims and physician reference data, never needed for the first brand, come into scope.
🔄 An NDC in the wrong market basket shifts both group totals, and nothing errors.

So build brand one on conformed physician, payer, product, time, and territory dimensions, and brand two adds rows and fact tables instead of a schema.

📈 That team absorbed the second indication's data work while its monthly hours trended down.
🎯 Celgene ran hundreds of data sets through one star schema, covered by more than 20,000 automated tests.
🔦 An Aggregate Balance test against the prior delivery flags both basket totals the day an NDC moves.

Read the full answer here:
https://hubs.ly/Q04C2TrV0

10/29/2026

📅 Webinar: Tuesday, November 17, 12:00 PM ET / 5:00 PM GMT

Your supplier changed a column, and your dashboard broke.

A data contract on a wiki page tests nothing. On November 17, Chris Bergh builds a real one live in open source TestGen, then shows where to run it so the next column change gets stopped at the door.

📉 A column type changes upstream, and bronze takes it without a word.
🕳️ The contract sits in a doc, so nothing ever runs it.
⏳ You find out when somebody in finance does.

TestGen writes 80% of the tests a contract needs in 60 seconds. You edit the rest in the UI.

🎯 At the supplier, before they send you the file.
🔦 On ingest, before bad rows land in bronze.
⚡ In CI, before a code change reaches your customer.

Register Now:
https://hubs.ly/Q04CsMxD0

10/29/2026

Venture-funded open source has a timer. Elementary's went off on October 19.

Boomi bought Elementary that day. Elementary raised its seed money in 2022, and a venture fund has to return its money within the life of the fund. The open source license keeps past releases yours. The roadmap, the maintainers, and every future version belong to whoever owns the company.

📉 Great Expectations raised $64.5M. GX Cloud went to FICO and shut down June 1.
🔒 HashiCorp moved Terraform off open source in 2023, then sold to IBM for about $6.4B.
🕳️ Metaplane went to Datadog, Databand to IBM, Synq to Coalesce. Kensu closed.

So before you build tests on an open source tool, ask who's waiting for the exit.

🎯 Who funds it, how much has gone in, and do the investors expect a sale?
🔑 A profitable company with no investors can still be sold. It just isn't on a fund's deadline.
📈 DataKitchen raised $0 and is profitable. TestGen is Apache 2.0.

Read the full story here:
https://hubs.ly/Q04CgTgR0

10/28/2026

Profiling five tables against a live transactional database slowed one customer's application for 20 minutes.

Aurora is usually an application's database first and an analytics source second. That order matters, and Aurora has the fix built in. The reader endpoint, the one with cluster-ro in its name, balances read-only connections across your Aurora Replicas.

🔒 TestGen needs two grants: USAGE on the schema and SELECT on its tables.
🧪 AWS Glue Data Quality doesn't support the RDS and Aurora table type, so it reaches Aurora through JDBC or a Glue ETL job.
⏳ No replicas? The reader endpoint connects to the primary, so profile off-hours.

So the answer walks through setup: a read-only role on the reader endpoint, then profiling, tests, and monitors.

🔦 Profiling captures 55 characteristics per column and flags 32 kinds of hygiene issues.
🎯 TestGen generates 34 test types from the profile, and every test runs as SQL inside Aurora.
🔑 Open source is free for one connection, with unlimited tables.

Read the full answer here:
https://hubs.ly/Q04C2SFC0

10/28/2026

The 2026 Data Contracts Software Landscape, open source and commercial. Few of the tools write the tests for you.

We put every tool through two filters: can an Open Data Contract Standard (ODCS) file go in, get edited, and come back out intact, and do the tests fail when a term breaks? Five open source tools and four commercial vendors pass.

📉 Many tools build a contract from the schema alone: types and required flags, nothing more.
🕳️ Ten tables with 20 columns each need at least 430 contract tests. By hand, that's 72 hours.
🔒 OpenMetadata and Collate drop servers, pricing, support terms, and custom properties on ODCS export.

So we scored all nine in two tables, with GitHub stats and published prices.

🎯 Profiling plus automatic test generation is what gets a contract past a schema check.
🔑 One contract runs in four places: at the supplier, on ingest, on delivery, and in CI/CD.
📈 The post has a calculator. Put in your tables and columns and see your number.

Read the full story here:
https://hubs.ly/Q04CwGrx0

10/28/2026

An LLM is trained to be plausible, not accurate. Nothing inside it checks whether what it said is right.

That's the line from Chip Bloche's AI 101 for data quality and governance people, and the recording is up. Chip built TestGen. He spends an hour on how LLMs, agents, prompts, and context work, and where each one turns into a data quality problem.

🕳️ Shadow pipelines show up in departments that never built one before.
🔄 Chat answers come from a pipeline that's gone the moment the chat ends.
🧪 The person who created something isn't the right person to validate it.

His answer is the one you already know: govern the context like data.

🎯 Accuracy, timeliness, and validity apply to the context an agent reads.
🔦 A good data prompt has goals, specs, deliverables, and verification steps. It sounds a lot like programming.
🔑 The full transcript, 41 chapters, and all 52 slides are on the page.

Watch the recording:
https://hubs.ly/Q04CgT8T0

Address

405 Waltham Street #101
Lexington, MA
02421

Website

https://datakitchen.io/

Alerts

Be the first to know and let us send you an email when DataKitchen, Inc. posts news and promotions. Your email address will not be used for any other purpose, and you can unsubscribe at any time.