Data-driven decision making means using observed evidence to improve a choice, then measuring what happened. I have come to think that this feedback loop is the main reason companies need data-processing capability. Storing more data has little value on its own, and useful AI depends on the same loop.
The value appears in the decision
A company is data-driven when current evidence is part of how it makes decisions. That evidence must arrive while there is still time to act, and its origin and meaning must be clear enough to check.
A study by Erik Brynjolfsson, Lorin Hitt, and Heekyung Kim examined 179 large publicly traded companies and found that companies emphasizing data-driven decision making had output and productivity 5% to 6% higher than expected from their other investments and use of information technology (Brynjolfsson, Hitt, and Kim, 2011). The sample was limited, and the authors were careful about causality. After accounting for general IT investment, they associated the measured difference with how decisions were made.
Stored data creates business value when it changes what someone does. It may alter a delivery route, a machine setting, a purchase order, a production schedule, or a price.
Three decisions improved by connected data
UPS: the order of today's stops
A delivery driver makes many small decisions on each route. Their effects add up across the delivery network. The order of 120 or more stops changes the distance driven, time spent idling, fuel consumed, and number of packages delivered in a shift.
UPS built its On-Road Integrated Optimization and Navigation system (ORION) around that decision. It combines package-level detail, custom map data, vehicle telemetry, and routing algorithms to produce an efficient route each day. Driver judgement remains part of the system.
In 2017, the first full year after ORION reached all targeted US routes, UPS reported 100 million fewer miles driven, 100,000 metric tonnes less CO2, and $410 million in operating-cost savings (UPS Corporate Sustainability Progress Report, 2017).
Google: the next cooling adjustment
Cooling a data centre is a continuous control problem. Pumps, chillers, cooling towers, weather, and computing load interact in ways that are difficult to capture with fixed rules. Google had already spent years making its data centres efficient, so the remaining improvements were hard to find through engineering intuition alone.
DeepMind used historical readings from thousands of sensors, including temperature, power, pump speed, and control setpoints, to train models that predicted energy efficiency, temperature, and pressure. The predictions were used to evaluate possible control changes against operating constraints before a recommendation was made.
In a live data centre, the system reduced the energy used for cooling by 40% and reduced overall power-usage-effectiveness overhead by 15% (Google DeepMind, 2016). Power usage effectiveness divides total facility energy by the energy used by computing equipment; its overhead is the amount above the ideal score of 1.0. The model relied on historical sensor readings collected with consistent meaning, joined to control settings, evaluated against a defined outcome, and connected back to the system being operated.
Scanfil: the next production introduction
Scanfil's Suzhou electronics factory had an ERP system, but parts of its equipment were disconnected, end-to-end material traceability was limited, and there was no reliable central system for defects and repairs. This made quality control and frequent line changes difficult in a high-mix, low-volume factory.
The site connected 236 machines and made 2,189 product routings available in a manufacturing operations system. That gave the plant a common view of materials, routing, defects, repairs, and live machine data. A Siemens case study reports that average new-product-introduction time fell from 40 hours to 9 hours, allowing four to six introductions in the same configuration, while overall productivity improved by more than 10% (Siemens and Scanfil).
A data-driven company is a feedback system
These examples operate at very different scales, but they have the same structure:
- A recurring decision is named precisely.
- The required observations arrive from the systems that produce them.
- Business rules or models turn those observations into an answer.
- The answer reaches a person or machine while an action is still available.
- The outcome is recorded, so the next decision can improve.
The five steps form a feedback system that can support a person, as ORION supports a driver, or control equipment within defined safety limits. The degree of automation is a design choice; both depend on the same data-processing capability.
Ordinary business processes follow the same pattern. A purchasing manager deciding whether to consolidate supplier spend needs vendor records from several ERPs mapped to the same legal entities. A production supervisor deciding whether to intervene on a loss-making order needs cycle events joined to the order, material cost, and labour rate. A controller answering a margin question needs the transformation and source records behind the number. The decision may happen in a meeting, a dashboard, an ERP screen, or an API. Its foundation is the data flow underneath it.
Timing determines whether the result can change the decision. A correct margin variance delivered two weeks after a shift is useful for accounting. The same variance delivered during the shift can change staffing, staging, or the next quote. We worked through that distinction in detail for per-order manufacturing margin.
AI starts with the same foundation
Data quality determines what an AI system can know and which actions it can support. Treating it as preliminary cleanup understates its role.
An AI system also needs the relationships between facts, the meaning of each field, and the boundaries within which it may act. This includes mapping customer identifiers from three ERPs to the right group structure, joining production orders to machine events through stable keys, and detecting source changes before a result is trusted.
Google could train and test its cooling model because thousands of historical sensor readings were available, the target measure was defined, the operating constraints were explicit, and its predictions could be compared with a live outcome.
We use language models to reduce the work involved in building and querying data flows. They can draft mappings, transformations, and queries from plain-language instructions. Beetl keeps that work visible and requires approval before a pipeline is deployed.
The foundation is mostly integration work
Most companies already have the individual facts, spread across ERP tables, operational databases, SaaS APIs, local machines, files, and spreadsheets. Each system makes sense on its own; useful business questions cross the boundaries between them.
Working across those boundaries requires a few ordinary capabilities: reliable source reads, source-faithful storage, explicit and versioned transformations, validation, queryable results, and visible failures. Rebuilding that path for every new use case is where the cost and fragility accumulate.
Beetl is our attempt to make this operating layer reusable. Today it brings data in from databases, ERP systems, files, spreadsheets, and local operational systems. SQL pipelines transform source-shaped data into validated datasets that can be inspected and queried within Beetl. Scheduling, state, credentials, deployment, monitoring, and recovery run on the same platform.
The source-specific work remains visible: how a system is accessed, which fields map to which business concepts, which rules produce an output, and how datasets depend on each other. Beetl reads source systems without changing them and retains a source-faithful copy before customer-specific transformation. Visible definitions and retained source data make an answer easier to replay, inspect, and reconcile.
Beetl uses the same operating model for recurring management reporting, supplier analysis, production-cost analysis, and other questions that rely on several sources. Each flow reuses the same deployment, monitoring, and recovery machinery while keeping its source-specific rules explicit.
What we want to make possible
We want companies to improve an important recurring decision with current, traceable data before they need a dedicated data-platform team. Sources, mappings, and operating history from the first flow can then be reused for the next one.
Start with one source, one useful dataset, and one decision that matters, then run the flow in production and measure whether the decision and outcome improve. The connected sources, explicit mappings, validated outputs, and operating history can then support the next flow and later AI systems.
We are building Beetl because operational facts still take too long to reach the decisions they should inform. Companies should be able to improve those decisions one data flow at a time and reuse the same work for analysis and AI.