Luma Energy had 800,000 smart meters and almost nothing to show for them, interval data arrived in four-hour file drops and outages were detected by customer phone calls. We built a streaming analytics platform on AWS that detects outages in 45 seconds and recovered 2.6 million euros in grid losses in its first year.
How we took Luma Energy from the problem that hurt to the numbers above.
A regulatory mandate had pushed Luma Energy to deploy 800,000 smart meters across its distribution area, but the investment stopped at the meter. The vendor head-end system exported 15-minute interval reads as files every four hours, which the billing team ingested overnight, so a network generating 77 million reads a day informed exactly one business process, a day late. Outage detection still worked the way it had for decades: customers phoned, a dispatcher triangulated from the complaints, and a crew went looking.
Grid losses were the quiet financial wound. Roughly 9% of energy entering the network was unaccounted for, a blend of technical losses, metering faults, and theft, but with consumption data trapped in files, nobody could say which feeder was losing what. Load forecasting for the day-ahead market ran on spreadsheets maintained by two analysts, and forecast misses were settled at penalty prices.
The engineering constraints were characteristically utility-grade: the OT network hosting the head-end was strictly separated from IT, the regulator's unbundling rules dictated exactly which business units could see which data, and the head-end vendor's roadmap moved on multi-year cycles. Replacing the head-end was off the table; the platform had to be built around it.
We began with a four-week consulting engagement rather than code, mapping the OT/IT boundary, the regulatory data-access matrix, and the actual latency budget of each use case, which established that near-real-time was necessary for outage detection but 15-minute batch was perfectly adequate for loss analytics, a distinction that shaped the whole architecture and roughly halved its cost. The head-end kept its file exports, now picked up every few minutes; a growing population of LTE-M meters streams directly through AWS IoT Core.
Both paths converge on Kinesis, where a validation-estimation-editing pipeline cleans reads before anything downstream consumes them, utilities live and die by VEE, and doing it in stream meant billing, analytics, and operations finally shared one version of the truth. Outage detection clusters meter last-gasp messages by feeder and substation topology: a median detection time of 45 seconds, against 41 minutes for the first customer call, with crews dispatched to a mapped fault segment instead of a neighborhood.
Loss analytics compares energy delivered at each feeder head against the sum of downstream metered consumption, ranked into a weekly investigation queue for the revenue-protection team. Day-ahead load forecasting moved from spreadsheets to gradient-boosted models on SageMaker, trained per substation with weather and calendar features, and the two analysts who owned the spreadsheets now own the models, that handover was a project deliverable, not an afterthought.
The platform is serverless-first because the load profile demands it: meter traffic arrives in synchronized 15-minute pulses with near-silence between them, which is exactly the shape Lambda and Kinesis absorb without paying for idle capacity. Hot telemetry lives in Timestream for operational dashboards; everything lands in an S3 data lake cataloged with Glue, where Athena serves the regulator's reporting queries and the data-science workloads. A unidirectional gateway bridges the OT boundary, data flows out of the head-end network, and nothing flows in.
Access control mirrors the unbundling rules in IAM: distribution operations, retail, and revenue protection each see only their permitted slices, enforced at the platform layer rather than by policy documents. The entire estate is Terraform-managed across two AWS regions, with the second region serving as a warm standby for the outage-detection path, the one workload whose downtime puts crews back on phone-call triangulation.
Technologies
“Idealdevs started by telling us which problems did not need real-time data, which probably saved us half the budget. The outage map alone changed how our control room works.”
Marta Vieira
Head of Grid Operations, Luma Energy
