Open source · Apache NiFi · 2026
From the machine to Power BI with Apache NiFi
- Problem
- In many plants OEE is worked out by hand in Excel once a month: by the time the number looks bad nobody remembers what happened, and each supervisor logs stoppages their own way.
- What I did
- A complete pipeline with made-up data: three simulated machines publish over MQTT whether they are running or stopped, the reason and the good and bad parts. Apache NiFi 2 reads it, validates the JSON against a schema, applies the business rules, adds the shift and the machine name from the PostgreSQL master table and stores it. Bad messages go to an error queue, each with its reason.
- Result
- OEE (availability × performance × quality) per machine and shift, downtime by reason and scrap come out of SQL views ready for Power BI. Everything starts with a single
docker compose upin about two minutes and, in testing, none of the 12,960 history messages was lost. It includes the NiFi flow template in JSON and a guide to scaling NiFi: back pressure, queue prioritization and when to move to a cluster.
Apache NiFi 2 · MQTT · PostgreSQL · Power BI · Python · Docker
Related service: Power BI for factories: OEE, downtime and scrap
If your team has tasks that keep repeating or data that doesn’t add up, tell me about it: info@arcedatalab.com