This blog post is a recap of a presentation held at the 2023 Knowledge Graph Symposium about BKW Energie's smart meter operations, the data challenges they experienced and how knowledge graphs supported this complex use case.
How knowledge graphs helped BKW resolve major data choke points for their smart meter operations
During the 2023 Knowledge Graph Symposium, Peter Hutzli, formerly IT Responsible - Energy Metering at BKW Energie, shared his experience in using knowledge graphs to resolve data bottlenecks in BKW Energies’ smart meter operations on the horizon of a major scaling project.
In this blog post, we’ll provide a recap of the presentation and examine how the final knowledge graph solution resulted in significant time-savings and improved efficiency for BKW Energie, leading to a sound foundation for scaling its smart meter operations. Keep reading!
Table of contents
About BKW Energie’s smart meter operations
BKW Energie is a Swiss energy production and distribution utility based in Bern, Switzerland, with a history spanning over 100 years. It has subsidies in Germany, France and Austria, and production facilities in several other European countries. While a leader in energy in Switzerland, the company has branched into building automation and large infrastructure projects in the last 10-15 years.
Smart meters are electronic devices that measure electric energy consumption inside homes, buildings and production sites, and automatically communicate these measurements to the energy supplier. The measurements are also used to calculate energy balances, ensuring there is always equal input and output to prevent any brownouts or network instability.
The smart meters deliver daily energy readings over a 4G network. The readings are received by the head end system (HES) and then pushed into the measurement management system (MDM), which is responsible for reviewing and addressing data quality issues. The readings are then sporadically delivered to the BKW SAP system, which generates the bill sent to the customer every quarter. In addition, smaller network operators have access to an online portal where they can access and review the data, providing metering as a service.
This process means that systems must be carefully preconfigured to accept these readings.
At the start of this project, BKW Energie had already installed around 80,000 smart meters across its service area and planned to grow this operation by a factor of five — aiming to install a total of 400,000 smart meters.
Data flow blockages and lack of visibility
Hutzli spoke of data issues that existed at the time when he joined BKW AG - before there were even plans to roll out more smart meters. “When I came to the company, there were a couple of issues, [that were a] challenge. In the fall of 2020, we had some rough times, with a tiger team having to fix data flow issues due to a snowball effect between a multi-threaded application and a database.”
There were also problems in the master data: discrepancies would sometimes emerge when data was being distributed from one system into another. As a result, replicated data started to diverge between systems. Even though there were very few incidents overall, those incidents had material consequences. Specifically, when BKW purchased new smart meters, they would end up sitting unused in the stock for months due to the existing data issues, resulting in financial losses and dissatisfied customers.
Troubleshooting issues was also cumbersome and tiring, often requiring the preparation of ad-hoc reports, for example. With no central platform available to oversee smart meter data, attempting to investigate and resolve any issues involved long, messy email chains between numerous individuals. Hutzli remembers up to 70 emails being sent back and forth to try to clarify one single matter.
The issue was that not one person could identify where issues existed or began because there was no overview of the data. Simultaneously, there was mounting pressure to resolve the master data issue quickly due to the impending smart meter rollout. Management made clear the urgency to fix these data bottlenecks as they would have a substantial impact on the launch — especially considering the scale of the project and the number of smart meters planned to be installed.
Proposing a path forward
After assessing the situation, a proposal was put forth to develop a knowledge graph-powered solution to resolve these persisting data challenges. The proposal involved aggregating data from four sources into a single graph so that all parties involved could immediately spot any discrepancies and inconsistencies in the information.
In this proposal, the team demonstrated how the SAP and Siemens data models they employed exhibited similarities but weren’t identical, highlighting the application-centric approach to data within the organization, which can foster an environment susceptible to data gaps. They would have to reverse-engineer the information in order to gain a comprehensive view of their data.
The proposal suggested not extracting all data from these sources, but rather just a slice of the most pertinent smart meter information on a business level. This subset would be combined into a single graph, which would allow for the evaluation of any overlap or gaps.
The knowledge graph solution
The knowledge graph solution proposed would consolidate master data from four sources into a central platform using ETL pipelines. Beyond data consolidation, utilizing a semantic knowledge graph enriches data with semantics and the valuable context and meaning that enables end users to gain a better understanding of data relations and extract hidden insights. metaphactory was used to build a low-code portal on top of the graph so that data could be visualized and presented in a way that all stakeholders could easily consume and interact with. By democratizing this data, different users and stakeholders would be able to quickly help solve data inconsistency issues.
To gain senior management’s approval of this project, a business case was presented that highlighted the potential cost savings the organization could achieve by taking on the solution. Their projections revealed that BKW Energie would break even on its investment after less than two years and then see continuous savings in the years following. The savings are realized by early detection of data issues and significantly accelerating troubleshooting, namely reducing the amount of unnecessary back-and-forth communications, reducing person-hours spent trying to resolve the issue and preventing financial losses from buying smart meters that would only be rendered unusable.
Specifically, maintaining the new solution would result in CHF 70k per year of additional expenses, contrasting with savings in operational efforts amounting to CHF 130k per year, resulting in net savings of CHF 60k per year. With an estimated project effort of CHF 157K for the project, this would result in a net present value (NPR) of CHF 100k over four years.
Image: Solution investment projection
One of the qualitative benefits of this solution includes restoring customer satisfaction. For instance, with the existing situation, customers had to endure the frustrating experience of waiting up to six months for their smart meter orders to arrive since BKW would have to first fix the data inconsistencies before being able to deliver the device to the customer.
Another motivation to employ the knowledge graph solution was the approaching smart meter rollout which required an even higher level of data quality.
Set up of the solution
The team was impressed by how quickly management was on board, receiving the approval after only a 20-minute pitch.
First, they built their data pipelines by leveraging the existing GoAnywhere File Transfer solution already established at BKW, which could interface easily with databases and FTP servers. Reto Gmür of FactsMission, a consultant, built a wrapper around tarql, an open-source transformation component. The wrapper provided webhooks for better integration into the pipeline, a quick conversion of the CSV files into Triples and the transfer of the triples into the blazegraph database. The visualization layer from metaphacts resided on top of the stack, providing the user interface.
They built a simple ontology that defined data classes, relations and attributes for more context. Examples of classes are “smart meter class” or “tenant class” (the latter being a result of each source system having its own tenant management system). They imported about 80,000 individual records and tenant data from each of the four source systems.
Once the data ingestion into the knowledge graph was set up, the final step was building the low-code portal. Using the metaphacts platform, they combined widgets with SPARQL queries to visualize the data in tables, charts, diagrams and icons. Through the portal, users could interact with the data in several ways, including viewing and navigating individual records or filtering the whole set of smart meter records. For convenience, users could leave a comment about a specific smart meter or choose to exclude it from the reporting, giving the operations team fine, granular control over the analysis.
The timeline for the project was as follows:
- February → Pitch: The pitch garnered immediate approval and the next three months were spent interviewing data managers and conducting negotiations on processes such as how to retrieve the data or which report or views were needed.
- July → Prototyping
- August → Infrastructure: They began building the infrastructure in August, while simultaneously setting up the CI/CD pipelines and the metaphactory instances.
- September & October → Implementation: September and October were the main implementation months for the portal and data ingestion. In October, they ran their first demo and made iterations based on feedback from colleagues.
- November → Hand-over and documentation
Hutzli remembers:
“The whole net effort in terms of invested time was about five person months. There was help from the infrastructure team [and] help from an external consultant that we used to build this wrapper, but overall it was about five person months. I do think this is quite efficient, compared to the complexity of the issue that we had.”
The new portal
With metaphactory, they were able to create a visual interface residing on top of the knowledge graph, resulting in a central portal that offered operations workers newfound visibility into the combined master data from four source systems. This was a massive improvement over the previous method, which involved manual data exports and stitching data in Excel spreadsheets, tediously reviewing each individual smart meter.
Once they built the application, the operations team gained a precise view of the master data and could interact with it effortlessly through the web portal.
Image: Portal overview of smart reader statuses
The portal offered a visualization that made it easy to assess the state of a smart meter at a glance. Users could access a comprehensive list of all smart meters and search and filter the list using back-end filtering. The portal also displayed the status of each smart meter, and by doing so, operations workers could quickly assess whether everything was running smoothly or if there were any concerns. For example, in the image above, different colors represent the up-or-down states of a smart meter. Source systems were also color-coded—e.g., SAP is shown as orange and Siemens is colored green. They also configured rules that would augment the data based on other dependencies, such as marking a specific meter with a check mark—green for good and red for issues. The purpose of the portal is to surface information to the operations team as quickly as possible and accelerate the analysis of the data.
Clicking on a specific meter would open a page with a photo and records about the meter aggregated from all systems. Each record was listed separately so users could also investigate records from each source system individually. The visualization not only showed the path of the master data but also the path for collecting reading data, enabling a reconstruction of the data flow of the readings. In the example above, you can easily view the source and initial record and see that the system would deliver data to this particular record, which would then deliver readings to another record in the SAP system.
While it seems like a fairly straightforward visualization, the brilliance of the portal became especially evident when ‘zombie’ records appeared, which are stale records associated with deprecated or faulty configurations, often confusing the operations team.
Image: Individual smart meter displaying source systems
Above is an example displaying two records for the same meter found in SAP where the active or inactive status of each meter.
Also, there are three records in the MDM systems. One is linked to the Head End system and one links to the portal. Because all the relevant data from all sources are aggregated into one graph, they could visualize the flow of readings so viewers could easily see the starting point, where the data travels and to which system.
Previously, the process of extracting this information was very tedious and time-consuming, while now, the information and especially the relations are presented on a silver platter.
Learnings
Reflecting on the learnings from this project, Hutzli said, “Most of you understand that data integration with RDF works really well. I love working with triples. You get the data, cut the data into little snippets, [and] throw them into the graph. You don't have to think about tables or JSONs that you have to create or fill, you can just throw the triples in there [and] they find each other. I really appreciate that part. It’s an enormous time-saver to integrate data.”
Additionally, the low-code environment provided by metaphactory sped up the project by 4-10x said Hutzli. “It's almost like you have two or three developers in addition [to] your team when you use this low-code environment on top.”
Another learning outcome was discovering the simplicity of the portal - even workers with non-technical backgrounds could contribute to and operate the solution, fostering collaboration and knowledge democratization among the various stakeholders.
After developing the application, management of the solution shifted from IT workers to business professionals. This allowed the business professionals needing to interact with the data the autonomy to do so independently, thereby saving time and additional steps. For example, the individual who took on responsibility for the project after its completion was a business professional with a background in geography and little IT experience. However, he possessed sufficient curiosity, which was all it took to seamlessly step into this role. While non-technical workers may not be directly involved in configuring or conceptualizing the future capabilities of the tool, they are perfectly capable of running and managing the solution day-to-day, leaving room for the IT workers and management to focus on big-picture data patterns and plans.
“One thing I have to admit is, [I’ve been working] 10 years, 15 years in Excel or [other] databases, [so] my head [was] still preconditioned [to] this “column” thinking. When I integrated all the data sources into the graph, the first thing I did [was] come back to this “column” thinking and make sure that all that the manufacturer serial numbers are the same, that all the operator serial numbers are the same, as I would do in Excel.
Only when I [talked] to the operations people, [and] got this rebuke of ‘Peter, I wouldn't want to be the person who has to sort this out,’ only then I thought that I could actually look for closed loops and open loops in the relationships between the records, representing a healthy or broken data flow for a particular smart meter. I realized that I could look for circles or triangles inside the data, rather than checking fields and strings and so on. I could look for shapes!”
For Hutzli, this closed-circle analysis of the data flow allowed him to rest easy knowing that the right data was always going to the right place.
Summary
After finding significant data choke points in its smart meter operations, BKW turned to a knowledge graph-powered solution to eliminate these bottlenecks and set BKW up to scale the operation — while discovering additional learning outcomes along the way. Utilizing a knowledge graph enabled BKW to streamline the data into a central system and gave managers newfound visibility into the data flow. The knowledge graph solution was able to gather data from four disparate sources and systems, and metaphactory provided a user-friendly visual interface that offered an overview of the data flow. This portal helped workers to identify issues quickly, as well as pinpoint the root cause of the problem. It also improved transparency and accountability when issues arose, thereby reducing excessive email chains and saving significant hours. Finally, it empowered operations workers to interact with the data independently and autonomously - no longer needing to rely on the IT team or resort to paper documentation or spreadsheets. BKW’s knowledge graph solution not only resolved its data issues, but it also established a central portal that led to remarkable time and cost savings.
Try it for yourself!
metaphactory is an industry-leading enterprise knowledge graph platform that helps you to transform data into consumable, contextual and actionable knowledge. Our low-code, FAIR Data platform simplifies capturing and organizing domain expertise, extracting insights from your data and sharing knowledge across the enterprise.
Our standards-compliant knowledge graph platform is equipped to support all the features and capabilities we mentioned above, including:
- Visual ontology editor for extending public ontologies and capturing additional internal or domain-specific knowledge.
- Low-code app building mechanism enabling the creation of a UX tailored to your specific use case.
- An intuitive data curation interface for easy management.
- A page rendering mechanism facilitating seamless integration of the app into the metaphacts website.
- Data publishing as JSON-LD for improved visibility on search engines.
Get your free trial of metaphactory, or speak with our experts to learn how metaphactory can support your specific use case!
