In this blog I am going to review the challenges that we face when trying to apply data science in management of renewable energy grids and to tell you a story about our journey toward the platform that can support future digital twins in this sector.

How renewables influence energy sector?

Before we start to talk about AI solutions let’s first take a look at the current situation in energy sector and its ongoing transformation due to introduction of renewable energy resources. It is very important to understand how this is going to change the world and how important is to harness AI to solve the challenges that are in front of us.

As we know, traditional power grids consist of a limited number of power plants that use high voltage transmission systems to transport energy across the countries. Distribution systems operators or DSOs then distribute electricity to every single house, building or factory within their micro-grids. Unfortunately, storing electricity is still very limited, so DSOs have one more role: to balance energy supply and demand between power plants and consumers.

However, with introduction of easily accessible renewable energy sources every household will become, not only a consumer, but also a producer of electricity, which will totally change the game. Can you imagine how complicated will be to balance supply and demand in situation when you have billions of producers and consumers all over the world. This transformation will consequently force DSOs to transform from energy transmitters into orchestrators of renewable energy grids.

Challenges for DSOs

Let’s see what are the main challenges that DSOs are facing right now.

The first one is volatility of renewable energy sources. As we know, solar collectors and wind turbines strongly depend on weather conditions, which makes the production from these sources intermittent and uncontrollable.

Another one is so called “duck curve”. Due to many social and natural factors, there are two peaks in daily energy demand. One is in the morning, and another is in the evening. However, the peak in production of solar energy is around noon, right in the period of low energy demand.

And third, new loads, such as electrical vehicles and heat pumps, introduce periods of very high demands which can overload the existing grids and cause failures and outages.

It is now clear that DSOs will need very powerful tools to deal with all these challenges, to predict energy productions and demands, and to optimize their operational plans. The solutions that scientists can offer are digital twins.

What are digital twins?

Digital twin is a dynamic, virtual representation of a physical asset, product, process, or system. Put simply, it is a software-hardware system that behaves exactly like its real-world counterpart. Using data from multiple sources, a digital twin continuously updates itself to represent the current working condition of the object or process. Such a system enables us not only to monitor the current state of the real system, but also to experiment with its behavior in different scenarios. It’s like a playground where we can play with a copy of the real system in order to find the most desirable behavior.

All that models

The core components of every digital twin are computational models. There are different types of models, but we can divide them roughly into two categories. Physically based and data-driven models.

Physically based models rely on our understanding of physics and our knowledge of physical laws. They are very robust and accurate, but require significant time to be created, and often are computationally very demanding, which is not acceptable in near real-time applications.

On the other hand, there are increasingly used data-driven or machine learning models. They use measurements from an object or process to obtain relationships between model inputs and outputs. Data-driven models are usually very fast and appropriate for near real-time applications, but require a vast of data to be trained, and sometimes can give predictions that are not realistic.

Therefore, in last few years there is an extensive work on making hybrid models that uses physical laws to improve learning process of ML models and to keep them physically feasible.

In this blog we will focus on ML models of renewable energy resources.

How to stay in tune?

Today, more and more experts become very familiar with making machine learning models using various methods, such as neural networks, decision trees, and so on. However, when it comes to real-world application of the models we have created, the main challenge that we face is how to stay synchronized with everchanging reality. Every data scientist knows very well how tedious is to create ML model, because there is a lot of parameters that you should adjust to obtain a model optimized for the specific problem. In addition, at the end you can never be sure if the model you have created is the best one. According to Gartner’s research, 60% of all ML projects never enter production stage due to complicated generation of ML models, difficult maintenance, and complexity of integration into existing processes.

So, is it enough to employ static ML models or we have to focus on systems capable of adapting to evergrowing datasets and new data sources in near-real-time?

The solution

Few years ago, we decided to develop a system that would be able, based on currently available data, to automatically generate a deep neural network optimized for the specific problem. To obtain optimal architecture of a neural network our solution uses evolutionary algorithms running on a distributed computing environment. Through processes of selection, crossover and mutation, a population of different neural networks evolve until they reach an optimal architecture for a given set of inputs.

And the Blackfox was born.

Blackfox is a platform for automated creation, monitoring and maintenance of ML models, based on an original combination of machine learning, artificial intelligence, and high-performance computing methods. Its Foxbrain AutoML module automates most of the tasks associated with constructing and implementing a machine learning pipeline that would normally be engineered by specialized teams. It significantly accelerates modeling process by automated feature selection, scaling and optimization of the model architecture and hyperparameters.

Blackfox in renewables

One of the successful applications of Blackfox in the renewable energy sector was the development of the model for prediction of hourly energy demand in Ontario state, which was conducted together with one of the world-leading smart energy storage companies. The goal was to predict the time and the intensity of the daily peak in energy demands, so battery owners can optimize their usage in order to avoid penalties for the participation in the peak.

Using Blackfox, a model created in only 12 hours has shown significantly better performance than the models made by the team of data scientists in a matter of weeks.

Asset and operational digital twins

So far, we were talking only about models of one single process or object, which enables us to develop so-called asset digital twins. An asset digital twin provides a window into single component, and can be used to enhance performance, to reduce operating costs, or for predictive maintenance. However, if we want to have integrated understanding of a system as a whole, we need so-called operational digital twins.

Operational digital twins offer visibility into the intricate and all-important relationships among machines and workflows. In case of renewable energy grids that means that we need a separate ML model for every single house, or even for every single solar collector, wind turbine or electrical vehicle charger in the grid.

To solve this problem, we have developed an extension to Blackfox, so Blackfox got a tail.

Foxtail MLOps module provides a full automation of production, management, and deployment of predictive models on industrial scale. Keeping thousands of models at hand it provides continuous data synchronization and models health monitoring. In case of drift detection, it automatically retrains the models or invoke Foxbrain to reoptimize model hyperparameters.

Coupled together, Foxtail MLOps module and Foxbrain AutoML module make the Blackfox a “Turn on and forget” ML solution that blends large-scale model management with a fine tuning of every single model. It can handle thousands of machine learning models of various assets, keep them always up-to-date and accurate, which paves us a path towards the application of operational digital twins in renewable energy grids.

Smarticity

Smarticity is our new platform for optimal management of renewable energy grids. It automates finding an optimal pattern in energy consumption and production in case of facilities with renewable energy resources.

Smarticity collects and stores all the relevant data about production and consumption within a grid, with the aim to use them for creation of machine learning predictive models of various resources of interest. Creation of the models is delegated to the Blackfox, which will automatically create adequate ML models based on the data acquired from the assets, but also based on publicly available data such as weather conditions, public holidays, social events, and so on. The models created are stored in the model storage, ready to be used for consumption and production predictions. Once we have the models that can accurately predict production and consumption, we can easily simulate any realistic or hypothetic operation plan and asses its effects on the grid efficiency and costs. Based on these simulations, our optimization service OSICE will search for the optimal operation plan in terms of the selected goals. The obtained optimal plan then can be manually or automatically applied to achieve defined goals.

Don’t forget that Blackbox continuously monitors all the models, so in case of changes in the grid or in the electricity market, the predictive models will be automatically retrained or totally reoptimized. This way, Smarticity provides DSOs not only with a reliable digital twins of their micro-grids, but also with a powerful operation optimization tool that can help them to find an optimal pattern of energy production and consumption in terms of efficiency, environment protection, safety, and profit.