Human-in-the-loop energy flexibility integration on a neighbourhood level: Small and Big Data management

Abstract

Modern buildings provide an enormous amount of data available from various sources ranging from modular wireless sensors to smart meters. As well as enhancing energy management and building performance, the analysis of these datasets can enhance the management of decentralized energy systems (electrical storage, PV generation, heat storage, etc.). To optimize the interaction between the building and the grid, it is essential to determine the total energy flexibility of the user and the building. A building has different possibilities for demand side management, energy storage and energy exchange for which a functional-layered approach is proposed from the user up to building and its interaction with the energy infrastructure. Central is the principle of the human-in-the-loop, where a bottom-up approach places the human needs as a central starting point for the energy interaction optimisation. The combination of Big Data with deep learning techniques offers new possibilities in the prediction of energy use and decentralized renewable energy production (e.g. from local weather data taking into account local phenomena such as urban heat islands). This combined with a more bottom-up approach of multi-agent systems with a gossip-based cooperative approach using Small Data offers decentralized control and monitoring autonomy to reduce the complexity of the energy system integration and transition. This makes it possible to relate the outcomes of the urban energy system integration on a neighbourhood level. The approach is being applied to a typically medium-sized office building. A first application of the human-in-the-loop controlling the lighting systems in the open-plan workplace of the test-bed office building showed some estimated annual energy saving of around 24%.

Practical application : Analysis of a large database containing so called Big Data of clusters of buildings seems promising. Therefor there is the need to study the potential impact of utilization of big building operational data in building services industry. Besides this there is also a need for a data mining-based method for analyzing massive building operational data of a specific building, Small Data. This work sets out a general framework and method for doing both and to combine the strength of both approaches. The presented combined approach and results will be of interest to engineers and facility managers wondering what the key constraints to optimal use data to optimize low energy/carbon control strategies might have within their work.

Keywords

BEMS Big Data energy flexibility neighbourhood energy management Small Data smart grid

Introduction

In relation to energy and sustainability targets, an energy transition is necessary for the maximized application of renewable energy sources. The necessary energy transition leads to drastic changes within the current energy system, new market structures and players (e.g. local or independent multi-carrier micro-grids, energy generation and storage at community level and smart buildings/homes) and changing end-user involvement as well as new technologies (e.g. storage and conversion between energy carriers). This will result in significant changes in the existing energy infrastructure. In the future multi-carrier energy grid context, the identification and prediction of building energy flexibility throughout the whole system from generation, distribution to end user is a challenging open question. As consumers will also produce energy,¹ they will become prosumers, thus necessitating two-way transport of energy: from and to the grid. The Netherlands has around 7.6 million houses and around 370,000 non-domestic buildings.² Integrating the total energy infrastructure for heating, cooling and electricity into one system would mean an explosion of complexity. In the future multi-carrier energy grid context, the identification and prediction of building energy flexibility throughout the whole system from central generation and distribution to the individual end user is a challenging open question.

End users account for 37% of final energy consumption² in the Netherlands and this is nearly the same throughout Europe. Houses with smart meters and modern non-domestic buildings with their Building Management Systems nowadays provide an enormous amount of data.^3,4 Unprecedented high volumes of data relating to user behaviour and building energy consumption are thus now available and accessible from these systems. The analysis of these datasets can provide useful insight for improved energy management and can also enhance the management of decentralized energy systems (electrical storage, PV generation, heat storage, etc.). This enables the estimation of the individual energy consumption based on the actual user behaviour and then aggregating it to predict the total building energy demand based on deep learning by data about end user’s behaviours in time and space. It would yield useful insights to facilitate the achievement of various set energy stability and environmental sustainability goals for process control as well as goals for urban energy infrastructure planning. There is a need for more energy system integration, while on the other hand, the autonomy of individual systems is necessary to cope with the otherwise exploding complexity.

Methodology

The potential for energy reduction within the built environment can be determined by the measures according to the Dutch Trias Energetica method, see Figure 1. However, in this approach, both user behaviour and energy exchange as well as storage systems are not included. With this new five-step approach model, the interaction with the electrical grid becomes more flexible by the new possibilities for energy exchange and storage. These optimal interactions lead to process control that is increasingly bottom up rather than top down. It includes the influence of the building user interactions transformed into efforts towards unitary performance metrics that capture both demand side (users) and power supply side (grid).

Figure 1.

The Trias Energetica method and the five-step method.⁵

Clearly, the energy demand characteristics of users, available from the analysis of the data from Building Energy Management Systems (BEMS) are very valuable information for determining energy flexibility and to perform grid interaction optimization. This specific user’s building energy flexibility is, according to the IEA Annex 67, the flexibility as defined by the ability of a building to manage energy demand and generation according to local climatic conditions, occupant needs and energy grid requirements. To distinguish all possibilities for energy storage and energy exchange within a building, a functional-layered approach is proposed, see Figure 2.

Figure 2.

IEA Annex 67 Energy Flexible Buildings approach.

The approach focuses on all important layers of the building, particularly on the room level where the occupant needs and requirements often have contradictory demands in relation to reducing the energy demand. To validate the approach, field experiments were conducted in an office building to evaluate methods for obtaining detailed information, particularly relating to space occupancy that enhances energy management.^6,7 Furthermore, a multi-agent system (MAS) coordination framework was developed and assessed in the test-bed office building for coordination of occupant behaviour on the room level and the building’s energy flexibility exchange with the electrical grid.

Standard BEMS control strategies rely on code-defined occupant comfort ranges⁸ and operate according to fixed schedules and assumed constant occupancy. This leads to energy usage for maintaining occupant comfort, which is inefficient as they do not incorporate the effects of real occupant behaviour. Once good actual information is obtained concerning occupancy and occupants’ comfort demands, there are many well-established control methods that can be directly applied to regulate the aggregated power response. Examples include⁹: an open-loop control method, Model Predictive Control, Lyapunov-based control, or simple inverse control that computes the control action so that the predicted output matches the given reference signal. However, all the aforementioned approaches have limitations that need to be addressed for realistic user demand response applications.

The new proposed solution will be focused on the level of individual occupants up to neighbourhood level in the existing built environment. The approach of combining individual occupants, houses and buildings on a neighbourhood level enables optimized energy management at a higher level of integration with more ways of optimizing the use of overall energy flexibility potential than an approach based on individual building optimization.

Holistic approach to energy flexibility

In order to stabilize the grid, the functionalities of an energy infrastructure need real-time energy management to make use of the flexibility of all grid-connected systems.^10,11 Therefore, new urban energy flexibility strategies are needed.^12,13 The energy demand characteristics of buildings available in BEMS represent crucial information for flexible optimization^14,15 to activate participation of buildings in the grid. Using all available flexibility within energy generation, distribution infrastructure, renewable energy sources^16–22 and especially the buildings themselves²³ is a new strategy. According to the IEA EBC Annex 67,²⁴ energy flexibility of a building is defined as “the ability to manage its energy demand and generation according to local climate conditions, user needs and grid requirements”. Energy exchange between the buildings is an emerging concept which can provide flexibility in a neighbourhood.

Clearly, the energy demand characteristics of buildings and their users, available from BEMS, are very valuable information for grid optimization.^25,26 Smart control of energy consumption and generation inside (nanoGrid) and around buildings (microGrid) can provide major contributions to address the imminent energy stability problems within the total energy infrastructure.

Power system flexibility is needed to mitigate the power grid uncertainties caused by the growing application of decentralized sustainable energy generation.²⁷ Energy flexibility control is a strategy, which describes the ability of a system to handle and respond to changing requirements.²⁸ From a grid environment perspective, power system flexibility is defined as a system’s ability to continuality balance the electricity supply and demand in an effective way, while maintaining the required comfort services towards the building’s users.¹⁹ The ‘active’ office building with ‘flexible’ loads requires demand-side management (DSM) to control building systems’ loads and realize optimization goals such as energy efficiency and cost reduction. DSM from the building (utility) perspective is defined as the planning and implementation of those electric utility activities designed to influence customer uses of electricity in ways that will produce desired changes in the utility’s load shape.²⁹ The performed DSM techniques lead to new energy profiles. DSM includes a variety of techniques to accomplish the objective load shape, examples being peak shaving and valley filling. In Macedo et al.,³⁰ six DSM techniques are described; this article focuses on two of them: load shifting and flexible load shaping, see Figure 3. Load shifting is the technique in which loads are rescheduled from peak to off-peak periods. Flexible load shaping presents controllable load systems during critical periods and relies on a set of actions that respond to the need of the grid to power system reliability.

Figure 3.

The six DSM techniques.¹⁹

Load shifting makes building systems operational during shifted periods of time, preferably off-peak. However, this is not always possible due to the ‘rebound’ of building load demand units.³¹ For example, the demand load chiller operates on highly reduced capacity for half an hour; within this half an hour, the indoor comfort is maintained. After half an hour, the chiller has to operate on full capacity to compensate the downtime and keep comfort parameters between the required boundaries, creating a rebound effect as shown in Figure 4. The actual resource savings are negative, so it is more a kind of backfire, because energy usage increased beyond the initial potential savings, as the rebound effect is higher than 100%. This situation is similar to the Jevons paradox. The improved energy flexibility efficiency increases the overall consumption of energy by making an intervention cheaper and thus more scalable or accessible.

Figure 4.

Rebound effect to compensate downtime.⁷

These new peaks may disturb the power grid balance and have to be taken into account for DSM technique development. The consequence of rebound demand may be significant at grid level as a result of cascaded effect of multiple buildings.³²

We take a more holistic approach to energy system flexibility, which looks at the potential interactions between new and traditional sources of flexibility.⁷ Starting from the neighbourhood perspective, an innovative approach is developed which has the potential to contribute to their occupants in their role as prosumers as they will get a more prominent position in the energy process control. It is based on a two-sided approach on neighbourhood level: bottom-up from building energy management systems towards neighbourhood energy management to effectively use building energy flexibility for efficiency, sustainability and grid stability; MAS with a gossip-based cooperative approach for additional decentralized control and monitoring functions for NEMS to connect to different BEMS. And top-down using the urban planning and neighbourhood structures; an exploration of how data from smart meters can be used to train deep neural networks for system monitoring and optimization.

MASs

New approaches are needed to increase buildings’ energy flexibility towards the Smart Grid. Traditionally, the energy approach towards the built environment is top-down (centralized energy generation/distribution through the Smart Grid). To be able to use a middle-out (control on building level by the BEMS) as well as a bottom-up approach (demand driven by the human needs for energy/comfort), see Figure 1. The energy infrastructure’s functional optimization leads to an energy management strategy making use of the flexibilities of all grid-connected systems, which will lead to a better balanced and controlled network at all levels. Intelligent agent technology offers intriguing possibilities to implement advanced DSM control strategies and energy flexibility management of the smart grid.^33,34 An agent is a computational system with a high degree of autonomy performing actions based on the information received from the environment. Within a MAS, agents interact to achieve cooperative (e.g. distributed problem solving) or competitive (e.g. coalition formation, auction) group behaviour. Agents with various functionalities achieve this by sharing a minimum amount of information between modules and asynchronous operation implemented via message exchanges.³⁵ The agents were designed using an open source web-based and fully decentralized agent design platform called EVE³⁶ and this made it possible to couple the agents to the BEMS.

Machine learning

Important within this project is the integral neighbourhood/energy design approach towards process control and mathematical data handling – in other words – from neighbourhood level to the deep detailed level of data processing. A wide range of machine learning methods, both supervised and unsupervised, was analysed.^37–41 It is, however, still a challenge to develop an accurate solution that could perform well for all type of situations/scenarios. Furthermore, self-stabilizing approaches to large-scale highly dynamic systems are quite limited in number, so this research provides a contribution in this direction. This self-stabilizing behaviour is essential to respond to all fluctuating and unpredictable circumstances due to the use of renewable energy such as wind and solar power. Otherwise, there might be a risk for black-outs.

Mocanu et al.³⁷ proposed a novel Internet-of-Things (IoT) framework, see Figure 5, to perform simultaneously and in real-time flexibility identification and prediction, by making use of deep learning methods for example Factored Four Way Conditional Restricted Boltzmann Machines (FFW-CRBM) and their disjunctive version.

Figure 5.

A general schematic representation of an IoT architecture used for the real-time identification and prediction procedure of the building’s energy flexibility.³⁷

More recently, there has been a revival of interest in combining deep learning with reinforcement learning. In Mocanu et al.,^42,43 deep reinforcement learning is introduced for building energy and cost optimization as the state-of-the-art method in machine learning⁴⁴ able to dynamically use information in an online manner. Therein, two optimization methods, Deep Q-learning and Deep Policy Gradient, are proposed to solve the same sequential decision problems at both the building level and the aggregated level. Francois-Lavet et al.⁴⁵ have proposed the use of Deep Q-learning for storage scheduling in microgrids. The type of strategies chosen for the optimization at the neighbourhood level could also increase the complexity of the problem. Hurtado Munoz^46,47 compared different types of strategies: centralized versus decentralized, as well as cooperative versus non-cooperative.

Gossip algorithms

In the present Big Data era, intelligent buildings are quickly becoming cohesive and integral inhabitants of cyber physical ecosystems.⁴⁸ This makes it possible to apply machine learning methods and to investigate their benefits. Traditional centralized control fails to work beyond a certain network size, reasons including limited bandwidth and real-time constraints violated mainly by the continuous changes in network topology. However, in many cases, an adequate control strategy relies on some form of measurement of global parameters of the network – in other words, estimating if the network is doing the “right thing” – and adopting corrective measures or not.^49,50 Gossip algorithms attempt to compute aggregate (i.e. global) values for important network parameters by relying exclusively on local exchanges of information,⁵¹ see basic principle Figure 6.

Figure 6.

Principle concept of gossiping after Kavis et al., 2013.⁵²

Most important is that due to the reduced sampling among the agents, the necessary communication time is enormously reduced which is a prerequisite for large real-time operating networks.

Different levels when integrating user-building Smart Grid

The research includes both analytical and experimental methods. On the building level, an existing building is used to verify concepts of data integration on the building level. On the neighbourhood level, a specific neighbourhood was selected as test case in the early stage of the research. The selected neighbourhood is representative of approximately 95% of neighbourhoods in the Netherlands. For these types, a number of relevant characteristics of the buildings were determined, such as energy demand, average degree of insolation, multi-storey or ground-based buildings, ownership and floor area. Taking the energy system of the built environment not as a whole, but as neighbourhoods, scales it down from eight million separate users to about 12,000 neighbourhoods differentiated by 15 types, according to year of construction, degree of urbanization and function.⁵³ This forms the starting point to analyse the different possibilities for energy system integration.

To understand the behaviour in the neighbourhood, it is of vital importance to know the demand profiles of the buildings. A detailed analysis of the load profiles could yield insights into the possibilities to identify the synergies between individual buildings in the neighbourhood. This leads to a better understanding of the added value of combining buildings for use of their total energy flexibility and energy exchange options. This paper examines the potential of energy exchange by examining their load profiles (heat load, cooling load and electricity demand) for some selected buildings in a specific neighbourhood in the Netherlands, namely Princenhage located in Breda. Princenhage is a neighbourhood with 8535 inhabitants, one of the largest districts of Breda. Currently, all buildings are connected to a non-renewable electricity and gas network. However, the municipality of Breda has the goal to be a CO₂ neutral city by 2044. According to the plans, this goal can be achieved if 50% less energy is being used compared to 2009 and the remaining 50% is generated sustainably. A load profile study to identify energy exchange possibilities in a neighbourhood can be found in Walker et al.⁵⁵ From this study, it was concluded that for the analysis of energy flexibility options, demand profile data with small time intervals are very important.

Consideration of research in energy and the built environment reveals a stark separation of the topics based on the scale of analysis: individual buildings and the urban scale.⁵⁶ The first scale of analysis, the individual building scale, is concerned only with the building itself and omits any relationship of the building to the larger levels within the built environment, such as for example neighbourhoods, districts or cities. It is related to the architectural design and operational systems. The second scale of analysis, the urban scale, focuses on entire energy infrastructures within the built environment rather than individual elements such as buildings.⁵⁶ This scale is related to the urban form and infrastructural networks. This separation by scale is problematic, as it ignores the actual pattern of operation and energy use: the building within existing cities. Assessing these currently missing patterns is crucial for a holistic analysis of energy use in the built environment and achieving future environmental goals. A combined bottom-up and top-down approach is proposed to address the current missing interaction between approaches on the two different scales. The approach aims for energy system integration (ESI) by aggregated value from the combination of Small and Big Data. It forms a new holistic approach to the scientific context of the problem on how effectively data can be used within the energy system of the built environment on different scales from building process control (Small Data) to urban energy planning (Big Data). A database using a subset of buildings, combining smart meters and BEMS with machine learning methods, is created and used for pattern/behaviour recognition and training. In particular, mathematical methodologies are used that support data-driven clustering of time series in order to identify underlying patterns and corresponding outliers. Functional energy control modules are being implemented in a multi-agent platform-supported NEMS system. A subset of buildings of the neighbourhood will be used to further test the concept.

Analytics of data and deep learning from home/building automation and management systems (Small Data) as well as smart energy meters (Big Data) can provide insights into the interactions and correlations between user behaviours (bottom layer) and the changing/future energy infrastructure (middle and top layers), see Figure 7.

Figure 7.

Schematic representation of the concept for energy system integration and data use.

New process control strategies are needed for improving the energy interaction within the building, its environment and the energy infrastructure by effectively incorporate the occupant’s behaviour. This research explored the use of increased sensor data and computational support for enhancement of energy management and energy flexibility in buildings. Specifically, the research focuses on the application of detailed information on building occupancy as well as MASs in enhancing the performance of traditional building energy management systems in office buildings, see Figure 8.⁶ This forms the basis for human-in-the-loop process control, see Figure 9.⁶

Figure 8.

Fine grained user information.⁶

Figure 9.

Control schematic human-in-the-loop approach.⁶

First experiment in a real office building as an example

To evaluate the MAS for coordinating of occupancy-based lighting control for enhanced energy management, a test case office building was used.⁴ This office building is a typically medium-sized Dutch office building built in 1992 and refurbished in 2009 and comprises a three-storey high building with around 50 employees. The office building is connected to a mid-voltage transformer station by two main connections, and all main power systems connected were measured. These comprise of: a mechanical ventilation system with heat recovery wheel (no recirculation of air); central cooling; electrical steam humidifier; heating by ventilation and two radiator zones, electrical lighting as well as electrical appliances. MAS, due to its flexibility, autonomous and communication properties, was considered a suitable computational intelligence tool that can be applied in buildings to enhance energy management as well as buildings’ interaction with the smart grid.^57,58 As noted from Figure 10, the agents represent all levels within the building including users and devices. This enables input from users as well as all appliances within the building, which, as depicted in Figure 10, is seldom utilized in traditional BEMS.

Figure 10.

Fine grained user information: determination of location, presence, count, identification, activity and tracking.⁶

The building has a peak occupancy of approximately 50, but daily average occupancy ranges between 25 and 30. As with a significant number of office buildings, as noted by the authors in Klein et al.,⁵⁹ the test-bed operates based on assumed occupancy profiles and schedules. The HVAC and lighting systems are operated continually during the building operational hours, which is typically between 7 am and 6 pm.

An experiment⁵⁴ was conducted in the open-plan workspace with the highest occupancy in the test-bed office building: 12 workspaces illuminated with 31 TL luminaries that are centrally controlled. To achieve controllability of the lighting fittings, wireless sensors nodes with switching and measurement functions were utilized. For occupancy detection, a combination of chair sensors and wireless PIR motion sensors was used. Data were collected for a period of three weeks between the 22 February and 11 March 2016 in the test-bed.⁴ During the first week of the experiment, the measured lighting electricity use was approximately 113 kWh. During the second and the third week of the experiment, when the MAS controlled system was applied in the test-bed, the measured lighting energy use for the test-space was approximately 81 kWh and 90 kWh, respectively. This represents a reduction of approximately 28% and 20% reduction in the space lighting energy used compared with normal controls. The estimated annual saving based on the use of this system in the test-bed office building is around 24%.⁴

Given that user behaviour is recognized as a major factor that influences energy performance and given its stochasticity and the uncertainty it introduces, clearer understanding of its impact on the changing energy infrastructure can facilitate an integrated energy management approach on the user-level through the neighbourhood level in such a way that is efficient, sustainably effective, flexible and resilient. The combination of Small Data, the small set of specific sensor data such as temperature, air speed, humidity and status,⁵² focusing on what the occupants need for energy, combined with the Big Data which focused on their actual energy usage leads to new opportunities for optimizing the energy consumption within the built environment.

Discussion and conclusions

Buildings are a key source of energy flexibility due to their high energy demand and their possibilities for decentralized generation of energy from renewable sources. Harnessing the energy flexibility of buildings, however, demands that buildings be considered collectively and that their energy behaviour is known. In the Big Data era, more and more machine learning methods appear to be suitable to overcome the limitation of not knowing the future generations and demands by automatically extracting, controlling and optimizing the energy consumption patterns. This can be done by performing successive transformation of the historical data to teach powerful machine learning models to cope with the high uncertainty of the energy consumption patterns. Then, these models will be capable of generalization and they could be exploited in an on-line manner (i.e. few milliseconds) to minimize the cost in newly encountered situations. Among all these machine learning models, the ones belonging to the area of reinforcement learning, as part of MASs, are suitable for the cost and energy minimization problems, as they are capable of learning an optimal behaviour, while the global optimum is not known.

Field experiments were conducted in an office building to evaluate alternative methods for obtaining detailed occupancy information that enhances energy management.^6,7 Furthermore, a MAS coordination framework was developed and assessed in the test-bed office building for coordination of occupant behaviour on the room level and the building’s energy flexibility.

The next step is to define Neighbourhood Energy Management systems. Grouping energy demand of end users and local renewable producers in neighbourhoods will enforce end-user involvement and automated load shifting which greatly improves the efficiency of advanced energy management. This allows the maximum utilization of flexible energy demand resources within neighbourhood for optimizing the grid interaction. This bottom-up approach for system integration of energy infrastructure starts from the user’s actual needs to support the Smart grid with the available energy flexibility of complete buildings. The impact of the solution is optimized energy (flexibility) exchange within the neighbourhood which leads to a more stable electricity network, so it stays reliable even when the decentralized renewable energy applications grow in future. The neighbourhood can grow towards a maximal level of sustainability. Optimized control also allows the comparison of energy use for different houses and enables benchmarking which will lead to increased efficiency of the existing installations.

Footnotes

Acknowledgements

The authors are grateful to the partners of NWO Project Energy System Integration and Big Data for the financial support, to make this research work realizable: the Municipality of Breda, Kropman Building Services, Foundation WOI, Foundation PIT, UNETO-VNI, De Dobbelsteen and Enexis.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research work is funded by NWO Project Energy System Integration and Big Data.

References

Šajn N. Electricity “Prosumers”. Brief Eur Parliam Res Serv, www.europarl.europa.eu/RegData/etudes/BRIE/2016/593518/EPRS_BRI(2016)593518_EN.pdf (2016, accessed 28 November 2017).

RVO. Monitoring Energiebesparing Gebouwde Omgeving 2016, Rijksdienst voor Ondernemend Nederland, November 2017, https://www.rvo.nl/sites/default/files/2018/03/Monitor-Energiebesparing-Gebouwde-Omgeving-2016.pdf (accessed 3 January 2019).

Labeodan

Zeiler

Boxem

et al.

Occupancy measurement in commercial office buildings for demand-driven control applications – a survey and detection system evaluation. Energy Build 2015; 93: 303–314.

Labeodan

De Bakker

Rosemann

et al.

On the application of wireless sensors and actuators network in existing buildings for occupancy detection and occupancy-driven lighting control. Energy Build 2016; 127: 75–83.

Gvozdenović

Zeiler

Massen

. Roadmap to nearly zero energy buildings M1 project, Eindhoven, Netherlands: TU Eindhoven, 2014.

Labeodan T. A Multi-agents and occupancy based strategy for energy management and process control on the room-level. PhD Thesis, TU Eindhoven, Eindhoven, Netherlands, 2017.

Aduda KO. Smart Grid-Building Energy interactions, demand side power flexibility in office buildings. PhD Thesis, TU Eindhoven, Eindhoven, Netherlands, 2018.

Shariatzadeh

Mandal

Srivastava

. Demand response for sustainable energy systems: a review, application and implementation strategy. Renew Sustain Energy Rev 2015; 45: 343–350.

Dong

Andrews

Lam

et al.

An information technology enabled sustainability test-bed (ITEST) for occupancy detection through an environmental sensing network. Energy Build 2010; 42: 1038–1046.

10.

Lopes

Lezama

Pineda

. Model based systems engineering for smart grids as systems of systems. Proc Comput Sci 2011; 6: 441–450.

11.

Acevedo

Molinas

. Identifying unstable region of operation in a micro-grid system. Energy Procedia 2012; 20: 237–246.

12.

Koirala

Avila

Gomez

et al.

Local alternatives for energy supply: performance assessment of integrated community energy systems. Energies 2016; 9: 981–1003.

13.

Sha

Aiello

. A novel strategy for optimizing decentralized energy exchange for prosumers. Energies 2016; 9: 554–576.

14.

Dave S, Sooriyabandara M and Yearworth M. A systems approach to the smart grid. In: The first international conference on smart grids, green communications and it energy-aware technologies, (ENERGY 2011), Venice, 22–27 May 2011, pp. 130–134.

15.

Wang S. Intelligent building electricity demand management and interactions with smart grid. In: Proceedings Clima 2013, Prague, 16–19 June 2013.

16.

D’hulst

Labeeuw

Beusen

et al.

Demand response flexibility and flexibility potential of residential smart appliances: experiences from large pilot test in Belgium. Appl Energy 2015; 15: 79–90.

17.

Kobus

Klaassen

Mugge

et al.

A real-life assessment on the effect of smart appliances for shifting households’ electricity demand. Appl Energy 2015; 147: 335–343.

18.

Kensby

Trüschel

Dalenbäck

. Potential of residential buildings as thermal energy storage in district heating systems-Results from a pilot test. Appl Energy 2015; 137: 773–781.

19.

Lund

Lindgren

Mikkola

et al.

Review of energy system flexibility measures to enable high levels of variable renewable electricity. Renewable Sustainable Energy Rev 2015; 45: 785–807.

20.

Masy

Georges

Verhelst

et al.

Smart grid energy flexible buildings through the use of heat pumps and building thermal mass as energy storage in the Belgian context. Sci Technol Built Environ 2015; 21: 800–811.

21.

Le Dréau

Heiselberg

. Energy flexibility of residential buildings using short term heat storage in the thermal mass. Energy 2016; 111: 991–1002.

22.

De Coninck

Helsen

. Quantification of flexibility in buildings by cost curves – methodology and application. Appl Energy 2016; 162: 653–665.

23.

Aduda

Labeodan

Zeiler

et al.

Demand side flexibility and building performance implications. Sustainable Cities Soc 2016; 22: 146–163.

24.

Østergaard S and Marszal-Pomianowska AJ. IEA EBC annex 67 energy flexible buildings. In: Proceeding Clima 2016, Aalborg, Denmark, http://vbn.aau.dk/files/233817779/paper_325.pdf (2016, accessed 23 February 2018).

25.

Zeiler

Aduda

Thomassen

. Integral BEMS controlled LVPP in the smart grid: an approach to a complex distributed system of the built environment for sustainability. Int J Des Sci Technol 2016; 22: 81–108.

26.

Aduda

Labeodan

Zeiler

et al.

Demand side flexibility coordination in office buildings: a framework and case study application. Sustainable Cities Soc 2017; 29: 139–158.

27.

Ulbig

Andersson

. Analyzing operational ﬂexibility of electric power systems. Int J Electr Power Energy Syst 2015; 72: 155–164.

28.

Saleh

Mark

Jordan

. Flexibility: a multi-disciplinary literature review and a research agenda for designing ﬂexible engineering systems. J Eng Des 2009; 20: 307–323.

29.

Gellings

. The concept of demand-side management for electric utilities. Proc IEEE 1985; 73: 1468–1470.

30.

Macedo

MNQ

Galo

JJM

de Almeida

LAL

et al.

Demand side management using artiﬁcial neural networks in a smart grid environment. Renewable Sustainable Energy Rev 2015; 41: 128–133.

31.

Palensky

Dietrich

. Demand side management: demand response, intelligent energy systems, and smart loads. IEEE Trans Ind Inform 2011; 7: 381–388.

32.

Motegi

Piette

Watson

et al.

Introduction to commercial building control strategies and techniques for demand response, Report, report no: 59975, 2007; 500: 1–107. Sacramento, CA: California Energy Commission.

33.

Jarvis

Rönnquist

et al.

Multi-agent systems and applications. Intell Syst Ref Libr 2013; 46: 1–12.

34.

Basso

Gaud

Gechter

et al.

A framework for qualifying and evaluating smart grids approaches: focus on multi-agent technologies. Smart Grid Renew Energy 2013; 4: 333–347.

35.

Kofler

Reinisch

Kastner

. A semantic representation of energy-related information in future smart homes. Energy Build 2012; 47: 169–179.

36.

de Jong J, Stellingwerff L and Pazienza GE. Eve: a novel open-source web-based agent platform. In: SMC 2013: IEEE International Conference on Systems, Man and Cybernetics, Manchester, 13–16 October 2013, pp. 1537–1541. Piscataway, NJ: IEEE.

37.

Mocanu DC, Mocanu E, Nguyen HP, et al. Big IoT data mining for real-time energy disaggregation in buildings. In: Proceedings of the IEEE international conference on systems, man, and cybernetics, Budapest, Hungary, 9–12 October 2016.

38.

Mocanu

Nguyen

et al.

A topological insight into restricted Boltzmann machines. Machine Learning 2016; 104: 243–270.

39.

Mocanu E, Larsen EM, Nguyen PH, et al. Demand forecasting at low aggregation levels using factored conditional restricted Boltzmann machine. In: Proceedings IEEE power systems computation conference, Genoa, 20–24 June 2016. Piscataway, NJ: IEEE.

40.

Mocanu

Nguyen

Gibescu

et al.

Deep learning for estimating building energy consumption. Sustainable Energy, Grids Netw 2016; 6: 91–99.

41.

Mocanu

Nguyen

Kling

et al.

Unsupervised energy prediction in a smart grid context using reinforcement cross building transfer learning. Energy Build 2016; 116: 646–655.

42.

Mocanu E. Machine learning applied to smart grids. PhD Thesis, TU Eindhoven, Eindhoven, Netherlands, 2017.

43.

Mocanu E, Mocanu DC, Nguyen PH, et al. On-line building energy optimization using deep reinforcement learning. In: IEEE Transactions on Smart Grid, 8 May 2018, p. 1. Piscataway, NJ: IEEE.

44.

Mnih

Kavukcuoglu

Silver

et al.

Human-level control through deep reinforcement learning. Nature 2015; 518: 529–533.

45.

Francois-Lavet V, Taralla D, Ernst D, et al. Deep reinforcement learning solutions for energy microgrids management. In: European Workshop on Reinforcement Learning (EWRL 2016), Barcelona, 3–4 December 2016.

46.

Hurtado Munoz LA. Uncovering demand flexibility in buildings: a smart grid inter-operation framework for the optimization of energy and comfort. PhD Thesis, TU Eindhoven, Eindhoven, Netherlands, 2017.

47.

Manic

Amarasinghe

Rodriguez-Andina

et al.

Intelligent buildings of the future: Cyber aware, deep learning powered, and human interacting. IEEE Ind Electron Magaz 2016; 10: 32–49.

48.

Dulman S and Pauwels E. Tweaking gossip algorithms for computation in large-scale networks. In: Proceedings 10th international conference on intelligent sensors, sensor networks and information processing (ISSNIP), Singapore, 7–9 April 2015.

49.

Dulman

Pauwels

. Self-stabilized fast gossiping algorithms. ACM Trans Auton Adaptive Syst 2016; 10: 1–29.

50.

Dulman S and Pauwels E. Enabling future smart energy systems, 25 years ERCIM challenges for ICST. ERCIM News, October 2014.

51.

Kavis M. Forget Big Data–Small Data is driving the internet-of-things, https://ercim-news.ercim.eu/en99/challenges-for-icst/enabling-future-smart-energy-systems (2014, accessed 22 August 2017).

52.

Luk

Albert Kai-Sun Wong

Lea

et al.

RRG: redundancy reduced gossip protocol for real-time N-to-N dynamic group communication. J Internet Serv Appl 2013; 4: 14–14.

53.

Schepers BL, Haber NR, Rooijers FJ, et al. Op weg naar een klimaatneutrale gebouwde omgeving 2015. CE Delft publicatie nummer 15.3A31.35, 2015.

54.

Walker S, Corten K, Labeodan T, et al. A load profile study of different buildings to identify neighborhood energy flexibility through exchange possibilities. In: Proceedings CISBAT 2017, Lausanne, Switzerland, 6–8 September 2017.

55.

Anderson

Wulfhorst

Lang

. Energy analysis of the built environment – a review and outlook. Renewable Sustainable Energy Rev 2015; 44: 149–158.

56.

Windham

Treado

. A review of multi-agent systems concepts and research related to building HVAC control. Sci Technol Built Environ 2016; 22: 50–66.

57.

Vallejo

Albusac

Glez-Morcillo

et al.

A multi-agent approach to intelligent monitoring in smart grids. Int J Syst Sci 2014; 45: 756–777.

58.

Klein

Kwak

Kavulya

et al.

Coordinating occupant behavior for building energy and comfort management using multi-agent systems. Autom Constr 2012; 22: 525–536.