Controlling Commercial Cooling Systems with Reinforcement Learning

About This Document

A technical paper by DeepMind, Google, and Trane on controlling commercial-building chiller plants using reinforcement learning (RL). The authors describe BCOOLER, a system that generates safe recommendations for setpoints and equipment configurations based on BMS data, energy-use forecasts, and operating constraints. In real-world A/B experiments at two sites, the system reduced energy consumption by 9% and 13%, respectively, compared with Trane’s heuristic Sequence of Operations (SOO).

Key Takeaways

• In real-world experiments at two sites, the BCOOLER RL system reduced chiller-plant energy consumption by 9% and 13%, respectively, compared with Trane’s heuristic SOO controller.
• An off-the-shelf RL algorithm is insufficient for industrial HVAC: the implementation required domain-specific constraints, BMS-data cleaning, operating-mode-dependent action masking, model-sensitivity testing, and HVAC expert involvement.
• Safety is provided through multiple layers: constraint-based action filtering, a conservative adjustment for ensemble uncertainty, BMS checks, and fallback to the SOO.
• The greatest optimization potential was observed at low outdoor wet-bulb temperatures and low load; at high temperatures and loads, the available efficiency margin is smaller.
• The work demonstrates the applicability to multi-component commercial chiller plants of an approach previously used for Google data-center cooling, but does not claim plug-and-play deployment.

Key Figures

Energy savings at site 1

approximately 9 %

Live A/B experiment: BCOOLER versus heuristic SOO.

Energy savings at site 2

13 %

Live A/B experiment: BCOOLER versus heuristic SOO.

Number of live-experiment sites

2 sites

A university chiller plant and a mixed-use commercial building.

Duration of each A/B test

3 months

Tests were conducted during the shoulder season; controllers alternated daily.

Data and control interval

5 minutes

One MDP step and the interval for generating a control action.

Decision-time requirement

1 minute

The decision must be calculated using fresh observations.

State-vector dimensionality

50 measurements

After feature engineering.

Dimensionality of the continuous action component

12 measurements

Setpoints are supplemented by discrete equipment on/off commands.

Action-value model inputs

62 features

50 observations and 12 actions.

Predictive model outputs

25 forecasts

1 target energy-consumption metric and 24 observation constraints.

Action-space constraints

59 constraints

Constraints that must not be violated when generating candidate actions.

Observed-parameter constraints

24 constraints

Forecast and conservatively checked before action selection.

Candidate actions after pruning

up to 100000 actions

Limit on the number of actions for subsequent scoring and ranking.

Share of exploratory actions

5 %

Optimistic ε-greedy exploration.

Low-load example

200 tons

At this load, two chillers consume more energy than one.

High-load example

800 tons

At this load, two chillers consume less energy than one.

Share of global electricity demand from space cooling

around 10 %

IEA estimate for 2018 cited by the authors.

Cooling-load ton equivalent

roughly 3.5 kW

Approximate relationship stated in the article.

Practical Value for Data Center Owners

For a data-center owner or project team, this provides a practical reference point for piloting AI optimization of a central chiller plant: savings on the order of 9–13% are not achieved automatically on top of any BMS, but require high-quality telemetry and actuation, formalized constraints, a safe fallback to standard automation, and extended validation across weather conditions and loads. The approach is particularly relevant to facilities with multiple chillers, cooling towers, pumps, and variable loads. It can be used to evaluate a supervisory layer above an existing BMS, rather than as a replacement for fundamental protections and local control loops.

Where It Applies

Topics