MIDS Capstone Project Spring 2024



The energy sector is on the cusp of a revolution. We're rapidly shifting from traditional, centralized systems to a more interconnected and renewable-based model. This evolution presents both unprecedented challenges and incredible opportunities. At the forefront are Distributed Energy Resources (DERs), like rooftop solar and electric vehicles. These technologies are fundamentally transforming power grids and presenting a critical challenge for utilities: limited visibility into consumer-owned resources.

This is where GridVue comes in. Our innovative machine learning solution provides utilities with visibility into the customer-side of the grid, traditionally opaque "behind the meter" activity. By synthesizing data from smart meters and weather stations, GridVue estimates a customer's energy use and solar generation. With this insight, utilities can navigate the new energy landscape more effectively, optimizing operations and enhancing grid reliability. Recognizing the value of behind-the-meter visibility, a landmark survey was conducted by Siemens AG and Oxford Economics, polling 100 utility decision-makers across North America. When asked about the potential benefits, the following percentage of executives highlighted: 60% think it can improve overall grid reliability, 53% believe it can improve capital allocation, 50% believe it can reduce operational costs, and 50% believe it can extend asset lifettime.

Basic Principle

Behind a typical utility meter, we can represent the key components as two nodes: a load node for customer demand and a solar node for photovoltaic (PV) generation. This creates three distinct signals which can be tracked:

  1. Gross Load: The total electricity consumption at the customer site.
  2. Solar Generation: The amount of electricity generated by the on-site PV system (represented as a negative value).
  3. Net Load: The combined power flow recorded at the meter, calculated as Gross Load minus Solar Generation. This represents the net power delivered by the utility, with Solar Generation acting as a negative contribution.

GridVue's machine learning model is trained to predict the current Solar Generation value based on a history of Net Load data and solar irradiance measurements from the closest available weather station.


GridVue's platform seamlessly integrates smart meter data with weather station data (from third-party or utility-owned sources). Utilizing advanced machine learning, we estimate photovoltaic (PV) generation and gross load at the individual meter level. By continuously analyzing real-time energy usage along with environmental factors influencing production, we uncover insights previously hidden behind the meter. Our solution provides two primary dashboards and two integration endpoints:

  1. An Operations Dashboard for real-time grid monitoring and operational decision support.
  2. A Planning Dashboard with essential data for grid planning, infrastructure investment analysis, and long-term analytics.
  3. A DataMart providing easy access to processed data via simple APIs for integration with external applications.
  4. A real-time Data Stream to feed systems requiring immediate visibility, such as Supervisory Control and Data Acquisition (SCADA) systems.

Under the hood, a multi-model approach

GridVue employs a three-step process to estimate behind-the-meter solar generation and determine a building's total electricity usage:

  • Solar Detection: First, the system analyzes a 2-day history of net load data from the smart meter along with corresponding solar irradiance data from nearby weather stations. A classification model examines this data to identify if a solar power system is likely present at that location.
  • Solar Generation Estimation: If the classifier detects a solar system, a separate estimation model takes over. Using the same net load and irradiance data, this model estimates the current solar power output.
  • Gross Load Calculation: Finally, by combining the estimated solar generation with the latest net load measurement, GridVue calculates the building's actual electricity consumption (gross load).


  • Overcoming Data Scarcity: Since real-world behind-the-meter solar data is limited, GridVue trains its models using simulated data from the National Renewable Energy Laboratory (NREL). The ResStock subset of NREL's "End Use Load Profiles for the U.S. Building Stock" dataset provides physics-based simulations of household electricity consumption, including scenarios with and without solar panels. This data, representing roughly 10,000 households across diverse locations over one year, allows the models to learn patterns across various home types, consumption patterns, and climate conditions
  • Testing for Real-World Performance: To validate performance in real-world conditions, GridVue utilizes the Pecan Street dataset, providing high-resolution electricity consumption and solar generation data for nearly 2,000 homes across the U.S., including 550 homes with solar panels. Since Pecan Street doesn't directly record weather data, GridVue integrates solar irradiance data from Solcast, a solar forecasting company with historical data dating back to 2007. This combined dataset enables comprehensive testing against real-world conditions.

More Information

Last updated:

April 17, 2024