Validation, Editing and Estimation (VEE) under AMI and Deregulation
I presented this paper back in 2000 at the AEIC Load Research Conference. The introduction is shown here, with a link to the complete paper below. (There’s a link to the 2000 AEIC Load Research Conference presentation as well.)
Load Research has had a long and varied history, having been carried on the coattails of a variety of applications since the early days of Load Research in the 1960’s and 1970’s. With the collection of load data associated with load research came the need for quality assessment, decisions on how (and whether) to fix problem data and, once data was sufficiently clean, expanding results to the population, whether the load data represented one customer or part of a group representing tens of thousands.
Whoever coined the phrase “garbage in, garbage out” was certainly thinking of situations encountered in load research. With the high cost of load survey equipment and operations, small carefully crafted samples meant that quality from each site was paramount. Consider that one sample site might represent the pattern of tens of thousands of customers and you can appreciate the implications of including problem data in an expansion of results to model population load characteristics.
Problem data and data loss are caused by a combination of factors, including equipment failure, data communications losses, human error, neglect, weather, computer failures and bad luck. Equipment and procedures have improved over time, but data problems have not disappeared. Those of us who have been in the industry for many years can look back on the many hours spent fixing data problems, only to have technology eliminate them, or so we think – and create new ones.
In the past few years, Automatic Meter Reading (AMR) and Automated Metering Infrastructure (AMI) technology and systems have become more prevalent, and many utilities have adopted such systems, or at least initiated pilot programs to test the technology and logistics, including customer response. These new systems enable more complex pricing for small customers, two-way communications, load control options and linking to Home Area Networks (HAN) for residential, as well as Energy Management Systems (EMS) in businesses. The potential for collecting interval data is almost limitless, and so is the need for validation, editing and estimation (VEE) to ensure that the data is valid!
Over the years, the reasons for collecting data have changed, from PURPA compliance, to conservation studies, load management, rate design, cost-of-service studies, demand-side management, technology assessment, billing, profitability, competitive threats, load profiling to enable reconciliation of sales from multiple suppliers within a service area and, most recently, dynamic pricing under AMI. The issues of data quality remain, with only slight differences, and the turnover of experienced staff has meant that some of the experience and techniques for validating, processing and analyzing load data has been lost.
This paper was written to refresh and reminisce for the old timers, teach the new generation, and highlight the similarities and differences associated with data validation, editing and estimation (VEE) and data expansion under the new deregulated utility environment as it will be affected by the new technologies involved in Advanced Metering Infrastructure (AMI) and Meter Data Management (MDM). As we march through the 2000’s, we need to take a fresh look at validation and editing techniques and their implications on the new applications for the data.
Published Paper: Lopes-Valeditaeic2k
2000 Presentation: valedit