Diagnostic testing for earnings simulation engines in the Australian electricity market

Abstract

This study has endeavoured to propose and implement a series of diagnostic tests to determine the appropriateness of electricity simulation engines (ESEs) for generating electricity load and price paths to be used as input in the determination of a retailer’s earnings distribution and the assessment of earnings-at-risk (EaR) measures. Additional diagnostic measures require development before a routine can be developed whereby a complete diagnostic report can be generated as output using simulated and historical data as input. This work includes:
(1) Further partitioning of output load and prices from an ESE into off-peak, peak and weekend periods to determine the subsequent effect on earnings.
(2) The diagnosis of simulated load paths. As simulated load was not supplied for all engines, the diagnostics developed in this report did not include an analysis of load.
(3) The building of a response surface to capture the interaction between temperature, load and price.
(4) Examination of the convergence behaviour of an ESE. Convergence in this context means the determination of the minimum number of load and price paths required from a simulator in order to return expected profiles that conform to industry expectations. This would involve the sequential testing of an increasing number of simulated paths from an ESE in order to determine the number required.

In conclusion, it is important to understand that each of the simulators that were diagnosed in this study were criticised according to industry expectations, and to the degree that the diagnostics employed here reflect those expectations. In fact, all simulators will attract criticism given that they are calibrated on historical data and are expected to generate future prices for market conditions that are unknown. The mark of an appropriate ESE is that the future load and pricing structure it generates is not too much at variance with industry expectations. A critical function of a simulator is for it not to overestimate or underestimate load and prices such that the risk metrics used to govern earnings risk faced by an electricity retailer are compromised to the extent that their book is either grossly over-hedged or under-hedged.