Validating an Operations Centre Model

Abstract

An airline operations centre determines the imminent and future operations of an airline by assessing the current situation, considering current constraints and then issuing a high level plan, describing which flights are operating, which are cancelled, which are delayed or rerouted, impacting aircraft, air crew, flight cabin crew, passengers and ground staff. Other departments of the airline transform the high level plan into detailed execution instructions. In the event, the system consists of a mixture of humans and machines both of which often provide and receive incomplete information, on which they make a decision, striving to execute as close as possible the ‘Ideal Plan’, which is the published timetable. The Airline Operations Decision Making Process is the ‘planning’ portion determining the next course of action, and the Airline Operations is the execution portion, putting the plans into action. In this context, the Study Group was asked to address the following problem: How do you validate a model of an airline’s decision processes to ensure a faithful representation of reality?

By analysing model outputs and actual data, we found that the model does not satisfy any of our proposed “validation” criteria and have therefore suggested a series of possible ways forward for the model development team. We have provided visualisation tools in both Matlab and R, which will be of use in analysing subsequent model versions. We have also provided a series of statistical indicators of model quality. Lastly, and perhaps most importantly, we have described in detail what can be done if additional information were to be made available, for instance further runs of the model or extra data from the operations centre.