Here's an example. Suppose an airline is scheduled to land planes at 1pm, 115pm, and 130pm. The landing process takes fifteen minutes. The first plane is fifteen minutes late.
If the airline is using 'on-time performance' as a key performance metric, it might allow the 115pm plane to land first. Since landing requires fifteen minutes, it must repeat the choice with the 130pm plane. Again, it might also allow the 130pm plane to land next to ensure its on-time arrival. The results of the two strategies:
1) Uses 'on-time performance'
115pm plane lands at 115pm - ON TIME
130pm plane lands at 130pm- ON TIME
100pm plane lands at 145pm - LATE by 45 minutes
45 minutes total delay
66% of planes land 'on-time'
***
2) Does not use 'on-time performance'
100pm plane lands at 115pm - LATE by 15 minutes
115pm plane lands at 130pm - LATE by 15 minutes
130pm plane lands at 145pm- LATE by 15 minutes
45 minutes total delay
0% of planes land 'on-time'Different types of travelers will prefer different approaches to the problem. As I wrote way back in the original post, those who incorporate slack into the travel schedule will likely prefer the intuitive system of option #2 over the statistical manipulations motivating option #1. Those who rely on the airline schedule to keep them 'on-time' will likely prefer to roll the dice with option #1.
One problem I see here is what will happen when a careless outsider looks at the options. A cursory glance at the aggregates - 66% to 0%! - will suggest an obvious choice. But those on the 1pm flight will surely become very upset if they find out their trip was delayed an additional half-hour just to fudge the numbers a bit. Is it worth the risk of potentially alienating up to one-third of the ridership in order to better market the airline to a far larger pool of possible customers?
The larger idea here is the result of failing to match a statistic or metric to the question at hand. Most customers assume airlines are trying to move planes from departure to arrival as quickly as possible. Further, they assume each flight performs independently of the other flights an airline operates. But having metrics that invoke the 'yes/no' nature of 'on-time performance' creates incentives for airlines to do otherwise. As the observer effect indicates, observing behavior is often enough to change behavior.
Or, as management guru (editor's note: and The Business Bro's personal hero) Peter Drucker once quipped - what gets measured gets managed. The mismatched behavior resulting from the strange incentives created by a seemingly innocent tool like 'on-time performance' shows this principle in action. These contradictions do not resolve themselves with good intentions or even trivial blog posts, however. From my long-term point of view, a company would be best to avoid unnecessary misalignment in measurement and performance whenever possible by trying to understand how the observer effect changes behavior and taking the necessary steps to correct for its bias.
Signed,
The Business Bro
