The Future Requires Maintenance

By Mike Koetting March 14, 2023

A couple weeks ago, one of the twenty-year old elevators in our building began stopping at random floors unbidden. The maintenance company addressed it. Two days later, it started again. They replaced the entire control panel. That worked—for three days. They replaced the entire panel once more and that seems to have done the trick. For now.

It’s certainly fair to question the overall condition of our elevators or the competence of the maintenance company. But I wonder if there are bigger issues lurking.

Two years ago, as part of changing billing parameters, our mobile phone company stopped sending us hard copy bills. As I was working on this year’s tax return, I needed to see the details of our billing. I couldn’t figure out how to do so from the phone company’s web so I called them. I bounced around for over an hour, the last half of which was with someone who always seemed to be on the verge of getting me access to my bills. But I eventually gave up and the next day took it to one of the provider’s stores. The sales representative spent almost half-an-hour and had literally given up, complaining that he couldn’t keep up with the changing systems. Fortunately, he had one more idea and that worked.

Certainly annoying, but no risk to life or limb.

Much more consequential issues were the meltdown of Southwest Airlines just before the holidays and problems in the Federal Aeronautics Administration pilot notification system in January. Both of these events scrambled airline travel for many thousands of customers. And both of these messes were due to aging computer systems.

The ground stop called by FAA in January–the first since September 11, 2001–was a result of the failure of the system that flags potential hazards, such as impending bad weather or local system glitches. Flights were suspended for only two hours, but the resulting chaos rippled throughout the day. More than 1,300 flights were cancelled and over 11,000 were delayed.

This fragility of this system was not a surprise. As long ago as 2008, the FAA started planning to replace it. The project has sputtered over the years as funding waxed and waned. Enhancements have been made to the system, but at bottom it is fundamentally outdated technology. In the meantime, new systems were added around it. The FAA was trying to maintain a mélange of systems that been developed over 45 years. It was, and remains, a recipe for disaster.

This is similar to the situation I found when I started at the Illinois Medicaid program in 2010 with the charge of expanding eligibility to comply with the Affordable Care Act. The core eligibility system dated from the 1970s. Technologically, it was generations behind the evolution of computer technology. Keeping it running relied on an aging cohort of programmers who understood all the ways the ancient COBOL system interacted with the newer systems that had been grafted on over the years. It would have been literally impossible to complete the ACA enrollment with that system. Our initial intention was to completely replace it, but we found there was no way that could be done and comply with the statutory start-date. So, relying on the same handful of programmers, we jerry-rigged a new system for processing applications on top of the existing system. It mostly worked, but also got the agency further out on the limb.

Illinois Medicaid continued with plans to completely replace the underlying system. That was largely completed about four years later, although the actual cut-over was accompanied by a period of extended frustrations for workers and clients alike. Of course, the “new” system is now six years old, which is a lifetime on the technology scale. The State of Illinois has neither the funds nor the workforce—not to mention the stomach– to replace that again. I no longer work there, but I have to believe the patchworking has already started.

Okay, some will say, what do you expect of government agencies? Maybe. (And that doesn’t make the consequences of those decisions disappear.) In any event, the epic Southwestern Airlines meltdown shows private entities make the same mistakes. When a major snowstorm hit parts of the country just before the peak Christmas travel period, the portion of their systems that scheduled crews was completely overwhelmed. For several days around Christmas, Southwest cancelled about two-thirds of its flights and another 20% were significantly delayed. Insiders were not surprised. For some time now, analysts and unions have lamented the state of their computer systems. Parts of it were almost as old as the FAA systems; the pieces didn’t work well with each other.

Here’s the Point

While it’s true these are a random collection of computer malfeasance, they point to common warnings. For openers, as we weave computers more and more into our lives, we are more dependent on them, for things small as well as large. There are advantages to that. We can do some things that were not easily possible before. For instance, we almost all take for granted the convenience of on-line banking and ATMS. I remember when you needed humans to cash a paper check if you wanted cash—which was virtually the only way to pay for things. We also take for granted the advantages of these possibilities and construct more and more complicated systems, embroidering them into global supply chains. And we get lower prices on some things because it’s typically cheaper to replace or supplement people with computers. This creates other problems, but in the meantime, consumers are happy to take advantage of the lower prices.

But with these advantages, we get vulnerability.

We can’t completely guard against this vulnerability—a major solar flare, sure to happen sooner or later, will cause massive computer outages—but on a day-to-day basis, individual systems are better protected if they stay current. And here’s the rub. It takes a lot of resources to stay current and maintenance doesn’t have nearly the pizzaz of new technology. (“We want to spend a lot of money to keep doing what we’ve always been doing” doesn’t have a particularly inspiring ring in a world where the ”new” is what gets attention.) Nor is it just money. It requires executive focus and willingness to risk business disruptions as changes are made, even if the “change” is just to keep the same functions running.  And it not simply keeping individual systems current, but keeping an entity’s entire suite of systems working together consistently and coherently.

And that perspective is within individual entities, be they corporations or governments. Operationally, society is a tangled knot of these entities working together. Individual entities need to keep current internally, but also need to stay more or less coordinated with the other entities with which they interact. A new technology anywhere in this broader network has the potential of introducing broader disequilibrium. In one sense, the pace of technological innovation is as much a function of the slowest adopter as the quickest. (A spokesperson for the airlines commenting on the FAA problems allowed it was hard to justify airlines spending more on new technology if the FAA wasn’t keeping up.)

There are two bottom-lines here.  First, I suspect we will be beset by increasing numbers of annoying to really difficult computer failures because as technology proliferates promiscuously there were will be more and more cases where the old and new technology don’t work together. (The U.S. Navy has come to this realization, concluding that putting too much new technology into ships leads to functional disasters because they couldn’t make it all work together.)

The second implication is that we have to spend enough on unglamourous maintenance if we are going to keep our systems functioning. In truth, the systems, mostly, still function exactly as they were intended to. The problem is that so much has changed around them that the systems are no longer compatible with the on-the-ground needs. Attention to maintaining systems is particularly difficult for governmental systems. Keeping current requires funds, legislative will, and effective longer-term executive leadership for something that has little voter appeal. But corporations face similar problems.

The future might be a lot messier than we are imaging.