The Modern World Is Too Complex

It's getting harder and harder to understand how the modern world works. That doesn't matter directly to most of us in our daily lives, until something breaks. Then we figure out what broke and either get it fixed or replace it. 

That is much harder to do when you're talking about more complex systems, like the software that runs a stock exchange (I have some experience with that), supply chain systems that manage the inventory for a large retail chain like Walmart, or the software that runs an airline's reservation system. It's often possible to create a failure chain for a complex system, but as soon as you connect it to another complex system you are in uncharted waters.

As Tim Maughan points out in this article from OneZero, this has consequences.

The average daily trading volume on the New York Stock Exchange generally spans between 2 billion and 6 billion shares, with the average daily trading value in 2013 being approximately $169 billion. And the only way to deal with a market of this size and complexity has been a relentless adoption of automation — and the increased handing over of day-to-day analysis and decision-making to software. And in an industry like finance, which is preoccupied entirely with growth, these systems have led to an exponential increase in complexity — while human traders would traditionally average five trades a day, high-frequency trading algorithms can make 10,000 trades every second.

And those platforms of technology and software that glue all these huge networks together have become a complex system themselves. The internet might be the system that we interact with in the most direct and intimate ways, but most of us have little comprehension of what lies behind our finger-smudged touchscreens, truly understood by few. Made up of data centers, internet exchanges, huge corporations, tiny startups, investors, social media platforms, datasets, adtech companies, and billions of users and their connected devices, it’s a vast network dedicated to mining, creating, and moving data on scales we can’t comprehend. YouTube users upload more than 500 hours of video every minute — which works out as 82.2 years of video uploaded to YouTube every day. As of June 30, 2020, there are over 2.7 billion monthly active Facebook users, with 1.79 billion people on average logging on daily. Each day, 500 million tweets are sent— or 6,000 tweets every second, with a day’s worth of tweets filling a 10-million-page book. Every day, 65 billion messages are sent on WhatsApp. By 2025, it’s estimated that 463 million terabytes of data will be created each day — the equivalent of 212,765,957 DVDs.

So, what we’ve ended up with is a civilization built on the constant flow of physical goods, capital, and data, and the networks we’ve built to manage those flows in the most efficient ways have become so vast and complex that they’re now beyond the scale of any single (and, arguably, any group or team of) human understanding them. It’s tempting to think of these networks as huge organisms, with tentacles spanning the globe that touch everything and interlink with one another, but I’m not sure the metaphor is apt. An organism suggests some form of centralized intelligence, a nervous system with a brain at its center, processing data through feedback loops and making decisions. But the reality with these networks is much closer to the concept of distributed intelligence or distributed knowledge, where many different agents with limited information beyond their immediate environment interact in ways that lead to decision-making, often without them even knowing that’s what they’re doing.

It's a sobering article. At the end, Maughan writes:

So what’s to be done about all this? Over the coming months I’m going to both locate ways that we can try to increase our knowledge of the seemingly unknowable, as well as find strategies to counter the powerlessness and anxiety the system produces. Along the way I’m going to be talking to a lot of experts about everything from automated shipping and algorithmic trading to financial regulation and political resistance, as well as taking deep dives into how emerging technologies like artificial intelligence and quantum computing could make things better — or a lot worse. I hope you’ll join me as we explore how our systems work, how their complexities impact our lives, and how we can regain some agency within them.

Future articles will appear in a column called No One's Driving. I am definitely going to follow what he writes.  

You might also want to check out my blog post from May of this year, which was triggered by a profile of statistician Nassim Nicholas Taleb.  

I hadn't really thought much about our modern reliance on optimization and how it affects supply chains until reading Vernor Vinge's, A Deepness on the Sky, a couple of decades ago. In this novel, an interstellar trader visits a world that by most standards would be called a paradise. But it relied on a highly optimized system for food production. A hundred and fifty years later, when the trader returned, the planet had reverted to savagery because of a series of unanticipated catastrophes. That got me thinking.