O’Reilly on Paradigm Shifts

Tim O’Reilly is one of the deepest thinkers about the continuing evolution of technology and its impact on economic and social domains.  Two years ago, he gave a talk at a Warburg Pincus gathering on "The Open Source Paradigm Shift".  A version of the talk will soon be published in a new book, Perspectives on Free and Open Source Software.  The talk pinpoints some of the fundamental changes re-shaping the software industry.  Early in his talk, O’Reilly drives home a different way of thinking about software:

I have a simple test that I use in my talks to see if my audience of computer industry professionals is thinking with the old paradigm or the new. "How many of you use Linux?" I ask. Depending on the venue, 20-80% of the audience might raise its hands. "How many of you use Google?" Every hand in the room goes up. And the light begins to dawn. Every one of them uses Google’s massive complex of 100,000 Linux servers, but they were blinded to the answer by a mindset in which "the software you use" is defined as the software running on the computer in front of you. Most of the "killer apps" of the Internet, applications used by hundreds of millions of people, run on Linux or FreeBSD. But the operating system, as formerly defined, is to these applications only a component of a larger system. Their true platform is the Internet.

It is in studying these next-generation applications that we can begin to understand the true long-term significance of the open source paradigm shift.

If open source pioneers are to benefit from the revolution we’ve unleashed, we must look through the foreground elements of the free and open source movements, and understand more deeply both the causes and consequences of the revolution.

Artificial intelligence pioneer Ray Kurzweil once said, "I’m an inventor. I became interested in long-term trends because an invention has to make sense in the world in which it is finished, not the world in which it is started."

I find it useful to see open source as an expression of three deep, long-term trends:

• The commoditization of software
• Network-enabled collaboration
• Software customizability (software as a service)

Long term trends like these "three Cs", rather than the Free Software Manifesto or The Open Source Definition, should be the lens through which we understand the changes that are being unleashed.

Much later in his talk, he highlights one of the key economic consequences of this paradigm shift:

The lessons of previous paradigm shifts show us a more subtle and powerful story than one that merely pits a gift culture against a monetary culture, and a community of sharers versus those who choose not to participate. Instead, we see a dynamic migration of value, in which things that were once kept for private advantage are now shared freely, and things that were once thought incidental become the locus of enormous value.

Towards the end, he urges us to consider the open software movement in a much broader historical, economic and social context:

In short, if it is sufficiently robust an innovation to qualify as a new paradigm, the open source story is far from over, and its lessons far from completely understood. Rather than thinking of open source only as a set of software licenses and associated software development practices, we do better to think of it as a field of scientific and economic inquiry, one with many historical precedents, and part of a broader social and economic story. We must understand the impact of such factors as standards and their effect on commoditization, system architecture and network effects, and the development practices associated with software as a service.

We are just beginning to understand the contours of a new software world where services are developed and managed collaboratively, building in layers on the contributions of others, pulled by users over networks to a broad array of access devices whenever and wherever they are required and tailored by users to meet their individual needs. Many believe this will lead to a fragmented world. That may only be partially true.  Fragmentation creates its own opportunities. O"Reilly appropriately reminds us of Clayton Christensen’s "law of conservation of attractive profits":

When attractive profits disappear at one stage in the value chain because a product becomes modular and commoditized, the opportunity to earn attractive profits with proprietary products will usually emerge at an adjacent stage.

In fact, I believe fragmentation will give rise to global and highly concentrated businesses focused on providing increasingly valuable services around testing, rating, reputation, mediation, aggregation and orchestration.