The End of (Human) Software

Human software is dying. Which is good. It’s probably useful to consider the implications of this terminal diagnosis and plan accordingly. Last Updated: 2024-06-04

Historical Context

First, what is software? While there is a (valid) semantic argument to be made here, it would be helpful to provide a sloppy, imperfect, but useful definition. Software Developers spend at least some of their time writing in programming languages. These programs are then compiled into other (lower level) programming languages, which are in turn ultimately compiled into binary - let’s exclude binary (as it’s hardware dependent and not human readable), but include all the other good stuff in our definition of software. This stuff – programs, scripts, applications – is what I mean when I say software. Software developers also spend lots of time doing other things (managing infrastructure, communicating with stakeholders, writing documentation, etc), all of which will also be impacted by AI, but for the purposes of this argument I’ll mostly be focusing on software. To understand why human software is dying, we should understand a brief history of progress in software development (this will give us some historical data points that, combined with some more recent data, will allow us to plot a rough future trajectory). The history of software development is characterized by abstraction (aside: this is probably the case for all disciplines, but is an especially useful framework in this context). The above pictured men were some of the earliest pioneers in this effort, abstracting the stateful properties of the universe into simple voltage (the transistor). Companies like Intel then abstracted a grouping of multiple transistors into a single unit, the CPU. Any ‘software’ developed had to be tailored for underlying hardware, making hardware and software inseparable and thus making it impossible to abstract software from hardware.

The CPU and the resulting development in personal and general computation allowed software to escape the grip of hardware. Assembly and the x86 instruction set provided a foundation, an interface by which to interact with the CPU. C, another product of Bell Labs (born in the 1970s), compiles into assembly. The core of Python, one of the most widely used programming languages, is CPython, an interpreter written in C. The history of software development is characterized by a constant upwards climb towards increasing levels of abstraction. And it’s easy to see why: productivity scales (roughly) with abstraction. Software is only ever instrumental to solving some problem, and abstraction allows for a more direct implementation of a solution. I’m thirsty - I can just walk to the sink and pour myself a glass from the tap, I don’t have to trek a half mile to the lakefront with some vessel and schlep it back. In this case, thirst is my problem and colossal efforts in civil engineering and modern capitalism is the abstraction that I’m leveraging to solve my problem. Walking 10 feet and opening a valve is a far more direct and effective means of solving my problem. Abstraction, as should be obvious, is powerful.

Current State

Having established what software is and the instrumental value of abstraction, we have the requisite context to understand recent developments in software development, and how their impact will ultimately kill software. The first technology to address is Github Copilot (and by extension, current foundation LLMs): trained on a corpus of (most) publicly available codebases, Copilot acts as a code autocomplete, suggesting content to the software developer. Copilot is in its infancy: despite this, preliminary research suggests that Copilot writes 30-50% of all lines of code produced by its users. From personal experience, Copilot excels in simple, structured environments (like writing test code), and helps to streamline more monotonous aspects of software development (refactoring, common logical steps, initialization, etc). Copilot still struggles in complex or uncommon environments. But overall, this is a useful tool that developers increasingly swear by, and it shows that LLMs can produce robust and useful code even at this early stage. In short, this is a proof of concept that software development is a tractable field for AI, and that AI powered tools can meaningfully improve the productivity of software developers today. But Copilot is just the beginning. Beyond simple code autocompletion, we’re beginning to see the first glimpses of AI software developers. Devin from Cognition AI or Coderabbit are examples of more fully featured AI agents that are attempting to automate much more complicated components of software development. These products are far from complete, and as of yet don’t pose a credible threat to software development as a human endeavor. While there are numerous reasons why Devin isn’t beginning to approximate a human software developer in capability, most of its shortcomings can be attributed to underlying model capabilities, and shortcomings specifically in the domain of reasoning quality. Because Devin, Coderabbit, and many other current agents rely on chain of reasoning style recursive prompting, reasoning deficiencies are often compounded over the course of a long chain of thought. Even if Devin reliably gives responses that are only ~5% worse than most human software developers, by step 4 or 5 in a reasoning chain it’s not uncommon for such agents to have completely lost the plot of whatever task they’ve been directed towards. What LLM agents generally do not lack, however, is source knowledge: most SOTA LLMs are drop in replacements for search engines w.r.t. most basic or intermediate technical questions. So one must imagine that as SOTA models improve (specifically in reasoning quality), it’s quite possible that AI agents like Devin become capable at accomplishing most, if not all, of the important tasks of software development.

The Future of Human Software Development

So how does this bode for human software developers (and by extension, all human intellectual labor)? We can take for granted that a sufficiently advanced AI would make completely redundant all human labor, but what about less advanced yet still rapidly improving AI in the near term? To this point it seems the answer is decidedly more complicated. Humans have always served in interstitial roles, filling in for the myriad and shifting deficiencies of non-human capital. This is tautological: just in the way that a medieval peasant farmer served to fill the gap between the scythe, plow, and oxcart, a software developer (or salesperson, business analyst, CEO, etc) fills the gap between internal or external software, hardware, and organizations. In the modern technocapital corporation, as in any time before, humans are engaged in exactly one or both of only two tasks: (1) helping to further automate things or (2) doing things that cannot be cost effectively automated (yet). To this point, AI is progressing substantially faster than most firms are automating, so there will almost necessarily be roles for clever human automators for quite some time, potentially right up until AGI. To reiterate: up until the advent of AGI, it is very likely that there will be strong demand for smart human software/AI orchestrators because essentially every organization on the planet is and will be moving slower than leading AGI labs.

The Future of Software

So if human software is dying, what will replace it? People will continue to spend more and more time glued to screens, so there will continue to be an insatiable demand for software. Here are some guesses:

• Software teams will shrink As the productivity and capabilities of exceptional devs increases due to access to improved tools, big software teams will become increasingly unnecessary and burdensome. Individuals will be able to architect, implement, maintain, and scale software systems that would previously have been infeasible for even midsize teams to manage. There will be billion dollar technology companies with headcounts in the single digits. Legacy organizations will continue to cut headcount, but will always lag substantially behind the efficiency of lean startups.

• Software languages get weird In the near/mid term, languages with the best corpuses of training data will see disproportionate use, as models will be able to produce code in those languages most reliably. What makes one language’s corpus better than another? The answer is not as simple as sheer quantity (although quantity is not the worst problem to have) - quality matters as well. It could be the case that corpuses for the most popular languages (like Javascript or Python) are sufficiently poisoned by bad code samples such that their disproportionate quantity is more than offset by their disproportionate quality (or lack thereof). Despite this, I suspect quantity is still the most important variable, but only time will tell. In the longer term, software begins to diverge from traditional languages altogether. “Programming” will happen entirely in natural language, at a conceptual level, and models will produce fully formed executables without any intermediary step. The purpose of a software language is, after all, to provide a more abstract and easily conceptualized interface to a human software developer - replace the human and all you’re left with is a redundant, slow, and needlessly leaky abstraction layer. Outside of very narrow applications where human verification is required (like cryptography), no task will necessitate writing “code” - instead natural language will become the sole (universal) interface.

• Software performance gets worse before it gets better Just as is the case with software languages, things will get weird/worse before they get better - the volume and quality of code produced by LLMs in the near term will introduce bugs at a greater frequency, produced unoptimized code, and allow otherwise bad developers to do more damage to codebases than previously. This will, however, quickly shift as models improve and increasingly less code is required. Once models are able to skip leaky abstraction layers and are able to write most/all of the code in an application, software performance will improve colossally. AI software agents will be able to produce perfectly formed assembly (or even binary), and implement what would to a human developer be incomprehensible optimizations. The currently substantial effort to make software maintainable, reviewable, and extensible will be made completely redundant.

• Personal software While not immediately obvious, AI already makes all of the most widely used software in existence personalized through tailored feeds and recommender systems. Personalization will continue to climb up the interface hierarchy, however, as users increasingly flock to the chat-centric interfaces of the most powerful LLMs. Just as LLMs are slowly cannibalizing search, they will begin to be increasingly used as content curators/aggregators, integrating with media and social networks. Eventually they will represent a new form of operating system altogether, a much more intuitive, predictive, and flexible interface to all underlying software and information. All current practical concerns about software (files, directories, downloading & updating applications, accounts/logins) will become optional, and ultimately fade away.

• API supremacy To this end, many existing user interfaces/frontends will eventually become outdated - why build a complex and costly interface when the end user will increasingly use an agent to extract and reformat data into a central, custom tailored chat-like interface anyways? In the same way that no user cares about seeing the raw feed of all uploaded Youtube videos or Instagram photos, users will become ambivalent to the source of information/content as it is abstracted away from their interactions. As such, novel data sources or services will begin to deprecate their user interfaces in favor of simpler, more easily scaled APIs.