Benedict Evans: AI Eats the World | Slush 2024

Thank you. Lasers, but somehow no one else has managed it. Now, I was asked to come and say insightful, sensible, clever things about AI and explain everything that's happening in AI, and not talk too fast while only taking half an hour. I'll probably manage two of those things, possibly one, but I'll see what I can do. I think a good place to start in thinking about where we are with AI today is this quote from Bill Gates from 18 months ago. He mentioned having seen two things that were revolutionary: the graphical user interface and ChatGPT, which is a pretty big statement.

And then on the other side, a month ago, two months ago, OpenAI raised at a valuation of $160 billion. I went and looked it up and worked out that it took Microsoft about 25 years to reach a valuation of $150 billion. OpenAI achieved that in just 12 to 18 months after launching their product. Again, this reflects just how quickly all of this has happened. This is a survey of people around the world who have used or at least heard of ChatGPT.

And this is kind of an unprecedented speed of adoption for a new technology. Something between a third and two-thirds of people have tried this or at least heard about it already. Some of that reflects the fact that you don't have to wait for everybody to go out and buy a phone or for telecommunications companies to build broadband; it's just a website that runs on the cloud, so you can go and use it. Nonetheless, it's a pretty amazing speed, growth, penetration, interest, and level of excitement around this phenomenon.

On the other hand, if you ask people what they actually mean when they say they've used it, you get a slightly different picture. The answer is that most people used it once and said, "Well, that was very clever," and then didn't go back. A much smaller number of people have figured out a way to make this part of their life every single day. You see something similar, I think, if we look at the enterprise. Every big company has some kind of pilot or trial, which might just mean that their Chief Information Officer is paying for a ChatGPT account, but a smaller number of companies have actually put something into deployment.

And of course, this is just about having one thing in deployment. It's nothing like moving your entire system over to generative AI. So we've reached a point where, on one hand, everyone is very excited and interested and thinks this is important, but we haven't actually reached the stage where everyone is using it or spending money on it—at least not yet. This summer, that led to a wave of people writing stories like this, questioning whether we are quite sure we want to spend half a trillion dollars on this when we don't have product-market fit yet.

And some of this is just kind of the normal hype cycle. For anyone who doesn't know the idea of the Gartner Hype Cycle, I asked ChatGPT to draw it for me. Apparently, it will take time to reach the peak of productivity. So we'll see how this evolves. But if we step back from that, this is probably the more relevant cycle to be thinking about.

So, we'll kind of see how this will evolve. If we step back, this is probably the more relevant cycle to consider: every 10 to 15 years, the tech industry goes through a platform shift. This shift becomes the center of all innovation, investment, company creation, and change in the industry. These shifts included mainframes from the mid-60s to the late 70s, then the PC, followed by the web, and then smartphones. Now, pretty much everyone in tech believes that generative AI is the next big platform shift.

However, that's about the only thing that we really think is clear. After that, all the questions are wide open. In fact, we're still trying to work out what the questions are. What I've tried to do in the rest of this presentation is to group some of the questions and determine what we might ask—if not what the answers might be—into three categories. For example, how far is this going to scale? This is, of course, the foundational question for everything else.

How are these models actually useful? What are you supposed to do with them? And then, of course, how do you actually deploy this? For the people in this room, that means: How do you build a product and a company from this technology? How do you take it to market? Looking at the first question, how far will this scale?

As I think we all know, or certainly everyone from the nodes, we made this work by making it much bigger than anyone thought was remotely feasible. We expanded it significantly, utilizing much more data and much more computing power, which led to better results. The question now is that we don't actually have any theory explaining why this works, but it has worked so far. So the question is, will it continue to work? One perspective suggests that no, it will inevitably scale down like anything else; it will slow down, and eventually, this will just be software.

And then there's another view that says, no, the scaling will just keep working, and at the extreme, that might mean that large language models can essentially do everything. They could replace all other software and everything else. You can spend hours of your life watching machine learning scientists argue about this on YouTube, and all you'll really conclude is that they don't know. There's Kevin Scott saying, "Well, it's worked so far," and Sergey Brin saying, "Well, yeah, but just because it's worked so far doesn't mean it will keep working." We just don't know.

In the last week, we've had a flurry of stories from inside some of the big labs suggesting that it may have already stopped working. There's a lot of debate about what that means, and it seems a bit premature to assume that everything has suddenly come to a halt. Of course, some of this is simply due to the fact that scaling these models presents a number of practical difficulties. It takes time to build gigawatts of additional power, and it also takes time to acquire the GPUs.

It's not clear how much training data is left and if we can use synthetic data or what other kinds of data we could use. And, of course, there's that foundational science question: If you make the model ten times bigger, will you get ten times better results? We just don't know. However, we're going to find out.

And so, both Google and Meta are stating that the downside of not investing is greater than the risk of allowing someone else to take over. The risk of spending a substantial amount of capital expenditures that ultimately proves unhelpful for several years is significant. A quote from Silicon Valley resonates here: "I don't want to live in a world where somebody else is making the world a better place faster than we are." This sentiment captures some of what's happening, particularly for individuals like Mark Zuckerberg, who has a controlling stake in the company. The fear driving this perspective is the apparent lack of a fundamental moat. There doesn't seem to be any substantial network effect or reason why one company would achieve a winner-takes-all outcome.

This is an internal memo from Google from last spring. We don't have a moat, and neither does OpenAI. However, it's not entirely true because the moat is capital. This is the CEO of Anthropic, very casually discussing $5 or $10 billion for model building. He is now talking about $100 billion for model building, which seems a little reminiscent of the 1990s.

And, slightly more tangibly, Mark Zuckerberg mentioned this summer that the next version of LLaMA will require 10 times more compute, because that's just how you build them. To provide some numbers, this is not a semiconductor presentation, but LLaMA 3.1 utilized 16,000 Nvidia GPUs, which amounts to about half a billion dollars worth of chips, plus the data centers on top. The clusters that people are using today are more like 100,000 GPUs. So, there's quite a lot of money, quite a lot of chips, and quite a lot of infrastructure involved. Of course, a significant portion of that money is going to Nvidia.

They updated their numbers last night, so the figures continue to rise consistently. This is kind of an unprecedented chart; you have to go back to around 1999 or 2000 to see charts that look like this. Most of that money is coming from the four big platform companies, which will spend something over $200 billion in capital expenditures this year—an almost $100 billion increase in capital expenditures from last year. They all expect to spend even more; at least, that's what they indicate at the moment. It's interesting to consider what that means for Microsoft in particular.

This is capital expenditure to sales for a typical telecommunications company, Verizon, in the USA. We think of telecommunications companies as being infrastructure companies. They spend about 15% of their revenue on capital expenditure every year. Microsoft will spend over 25% of its revenue on capital expenditure this year. It has transitioned from being a company that sold you a $1 CD for $100 in a cardboard box to a company that actually has to build infrastructure in order to sell its products.

Of course, when you have these kinds of surges of money and people talking about hundreds of billions of dollars in construction, private equity, bankers, and Wall Street start to get interested. A little company called Andreessen Horowitz, you might have heard of, apparently has 20,000 GPUs in order to win AI deals. And all of this, of course, is happening in the context of a market where we still don't really know how this is going to work. All of the science is still changing, and all of the engineering is still evolving. This started working two years ago, and that was amazing; then what?

Well, we have a huge amount of investment into creating better results, agent models, and scaling, of course, that I've talked about, as well as multimodal models. On the other hand, there is a significant investment in trying to make this slightly less expensive, as we've applied a large amount of data center engineering to what was previously a science project. That cost, I think, is really important here because the last time the software industry had marginal costs was back when this is what software looked like, kind of before most of us were born. This was the last time it cost you money every time a user pressed "OK" and waited for something to happen. The whole consumer internet has grown up on the premise that you basically don't have marginal costs unless you're YouTube or someone similar.

But LLMs, at least for the moment, still kind of do. This leads us to charts like this one. The horizontal axis represents cost, while the vertical axis indicates quality for the succession of models that have been released by OpenAI. There’s a relatively small increase in model quality, but a huge increase in model efficiency—certainly an order of magnitude, and perhaps even more than that—in how inexpensive these models are, depending on how you analyze it. In parallel, there are also many other models available now.

So, these are just the models from the other three big companies. I could probably add, I could probably double the number of points on this chart if I included everything that's been released in the last two years. But what's really happening amidst all the noise is that we're seeing a convergence. On the one hand, everybody has a top-tier model, and then everybody has an affordable model. The thing is, you can have a model that's almost as good for about 5% of the price.

So, you've got a very steep cost curve. What you also have on this chart is Meta, which is giving the model away for free or almost free. There's an old saying in the tech industry: everyone in tech is giving somebody else's business model away for free. That's what Meta is trying to do with open source. They're attempting to transform large language models (LLMs) into commodity infrastructure available from anyone and sold at marginal cost.

Cost. Apple is doing something similar from the opposite direction. Apple sells high-performance edge computing, so they want to turn large language models (LLMs) into commodity infrastructure that runs on your iPhone, making it just another API call. In the last two years, we have experienced a sort of model boom, where you can choose better, faster, and cheaper options. You can have the best model, the best inexpensive model, or the best model that fits on the edge, and that's a significant story for the upcoming year.

And of course, there's another model that improves every couple of weeks. Now, if you step back from this, especially if you're not a machine learning scientist or a data center engineer, I think there are a couple of fairly simple observations one can make. Here we have Michael Corleone telling us that if anything in life is simple, it's that you can kill anyone. I think the technology industry's equivalent of this is that semiconductors are a cyclical industry.

Commodity technology tends to converge on marginal cost, making it even more of a commodity. Meanwhile, every new technology tends to produce a bubble. It's very easy to identify a bubble, but much more difficult to predict when it will burst. However, it's relatively straightforward to point to the trends we've observed in the past.

Now, if we set all of that aside, we can ask, "Okay, this is great; there are going to be lots of models, and they're going to get cheaper, but what are we going to do with them? How exactly is this useful?" To answer that question, I think it's helpful to revisit the last wave of machine learning that began nearly a decade ago with image recognition. I would attend events like this, show people demos, and say, "Look, isn't this cool?" The big companies in the audience would respond, "Well done."

We're very happy for you. You can recognize a dog. We are a very large German reinsurance company. We don't have pictures of dogs. What are we supposed to do with this?

It took a while to work out that the right level of abstraction, the right way to understand this, was that this is pattern recognition. Once you understand that this is pattern recognition, you can start thinking about what we can turn into pattern recognition. For the last decade, every software company has basically been saying, "We can turn that problem deep inside this industry, or that department, or that corporation into pattern recognition. This is how we get the data, and we'll build a company around that." We all understand this now. I think we're at the same point now with generative AI and large language models.

Okay, that's a great demo, and I can kind of see why that's cool, but I'm not quite sure how I should understand this at a conceptual level. What is this? How do we think about what we could do with this? We also have to remember the things that we can't do with this. You can't use this as a database.

This is still my favorite example of hallucinations. Air Canada used a large language model (LLM) to build a customer support bot. A customer asked about the return policy, and the support bot provided them with a return policy. It was a very good return policy.

Unfortunately, it wasn't Air Canada's return policy. The customer had to go to court to get a refund because Air Canada claimed that the customer should have read another part of their website, not the part they had referred to. The judge did not find this argument very impressive. There is a challenge here in understanding what these systems are doing and how to build products that can automate tasks but might not always get it right.

One answer is, well, just make the models better, which would be nice, but we don't really know if we can do that. It's not particularly clear that a probabilistic model can produce deterministic answers anyway. That seems a little bit like Zeno's paradox. So you ask, on the one hand, what are the use cases where there isn't a wrong answer? Or use cases where the errors are easy to see?

On the other hand, how do you abstract this away? How do you build product design around the fact that it might be wrong? How do you manage that, providing tooling, analytics, metrics, and user experience to address those kinds of questions? The big area that is particularly interesting at the moment is asking whether you can use this for general search. Can you use this to replace Google?

And there are many unknown answers to that. What kinds of queries matter if they're wrong? Where does it matter if you can't tell that something is incorrect? How much filtering is needed for this? We don't really know, and neither does anyone at OpenAI, but Alphabet made so much money from search that it seems worthwhile to try to find out.

And so, if we go back to my question: how do we work out the right level of abstraction to understand this? We can say, well, it's reasoning, synthesis, and summary, or it might be. We can step back and say, okay, this automates a class of tasks that we couldn't automate before, and we're trying to determine which tasks we can automate. One of the ways I used to talk about the last wave of machine learning was to say that this provides you with infinite interns. For example, you would like to listen to every call coming into the call center and determine if the customer is angry.

You don't need an expert for that. You could just use a 10-year-old, except you could never automate it. Machine learning allows you to automate a whole class of tasks that typically require a mammal brain. You can analyze generative AI by examining its conceptual pivot points. This raises the question: what does it mean that creating certain kinds of content is going to be free?

What does it mean that translation will now be more or less perfect and free? What does that have to do with science, pop culture, or the nature of the web? You can also approach it from a top-down perspective, like a macroeconomist. Additionally, you could discuss which industries have low labor productivity. Which industries seem to be filled with many people doing repetitive, mundane work that we were unable to automate before but will likely be able to automate now?

And then you can kind of go across industries and produce a plot like this. I think the challenge with this kind of exercise is to imagine that you've done this with the internet in 1995. What would you have gotten right, and which things would you have completely missed? Can you really model the evolution of a transformative technology like that? I’m not sure that you can.

And meanwhile, if you're actually sitting in a big company saying, "Why should I double my Microsoft spending?" you come across quotes like this. The CEO of Chevron employs two or three hundred thousand people, and Microsoft is saying, "You should give 200,000 people a license for Copilot and double your Microsoft spend." And he's thinking, "Yeah, but why? What value are we actually going to get from that? What are they going to do with this?"

I think perhaps a slightly more useful way to look at this is to try to unbundle it a little bit. This is a wonderful advertisement from the 1970s for VisiCalc, which was the first successful software spreadsheet. What you do with this is type in a grid of numbers, and then you can change a number here, and all the other numbers on the screen change. When we look at this today, we think, "Yes, but if you were an accountant in 1978, that was your entire life." You could have spent weeks doing that, and now this software just does it for you in about ten minutes.

The founder of this company has many stories about accountants who would complete a one-month project in just two or three days, then take three weeks off to play, and return to say, "I've finished now." If you were an accountant and witnessed this, you would definitely want to get involved. However, if you were a lawyer, you might think, "That's very clever; maybe my accountant should see this, but I don't have that use case. That's not relevant to my work." I believe this is how the other 90% of people react when they see ChatGPT.

They say, "Well, that's very clever, but I don't do that all day." So if we ask who has that use case today for generative AI, particularly for ChatGPT, there are some areas that clearly have that use case. Software development is working right now, and companies are already discussing 20% to 30% efficiency gains. People in marketing are saying something very similar. They don't really have wrong answers, and if there are wrong answers, they're kind of easy to see.

A lot of people in customer support are very interested in this. However, as I mentioned earlier, you have to be careful about that. Then you have early adopters across every industry who are starting to experiment with it and figure out how they can utilize it. This leads to some statistics, such as how many employed people in the USA are using generative AI and the extent of their usage. You can see that in some fields, around 20% of people are at least willing to claim that they are using generative AI.

Now, of course, you could suggest which fields people are most likely to exaggerate in, and is anyone in management going to admit that they're not using ChatGPT yet? Maybe not. On the other hand, in a field like law, it really matters if this tool provides information that isn't quite right. So, we see this kind of widespread adoption. For everyone else, I think we have a problem: we have a technology, and we're asking the user to figure out what to do with it, which brings us to this great quote from Steve Jobs.

That's not normally the way we go about deploying technology. We typically start from the other end. We begin by determining what the customer experience should be and then work back to the technology. Right now, however, we're trying to do the opposite. This brings me to my third section: how do we deploy this?

How do we build products with this? How do we build companies with this? As we ask that question, there are a number of common patterns for every technology. How do we consistently approach this? First of all, you try to absorb it.

The incumbents try to make it a feature. We bolt it into the existing business. We automate the things we already do. Then, over time, we start changing the way we do business. We begin creating new products.

You start creating new ideas and maybe unbundling the incumbents. You unbundle something out of SAP, Google, or Oracle. Then, every now and then, someone will come along and fundamentally redefine what the market is. For example, Airbnb comes along and changes what you mean when you say "hotel." That's the aspect that is kind of hard to predict.

And so you can kind of ask, as you look at this, are you asking an Accenture kind of a question or a Bain, BCG, or McKinsey kind of question? Is this a question for your CIO or is this a question for the CEO? Is this top-line innovation or bottom-line innovation? The answer is that it might be all of those, depending on which part of the company you're talking to, which part of the industry, which kind of consumer, and which kind of problem. It might be all of those at once.

The first answer, of course, is that you call up McKinsey. Call up Accenture and give them an RFP. Accenture reported that they are now generating a billion dollars a quarter from generative AI, although I'm not quite sure what they define as generative AI. Regardless of what Accenture calls generative AI, they are indeed making a billion dollars a quarter in that area. This leads to some rather conventional enterprise procurement questions: Do you buy or build? Do you choose big companies or small companies?

Will Google own the whole thing? And the most important question of all: What does this do to our EPS? Questions like these lead to charts like this one. For most people in the world, the cloud is old, boring, and finished—it's something their parents worked on. However, in actual large companies, the cloud still accounts for only about a quarter to a third of enterprise workflows.

Even though CIOs, of course, always desperately hope that they'll be able to get the budget to double that in three years' time, they somehow never seem to manage it. We see something very similar regarding what CIOs think is going to happen with generative AI. Perhaps a quarter of them believe they'll have something deployed by the end of this year. Another quarter think it probably won't be until 2026 or later. It takes time to deploy this technology.

It takes time to figure out what to do with it. It takes time to build it into products. When you accomplish that, you create a tidal wave of new products. The Software as a Service (SaaS) industry, as I mentioned, is still only about a quarter of enterprise workflows, which means that the typical large company department now has 50 to 75 applications. Overall, a typical large company has between 400 to 500 SaaS applications.

What all of these companies are doing is unbundling Excel, email, SAP, Salesforce, or Google and turning those into workflows that they've managed to automate. We saw this with machine learning over the last 10 years, and we will see it again with large language models (LLMs) as people use them to unbundle some tasks and automate them once more. Hence the classic quote from Jim Barksdale: "There are two ways you can make money: bundling and unbundling," and that's what startups do. Now, the counter to all of this, what you might call the AI maximalist view, is to say, "Yes, but what about the scaling?"

So let's come back to this slide. Is this stuff going to keep scaling or not? If the scaling continues to work, then what happens? Well, it might be that the LLMs just become the whole thing. The LLM sits on top, runs everything else, and makes everything else an API.

And you might have radically more tasks being accomplished with software without needing radically more apps, companies, or products. You could simply ask ChatGPT to do it for you. On the other hand, what we have now is conceptually close to where we are headed. The LLMs (large language models) are likely going to be just another API call. You will build software, and you will use an LLM to perform a task for it, similar to how you use API calls for storage, calculations, or even image recognition.

And that brings me to another Steve Jobs quote, which is: "It's not actually the customer's job to know what they want. It's not the customer's job to work out what to do with this." I think there's a challenge in giving everyone at Chevron an LLM today. The classic way we determine how to use technology is that startups adopt the technology, invent use cases, identify problems, and find innovative approaches to utilizing it. However, when you provide everyone with an LLM, you are forcing the users to create the use cases and figure out what they are supposed to do with it, which isn't really how innovation works. If you look at charts like this, which track startups going through Y Combinator, it shows a bet on there being thousands more companies and thousands more use cases, with LLMs becoming just another API call.

Because if ChatGPT could do the whole thing, then you wouldn't need all these AI companies. Now, the extreme case here, of course, is that this just becomes a feature. On the right here, we have a screenshot of Apple's new AI-powered writing tools. I look at this and I think this is spellcheck, or this will become spellcheck. Rewrite this, proofread it, summarize it, check the spelling, and check the grammar.

These just become features that disappear into the background. I think there's a progression here with AI. To begin with, it's AI, and it's amazing, sexy, exciting, and cool. Then, a little bit later, it's just smart—smart suggestions, smart formatting, smart recommendations, smart layouts, smart summaries. Eventually, it becomes automatic.

So it's autocorrect or auto format, and then it becomes just software. It's just what's always been there. I think if we try to imagine some models for what products we could develop with this technology, I suggest three categories. The new features could include rewriting my emails, summarizing reviews, suggesting ideas, or creating new content.

There will be completely new tools, capabilities, and features, just as we have had for the last 50 years. Then there's this kind of maximalist view that says, "I can just go to JackGPT-7 and say, 'Yeah, so I'm moving to Singapore. Can you buy me a house and sort out the visa?'" And it will just do it. That’s not quite science fiction, but it’s still definitely an outlying perspective. Hence, if I pull all of these threads together, I've spent a lot of the last 18 months talking to big companies and groups of people.

I have a lot of questions about AI, particularly generative AI. It seems to me that all questions about this topic have one of two answers: either it will work just like every other platform shift, or we don't know. So, should we buy from Google? Will Google control the whole thing?

Will there be any startups? Do we need our own national strategy for this? Do we need our own national strategy for AI? To me, that's like asking if we need our own national strategy for this. Do we need our own national strategy for AI?

To me, that's like saying, "Do we need our own national strategy for SaaS? Do we need our own national strategy for SQL?" That seems like the right way to understand it; this is just another platform shift. The other set of questions are: How much will the model scale? What happens to error rates?

Is there enough data? How much energy will this need? Will we be able to have models that can train themselves continuously? There are all sorts of scientific questions regarding how effective this will become. The answer is, I don't know, but neither does anybody else. No one knows, and we'll just have to wait and find out in a couple of years.

Meanwhile, it is worth noting that all the things we were excited about two years ago, before ChatGPT 3.5 launched, are still relevant and ongoing. One way I tend to think about this is that the tech industry is always enthusiastic about what is expected to happen in 2025 or 2030. Two or three years ago, for example, that would have included cryptocurrency. Obviously, no one here believed in crypto, but there were others who thought it would become a significant phenomenon. Additionally, there was excitement surrounding the metaverse, as well as augmented reality (AR) and virtual reality (VR).

Now, of course, it's all generative AI. Meanwhile, most of the actual software industry is deploying ideas from 2010 and 2015: SaaS, cloud, automation, workflows, business process management, and collaboration. Google Docs but for video, Google Docs but for other applications. We identified this problem deep inside the HR departments in the finance industry and built a SaaS company to solve it, which we believe will become a billion-dollar company. That's what most actual software companies are doing.

And then the rest of the economy is being overturned by ideas from 2000, like wild, crazy, bizarre ideas—like maybe people will watch TV on the Internet. This was definitely a crazy idea in 2000 and took 25 years to happen. So, if we think about what some of those things look like, firstly, I did this chart last year: the metaverse is still a focus for Meta. Meta has invested $17.5 billion in the last 12 months in the Reality Labs division, which is significantly more than Apple spent to launch the iPhone.

But meanwhile, there’s a bunch of other stuff that’s still going on. So this is kind of the most boring chart in the tech industry. It went up one percentage point every year for 20 years, except for that little bit in the middle. But that’s just now 20-25% of retail. It’s trillions of dollars of value.

And when that happens, all sorts of other things change. So Xi'an is now almost certainly the world's largest apparel retailer, aggregating Chinese manufacturers and shipping directly to the West, with $45 billion in GMV last year. Their IPO prospectus will come out any day now, and then we'll find out where they are today. Shopify did over $150 billion in GMV last year, which makes it about a third of the size of Amazon, and that's enabling tens of thousands of small businesses to produce the best-in-class e-commerce experience.

Amazon itself now makes more money from advertising than almost anything else, certainly more than it makes from retail. It generated over $50 billion from advertising on the Amazon.com site in the last 12 months. The software industry is overtaking television. YouTube generated $50 billion in advertising and subscription revenue in the last 12 months, which is significant—more than any global media company except Disney. If you were to exclude sports from this comparison, YouTube would probably surpass Disney as well.

Software is also eating cars. I'm old enough to remember when we were excited about autonomous cars. Waymo is actually doing 300,000 robotic taxi trips a month here, so Waymo has actually kind of got something working, maybe, which is more than some other companies we could mention are talking about. The interesting question, though, as we look at cars, is that the car industry is going to transition to electric, and that's going to happen. But who is it that's going to lead that change?

Is that going to be software companies, or is it going to be a whole new generation of car companies? How is this going to work? I think there's a generalized question here, which is that technology changes all the parameters. Technology alters all your assumptions about how these industries work, but then all the questions for that industry are no longer tech questions. All the questions for TV now are TV industry questions.

All the questions for Xi'an are apparel questions. Is the Chinese car industry going to do what the Japanese car industry did in the 1980s? This is a car industry question; this is not a software industry question. Coming back to a quote I used earlier, Larry Tesler, who is a pioneer of computer science and AI, said, "Intelligence is whatever machines haven't done yet. AI is whatever doesn't work yet."

And he's partly making a philosophical point here: we tend to redefine intelligence as anything that machines do. If machines can do it, it can't be considered intelligent. He also points out that as soon as something works, people say, "Well, that's not AI; that's just software." In the 1970s, data was not considered AI; basic operations were seen as AI. Now, they're just referred to as software.

Ten years ago, image recognition was considered AI. I don't think image recognition qualifies as AI anymore. I believe image recognition is now just software. The same goes for voice recognition and pattern recognition. Therefore, by default, the process we will undergo in the next couple of years is that large language models will similarly come to be viewed as just software.

They won’t be intelligence. They’ll just be something that computers do. And so I think you could propose a more general version of this thesis: that technology is, indeed, whatever machines haven’t done yet. I’ll finish with one final question and a final chart, which I will use a great deal because it took me hours to type this data in, and I couldn’t get ChatGPT to do it for me.

This is the number of people employed in the USA as elevator attendants. To begin with, there are more elevators, so you need more elevator attendants. Then, the elevators become automated, and you don't need as many. So, I'll ask you a question: When is the last time you used an automatic elevator?

When is the last time you got into an elevator, pressed a button, and said, "Ah, I'm using an AI elevator today"? In fact, an electronic elevator, an automatic elevator, or was it just an elevator, and that's just the way the world has always been? That's probably how all of this is going to evolve as well. With that, I will say thank you.