AI Snake Oil—A New Book by 2 Princeton University Computer Scientists

Hello, it's Eric Topol with Ground Truths, and I'm delighted to welcome the co-author of a new book called *AI Snake Oil*, Sayesh Kapoor, who has written this book with Arvind Narayanan of Princeton. So welcome, Sayesh. It's wonderful to have you on Ground Truths. Thank you so much. It's a pleasure to be here.

Well, congratulations on this book. You know, what's interesting is how much you've achieved at such a young age. Here you are, named in Time 100 AI, the inaugural edition, as one of the eminent contributors to the field. And you're currently a PhD candidate at Princeton, is that right?

That's correct. Yes, I work at the Center for Information Technology Foundation, and you're currently a PhD candidate at Princeton. Is that right? That's correct, yes.

I work at the Center for Information Technology Policy, which is a joint program between the Computer Science Department and the School of Public and International Affairs. So, before you started working on your PhD in Computer Science, you were already involved in this field, right? That's right. Before I started my PhD, I worked at Facebook as a Machine Learning Engineer. You're taking it to a more formal level here. Before we delve into the book itself, can you provide some background? I know you described it in the book, particularly why you decided to write one entitled "AI Snake Oil: What Artificial Intelligence Can Do, What It Can't, and How to Tell the Difference." Absolutely. For the longest time, both Arvind and I had been examining how AI works and how it doesn't work.

What are cases where people are somewhat fooled by the potential for this technology and fail to apply it in meaningful ways in their lives? As an engineer at Facebook, I had seen how easy it is to slip up or make mistakes when deploying machine learning and AI tools in the real world. I had also seen that, especially when it comes to research, it's really easy to make mistakes, even unknowingly, that inflate the accuracy of a machine learning model. For example, one of the first research projects I undertook when I started my PhD was to explore the field of political science and the subfield of civil war prediction. This is a field that attempts to predict where the next civil war will occur in order to better prepare for civil conflict.

And what we found was that there were a number of papers claiming almost perfect accuracy in predicting when a civil war would occur. At first, this seemed astounding. If AI can really help us predict when a civil war will start years in advance, it could be game-changing. However, when we dug deeper, we discovered that every single one of these claims, which stated that AI was better than two-decade-old logistic regression models, was not reproducible. This revelation set off alarm bells for both of us.

We sort of dug in a little bit deeper and found that this is a pervasive issue across fields that are quickly adopting AI and machine learning. We discovered, I think, over 300 papers. The last time I compiled this list, I believe it was over 600 papers that suffer from data leakage. This occurs when you train on the set that you're evaluating your models on.

It's sort of like teaching to the test. A machine learning model seems to perform much better when you evaluate it on your own data compared to how it would really function in the real world. Right? You mention in the book that the goal is to identify AI snake oil and distinguish it from AI that can work well when used properly. Now, I have to tell you, it's kind of a downer of a book if you're an AI enthusiast because there's not a whole lot of positive information presented here.

We'll get to that in a minute. But you break down the types of AI, which I'm going to challenge a bit, into three discrete areas: the predictive AI, for which you take a really harsh stance, saying it will never work; then there's generative AI, obviously—the large language models that kind of took the world by storm, although they were incubating for several years before ChatGPT came along; and finally, content moderation AI.

Sure! Here’s the text with correct capitalization and spelling: "So maybe you could tell us about your breakdown into these three different domains of AI?" "Absolutely. I think one of our main messages throughout the book is that when we talk about AI, often what we're really interested in are deeper questions about society. A breakdown of predictive, generative, and content moderation AI reflects how these tools are being used in the real world today. For predictive AI, one of the motivations for including this as a separate category in the book was that we found it often has nothing to do with modern machine learning methods."

In some cases, it can be as simple as decades-old linear regression tools or logistic regression tools. Yet, these tools are marketed under the umbrella of AI. Advances being made in generative AI are often presented as applicable to predictive AI as well. As a result, we are seeing companies across dozens of domains, including insurance and healthcare, education, and criminal justice, selling predictive AI with the promise of replacing human decision-making. I believe that this last part is where many of our issues truly lie because these tools are marketed as being capable of far more than they actually are.

These tools are being sold as if they can enable better decision-making for criminal justice. However, when people have tried to interrogate these tools, what we have found is that they often work no better than random chance, especially regarding consequential decisions such as job automation. This includes determining who gets called for the next level of a job interview or who is rejected right after submitting their CV. These are very significant decisions. We believe there is a lot of "snake oil" in the field, in part because people do not distinguish between applications that have made significant advances, such as those in generative AI, and those that have been around for decades but do not perform as claimed by the developers. The metaphor of "snake oil" is a tough one, and you even reference an advertisement from 1905 featuring snake oil in your book. You are really addressing predictive AI and how it relies on outdated tools while marketing itself as some kind of breakthrough.

And I do. Before I challenge that, are we going to be able to predict things, by the way, using generators, not in the way you described? I would like to go through a few examples of how problematic this has been. Since many of our listeners and readers are in the medical or biomedical field, I'll try to focus on those. One of the first examples you mentioned, which I completely agree with, is the prediction of COVID-19 from chest X-rays. There were thousands of studies that emerged throughout the pandemic.

Maybe you could comment on that one. Absolutely. Yeah. So this is one of my favorite examples as well. Essentially, Michael Roberts and his team at the University of Cambridge, about a year after the pandemic, looked back at what had happened.

I think at the time, there were around 500 studies that they included in the sample. They looked back to see how many of these would be useful in a clinical setting beyond just the scope of writing a research paper. They started out by using a simple checklist to determine if these tools were well validated. They asked questions such as whether the training and testing data were separate, and so on.

So, they ran through the simple checklist, which excluded all but 60 of these studies from consideration. Apart from those 60 studies, none of the others even passed a very basic criterion for inclusion in the analysis. Now, for these 60, it turns out that if you make a guess about how many were useful, I'm fairly confident that most guesses would be wrong. There were exactly zero studies that were useful in a clinically relevant setting.

And the reasons for this varied. In some cases, the reasons were as bizarre as training a machine learning model to predict COVID, where all of the positive samples of people who had COVID were from adults, while all of the negative samples of people who didn't have COVID were from children. Essentially, claiming that a classifier can predict who has COVID is bizarre because all the classifier is doing is looking at the chest X-ray and predicting which X-ray belongs to a child versus an adult. This illustrates the kind of error that can occur. In some cases, we also saw duplicates in the training and test sets.

So you have the same person being used for training the model and then also for evaluating the model. Simply memorizing a given sample of X-rays would be enough to achieve a very high performance. Issues like this, I think all 60 of these studies have proven to be not useful in a critically relevant setting. I believe this is the type of pattern we have seen over and over again. Yes, and I agree with you on that point.

I mean, that was really a flagrant example of how that would fulfill the title of your book, which is, as I said, a very tough title. On page 29, and we'll have this in the post, you have a figure titled "The Landscape of AI: Snake Oil, Hype, and Harms." The problem is that there was nothing good in this landscape. So, on the y-axis, you have "Works," "Hype," and "Snake Oil" going up, and on the x-axis, you have "Benign" and "Harmful." Thus, the only elements you have are "Works," "Hype," and "Snake Oil" on the y-axis, while the x-axis displays "Benign" and "Harmful."

So the only thing you have that works, and is benign, is autocomplete. I wouldn't say that works. Then you have facial recognition for surveillance, which is harmful. This is a pretty sobering view of AI. Obviously, there are many things that are functioning that aren't represented in this landscape.

I would like to challenge your perspective. Are you focusing too much on the negative aspects? This diagram presents a rather bleak view. There has been so much progress in AI. For instance, you mentioned predicting civil wars, but we also have advancements in areas like cheating detection and criminal risk prediction.

I mean, there are a lot of problems, like video interviews that are deepfakes. However, you don't present any good things. To be clear, I think both Arvind and I are somewhat paradoxically optimistic about the future of generative AI. The decision to focus on Snake was a very intentional one from our end. In particular, I believe that at various points in the book, we outline why we're optimistic and what types of applications we are optimistic about as well.

The reason we don't focus on certain topics is that, quite frankly, no one wants to read a 300-page book about the virtues of spellcheck or AI for code generation. However, I completely agree and acknowledge that there are many positive applications that didn't make the cut for the book. This decision was made because we wanted readers to approach the subject with a sense of skepticism, so they are not misled by the hype. Essentially, we see even these positive uses of AI being overshadowed if people have unrealistic expectations about what an AI tool can achieve. Therefore, pointing out "snake oil" benefits is almost a prerequisite for effectively using AI in a work environment.

I can give a couple of examples of where or how we've manifested this optimism. One is AI for coding. I think writing code is an application that I do; at least I use AI a lot. I believe almost half of the code I write these days is generated, at least the first draft is generated using AI. And yet, if I did not know how to program, it would be a completely different question, right?

Because for me, pointing out that "oh, this syntax looks incredible" or "this is not handling the data in the correct way" is as simple as looking at a piece of code, since I've done this a few times. However, if I weren't an expert in programming, it would be completely disastrous, because even if the error rate is just 5%, I would have dozens of errors in my code if I were using AI to generate it. Another example of how we've been using AI in our daily lives is that Arvind has two little kids, and he's built a number of applications for them. I think he's a big proponent of incorporating AI into children's lives as a force for good, rather than having a completely hands-off approach. These are just two examples, but I would say a large amount of our work these days occurs with the assistance of AI, so we are very much optimistic.

And at the same time, I think one of the biggest hindrances to actually adopting AI in the real world is not understanding its limitations. Right? Yeah. You say in the book, "The two of us are enthusiastic users of generative AI, both in our work and personal life." It just doesn't come through as far as the examples. But before I leave the troubles of predictive AI, I'd like to discuss a few more examples because that's where your book shines in convincing us that we have some trouble here, and we need to be completely aware. One of the most famous examples, or rather, a couple we're going to get into, is the prediction of sepsis in the Epic model, which I would like to review with you.

As you know very well, Epic is the most widely used IT and health systems electronic health record. They launched an algorithm that had never been published, which was meant to indicate whether a patient was hospitalized and if they actually had sepsis or were at risk of sepsis. Perhaps you could take us through that, as you do in the book; it truly was a fiasco. Absolutely. I think back in 2016 or 2017, Epic developed a system to help healthcare providers predict which patients were most at risk of sepsis.

And I think, again, this is a very important problem. I believe sepsis is one of the leading causes of death worldwide and even in the U.S. If we could address this issue, I think it would be a game changer. The problem was that there were no external validations of this algorithm for the next four years. So, for four years, between 2017 and 2021, the algorithm was in use by hundreds of hospitals in the U.S.

In 2021, a team from the University of Michigan conducted a study in their own hospital to assess the efficacy of the sepsis prediction model. They discovered that Epic had claimed an AUC (Area Under the Curve) of between 0.76 and 0.83, while the actual AUC was closer to 0.6. An AUC of 0.5 indicates random guessing, so this performance was significantly worse than the company's claims.

And I think even after that, it still took a year for sepsis to roll back this algorithm. At first, Epic's claims were that this model works well, and that's why hospitals are adopting it. However, it later turned out that Epic was actually incentivizing hospitals to adopt sepsis prediction models. I think they were offering credits of hundreds of thousands of dollars in some cases if a hospital satisfied a certain set of conditions, and one of these conditions was using a sepsis prediction model.

And so we couldn't really take their claims at face value. Finally, in October 2022, Epic essentially rolled back this algorithm. They transitioned from a one-size-fits-all sepsis prediction model to a model that each hospital now has to train on its own data. This approach, I believe, is more likely to work because each hospital's data is different; however, it is also more time-consuming and expensive for the hospitals. All of a sudden, they now require their own data analysts to implement and monitor this model. This situation highlights many of the broader issues with predictive AI. These tools are often marketed as replacements for existing systems.

But then, when things go bad, they are essentially replaced with tools that do far less. Companies often revert to the fine print, stating that, oh, we should always deploy it with a human in the loop, or oh, it needs to have these extra protections that are not our responsibility, by the way. I think that gap between what developers claim and how the tool actually works is what is most problematic. Yeah, I mean, it's an egregious example.

And again, it fulfills what we discussed about the chest X-rays, but even worse because it was marketed and incentivized financially. There's no doubt that some patients were completely miscategorized and potentially harmed. The other classic example that went south is the Optum United Health algorithm. Perhaps you could take us through that one as well, as it is yet another terrible case of how people were discriminated against. Optum, another health tech company, created an algorithm to prioritize high-risk patients for preemptive care.

I believe it was around the time Obamacare was being introduced that insurance networks began exploring ways to reduce costs. One of the main strategies they identified for cost reduction was to proactively care for patients who are extremely high-risk. In this context, they decided to focus on approximately 3% of patients in the high-risk category and developed a classifier to identify those at the highest risk. By doing so, they aimed to treat these patients proactively, potentially leading to fewer emergency room visits, fewer hospitalizations, and other positive outcomes.

So that’s all fine and good. But what happened when they implemented the algorithm was that every machine learning model needs a target variable, which is what is being predicted at the end of the day. What they decided to predict was how much the patient would pay, and how much they would charge. They also considered what costs the hospital would incur if they admitted this patient.

And they essentially used that to predict who should be prioritized for healthcare. Now, unsurprisingly, it turned out that white patients often pay a lot more or are able to pay a lot more when it comes to hospital visits. Maybe it's because of better insurance or better conditions at work that allow them to take leave and so on. But whatever the mechanism is, what ended up happening with this algorithm was that Black patients with the same level of healthcare prognosis were half as likely, or much less likely compared to white ones, to get enrolled in this high-risk program. So, they were much less likely to receive this proactive care.

This was a study, a fantastic study by Obermeyer et al., published in *Science* in 2019. What I think is the most disappointing part of this is that Optum did not stop using this algorithm after the study was released. This was because the algorithm was working precisely as expected; it was designed to lower healthcare costs.

It wasn't an algorithm that was meant to provide better care for patients who need it most. Even after the study was rolled out, I believe Optum continued using this algorithm as it was. As far as I know, some version of this algorithm is still in use across the network of hospitals that Optum serves. It's horrible that this was exposed by Ziad Obermeyer's paper in Science and that nothing has been done to change it.

It's extraordinary. I mean, it's just hard to imagine. Now, you do summarize the five reasons predictive AI fails in a nice table. We'll put that up in the post as well. I think you've reviewed that with these case examples.

So now I get to challenge you about predictive AI. I don't know that there is such a fine line between that and generative AI or large language models. As you know, the group at DeepMind, along with others, has done weather forecasting with multimodal large language models and has achieved some of the most accurate weather forecasts we've ever seen. I’ve written a piece in Science about medical forecasting, again taking all the layers of a person's data and trying to predict if they are at high risk for a particular condition. This includes not just their electronic records, but also their genomics, proteomics, scans, labs, and other factors, such as exposures and environmental influences. I want to get your sense on that because this represents a coalescence of where you took down predictive AI for good reasons.

And now, we have much more sophisticated models that are integrating not just large data sets, but truly multimodal data. Some people think multimodal means only text, audio, speech, and video images. However, we are talking about multimodal layers of data for applications like weather forecasting models, earthquake prediction, and other areas. Let's get your views on that, as these perspectives weren't really presented in the book. I believe they represent a positive step, but I want to hear what you think.

No, absolutely. I think maybe the two questions are slightly separate in my view. For things like weather forecasting, I think weather forecasting is a problem that's extremely suitable for generative AI or for making predictions about the future. I think one of the key differences there is that we don't have the problem of feedback loops with humans when making predictions about individual human beings.

We are primarily making predictions about geological outcomes. We have good differential equations that we've used to predict them in the past, and those have proven to be quite effective. However, I do believe that deep learning has taken us a step further. In that regard, I think it's an excellent example of something that doesn't quite fit within the context of the chapter, as we are focusing on decisions related to individual human beings.

And you rightly point out that that's not really covered within the chapter. Regarding the second part about incorporating multimodal data, such as genomic data and everything related to an individual, I think that approach is promising. However, I will say that, so far, we haven't seen it used for making individual decisions, especially when it comes to consequential decisions about human beings. Often, what ends up happening is that we can make very good predictions; that's not in question. But even with these good predictions about what will happen to a person, sometimes intervening in the decision is difficult. This is because we often treat prediction as a problem of correlations, whereas making decisions is a problem of causal estimation.

And that's where those two approaches begin to disentangle a little bit. One of my favorite examples of this is a model that was used to predict who should be released before screening when someone comes in with symptoms of pneumonia. For instance, if a patient arrives with symptoms of pneumonia, should you discharge them on the same day, keep them in the hospital, or transfer them to the ICU?

And these machine learning researchers were basically trying to solve this problem. They found out that the neural network model they developed—this was like two decades ago—was extremely accurate at predicting who would have a high risk of complications once they developed pneumonia. However, it turned out that the model indicated that anyone who came in with asthma and pneumonia symptoms was considered the lowest risk patient. Now, why was this? This was because, in the past training data, when such patients came to the hospital, they were often transferred directly to the ICU. Healthcare professionals recognized that this could be a serious condition.

That actually, patients who had asthma and came in with symptoms of pneumonia were the lowest risk among the population because they received such good care. However, if we use the prediction that a patient presenting with pneumonia symptoms and asthma is low risk as a basis for deciding to send them back home, that could be catastrophic. I think that’s the danger of using predictive models to make decisions about individuals. Again, the scope and consequences of decisions also vary. You could consider using this to identify interesting patterns in the data, especially at a slightly larger statistical level, to observe how certain subpopulations behave or how specific groups of people are likely to develop symptoms or other outcomes.

But I think that as soon as it comes to making decisions about people, the paradigm of problem-solving changes. As long as we're using correlational models, it's very hard to predict what will happen if we change the conditions or if the decision-making mechanism is significantly different from the one where the data was collected. Right? Where we agree on this is that, at the individual level, using multimodal AI with all these layers of data that have recently become available—or should be available—needs to be compared, ideally in a randomized trial, with the current standard of care, which does not utilize any of that data. We need to see whether the decisions made actually alter the natural history and if there is an advantage to this approach.

That's yet to be done, and I agree. It's a very promising pathway for the future. Now, I think you have conducted a very comprehensive overview of the predictive AI failures. You've mentioned here in our discussion, as well as in the book, your enthusiasm for and the positive features of degenerative AI—hope and excitement, perhaps even.

You haven't really provided much information yet; we haven't discussed the content moderation AI that you have discreetly categorized. Perhaps you could give us a brief overview of your thoughts on it. Absolutely. Content moderation AI is technology used to help clean up social media feeds. Social media platforms have various policies regarding what is allowed and what is not on their sites.

Spam is obviously not allowed because, for example, if people start spamming the platform, it becomes useless for everyone. Additionally, there are other things such as hate speech, nudity, and pornography that are also disallowed on most, if not all, social media platforms today. I believe that many of the ways in which these policies are enforced today involve the use of AI. For instance, you might have an AI model that runs every time you upload a photo to Facebook, and there may be hundreds of such models designed to detect nudity, hate speech, or other content that might violate the platform's terms of service. Content moderation AI is the technology used to make these decisions.

And, as has happened very often in the last few years, we've seen that when something gets taken down—such as when Facebook deletes a post—people often blame the AI for having a poor understanding, particularly of satire, or for not comprehending the content. They essentially argue that their post was removed due to inadequate AI. Many claims have been made that content moderation AI will resolve social media’s issues. In particular, we've heard assertions from Mark Zuckerberg, who, in a Senate testimony back in 2018, stated that AI would solve most, if not all, of their content moderation problems. So, our take on content moderation AI is basically this.

AI is very, very useful for solving the symptomatic parts of content moderation. What is a simple part? Essentially, the simple parts of content moderation involve having a large training dataset of the same type of policy violation on a platform like Facebook. If you have large datasets that have a clear line in the sand, such as nudity or pornography, it becomes relatively easy to create classifiers that will automate this process. On the other hand, the hard part of content moderation is not merely creating these AI models.

The hard part is drawing the line. When it comes to what is allowed and not allowed on platforms, these platforms are essentially making decisions about speech. That is a topic that is extremely fraught. It's fraught in the U.S. and also fraught globally.

And essentially, these platforms are trying to solve a very challenging problem at scale. They are attempting to develop rules that apply to every single user of the platform, which encompasses over 3 billion users in the case of Facebook. This inevitably involves trade-offs regarding what speech is allowed versus what is disallowed, which are difficult to categorize definitively. They are not black and white. Our belief is that AI has no place in this complex aspect of content moderation, which fundamentally requires a human touch.

It's essentially about adjudicating between competing interests. When people claim that AI will solve the problems of content moderation, I think what they're often missing is that there's an extremely large number of things you need to do to get content moderation right. AI solves one of these dozen or so things, which is detecting and taking down content automatically. However, all the rest of it involves human decisions. This is the brief gist of it.

There are also other problems. For example, AI doesn't work very well for low-resource languages. It struggles with nuances and other aspects that we discuss in the book. However, we believe some of these challenges are solvable in the medium to long term.

But these questions around competing interests of power, I think, are not limited to, or beyond, the domain of AI, even in the medium to long term. No, I think you nailed that. This is an area that you've really aptly characterized and shown the shortcomings of AI and how the human factor is so critically important. What's extraordinary here is you're just 28, and you are, you know, excelling with publications all over the place on reproducibility, transparency, and evaluating general AI safety.

You have a website on AI snake oil where you're collecting and writing more content. Of course, you have experience working in the IT world with Facebook and also, I guess, Columbia. So you're really off to the races here as one of the young leaders in the field. I'm struck by that. Perhaps you could comment on the inspiration you might provide to other young people.

You're the youngest person I've interviewed for Ground Truth, by the way, by a pretty substantial margin, I would say. And, you know, this is a field that attracts so many young people. So maybe you could just talk a bit about your career path and your advice for people. They may be the kids of some of our listeners, but they also may be some of the people listening as well. Absolutely.

First, thank you so much for the kind words. I think a lot of this work is with collaborators, without whom, of course, I would never be able to do this. I believe Arvind is a great co-author and supporter. In terms of my career path, it was sort of like a zigzag, I would say. It wasn't clear to me when I was in undergrad whether I wanted to go to grad school or enter the industry.

And I sort of, on a whim, went to work at Facebook because I'd been working on machine learning for a little bit of time. I just thought it was worth seeing what the other side has to offer beyond academia. I think that experience was very, very helpful. So, one of the things—I've talked to a lot of undergraduates here at Princeton—one of the things I've seen people be very concerned about is what the graduate school they're going to get into will be like right after undergrad.

And I think it's not really a question you need to answer now. I mean, in some cases, I would say it's even very helpful to have a few years of industry experience before getting into grad school. That is definitely sort of, at least that has been my experience. Beyond that, I think working in a field like AI, it's very easy to get caught up with all of the new developments happening each day. So, I'm not sure if you know, but AI has over 500 to a thousand new archive papers published every single day.

And so, with this rush, I think there's this expectation that you might put on yourself, like being successful requires a certain number of publications or a certain threshold of achievements. More often than not, that mindset is counterproductive. It has been very helpful for me, for example, to have collaborators who are thinking long-term. This book, for instance, is not something that will be valued highly within the computer science community. I would say that the computer science community values peer-reviewed papers much more than they do books.

And yet we chose to write it because I think the staying power of a book, or the longevity of a book, is much greater than what any single paper could achieve. Another concrete thing I found very helpful is optimizing for a different metric compared to what the rest of the community seems to be doing, especially in fast-moving fields like AI. That last piece of advice is important because I think too often people—whether they are computer scientists, life scientists, or others—do not realize that their audience is much broader. Reaching the public through mediums like books, op-eds, or essays offers varied ways to engage in public discourse, rather than solely targeting computer scientists. That's why I believe the book is a valuable contribution.

I don't like the title because it's so skewed, and all the content is really trying to hammer that point home. I hope you all consider writing a sequel that explores the positive sides of AI. I wanted to ask you, when I read the book, I felt as though I could hear your voice. It seemed to me that you had written it, and perhaps Arvin did some editing, as you wrote about Arvin this and Arvin that.

Is that did you write the first draft of the book and then he kind of came along? No, absolutely not. We basically started writing it in parallel. I wrote the first draft of half the chapters, and he wrote the first draft of the other half. This process continued throughout the writing of the book. We would write a draft, pass it to the other person, and then keep doing this until we sent it to our publishers.

Okay, so I guess I was thinking of the chapters you wrote where it came through. I'm glad that it was a shared piece of work because, you know, that's good—because that's what co-authoring is all about, right? Well, it's really been a joy to meet you, and congratulations on this book! I obviously have expressed my objections and my disagreements, but that's okay because it stimulates discussion. This book will feed the skeptics of AI. They'll love it, okay? I hope that the positive side, which I think is underexpressed, will not be lost and that you'll continue to work on this and be a conscience in the space. You may know that I've interviewed a few other people in the AI space who are similarly like you, trying to assure its safety, transparency, and address the ethical issues.

And I think we need folks like you. I mean, this is what helps get it on track, keeping it from getting off the rails or doing what it shouldn't be doing. So keep up the great work. Thank you so much for joining. Thank you very much.

It was a real pleasure.