Pydantic AI - The NEW Agent Builder on the Block

Okay, so many of you will know Pydantic. Pydantic is basically a validation library that allows you to build classes to check whether your data conforms to a particular class or data model. Originally, it was used with APIs to clarify that the inputs coming into the API were correct and that the outputs being returned were also correct. You can define a schema for what you wanted as input and what you wanted as output, as you can see here. This made it much simpler to perform this kind of task in Python.

Now, fast forward to when people are building large language model apps and agents; they also wanted a way to get the outputs to conform to a particular schema. We have seen this classically in function calling and JSON output. People want the model to be able to follow a specific schema. Back in the early days, when people were trying this out, they discovered that Pydantic was perfect for this use case as well. I think the first use of this was actually in Instructor, although the idea may have been around for a bit before that.

Certainly, LangChain and LlamaIndex adopted this idea, and you saw Pydantic being used a lot in frameworks like LangChain and pretty much most of the frameworks that have come after this. One of the key takeaways from this is that, in many ways, this is really what you want from an agent framework or an LLM framework: a way to ensure that the outputs of a model conform to a particular standard so that you know the output can be used programmatically elsewhere. Fast forward to today, the team behind Pydantic has created Pydantic AI. This is a whole new LLM library with Pydantic at its core, building upon these concepts.

From Pydantic, rather than just plugging Pydantic into a framework that's built for different purposes. With this launch, they are demonstrating how their framework can be used for a variety of tasks, ranging from chat apps to structured output to Retrieval-Augmented Generation (RAG) to agents, and a whole range of application types. Additionally, from day one, they are supporting a diverse array of models. We have the OpenAI API models, the Google Vertex AI models, and the Grok API models. I believe they will also add support for the Anthropic models very soon.

So this is their whole take on an agent framework, and it really is building a sort of LLM app framework around their original Pydantic library, but making it easy to use things like system prompts, tool usage, retrieval-augmented generation (RAG), etc. This seems to be the product of the team recently raising a significant amount of funding and transitioning from being a single open-source library to actually becoming a company with revenue-generating products. In line with that funding, they announced a feature called Log File, which is essentially an observability platform that allows you to track inputs, outputs, and more. We have seen a number of these for large language models; obviously, LangChain has LangSmith, but there are quite a few options available, including open-source solutions, fully proprietary offerings, and some that fall somewhere in between. Following up on that release, today they have introduced Pydantic AI.

So let's jump in and have a look at this. First off, why would you want to use Pydantic AI? As they rightly point out, pretty much every agent framework out there, in some way, shape, or form, has Pydantic in it for validations. This is basically used to validate the responses back from a language model. To decide whether you need to call the language model again to fix something.

That kind of thing. As I mentioned earlier, I think LangChain and LlamaIndex were some of the first to do this, but certainly a whole bunch of different libraries have gotten on this bandwagon. It makes a lot of sense. Pydantic has been such a strong, reliable Python library for a long time. On top of that, they mentioned that this framework is model-agnostic and currently supports OpenAI, Gemini, Grok, and Anthropics is coming.

It is type-safe by design. This provides many advantages and allows you to seamlessly integrate your data into various tools, databases, and more. Another significant benefit is that the entire control flow and agent composition are managed using standard Python. This simplicity is something I appreciated about the Swarm project; in the end, it was quite straightforward, and if you understood Python, you could easily integrate it.

Now, this becomes really important when you want to be able to put something into production. Even though I prototype a lot of the stuff I do in LandGraph, often when I actually push something to production, I just recreate it in Python. Once I've figured out what's going to go where, I find that for each agent, you often want to store the state a little bit differently. You want to handle a variety of tasks in different ways. So for me, one of the advantages of this, along with Swarm, is the ability to store the state a little bit differently.

You want to do a variety of things a bit differently. One of the advantages of this, along with Swarm, is the ability to do everything in Python from the start. They mention that they support structured responses from the models, which is a huge advantage, and all that validation is done with Pydantic. One of the cool things about this is that even if you're streaming the responses, you can still use Pydantic for that.

The way they’ve set it up with dependency injection makes it very simple for you to inject system prompts and tools into these applications on the fly. You can imagine that as a conversation changes over time and you want to modify the tools, you don’t necessarily need to move to a whole new node. Instead, you can simply inject a new tool or have a dynamic system prompt that evolves as the conversation or agent workflow changes. Additionally, they are also supporting their LogFire integration.

Although from what I see, that's certainly not needed; it's not a compulsory thing that you need to do. So I think the simplest thing is to jump into the code, have a little play with what it can do, and just get a simple sense of how to use this with different models and tools, etc. Okay, so one of the key things about this is that you want to make sure that you install the `nest_asyncio` library to make it run in Colab because Pydantic AI is an asynchronous library. This is what you're going to need to help it run, and you need to have this installed to ensure that the event loop works in Colab. All right, the first time you run this, you will need to restart the notebook to get it working. I'm just bringing in some keys for LLMs here.

I will kick this off. First, we are going to set up an agent and make a call to it. Essentially, we are just calling the LLM here; we’re not doing anything fancy. You can see that we set up the agent with Gemini 1.5 Flash.

In this case, we've got a system prompt. Then, we would pass in what we want to have as our user prompt. This is reasonably similar to something like LangChain. So, okay, we've done Gemini 1.5 Flash. We've asked it to be very concise and to reply with only one sentence.

Sure enough, we've got our one sentence back. Now, we can also change the model simply by using `agent.model`. However, we can't change `agent.system prompt` directly; we need to inject that instead. So, here I've just wrapped this prompt with a decorator that we're going to pass into it.

This is going to be a long one-paragraph answer, and I will make it dense. When I run the exact same query that we had before, we should receive a much longer paragraph as a response. Sure enough, we do; we can see that it's actually working as intended. You can always check the agent to see what the prompts are.

And I'll show you that in a second. All right. The next thing is looking at basic structured output. I really feel this couldn't be easier. So, you define some kind of model.

That you want the output to have. So I've taken their basic sort of example, and their basic example was going to have a city and a country. You define the model. So when I say "model" here, I'm talking about the class, right? That is the model of what we're going to get back as our response.

And we can see that this is actually going to be our result type that we get back. Now, the model in this case, we're using OpenAI's GPT-4. We're going to set up the agent so the agent will take in this model, and we'll just pass in a prompt here. In this case, it's going to give us back nicely formatted data. So, we can see that we passed in "the Windy City" in the USA, and we got back "Chicago" and "Country: USA."

We can look at the cost to see how many tokens we used, etc. We will try this with a different city. Sure enough, Merlin, we get Singapore in there. I want to show you something really cool: we can just come in here and add things to this.

So I can just say, okay, we've got city, country, reason. What about if we include something like a famous person from the city? Okay, we'll skip the famous person from the country. We'll just go with a famous person from that city. So now we've got four things, right?

Yet you'll see that by simply adding those two lines to the model, we've now included the city, the country, and the reason or explanation. I guess we could have mentioned that Chicago is commonly referred to as the "Windy City," a nickname that originated in the 19th century. Then, at the end, we should mention a famous person from that city. I'm not sure if Kanye West is from Chicago or not.

And it will be different sometimes. So sometimes, from this one, you'll get other names back in here. Now that we have that, we can run it through a whole bunch of different cities really easily. From Singapore, we have Lee Kuan Yew as the famous person. If I say "the cold city in the south," it's interesting; I didn't know what the answer would be.

I experimented on different continents to see what results I would obtain. If I leave it in the south, it basically selects a city in Argentina. This city is the southernmost in the world with a cold climate. It then provides us with a person who is from there. I'm not sure if that's hallucinated or accurate, but you can see we've got these results that are nice and neat like that.

Of course, we can index into these as well. You can see we've got these results that are nice and neat like that. If I simply say "result data famous person from city," I will get the last result from the last run.

I can use these in a whole variety of different ways. One of the best uses of structured data is to experiment with different inputs like this. This library certainly makes that super easy. If we look at the agent we had there, we can see that it's using the OpenAI model, GPT-4.0.

We can see the last messages run in its use of the OpenAI model, GPT-4.0. We can observe the last messages exchanged during our calls to the south, as well as the responses we received. If we want to consider developing some type of chat application, this is where things become quite different. Instead of thinking about things in the traditional manner of having a user-assistant role, you can focus specifically on this aspect.

So you can see here, I can just basically say, "Right, I'm going to set up a system prompt, 'Be a helpful assistant,' and I say, 'Tell me a joke.'" And sure enough, it comes out with this joke. Obviously, I could pass in a model there and get other information out if I wanted to as well. Once this is done, though, I can actually see the messages. So if I come in here and pretty print these messages, we can see that the system prompt was, "Be a helpful assistant."

Then the user prompt is, "Tell me a joke." The model's text response is what we received here. If we look at this, we can see that it's just a simple list. We could repurpose this; we could do a bunch of things with this.

As we go along, if we just want to see the most recent messages, we can look at the last user input and the model's response. Now, if we want to send another message, we can keep the same agent. I would say, "Okay, explain the joke. Please explain it."

And I'm just going to pass in the message history with these new messages. So the system prompt is already in there. You'll see that it basically comes back and explains the joke, including details about the scarecrow, etc. Now, if we look at all the messages, we'll see that we have the system, user prompt, model, and the new user and model included.

If we just want to see the most recent ones, we can easily retrieve them here. Again, we can take the most recent ones from the last call, call that again, and you'll see that we're able to extract this information. Now that we're processing this, we can examine where the joke originates, and it helps to explain it a bit more, I guess. We can see all messages as a list of that entire conversation, and we can decide to keep a certain number of those to include as the history.

Because you can see each time we've just passed in a message history, which is just a list of messages. This makes it very easy for us to trim the messages down if we want to, such as keeping only the last one or having all of them. Additionally, we can also get this in JSON format. All messages can either come as a list in Python, or we can retrieve them as JSON. This format aligns more closely with the user-system role structure seen in many of the other frameworks you will be using. You can even change the LLM partway through.

So here we've got result one. Tell me a joke. Then we've got the part where I'm printing that, and that's going to be probably this. We decide, okay, we're not going to use OpenAI for that second part; we're going to use Gemini 1.5 Pro. Remember, this framework is model-agnostic or LLM-agnostic, I guess I should say.

And you can see that the second part is essentially taking the history from one LLM and passing it into the other, and we've utilized that just fine. We have no problem doing this. We can see this entire history, even though it has come from two separate LLMs here. All right, the last example I want to show you is one that demonstrates the use of function calling and tool use. This is their example, which employs tool use to obtain weather details and similar information.

Now, it's actually split into two tools. I don't have API keys here. If you don't use an API key, it's just going to give you a sort of default answer, but it still shows us what's being called here. We're going to have our weather agent. These dependencies are what will be included.

We're going to have the weather API key and the client incorporated here. Next, we'll add the tools. You'll see that once we've set up this agent, we can configure the number of retries; I believe I mentioned that in one of the earlier examples. Once we have this agent, it will take in the context of the dependencies, which will likely include some form of location description, probably the name of the city or something similar.

And this first tool is to get the latitude and longitude. Now, if there is no API key, it will just return a dummy response. However, if there is an API key, it will actually go out and make the call. You can see that it essentially returns a dictionary with latitude and longitude. The second tool will take the latitude and longitude that we've obtained and then send it off.

Again, if we're missing an API key, it's just going to give us a dummy response. However, it will eventually return the actual weather data, along with some code lookups and other related information. The API actually has a variety of codes that you can use to perform lookups and gain a more descriptive understanding of the data. Finally, it will return the temperature and either a description or the code lookup. This process will convert the information into natural language. Essentially, we have extended the functionality of this agent by simply using this decorator.

This decorator is located in `weatheragent.tool`. We can utilize multiple tools here. The main function will retrieve your API keys, if you have them. It will then pass these keys in and execute the function with the user's prompt and the necessary dependencies. If we run this, we should receive a response indicating sunny weather with a temperature of 21.

We'll look at the end. I'll come back to this, sure enough. We're getting the current weather in both London and Singapore, which is 21 degrees and sunny, right? So that's just a dummy response. But we did ask, "What is the weather like in London and Singapore?" All right, let's look at the actual output to see what really happened.

So, we can see that, sure enough, it went through this and decided it had the user prompt. From that, it provided us with a model-structured response. That was a tool call, and we can see that the tool call was essentially to get the latitude and longitude for London as the first request, and then another one to obtain the latitude and longitude for Singapore.

So, that's two separate tool calls that it made there. In this case, it returned the same latitude and longitude for each of those. Therefore, we should get this result of 51.50. Sure enough, the tool returned the latitude and longitude as 51.1 and -0.1. We received that for both of them, as both tools used dummy data.

And now, its next response is to call a tool again to get the weather, passing in the latitude and longitude that we have. It's doing this, obviously, once for London and once for Singapore. The response indicates that the temperature is 21 degrees and sunny for both locations. Once it receives this information, it provides a model text response back. The tool calling is complete, and the model text response states that the weather in both London and Singapore is currently 21 degrees and sunny.

And then we've received detailed data about how many tokens we used. You can see that it even includes specifics, such as, if we were using the O1 models, it would show us the number of reasoning tokens or audio tokens, etc. in there as well. So that's an example of setting up a very simple agent that can perform tool use and function calling with just a short amount of code. Of course, the length of your agent's code will depend on the actual tool you are implementing.

So, that would be almost the same for every framework. However, here you can see we've set this up really simply, and we haven't had to pass in a list of tools or anything. We've just appended these onto this. Finally, I included another one of their examples for just streaming markdown, which illustrates how it can effectively utilize the structured output to identify the different parts of the code. This allows it to print everything out nicely. This is using a couple of different models, and it is printing out the code with the prompt, "Show me a short example of using Pydantic." It has some styling for logging so that you can see the different components involved. Anyway, this certainly shows a lot of promise as a new framework. It allows for very simple abstractions to choose different LLMs, change system prompts on the fly, inject different elements, and set up a history for memory management—all of which can be done very easily.

It's something that's done in a very Pythonic fashion, and in many ways, it's much easier to understand than some of the things like LangChain, LangGraph, LamaIndex, etc., for these kinds of applications. If people are interested, I may do another video on setting up a RAG system and just showing you how to use this for a RAG system. Otherwise, I definitely recommend that you check it out, play around with it, see how it works, and explore its capabilities. My guess is that you can probably achieve a lot with far less code than you're using for some of the other agent frameworks by just using something as simple and Pythonic as this. As always, if you've got any questions, I'd love to hear your comments about what you think of it. Please put any questions you have in the comments below.

If you found the video useful, please click like and subscribe. I will talk to you in the next video. Bye for now!