Introduction to the Actor Model for Concurrent Computation: Tech Talks @ AppNexus

Thanks for coming out, everybody. My name is John Mary, and today we're going to talk about the Actor Model. I'm just going to go ahead and jump right in. So, just to provide a roadmap for our discussion today, we will start with some theory.

I'm going to introduce it kind of in the abstract. Then we're going to look at some practical usage and have some live coding that I'll be doing, so I look forward to that. At the end, I'll provide some resources if you're still interested after the talk, so you can go and learn more. Just before we really get into it, I want you to know that the actor model isn't just an abstract concept. It is something that, after today's talk, if you're interested, you can use in your favorite programming language. Here are some popular ones.

There's Erlang and Elixir, which are among the first truly production-ready implementations of the Actor model. If you're using the JVM, you might want to consider Akka for Java and Scala. For .NET, there's Orleans. A fun fact about Orleans is that the Halo multiplayer backend was actually written using it. So, you know, that's pretty cool!

And then, you know, there are other languages: C++, Ruby, Python, or whatever your favorite language is. If it's moderately popular, you're probably going to find some sort of actor model implementation. There are also other types of concurrency models, such as futures, which are gaining popularity, and coroutines.

There are CSP-style languages. So, if you're looking at something like Go (Golang), you're going to encounter those kinds of semantics. There are many others as well. If you search on Wikipedia, it's not very hard to find a variety of different concurrency models. It's definitely worth exploring.

All right, let's jump right in. What are actors? We kind of know what this is, but what are actors in the model? Let's look at some properties of an actor. The first one, which is interesting, is that actors are persistent. This is different from, say, a thread, a process, or a future.

When I start a thread, when it's done doing whatever work I've given it to do, it terminates, unless it performs some unusual blocking operation. The same goes for a future. I provide a future with a unit of code to execute. Once it finishes executing, the future exits. It no longer exists.

In an actor model, an actor is different. Whether it's doing something or not, it still exists. It continues to exist until either I tell it to stop existing, I shut it down, or there's some sort of exception, crash, or failure that causes it to no longer exist. The next interesting aspect is that it encapsulates internal state. Not all concurrency models have internal state.

So, if you think about futures, right? Futures don't really have state. Threads can have state to some extent if you use thread-local storage. Coroutines have state, and actors have state as well.

They also encapsulate it, which means that there has to be some sort of semantics for how we access and modify that data, which we'll see in a little bit. We have already mentioned that actors are asynchronous, right? So when you have multiple actors, you have concurrency, making it logical that they operate asynchronously. So, what can actors do? First and foremost, they can create new actors.

This makes sense, right? When I have a threaded application, it doesn't make sense if my main thread can't create more threads. This is kind of the first fundamental property. Interestingly, they can receive messages. This is where we kind of get into the meat of it.

And in response to that, they can make local decisions. So this is how we kind of get to that local state. We alter that local state. They can perform any sort of arbitrary side-effecting action, maybe writing to a database.

Maybe they're writing to a log file—anything that affects the global state of your application. They can send messages, including messages to other actors. Files can also impact the global state of your application.

They can send messages, right? So they can send messages to other actors, and they can respond to the sender zero or more times. Now, this is interesting because it differs a little from your typical kind of procedural model, where if you call a function, you're either going to get nothing back or you get one thing back. With actors, I could send a message, and that actor would respond with possibly no messages or one message, as we usually expect. However, I could also receive five messages, ten messages, or even an infinite number of messages.

This is quite interesting in terms of how we work with the actor model. Lastly, we process exactly one message at a time. If you send an actor multiple messages, they will go into what can be described as a mailbox, or a queue, that we will explore further later. The actor will process one message from the queue at a time. This means that within an actor, we do not have additional concurrency, right?

If we're talking about a pure actor model, okay? This is something I derived from the Effective Go website, and it applies quite well here to actors as well. We don't want to communicate by sharing memory; this is very typical of a threaded model. Instead, we want to share memory by communicating.

This means we want to build everything on top of this message-passing paradigm. Okay, so I want to jump into an example. We've kind of seen a little bit about what actors are, but let's get some concrete. I’m going to use a very classic example you've probably all seen in school or online somewhere, where we're going to have a checking account. We’re going to try to perform multiple simultaneous withdrawals from this checking account, and we're going to require some sort of coordination to ensure that we don’t overdraw the account.

We can assume here that overdrawing the account is going to be an invalid statement. The balance returned true. Returning true means that we have successfully withdrawn the amount of money we wanted, and we were successful. Otherwise, we returned false, indicating failure. Now, what does this actually look like?

In a single-threaded approach, or in a single-threaded concurrency model, Person A retrieves the balance, which amounts to $80. They then perform a check using an if condition, as seen earlier: if they have enough money. If they do have sufficient funds, they will go ahead, retrieve the balance again, update it, and store it back. This represents the decrement operation, or the minus equal operation, when expanded. However, once we introduce another person, things begin to get considerably more complicated. Both Person A and Person B retrieve the balance.

They both get back $80, right? Because they're both doing it at the exact same time. This means that they both pass that if condition check and they both attempt to update the balance simultaneously. In this case, person A wins. They update the balance first.

But Person B has already done the check. They've already said, "Yeah, there's enough money in the checking account," and they perform an update again. We end up storing a negative balance. We can end up with a lot of different combinations here, which could lead to various invalid states. So if we were taking a typical approach, we'd say, "Okay, let's introduce some sort of lock."

To introduce this mutex, we lock it on line six. From that point on, we proceed with the normal function we observed earlier. Once we obtain the lock, we know that we are the only one in that particular area of code. We perform our withdrawal and then release the lock. Right?

Return successful. This ensures that only one person can tell me to update at a time. Okay, problem solved. However, what this means is that while it is correct, it does get a little more complicated, right? In the real world, things are not so simple.

You may need to obtain multiple locks, and there may be a specific order in which to acquire them. Deadlock conditions can arise, and it's important to ensure that any new team members are familiar with the proper procedures to avoid introducing race conditions in the code, which can be difficult to diagnose. Now, let's examine this from an actor's perspective, shall we?

So, we are going to define this checking account, which is going to have a balance of $80. We will also define a receive method. This method is similar to looking for incoming messages. In this case, we are specifically looking for an incoming message related to a withdrawal of a certain amount. Then, we will implement the typical naive approach, right?

We check to ensure we have enough in the balance. We update the balance and send a message back indicating true or false. The only significant difference here is that this operates within an actor, and instead of using "return," we use "send a message." If we were to call the code, we would say "checking send message 50 and 60." Assuming we sent these at the same time, we would get a sequence diagram that looks like this, correct?

So both Person A and Person B send a message, and they go into the mailbox, right? This is the thing that sits in front of the actor. This is the queue. From there, the checking actor is going to process one message at a time, right? That's one of our properties, our rules.

So we go ahead and process it. This is going to update the balance, perform the decrement, and send back a message saying, "Yep, you're good to go." Then we're going to process the next message. Obviously, if there's not enough money, we're going to fail and send the message back as false. This gives us the same correctness as the locking example, with the difference that no one can mess this up.

If a new person comes along, they simply send a message. They cannot access that actor's internal state, so they are unable to circumvent us and create a race condition. This concept is very simple and easy to understand. The example we discussed is straightforward, and it leads to happy developers.

You don't have to worry about locking in a race condition. Okay, so we've seen actors; we saw a very short example, and we noted that communication is really important, right? We kind of based everything on sending and receiving messages. Therefore, we need to define some properties of communication to understand the actor model fully. The first property is that actors don't use any sort of channels or intermediaries, such as a CSP-style language. For instance, if we think of Go, you may have a channel you share between two goroutines, which can complicate matters. But remember, actors do not use any form of channels or intermediaries like in a CSP-style language.

You know, if we're thinking about Go (Golang), you may have a channel that you share between two goroutines, and it's going to include semantics on that channel, such as bi-directional or unidirectional. You may have types on the channel and features like guaranteed delivery or buffering, things like that. Actors don't have any of that, right? It's a very, very simple form of communication.

And because of that, they're going to give you what's called a best-effort delivery. This means that, regardless of the protocol they're using to send the message—whether it's within the same process and involves some internal protocol your framework uses, or across machines using TCP or UDP, or maybe a socket—whatever that protocol indicates, if it states that the message was sent successfully, then we're good to go. The actor model trusts that, and that's what is referred to as best-effort delivery.

From this, we can see that we're going to get at most once delivery because the actor model is not going to implement any sort of timeout. It will not perform retries or anything beyond what the underlying protocol indicates is valid. If the underlying protocol happens to be UDP, then obviously, you may not receive the message at all. This is the reason we have at most once delivery.

Messages can take arbitrarily long to be delivered. This is partly because the actor model, in a theoretical sense, does not have a concept of time. There are no timeouts, and there are no timestamps. So, it makes sense that if I send a message and there is some DNS wonkiness, causing it to be sent halfway around the world just to reach a server that's within the same rack, then, from the actor model's point of view, theoretically speaking, that's perfectly acceptable. Additionally, there are no message ordering guarantees.

We can see that because we already make our best effort and there are no timeouts, messages can take an arbitrarily long time to be delivered. This leads us to our next point: we cannot make ordering guarantees. Since we don’t know where messages go, we don’t track them, and there are no timeouts. Now, the next important thing is an address. An address identifies an actor. For example, if we think of a web service.

So, you know, some Gmail.com addresses, right? That is an address that identifies a web service, right? In the same way, you have addresses that identify actors. They may also represent a proxy or a forwarder to an actor. It's kind of the same thing.

If I go to gmail.com, it's very, very unlikely that I'm going to get one specific server every single time. Right? What am I going to get? I'm going to get some sort of load balancer or proxy forwarder that will eventually get me to a Gmail server. So the same is true of actors.

An address may not represent a single actor; it may be some sort of proxy or load balancer. The address will contain location and transport information. All right, so think once again—just keep thinking of web service addresses. The DNS or the DNS address, or the IP address, will serve as the location information, maybe with a port. The transport information will be the protocol you're using.

Maybe it's TCP, maybe it's UDP, etc. What this gives us is location transparency, right? I don't really care where the actor lives, as long as I can send its TCP, maybe it's UDP, etc. This is the advantage of having location transparency, right?

I don't really care where the actor lives. As long as I can send a message and that address has all the information about where it needs to be delivered, that's great. That means that I can have an actor in the same process or an actor on a different machine, you know, a different process on the same machine. But the communication protocol and the semantics for how I communicate within an actor system, using the actor model, are completely the same across all of them.

All right, so this gives us some really interesting ideas about how we would scale a system from one machine to multiple machines because the way we send messages doesn't change. Now, I kind of already mentioned this: one address can represent many actors, right? We have some actors in that address, you know, right? One address can represent many actors. Similarly, one actor can have many addresses, right?

So, we have an actor in the same process, a different process, and on a different machine, right? A, B, and C are all pointing to that actor in the middle. It's the same actor they're referencing, but they're all using different addresses to access it. Okay, so we have these actors. They exist.

They're, you know, whether they're processing messages or not, they're persistent. They constantly monitor the running state of an actor, right? Is it processing messages? Has it stalled? Has it encountered an exception that it doesn't quite know how to handle, right?

It can perform actions based on the state of the actor, right? Was there an unhandled error? If the actor encounters an unhandled error, maybe it's just like, "I don't know what to do, supervisor. What should I do?" And the supervisor can then take action regarding that actor, right? So maybe it's just like, "You know what?"

Like you messed up; just start over. Maybe it’ll go better this time, right? That would be great. But then the question arises: okay, who supervises the supervisor? And we start to consider the supervision hierarchy.

We can think of this like an organizational structure within a corporation. You have managers, and your managers have managers, and they have managers, and so on and so forth. Each manager knows how to manage their direct reports, and they are managed by their own manager. This creates a hierarchical structure, resembling trees. At the top, you usually have something that, in practice, is a kind of magical entity provided by the framework that never dies, and you don't really have to worry about it.

It just kind of has some default behavior for how it handles exceptions or other errors at the top level. This supervision, along with what we observed with addresses, provides us with transparent lifecycle management. There are a couple of points to note: addresses don't change during restarts, which makes sense.

You restart your web server, and your address doesn't change. DNS doesn't change. That would be terrible, right? And your mailboxes are persistent outside of the actor instances, right? So we talk about this mailbox being a queue in front of the actor.

We kind of have something like this. The address encapsulates both the mailbox and the actor, right? Messages flow through the address into the mailbox. The actor processes messages from the mailbox and responds as defined. If that actor instance encounters an issue, the supervisor can simply say, "Oh, well, continue processing and ignore whatever issue you encountered."

It could even just say, 'You know what, I'm gonna kill that actor and replace it with a whole new actor instance and just kind of shove it in there.' To the outside world, this is completely transparent because you're sending messages; the mailbox still exists, right? And the actor can be replaced, and you would never know. Okay, so let's discuss actor use cases.

When is it a good idea to use actors? The first scenario is when you have some sort of processing pipeline. This builds on the fact that actors operate based on messages—they receive a message, perform some processing, and then send out a message. You can see how having this basic setup allows you to dynamically create processing pipelines with different actors and compose them to generate a processing workflow. Along with that, you also have the concept of streaming data.

Right, so you're reacting based on messages. You can possibly filter things because you don't have to respond to every message you receive. You could create more messages for yourself, similar to processing streaming data. You can decompact streaming data by sending out more messages; this allows you to create additional messages for processing that streaming data.

You could decompact streaming data by sending out more messages than you receive. You can do a lot of things that way. Many systems, like Spark, are built on the idea of actors, even though they don't use actors directly. This relates to multi-user concurrency.

This one is interesting. It's probably my favorite because it's just kind of cool. It plays on the idea that actors are persistent. So, let's say you have a gambling website. A user logs in.

They want to spend some money. They'll probably lose some money too. Hopefully, they'll lose more than they spend if you're running a profitable website. You want to track how much money they are making and spending, kind of like we saw with that checking account earlier. You want it to be transactional.

You definitely want to make sure you're safe about it. You don't want to erase conditions when you're talking about money. What you can do is create an actor when the user logs in. This actor represents the user in your system and exists for the entire session or the duration that the user is logged in. Any actions they perform within your gambling website will be represented through messages that are sent during their session.

This is updating how much money they spend and make. This allows us to handle everything transactionally, while also being very scalable. Therefore, we are not relying on a single centralized database to do this for us. Another interesting aspect is systems with high uptime requirements. This ties into the earlier concept where restarts and lifecycle management can be largely transparent to the user interacting with the actor.

And this allows us to build systems that are kind of self-healing, right? If there's an error, the supervision kicks in, handles the error, restarts the actor, and it prevents any unexpected exceptions from bubbling all the way up and crashing your application. Ericsson, the company that created Erlang, is a telecommunications company that builds network nodes. The story goes that they were trying to find a language that would provide them with very high uptime. They developed their own language and have reported nine nines of uptime, including planned downtime for their network nodes.

And, in part, they attribute that success to Erlang and its ability to handle failure in a very graceful way. Then we have applications with shared state. We can also use a checking example. If you have shared state, actors are a fantastic way to wrap that shared state and ensure safe access to it. Okay, so I'm going to talk through a little bit of an example here with batch job processing.

So, this is something that we encountered a while back at Abnexus. Our clients expressed that, while they could create campaigns and creatives through the API, they wanted an easier way to create a large number of campaigns simultaneously. They asked for the ability to create around 5,000 campaigns at once using a simple method—essentially, they wanted to provide a large blob of JSON containing all 5,000 campaigns.

Go off and create them. Come back and tell me, how did this go? So we designed a system that looks something like this: we have an HTTP actor, with each circle representing an actor that pushes into a queue actor. This queue actor then distributes jobs among bulk processing actors. These actors handle large bulk jobs, right? So a request comes in, as I mentioned, and it goes to the queue.

You know, the queue is basically just kind of doling out work as we have actors available. The bulk processing actor then has this giant job. It's like, okay, I've got to process this giant job. And then we have these small job processors, each with their own pool, and they're each doing one thing—creating one campaign and creating one creative in our advertising system.

And the bulk processor just keeps track of all this, right? It's very stateful. It keeps track of what we have done, what succeeded, and what percentage we are at in completing the job, right?

This is all information that can be reported back. Once we're done, the bulk processing actor can respond directly to the HTTP actor, allowing us to send our results back to our client or perform whatever action we want. In this sort of system, we can not only define this abstract data flow through our system very easily using actors, but we can also control the level of concurrency we want to support, ensuring that our system does not become overloaded.

All the actors kind of use this push-pull mechanism, so we have back pressure and the whole system won't be overwhelmed. If we want to support more throughput in the system, we can add more instances or increase the level of concurrency that our application is using. We will create a list of actors, and then we'll say, "Okay, when the prime finder actor starts, I want to go ahead and start my actor pool as well." So, when starting, we'll define some actions to run.

We'll say one to five; we'll create five workers. Then we'll loop through here. Whoops. For each of these, we'll do the same thing we did with the prime. So, let's say `val workerActor = actor`.

And then we'll discuss the context. That's just something that's particular to the ACCA system here. We'll also make similar efforts to look for the messages. In this case, we're searching for messages that are simply going to be an integer, right?

We're not actually going to process a range. We're just going to process one integer value. We have a simple thing here. We're just going to say, you know, if the value is prime, right?

Then I want to send a message. Now, the exclamation point here is a semantic symbol for "tell." I'm informing another actor. I'm sending a message. So I'm sending a message back to the sender, and I'm going to send them a two-tuple pair here.

So, this is going to be the value that we received, and true. So, true is, you know, yes, we found a prime number. Alright, we’ve defined our worker. Down here, we found a prime number. Alright, we’ve defined our worker.

So down here, we can refer to it as the worker pool, and we’re going to pin the worker actor that we created to the pool. Alright, cool. We have our workers, and we have our worker pool. Now, down here, I left this blank. What are we going to do when we receive a range?

All right, so the first thing I want to do is define our workers. What I'm going to do here is take our worker pool and turn it into kind of an infinite iterator, similar to a circular array. When we reach the end, we'll go back to the beginning. Then, I'm going to use `range` to iterate through the range.

And for each number, I'm just going to say `workers.next` and I'm going to send it the number. Okay, so we've sent all the numbers off to our worker actors, but they're sending information back, right? They're sending that message back of either, you know, I found something and this is what's prime, or they're not sending back anything at all, right? They're sending that message back of either, you know, I found something and this is what's prime, or they're not sending back anything at all. Right?

So down here, we just need to look for another message, right? We'll create another case and say that this time we're going to receive a value that's an int and a true, right? That's how we know that what they found is prime. We'll just call out a print line and say, "That's the value that was found." It's fairly simple, but you can see how we're sending messages.

Right? Okay, so we've defined the Prime Finder actor, right? We've defined it; it's fairly simple, but you can see how we're sending messages. So, we've defined the Prime Finder actor. It exists and is persistent, as we've seen.

But we haven't actually sent it a message. So if we want it to do anything other than just exist, we need to send it a message. If I go in here and say "prime finder," I'm going to send it a message, and I'm going to send it 1 to 100. I'm creating a range of 1 to 100, sending it to the actor, and it should print out and tell me what was found. So let's go over here.

You'll see this. Just tell me what was found. Now, let's go over here. You'll see this. Just make sure this compiles quickly.

It compiles. All right, cool. So let's go ahead and start this. And we're done. Now we can see it printed out all the lists of prime numbers from 1 to 100.

And you can see that it didn't quite show up in the order that the range is, right? Because you have five actors processing at once, so you get them a little out of order. We got 3, 2, 5, 1, 7— a little bit out of order. So that's kind of the demo.

Now you've kind of seen what actors are. We've explored some of the use cases for when they're useful. We've had a demo, though it was a bit of a contrived example.

But what are the anti-use cases, right? When do you not want to use an actor system? When is it not appropriate? Well, the first is probably the silliest one. That's when you're working in a system where you cannot have concurrency.

That makes sense. Don't use features, don't use threads, don't use actors, right? Because they are concurrent. When you have performance-critical applications, this isn't to say that actors can't be fast; many actor implementations are indeed very fast.

But it's important to note that they are an abstraction. They exist on top of processes, threads, and servers. If you have a very performance-critical application and you need fine control over the threads that are running—how they're running—maybe you need to interrupt processes at certain times because you have a very tight latency window that you need to meet, then actors are not going to solve that problem. They abstract all that information away from you, and you usually don't have much say, depending on the implementation, about how those actors actually execute. The next consideration is when there's non-concurrent communication involved, and I’m going to pair this with the absence of mutable state.

Sure! Here’s the text with correct capitalization and spelling: "Right? So let's actually, I'm going to explain this one with an example. Right? So we have this actor who is going to receive some requests, and then within that, it's going to make a database query. Based on the result of that database query, it's going to make another database query. Then, based on the result of that database query, it's going to get some value from Redis."

And we can assume that the database query and the Redis operation are both going to be blocking operations, right? We also notice that this actor has no state. Essentially, if we created this actor and sent it a bunch of messages—let's say 10 messages—then we would experience little to no concurrency. Each one of those messages would be processed one at a time, and we wouldn't really gain anything because we don't have any shared state that we are accessing.

So there's no real reason to use an actor here. I could create a pool of actors, kind of like we did with our example, but, as I said, the example is a little contrived. We could create a pool of actors, and maybe that would get us what we wanted. However, there are also better approaches, right? In this case, a future is probably more appropriate to use.

Futures are stateless. When I create a future, if I call this method five times, I get five futures. This means I have five threads of execution or parallelism, depending on how many futures I create. All right? Okay, so if you're considering threads of execution or the level of parallelism based on the number of futures I create, that’s correct.

Okay, so if you're going to use an actor system, what are some of the drawbacks you need to be aware of? The first one is having too many actors. While you may not be able to have too many kittens, you can certainly have too many actors.

Right? Now, there are some people out there who are a little actor-crazy, who will say that if you turned an object-oriented system into an actor system, then every class should be an actor. Right? That’s a little extreme because, as I mentioned, actors are asynchronous. Actors rely on message passing.

So tracing that and understanding that when you have so many actors, it can be really hard to see what's going on, how they're composed, and it just becomes a little difficult. This kind of leads into testing, right? When you have so many actors sending numerous messages, and all these actors are potentially stateful, things can get really hard to test. How do you know you're testing the right actor? How do you know you're not unintentionally testing a bunch of downstream actors because of messages that might get sent?

Additionally, since actors are concurrent, you typically need some sort of mechanism in practice. You don't want to introduce complexities such as thread sleeps and timing-based issues into your testing, as that can lead to poor outcomes. For example, Akka has an Akka Test Kit that allows you to test actors in a way that appears to be synchronous. So, definitely take testing into account when you're building a system with actors. Lastly, debugging can be difficult under any circumstances.

Anytime you introduce state with concurrency, it can be a bit difficult. However, it's important to note that actors don't really solve that problem. State and concurrency together are always going to be challenging to debug. Additionally, if you're debugging in production and, say, you open up something like `top`, right?

You want to see what processes are running. Maybe one of them is going crazy. Perhaps you want to see the threads running on your machine. What you're not going to see, right? You're not going to see which actor is going crazy.

You're going to see threads going crazy. As I mentioned before, actors are an abstraction on top of threads and processes. If you notice a thread behaving erratically, you might take a thread dump in hopes of identifying the actor that is causing the issue. However, by the time you've done that, the actor may have moved to a different thread because the scheduler has determined that this thread is too congested and has decided to move the actor elsewhere.

Things can get a little hairy. All right. So if I haven't scared you away too much, there are additional resources. These slides are going to be on my GitHub, so you can click all these links and take a look at them. My two favorites are the wacky ones at the bottom, which remind us not to use actors for concurrency, right?

This is kind of a provocative title, but it's really just about when to use actors and when not to use them. It's a pretty good discussion. The other topic is, you know, why the actor model has not succeeded. I obviously disagree with this perspective, but it is from an older poster, from '97, and it does make some interesting points when discussing running actors in production and addressing performance concerns. Obviously, there are some books and talks that you can read that are quite good as well.

ACC Professor in Production is from the Pacific Northwest Scholar Conference, and it's a really good talk about building a large-scale metric system and doing a lot of streaming work with ACCA. If your questions don't get answered today, you can reach out to me through Twitter, email, or GitHub. The GitHub link is where you can find the repository for the slides from this talk. If we have any questions, I apologize for not mentioning this earlier, but there is someone with a microphone walking around, so if you raise your hand, you can speak. I believe we have someone already in the middle back there. I'll just be pleasantly walking around.

I'm not really going to run. So, you mentioned actors for pipelines. I use Scala. Is there any advantage of actors for pipelines that aren't solved by Akka's own streaming engine, for example? Because it has a much higher-level interface.

Based on my experience with actors, it's exactly as you mentioned. It can be a bit hard to follow what's happening. Is there any advantage to using ACA? The second question I have is about persistence. What happens if an actor dies, for example?

Of the state of the actor, can it be recovered, or is it lost completely? How does that happen? Okay, so the first question about streaming: I don't think I've used the Scholar Streams enough to really know. The main benefit with ACCA is just being able to compose different actors together. If you can achieve the same semantics with streaming, then I would say that streaming is probably a little bit better in terms of formulation. What about the second question?

Oh, actors dying. Okay, so if an actor dies, we already saw that the mailbox will stay, right? Because it exists outside the actor. The internal state of the actor itself, in the pure actor model, I don't think it actually describes what will happen to that state. I know for some frameworks, like Akka, there are persistent actors that will attempt to keep that state if the actor restarts. I believe they do some snapshotting as well.

That may be more of an implementation-specific thing. From what I've seen in most actor systems, the default behavior is that if an actor encounters an exception and is restarted, it actually loses its state. A lot of this is based on the idea that that state might have actually been the problem that caused the issue. For instance, in Akka, I'm not sure about other implementations, if you have an exception on a particular message, that message will not be replayed against the actor when it is restarted, because that message might have caused the issue.

Right? And it's up to the supervisor to determine if it's the message plus the state that causes the issue. The supervisor might simply say to ignore the exception and continue processing, or they might decide to restart the actor instance. It was a little unclear to me when you were discussing message passing between the supervisor and the actor where the responsibility lies. Is the supervisor polling the actor for information about its internal state and current processing? Is the supervisor responsible for notifying all other supervisors? It seems there's no direct communication relationship between the actor and the supervisor.

There's usually some sort of special relationship between the two. In the case of Akka, for example, I'm speaking from my experience with Akka. When there's an exception, it's actually wrapped up, and the framework takes care of invoking the supervisor's handler code. No exception escapes the actor. In other systems, there are different ways of handling it. When an actor goes down, there might be some default functionality provided by the framework, where a message is sent to the supervisor to notify them that the actor has gone down. Akka has a similar concept known as a death watch, where you may not be supervising an actor, but you may care if it dies.

And you can kind of watch that actor if it dies, and then you get a message that says, "Hey, this actor has been shut down." Depending on how you set up your application, the actor might not reach out to you when it has been shut down. You can then perform some action based on that. So, there are different relationships.

A lot of it's going to be dependent on how your framework is implemented. The actor model, in general, doesn't specify a whole lot about the relationship between those two. But, yeah. One more in the back, and then, okay, one more over there. Any farther away, huh?

I just want to know for Node.js developers, what does it mean? What does this model mean? Is it like the default Node.js? Is it the default model of Node.js? Yes.

So, as a Node.js developer, what advantages would I gain? Sure. If you're talking about Node.js, it has its own model for concurrency. I don't think it quite fits into the actor model for concurrency because it doesn't have that message-passing paradigm already. In terms of how it's applicable, to be honest, I'm not really sure how you can apply it to JavaScript in general, especially since a lot of JavaScript operates in a single-threaded manner.

Yeah, I'm not really sure, to be honest. If you want to talk to me more later after the talk, maybe we can chat. Is there a different toolchain that you use for monitoring and logging when your app is structured to use actors? Yes, there are tools available; metrics, tracing, and debugging can be quite challenging. There are different tool sets out there that allow you to instrument actors and see message flows through your system.

Not a lot of my scenes are really mature; they are either a combination of different tools put together to support an actor model implementation specifically. So, it's definitely a challenge when you're working with actor models. For Akka, there's one particular tool for actors that helps trace a lot of information, but I can't seem to remember its name.

But it's difficult, especially when you're looking at actors that span multiple servers. Then you have to trace that request across servers, and so you definitely need some really good metrics. However, there's nothing that I've seen that's really, really mature. Okay, I think that's it, so I'll hand it back off to Ben. Thank you.