Designing The Lustre Web Framework (with Hayleigh Thompson)

Web app architecture is still an evolving field. I would say the front-end user experience feels like it has stabilized. We all know roughly what a web UI is supposed to look like and how it should feel. However, the architecture underneath is definitely still in flux. Should we be using REST, WebSockets, or GraphQL?

How much rendering are you supposed to do in the browser, and how much should be done on the server? How do you manage the state of a web page when, nowadays, the state can be so large that it definitely needs some management? Web pages today are complicated entities, and we need good architectural practices to handle them. Joining me to discuss some possible next steps in the evolution of web app architecture is Hayley Thompson. She's a React developer and an Elm developer, using two quite different frameworks in two very different languages.

Synthesizing her expertise into a new framework called Lustre, this framework incorporates some inspiration from the Actor Model. It has connections to Erlang, and there are parts of Lustre that resemble location-agnostic actors more than traditional web pages. How does that work, and why pursue this approach? What are the benefits?

The upside is all in pursuit of the perfect architecture. So let's see where she's taking this. I'm your host, Chris Jenkins. This is Developer Voices, and today's voice is Hayley Thompson. My guest this week is Hayley Thompson.

Hayley, how are you? Yeah, I'm really good, thank you. How are you? I'm very well, very well. You've picked one of my favorite topics, which only became my favorite topic about halfway through my career. So, I got into front-end programming probably around the early 2010s, just out of sheer frustration that the quality of engineering, in my mind and based on my experience, was pretty terrible. If it worked, you weren't supposed to complain about anything else.

I found AngularJS, which, in hindsight, was not the greatest design, but at least there was a design. That’s when I really started getting into not just front-end programming, but how we can make the engineering of it sane. That was my starting point, but it's about you. So, what's your starting point in the front-end world? Well, actually, my background is in music computing and creative coding.

So I actually got into web development through the Web Audio API. Yeah, I was teaching myself programming, and the idea of learning C was kind of off the table; it was a bit too complicated for me.

So, when I discovered that JavaScript had this API for synthesis in the browser, I kind of just ran with that. I probably started around 2016. Back then, I was doing a lot of C++ with OpenFrameworks and Java with Processing—creative visual coding. Okay, Processing is an audio framework in Java, right?

It's more of a visual framework, actually. There's an audio module, but at the time, I was working on generative art and concurrently learning web development because it sounded interesting. Then, I discovered the Web Audio API and started experimenting with it. I picked up Vue in 2018.

I was doing that for a while. I leapfrogged over React and TypeScript and all of that. At the time, npm, Node, and all of this were super intimidating to me. I really liked that I could just drop Vue in a script tag and get to work. I didn't really understand package management or anything like that at the time.

Right. And then in 2019, I came across Elm. Yes, I took a functional programming class at university that taught me Haskell.

And I had seen Elm previously, maybe a year before, and bounced immediately off it because it looked super weird; it was very strange. After I did this cast, I kind of came back to Elm. There was this big push at the time, around 2019, with a lot of the big names in the Elm community giving a lot of conference talks.

I was receiving a lot of recommendations on YouTube about how Elm was solving many significant problems, such as combining TypeScript, React, Redux, Immutable.js, and Webpack into one thing. I decided to explore it further, and the rest is history. I've been working with Elm for five years now. For about a year, I've been conceptualizing what Elm 2 would look like in Gleam.

Right, which is luster, right? Exactly. So we're going to get into luster, but elm 2 in gleam. Right, which is luster, right? Exactly.

Okay, so we're going to get into Lustre, but the fact that you've created Lustre positions you as an architect of front-end design patterns. So, I'll give you the interview question: how would you describe and contrast the way Vue does it to the way Elm does it? Yes, that is a very good interview question—sorry, this is an interview, but not that kind of interview. I think what a lot of current front-end frameworks come down to is this very component-driven architecture. In many ways, you’re thinking at quite a small scale. You're encouraged to focus on components. Vue is a really good example, actually, because they have these Single File Components (SFCs). You have the markup for your component, a script tag that encapsulates all of the logic, and a style tag that styles the component. The idea is that by focusing at the component level, you create rich, interactive, sophisticated experiences, which you then compose and piece together into something much larger. In stark contrast, Elm takes a very holistic view of your application. What do you think?

Okay, what are the main domain types in the application line building, for example? What does my model represent? Your model represents your entire application state. From there, your UI functions are pure functions right now. You have no component state, no local component state.

Instead, you have simple building blocks to piece together your UI, which is largely divorced from your application logic and business logic. I think these two approaches are very different. It represents a dividing line between object-oriented programming (OOP) and functional programming (FP). In one, you think of the world as being broken up into little islands of objects, while in the other, you view it as data and the operations you perform on that data.

It's interesting because in the Elm guide, there’s actually a footnote written by Evan that addresses a common question: "Where are components in Elm?" This footnote explains that components are objects, and we don't have objects in functional programming, and so on. How did you find the transition from one model, which is probably a more popular object-oriented model, to the functional programming model? Actually, the transition was, in some ways, predictably difficult because it's a significant shift in your worldview, I would say. However, I was early enough in my journey overall that the scale of the things I was building and working on was small enough that the transition to this Elm approach, where you have a model that describes your whole application, actually worked really well for me.

And I actually found that transition very smooth in a lot of ways and now it feels very, very natural. Okay, okay. It sounds like you found it easier than I did, but I was more immersed in the whole "objects are everything" worldview. Yeah. Okay, so I think these things are less stark.

Different at small scales, as you say you're dealing with small applications. Yes, they are different, but the differences don't really become apparent until things scale up. Have you worked on large Elm projects and large Vue projects? I've not worked on large Vue projects, but I am also a React developer these days. Okay, okay.

So, I have worked in both large Elm and React codebases lately. Do you want me to compare and contrast them without being too biased? Actually, I don't think I'm that biased; I'm a really big fan of React. After working with Elm for four years before I even looked at React, I found that when I came to React, it offered me many features I wished I had in Elm.

Oh, okay. So I think both approaches do scale, I would say. In my experience, I've worked in big codebases in both, and both of them have been pleasant enough. I think they each have their own pain points. However, I believe there's a significant shortcoming in Elm's model, which is its complete rejection of encapsulated state. This, I would say, is a very dogmatic approach to encapsulated states as objects. Objects are a part of object-oriented programming (OOP), and OOP is viewed as bad. This perspective breaks down quickly when you consider the DOM and the HTML that you're already rendering.

Explain that to me. The example I always give is the input element, the native HTML input element. This element has so much state that you don't ever interact with it, right? The native HTML input element has a lot of internal state that is not typically manipulated directly.

Like we set the value of it. But think about managing focus. For example, if you tab into it and it is now focused, this is an internal state. Consider handling keyboard interactions or even something like blinking—the exact placement of the cursor or the cursor blinking when you make a selection. All of this is handled by the browser, but it is managed as an encapsulated, stateful thing. In our maps, we say, "Well, we don't worry about that; we're going to render the marker." I think there are times, especially now in modern web development, where we are really crafting quite sophisticated applications.

We've moved beyond documents, or at least the industry that I'm in has moved beyond documents into applications. Here, you have very rich interactive elements on a page. If you really want to craft a native experience, I think it very quickly falls down in Elm's model, where you have to store all of that somewhere centrally. Yes. Okay.

I can buy one place I've experienced that is trying to do masterminds centrally. Yes. Okay. I can buy one place I've experienced that is trying to do maps in Elm, and they are sophisticated. Efficiently rich, but it does start to become very difficult.

But, okay, so you're not supplying enough controversy, so I'll do it, right? The polarizing line—and all programmers try to be polarizing at some point—is that Elm tries to reject the existence of state except for one big global state. I think that works quite well generally, highlighting some problems. On the other end, where every tiny little thing becomes a component, I find that the individual units work well, but there's a hidden graph of interactions that becomes impossible to debug.

If you're saying components are sometimes good, I think that's what you're saying. How do we know where to divide up those lines? I think a lot of it comes down to culture. I mean, I’m going to keep emphasizing Elm because it's the thing I'm most familiar with and also the thing I have the most opinions about. Okay.

And so, as a culture, as Elm developers, we have this rejection of encapsulated state. However, if you look at large Elm codebases or complicated packages, there is this implicit recognition that some behavior is complex enough to warrant encapsulation. Therefore, we create an opaque type that we instruct users to store in their model. Additionally, we provide functions to update this state and a function to render it. What we are doing is acknowledging that this is sufficiently complicated that we want to hide its inner workings from our users and consumers. However, Elm is not expressive enough to achieve this encapsulation without breaking it into three isolated pieces and requiring our users to connect them. As Elm developers, I believe we have a good intuition for when we should take this approach.

Because the language and the framework itself make it quite tedious, it's not an approach we reach for immediately. Whereas in React, Vue, Angular, or Svelte, it's very easy to create a component with encapsulated state. The culture there is focused on creating many encapsulated components. I think it's quite obvious to say, but it comes down to what the language makes easy. If you make something hard, your users will only do it when they need to.

I'm going to play devil's advocate for the Elm team here and say that the reason it makes creating components hard is that they're a bad idea. But I would say that it's clear. Would that be the fair Elm line? I think it's the fair Elm line. However, if you take just half an hour to look at things being built with Elm, you will see that they are still making components.

They're just creating structures that are technically not components, but are functionally identical to one. If you publish a package or work internally, and it isn't technically a component, but operates just like one, that's the idea, right? For instance, if you have a module with its own model type, a message type, an update function, and a view function, you have essentially created a component. However, you can't assemble those pieces yourself, so you provide the individual parts to your users and instruct them to wire it up, but to you, it's a black box. Yes, exactly.

Okay, yeah. I'm now thinking of a table library I've used in Elm. It's not as customizable as I'd like, but it's great when you just want to create a table with sortable columns. I always have to double-check and triple-check how to install it because I'm installing it in three different places.

Exactly. The ARM community has a strong infatuation with web components, which is somewhat uncommon among other front-end frameworks. I think, as a front-end community, we have largely concluded that web components aren't as beneficial anymore. However, the Elm community, in particular, is very bullish on the idea that when you have encapsulated state, you should put it in a web component and render it in your view. I've seen people experiment with building maps as web components, and I believe that indicates there are individuals who recognize that when you have sophisticated logic combined with Vue, there is value in the concept of packaging it up in a black box.

Because this web component is very much like the input tag, right? Exactly. In that you just treat it like HTML, but it actually hides a whole bunch of interesting states. I can see why, philosophically, Elm people would be attracted to that idea since they're already using that concept in even simple form fields. However, you seem to think it's not such a great idea?

I think it's an excellent idea. Oh, okay, I've misinterpreted you. Yeah, I'm a big fan of that idea, and in fact, this is how components work in Lustre. Just to give a quick overview, Lustre follows a similar model-view-update pattern as Elm. You have a model of your application state and a message type that represents all the ways the world can communicate with you.

You update the state, and you have a kind of pure view function. Born out of this frustration with Elm, what you can do in Lustre is take those building blocks from the model-view-update package and treat this as a web component. It registers as a real web component that you can render in your application like any normal HTML element. You communicate with it in the same way that you would with native HTML.

So you set attributes, and the component sends you events. There isn't this split idea of, "Well, I render HTML this way and I pass data into HTML this way," versus "My framework has components, and I have a props object that I pass in." It's quite common these days for the idea of a component to be a framework-specific concept that is detached from the idea of HTML. In Lustre, these two aspects are operationally equivalent at the point of HTML, and in Lustre, they are also operationally equivalent at the point of consumption.

Does that bring back the problem of once you've decided? Because there is a nice benefit in separating the behavior from the HTML, in that you can render it in multiple ways. You can have the same underlying logic for multiple presentation types. It sounds like you're losing that as you tightly couple the HTML tag to its functionality. I think it will always come down to API design, and that is unavoidable.

So, I would say that yes, if you have committed to treating something as a component, then you have lost the ability to treat it as constituent parts. But if that functionality was valuable to you, you didn't have to make it a component, right? Yeah, okay. I mentioned previously that it's about culture, and it's about what the framework makes easy. So, there is a bit of overhead to creating a component in Lustre because you do need to build a full application.

You do need that model type, that init function, and that message type, right? You need all of those bits. Only once you have those bits can you make them into a component, which is quite different from conventional front-end frameworks. In those frameworks, you might initially start with a function that just returns some markup, and then, when you decide that you need state, it's very easy to add state, right? It's very easy to turn it into a component.

But the fact that Lustre kind of adds a not insurmountable but obvious hurdle means you’re discouraged from making this approach the default. The default approach is still always to write pure view functions. Okay. So you’re saying if I come into Lustre, I will get largely the Elm experience, which is to write some data describing my idea of the world and the way it can change, and then write some rendering code. However, the option to turn it into a web component—like a live HTML tag, a smart HTML tag—is very much open to me.

Yes, that's pretty much the ideal scenario. Okay, is that novel? I mean, I've never actually tried to create web components in other frameworks with anger.

Is that common? Is it well supported? Do you think you do it better than anyone else? Yeah, I think the best example I can think of is Svelte.

So you can compile any Svelte component into a web component, at which point Svelte has disappeared, and you could use it anywhere. This is the same in Lustre. I don't think that idea is novel. I think the novelty for Lustre is actually that we have this idea of universal components. Oh yes, I want to get into that, but first we've got to have a quick sojourn into Gleam and what that offers to build it up, right?

So, "Lust" is written in Gleam, which is a language I know you love. Why Gleam, and what does it offer as a front-end developer first? Yeah, it's actually—I think that's a great question. However, I believe my answer may be somewhat dissatisfying. Okay.

I think there are a lot of exciting and mature technologies available. As a front-end developer, you are spoiled for choice. In my opinion, the strength of Gleam lies in the fact that if you're a back-end developer, you gain access to the front end, which is less common. You can access the front end through Beam's JavaScript target.

So, for people who don't know, Gleam compiles to both Erlang and runs on the BEAM, the Erlang virtual machine. It also compiles to JavaScript. While it's not exactly novel to have a full-stack language, I do think that if you are a front-end developer, you're spoiled for choice. However, there are reasons to choose Gleam. I would say that the biggest one is...

The big one for me is that Gleam is just much semantically simpler than TypeScript. The experience of using it is much lower in complexity; there's a lower cognitive burden, I would say. However, I think the real superpower is for back-end developers who don’t really care about front-end development but need a front end because everything requires one now. The selling point, then, is that you get access to a front-end framework created by an enthusiastic front-end developer who, hopefully, knows what they're talking about.

And you get to use the same language, the same concepts, and the same primitives. So, is Luster mostly about bringing good front-end ideas into the world of people who don't necessarily have access to many front-end concepts? That's definitely been my strategy so far. Interestingly, front-end developers are a naturally curious bunch, I think. If you're into front-end development, you've almost definitely explored a dozen languages that compile to JavaScript and tried a hundred different frameworks.

And so, actually, the number of people using Lustre right now, especially about four months ago when Gleam had just become version one, consisted mainly of front-end developers. There is already a community of knowledgeable people. However, my focus as a brand strategy is really on targeting developers on the BEAM. Specifically, I am interested in Erlang and Elixir developers. The reason I believe this is a viable, interesting, and exciting strategy is that I like Elm's idea of how front-end development should work. The Model-View-Update architecture is strikingly similar to what Erlang and Elixir developers are already implementing on the BEAM with actors. Yes, there is a huge parallel there, isn’t there?

There's a huge parallel there, isn't there? Because it's like—yes. It's all about treating the world as a state machine. Exactly, it's entirely message-based functional updates of state. It's just the Elm and the Luster thing that we use to add a view function to that building block that you already have. We just need to add a pretty way to look at your actor. Exactly, I've mentioned in talks I've given in the past that I frame it as your front end being another actor, but not an actor in the Beam.

Now this actor lives in the browser, but it's the same. You still send messages to it. It still has an update loop. I think for a lot of those kinds of developers, that's a real light bulb moment because it's like, "Hey, this kind of approach to building applications that I'm already familiar with, I don't need to learn a new idea to build the front end." I still need to learn how to put HTML together, but in terms of creating an application that has behavior, you're already doing what you need to do. Yes, and it's interesting because that's exactly the flip side of the coin from where I started out, where people knew how to generate pretty-looking HTML, but there was no fundamental underlying architectural pattern to make it sane.

You're turning that on its head and saying, "Here's the architectural pattern; now all we need to do is make it look good." I always outsourced that to a designer, often relying on them to provide me with Photoshop documents anyway. So, in the end, you're just left having to center a div, but you know that you can make that happen eventually. Actually, I have a companion library. I have a UI library that is currently a bit in flux, but the idea is that it's just a UI kit. If you don't know or care about front-end development and just want something consistent and visually appealing, is that something like Bootstrap or Tailwind, conceptually?

Yeah, I suppose so. Conceptually, yes. Conceptually, it's like Bootstrap in the sense that if you just need something that looks good and a way to put it all together, it's perfect. Yeah, it's absolutely great for the prototype, and we're aiming for a higher design.

As if anyone complains. Yeah, pretty much. Okay, so I can certainly see that sell, especially for people who are on a backend like Beam, for which there is no obvious frontend framework to go to, as far as I'm aware. The major architectural difference there is that if I'm trying to think like an Erlang Beam backend Glean developer, I'm expecting to have lots of different actors running my application. It seems like the frontend is going to largely be a single actor.

Is this where the idea of making web components comes in? How does that play out—the proliferation of actors for tasks? I mean, that is largely it. So you would expect to have many actors in your application, for sure. In most cases, your front end, your view, will be one actor, right? The browser. But I think the draw of the Lust approach to components is that if you do need more, you are still proliferating that same idea—the same update loop idea—into components now.

And they do compose. So, like in your Erlang applications, your actors form a graph, right? You have supervision trees, but in terms of communication, you essentially have a graph of actors that can send messages to one another. The only difference with the front end, then, is that rather than a graph, you have a tree, right? Because you're ultimately rendering HTML.

So there is, but the building blocks are the same. I'm going to bring it up again because I think the most exciting part is that, if you have this application type, which just bundles up the idea of an actor with a view function, you can use it as your application. Additionally, you can embed or nest them as components. This idea is already very familiar to the Beam. So, why can't you just move that whole bundle as an actor on the Beam? Which is exactly what Lustre lets you do.

Take me through that. Yes, we need to dive into that. So you've got an actor running on the front end generating HTML from a certain point of view. You want to move that to the back end. Why?

I think the most compelling argument is that if you need or want direct access to some sort of backend service, right? Like maybe you're doing real-time data visualization, or you have a requirement for real-time aspects, which is the main appeal of this ability. Additionally, this capability eliminates the need to worry about network transport. You don't have to craft an API layer on top of your database, perform JSON serialization, deserialize it, and then convert it into HTML. You can conceptually run your component next to your database, make database queries, and transform the results directly into HTML.

Right? Run your component next to your database conceptually, and just make database queries, and then turn it into HTML. So, before we get too deep into that rabbit hole, let me provide a little bit of grounding or anchor it to something else that already exists. The Elixir community has LiveView, which is the idea of running the entire front end as an actor on the BEAM. But this is for entire applications.

So, you would have a live view application, and what you do is send a minimal runtime to the client, which sets up a WebSocket connection. Then, your entire front end runs on your back end and just sends DOM patches—essentially HTML divs that get live updated. Right? So the front end is just saying, "User clicked this button," and the back end is sending back the updates.

You need to insert a new div here. Exactly. This is how LiveView works. LiveView encourages you to build your entire application this way, which is a very popular approach among Elixir developers. They specifically prefer not to deal with the front end, right?

This lets them reuse everything that they already know about Elixir to create interactive front-end experiences, which is great. I think it's really, really cool technology, and it's very, very exciting. It's somewhat confusing that it never really caught on outside of Elixir. I believe there's a PHP library that does something similar, but I think it has a glaring problem: now your entire front-end interactivity is mediated over a WebSocket connection, which, as a front-end developer, seems absolutely wild to me.

As people in the UK who have interacted with services that only exist in California, the latency can be suboptimal the further away you are. Exactly. I think this approach is very popular in areas where there is excellent internet, as well as for internal tools. Yes, with a good internet connection, you wouldn’t notice the difference. Exactly.

So, Lustre has essentially looked at that and thought, "Well, that is an excellent idea." We already have this concept of components that is compatible with how things already work on the beam. Instead of shipping our entire frontend to the backend and running it over a WebSocket, why don't we take the idea of a component and say that you can render it on the client as a web component, or you can run it on the backend as a server component? This allows for a seamless experience where sometimes it's actually running in the browser, and other times it appears to be running in the browser, but it's actually DOM patches being sent over the wire. Exactly.

The idea is essentially that, in cases where you aren't writing platform-specific code, you can achieve a more flexible application design. For example, if you write some code that uses an Erlang library, you obviously cannot run that in the browser; that's fair, and I’m not going to solve that problem. However, if you write your application in a platform-agnostic way, you can provide a JavaScript implementation as an alternative for the JavaScript target. The idea is that you can take your application, encapsulated in a Gleam type that includes your application update function and view, and the only thing that needs to change is how you start it.

You can start it as a single-page application or a SPA by simply calling "Lustre.start" in the browser. You can register it as a web component, or you can register it on the backend as a server component. The idea is that your application code never changes. You could place that in a separate project or publish it as a package, allowing others to choose where it runs. Now, let me think of a scenario. I could start off with a project that is hosted on a local intranet, rendering everything on the server-side.

Eventually, I decided to spin out some parts of it to the public, and then, 18 months down the line, I started to push some, if not all, of the application out to be genuinely on the front end. Sure. This is when you would lean into components. We had that discussion, right, kind of debating whether the idea of encapsulation of state is a good or a bad thing, whether it is a naughty or a dirty thing. But actually, when you draw that hard component boundary, you say this is a black box that is encapsulated. The benefit of making a black box is that, from the parent's perspective, it doesn't need to understand its internals. If you wanted to start with your application fully rendered on the client, for example, as a single-page application, you could do that. Then you could say, "Okay, actually, I have some complicated piece of functionality, so I'm going to make that into a component," because there are engineering benefits to doing that, right?

It's easier for me to think about this as an encapsulated thing. Mm-hmm. And then nine months down the line, you think, "Actually, this would be way better if I could just run it next to my database." Because you already drew that component boundary, the parent already doesn't care. It's not the parent's concern where the component is rendered anymore.

It just says, "I want to render my counter component." The counter is a really poor example because you would never create a counter component. But you get the idea. Yeah, yeah. And so at that point, it's just an implementation detail of the counter where it renders.

So you just say, "Okay, well, now I'm going to swap out that for a server component and I'll move my counter to the backend." What does that actually involve in practice? I mean, what things am I typing to make it now on the backend instead? In Gleam projects, full-stack Gleam projects are essentially organized as a monorepo. So you have a backend Gleam app and a frontend Gleam app.

And typically I suggest people have a third Gleam project for their shared code, such as types and serialization, and that kind of stuff that you want to share between both projects. So, let's say you have this counter component. It's in your front-end project because that's where it was. You have a module that includes the init, the update, and the view functions.

It's all neatly bundled up into its own thing. Currently, in the client’s main setup function, you have a call to `Lustre.register`, where you specify your component and provide your component app. Now that's registered as a custom element. The process of moving that to the backend involves removing that call because you no longer need to register it as a custom element.

Copying that counter is probably to your shared project now, or you could move it straight to the backend. You would move the code somewhere else so that you have access to it from your backend. Then, from your backend, you set up a WebSocket handler. When that WebSocket initializes, you call the Lustre start server component. Now, that's been an actor that communicates over a WebSocket.

And then the only thing that changes in the client is that you render a custom element called "Lustre Server Component" and point it to your WebSocket endpoint. If you organize this properly, that would essentially be hidden through a function call. You would have a function that is like `counter.view`, right?

Just from the counter module, you would call that, or counter.element, or whatever you decide to name it. You would simply swap out the implementation there, so the parent doesn’t even need to change at that point. Okay, so let me ask you a couple of quick practical questions on that. Do I need one WebSocket per component, or am I reusing a single WebSocket across the whole project? Currently, well, actually compared to LiveView, this whole approach in Lustre takes a step back from being tied to any particular web server, data transport, or protocol.

And so, you could actually use the same WebSocket for anything, because it's your responsibility to essentially hand the data from the WebSocket to the server component runtime. This is a negative in the sense that it requires a bit more boilerplate as an author, but it's a positive because now you are in complete control over how that works. You could swap out the WebSocket for HTTP long polling or server-sent events, or, I mean, if you were really ambitious, you could wire the whole thing up as a REST API. The server component runtime doesn't actually care.

Okay, I'm going to run it over a Kafka topic then, or a RabbitMQ queue, something like that—something crazy. Yeah, typically, you would use one socket per component, but only because that would be the easiest or least complicated approach. However, there's nothing technically stopping you from using one socket for all components on the page.

Okay, okay, so multiplexing is my job, but it doesn't sound like that hard of a job. The other question is about sessions. Are you automatically taking care of the fact that 50 front-end users are running 50 components on your server? No? Okay. At this point, basically, what Lustre gives you is an actor that represents your character.

Your application runtime, and the rest of the job, is related to Luster’s role, which is purely about rendering. Essentially, I'm expected to focus on the back end. Sorry, exactly—yeah, yeah. Luster really covers the frontend application aspect. The fact that it's running on the backend is somewhat of an implementation detail. As an author, you might not know as much about frontend rendering, so you utilize Luster. However, you have more expertise than I do in building your backend, so I won’t have opinions on that.

I think this is a benefit. Gleam's general approach, both in language and as a community, is to minimize magic. This often manifests as slightly more work for the user and slightly more boilerplate. However, if you can get over that initial gut reaction of thinking, "Oh, boilerplate is the worst; I want all the fancy things to do it for me," you will find that what you get is a very simple experience, one that gives you a lot more control.

Generally, in the long run, I favor that. Actually, what you get is a very simple experience, one that you have a lot more control over. Yep. Okay, so what about the practicalities? I should do a couple more checks on this.

So what about continuously nesting? It sounds like I'm okay if I take a client-side component that contains a server-side component, which in turn contains a client-side component. Is it just going to work? Exactly. It will just work.

Okay, okay. That's an easy one, which I think is, like, so magical. Is it just going to work? Exactly.

It will just work. Okay. Okay. Which I think is so magical. I believe that is really the coolest part.

The idea or scenario that I have in my head, which I always tell people, is like this: imagine you have an admin dashboard, or some sort of admin dashboard, right? You have it set up as a single-page application because you want snappy navigation, and it makes sense to render it on the client side first. However, within that dashboard, you have, say, a really data-intensive table. For instance, you might have a table that displays the million users registered to your startup or SaaS app, right? So, you turn that table into a component because you want to handle filtering, searching, ordering, and that kind of functionality.

You just want to encapsulate that, so you make it a component. Then, inside that component, you render a search input that handles focus trapping, autocomplete, and suggestions. You can make that a component as well, right?

Right now, you have your single-page application with a table component and an autocomplete search feature. The really neat thing is that, while you're spending a lot of time creating a whole backend service for querying your users—including adding pagination and filtering, building a REST API, and handling JSON encoding and decoding—you can think, "Okay, why don’t I take this table and render it as a server component?" By doing so, you can skip that entire layer and make database queries directly, allowing for more efficient communication with the database.

Yeah, yeah, yeah. But the table is still rendering a client component, right? It's still rendering that autocomplete input. So, you have all of the benefits of client rendering there, you know, like not having to worry about WebSocket latency at that point or anything like that, while still getting all of the advantages of the renderer.

Being next to your database while enjoying all the benefits of your application still running on the client—yes, you might have just found a solution to the age-old debate of thin client versus thick client. Each approach has its pros and cons, and you're suggesting that you can select and customize which advantages and disadvantages you want on a component level. Yes.

And because components, um, because components are conceptually larger than they are in React and similar frameworks, you don't get stuck in the same sort of debates or pitfalls that you might encounter if you had this ability in, say, Vue. It would be crazy if you could turn every single React component into a real-time thing that ran on the server, allowing for arbitrary nesting. That would create an absolute explosion of complexity and engineering effort, right? Right.

Because components are working on a far too granular scale in those frameworks, Luster components encapsulate much more, right? As a result, some of the operational downsides to this approach—such as a bit of setup—feel significantly lower because you're getting much more out of one smaller component. So, are you suggesting that there's a natural push away from components? It isn't a law, like it almost is in Elm, but when creating components isn't made the easiest path, their proliferation may not be the same. However, the option is still there; you just need the power.

Yeah, that's exactly it. And honestly, I find that when I'm making things in Lustre, I use components sparingly. There's a bit of setup involved, and also, you just kind of don't need it in, say, 80% of scenarios.

The Elm approach works extremely well for about 90 to 95% of things. So, don't fix what isn't broken. However, the part that is lacking is the 5% where you truly need a component, and you can't have it. Yes, okay.

So, do you have any guidance on where those boundaries lie? This is a question that every framework faces: what is the component boundary? I'm not sure I have any real rules right now. The framework is still young, and the idea of components is also still in its infancy. When I first started working on Lustre, we didn't have components.

It was a pretty faithful Elm implementation. The framework is now technically about a year old, but practically more like six to eight months old. I think there's a lot of learning happening within the community as we work out where those boundaries lie. There is some overhead to setting up a component; you do need a message type. I believe the message type is the crucial element that determines whether something truly needs to be a component. If you only have, say, two messages, you're not really encapsulating a lot of complicated interactions here.

And so the benefits of setting this entire thing up as an application are quite slim. You've gone through all that work just to hide the fact that this component responds to button clicks or something. Yeah. But as soon as—like, I don't really want to give a precise number because I don't think that's the kind of heuristic that is practical— but as soon as you get to, say, five messages, five, seven, or ten messages, at this point you're clearly dealing with something that operates in its own world and receives messages from the outside world specific to it.

And then I think it starts feeling a bit more natural to turn this into a component. Yeah, I see that. Which is a very unsatisfying way of saying I'm not entirely sure what that boundary is. But I do know that Lustre pushes that boundary further away than React, Vue, or Svelte do. I think that's a good thing.

Sure! Here’s your text reformatted with correct capitalization and spelling: "Yeah, it puts it on a high shelf that you can still reach, but you won't have it. It's like that trick where you put the biscuits, or cookies, on the shelf you can't quite reach. Yeah, exactly. If you really, really want the cookie, if you really want the cookie, you can get it. Yeah, but you won't do it unthinkingly. Um, so I have one more question about this in practice. If I've not really had a great experience trying to write web components, and if I just thought, 'Okay, I'm going to get into this just for the one feature of hopefully an easy way to make web components and publish them in a language of my choice for other people who don't care what language it's written in to use,' is Lustre on Gleam a good choice? It's certainly a choice."

I would say that this functionality or this particular scenario is something that I did have in mind when I was working on this feature. To rewind a little bit, Lustre has a companion dev tools package that runs on Erlang. The main goal here is that most of these developers do not need Node or npm, which is a selling point. There's a dev tools package for Lustre that includes a bundler, a dev server, and some other tools. Relevant to this conversation, it includes a bundler and specifically a command to bundle a web component. This means you can take any Lustre component, say "Hey, build this as a component," and you will receive a JavaScript bundle that you can include on any web page, and it will just work.

And then I just write a regular old HTML tag, and it's magic. Exactly. Okay, so there are some caveats. There are some caveats to that and some problems inherent to web components, I would say.

So Lustre puts in a bit of work to ensure that you, as an author, don't need to know that a component is a web component. Styling web components can be challenging in the web component world. What I mean by that is the term "web component" refers to more than one specification. There is the Custom Elements specification, which involves custom HTML tags with their own logic, among other things.

It also refers to the idea of the Shadow DOM, which is a way of hiding what the component renders to the parent so that you can't reach in and manipulate it using something like a query selector, for example. Iframes 2.0. Exactly. The Shadow DOM is excellent; it's really, really good.

But when you register a shadow DOM, what you have actually done is isolate the styles of the component from the parent. This is both a blessing and a curse. On one hand, this is excellent because it means that you can create a component with its own styles, and then you can drop it into a different web page, guaranteeing that it won't be affected by that page's styles. It looks exactly how you authored it.

Yes. But the problem, of course, is that if you want to use Shadow DOM and you want to use it in your own web application, you're not actually worried about conflicting styles. Now, typically, as an author, if you are building these in JavaScript, for example, you have to pay attention to how they're styled. There are ways to pierce the Shadow DOM. Typically, you would use CSS variables, but there's also a Shadow Root CSS selector. The point I'm making here is that you have to be aware of this, and then you have to essentially work around it.

So Lustre does some work to work around this for you so that you never need to understand or even know. The way it manages to do this is essentially by adopting stylesheets from the parent document. When it can't do that, it clones them into the component. For example, if you have an external link tag for a stylesheet hosted somewhere else, that gets cloned into the component so that the component has access to all of those styles as well.

Which works great if you're building a Lustre app and you want components. It works less well if you're creating a component without an app and you want it to be styled in a self-contained way. So I would put this back to you and anyone else who's interested in this scenario: please try it, and then please tell me what you need to make it work nicely because I don’t think anyone is doing this right now. The functionality is there, and the dev tools have what you need to make it work well. However, I don’t believe anyone is actually implementing it at the moment. The functionality is available, and the dev tools have thoughtfully included the means to bundle them, but again, I don’t think anyone is currently doing it.

Right. We need to conduct testing. Yes. Okay. That brings us to what must be the last and inevitable question.

If someone wants to get started with Lustre, where should we begin? The documentation is something I have put a lot of effort into. It's still a work in progress, but I have invested significant time in creating it. If you go to lustre.build, it will direct you to the package API on Hex.

And there is a guide, a long-form written guide that will take you from zero to a complete app that handles side effects and HTTP requests. There is also a quick start guide, as well as a series of examples that progressively increase in complexity. If you prefer not to do a lot of reading and just want to look at code, there are plenty of examples available as well.

And there are cheat sheets if you're coming from React, LiveView, or Elm. Oh, wow. Okay, you have done your homework. Yeah, I really want people to use this thing, you know. Cool.

So, one last question then: what are you using it for? Are you using it in Angular or in anything in production, or are you just the library author sending it out into the world to be used? Right now, I actually don't have the time to use it myself.

Outside of work, I'm spending all of my time developing the library, but I do know a handful of people who have successfully put this into production now. I know some individuals who are building their startups around it. Yes, I'm not using it that much right now, which is why I need more people, more experienced reports, and more users. Right.

Well, then hopefully this little conversation will inspire a few people to go and give it a try and then track you down with feedback. Yeah, I hope so. At this point, we should probably let you get back to writing the library and let them get started. Hayley, thank you very much for your time. Thank you very much for having me.

Absolute pleasure. Cheers. Thank you, Hayley. If you want to give Lustre a try, there are links in the show notes. And if you're googling for it, it's L-U-S-T-R-E.

If you want to give Gleam a try, Lustre is a very nice gateway into Gleam's world and, thereby, into the world of Erlang. So, give that a try. If you just want a nice place to hang out, the Gleam Discord has some great people, including Hayley and others. As this episode goes live, I will be at the GOTO Conference in Copenhagen.

So just on the off chance that you happen to be there, come and say hi. And if not, I have every intention of coming back with some new guests, so you won't miss out. So stay tuned for that. Make sure you're subscribed; we'll be back soon. I've been your host, Chris Jenkins.

This has been Developer Voices with Hayley Thompson. Thank you for listening.