Side Projects: The Unsung Heroes of Software Development - Rasmus Kromann-Larsen - CPH DevFest 2024

I think we are also on time, so let's get started. We are now at the end of a very long day, at least the conference part, so hopefully I can keep you awake during this last talk. This will be more of a collection of stories, so even if you zone out for a bit, it should still be okay, and you can pick up where you left off when you wake up again.

My name is Rasmus. I live here in Copenhagen, or just north of Copenhagen, with my family. I've been writing code for about 30 years now. Eighteen of those years have been professional, or at least done full-time. I'm currently at Templify.

Previously, I've been at Microsoft and Xamarin, and I'm basically always building something. That's also the reason I'm giving this talk; I'm fascinated with the whole idea of software development. You don't really need much—just sit down with your computer and your imagination, and you can create something from nothing.

And, you know, then I see colleagues and friends, and they might not be building software outside of work. And that's also perfectly okay. But at least I feel like I've gotten a lot out of it. So I wanted to share my story, at least about side projects. This is the story about the side project that I have put the most time into.

So it's a website that I started eight years ago. And then, of course, hopefully you'll get some of my advice, my experience, and what I've learned over the years. Now, side projects—what is a side project? From the name, it's pretty clear that it's something that's not your main occupation, right? It's something that you're doing outside of work, studying, or whatever else you’re engaged in.

And there are usually a lot of different reasons why people undertake side projects, right? I, at least, have been burned by this a few times in the past, and my reflection has been that it's always good to briefly think about what I am trying to achieve. Why am I writing this code? Understanding that will help you with whatever you are trying to accomplish. The problem is that when you mix these motivations together, you're likely to have a negative experience. Everything is going to take a lot longer when those reasons are not aligned.

And so I usually like to put the side projects into four buckets. These could be four main reasons, at least for me, that I would try to create something. The first two are more like internal reasons for me. One reason could be that I've been looking at Rust on Hacker News for the past three months; it has a lot of hype.

I want to try it out. I want to challenge the ownership model and everything, so I want to learn something. That's the reason I'm sitting down to write this code. It could also be that I genuinely want to create something.

You have something in your life that makes you think, "I want to automate this." You want to create something that can help someone. Whatever it might be, you're trying to create something and move from an idea to something that actually works. It could be a showcase or a demo for a presentation.

It could be you trying to show off your skills for job hunting or other purposes. What I find with showcasing side projects is that they're usually very focused. There's often less code involved, but potentially higher quality or at least demonstrating a different aspect of what you're trying to present. Of course, it could also be that you are starting a side project because you want to earn more money for profit. While there may be elements in this talk that can assist with that, there's also a wealth of information available on starting businesses and startups.

And I think things here will mainly focus on the creation goal. Some of it will be useful regardless, but I would recommend also looking into the overall information on startups. What I find is that I often mix up the internal reasons for wanting to create something.

I want to sit down. The reason this frustrates me is that if I want to learn something while creating something at the same time, I often encounter friction with the new technology I’m bringing in. This means it takes me much longer to create something. My advice for creating something is to use a technology stack you’re already familiar with. One thing that has worked really well for me is utilizing the stack I use at work, as it has its own benefits and is a stack where I am already productive. There might still be some new technology required for your creative idea, but I recommend trying to reduce the number of new tools you introduce at the beginning. The same principle applies when learning: avoid overwhelming yourself with too many new technologies all at once. Instead, pick one new technology—like Rust or WebSockets—and combine it with some technologies you already know. This doesn't need to be a lengthy process; a brief reflection before you start can help clarify your goals. Ask yourself: What am I trying to accomplish? Do I want to create something or learn something? Typically, you’ll have some constraints, often time-related. So, what might these side projects look like?

I have, I don't know, maybe created a hundred side projects over the years, right? Some have taken just an evening, while others consist of code that I've been using for a long time. One example could be that I created a browser and a Windows launcher for myself many years ago—about 10 years now—and I still use it every day. I think I'm the only user now, but it's available on GitHub if you want to try it out. There are lots of Windows launchers these days, but I'm not sure. I had some users at work once, but yeah.

Another idea could be to try and create an old Google AI competition. This side project is called Space Battles. It started as a hackathon weekend with Jasper, who is in the audience, and he is actually the one running it now. Additionally, this is the only reference to AI in this talk, but there used to be these competitions called AI competitions that focused on writing bots to fight each other in a space world. Another application you might consider is creating a tool for managing your life—notes, projects, tasks, or whatever else you need—if you haven't been able to find the right tool for yourself.

And this, if you look at the list over here, you can also see that this is one where I've basically been burned a lot on the technology front. I'm on the nth rewrite of this app, but I'm actually pretty happy with it now. The main problem with an app such as this one is that you want something that's both cross-platform, but you also want your data to be local. That turns out to be pretty complicated, especially if you're limited on time and effort. So, these are just three examples of side projects.

There could be many more interesting ones as well. This brings me to the main side project that I'm going to discuss today, which is called Pue.Ninja. It happened somewhat by accident, but that is actually the full domain name of the site.

It is a website that tracks statistics in a video game called Path of Exile. It's a bit random, but that's what I've been building for the last eight years. Just for reference, who actually knows about Path of Exile? Okay, there are a few people at least; that's good. I will give a short introduction to what Path of Exile is.

Path of Exile is an ARPG, or action role-playing game. If you are familiar with Diablo or Last Epoch, then Path of Exile is quite similar. It's a relatively old game, with its first version released in 2013 by a company called Grinding Gear Games from New Zealand. I believe what Path of Exile does really well is that it leans into complexity and offers players a vast array of options.

And I think what Path of Exile does really well is that it leans into complexity and gives you a lot of options. Some of the main aspects it focuses on are deep character building. The premise of the game is basically that you need to build a character who becomes increasingly better at killing monsters. That's the point. It provides you with a lot of tools to achieve that.

One infamous example may be difficult to see on the monitor, but this is the passive skill tree from Path of Exile. As you can see, it's a massive graph. Essentially, one of the things you do when creating a character is chart a path through this tree and acquire various bonuses that you desire. There's a quote from someone on Reddit, which is a sentiment expressed by many: "I have 11,000 hours in this game, and I'm still learning something new every day."

That highlights the complexity of this game. When something is complex, it usually means that software developers like me, who may not be very good at playing the game, feel like, "Well, maybe I can at least play with some data and have fun." To give you a brief idea, here’s a quick 20 seconds of gameplay, in case you haven’t seen the game before. Let's see if this works.

I took one that’s not too crazy in terms of movement and other mechanics, but this is what the game looks like, right? You have some health, some mana, and then you basically run around and kill monsters. And it speeds up, essentially. All right. So, poe.ninja.

Eight years ago, I was playing a game with my friend. Path of Exile has a player-driven economy, which means it can be quite challenging to determine the value of items. My friend was frustrated; he wanted to buy a particular item but had all these things and didn't know their worth. He asked, "How am I going to get from here to actually buying the things I want?" Then he said, "I wish someone would just create an overview that would show me what things are worth."

And the game had recently released an API, and I thought, you know, that sounds fun. I'll sit down for a weekend and try to build something. And I did. Then I posted it on Reddit, and people really liked it. I thought, okay, that was good.

And it was useful to me as well. So, I thought, okay, let's just keep building this thing. I continued to expand it, and it was helpful to me as well.

And I kind of just kept on expanding it. Here we are, eight years later, and I'm still building it. There have been on-and-off times, of course, but it's been my entertainment for the last eight years when I haven't been working. To give you an idea of the scope of the project, it's written in C#, uses .NET Core, and has a React frontend in TypeScript. It has processed nearly 500 terabytes of data from the API at this point.

And it's served a lot more, for sure. It's about 50,000 lines of code. Usually, the split is pretty even between C# and TypeScript, but I think I deleted a lot of TypeScript recently, so it's a bit heavy on the C# side now. The graph you see at the bottom here represents the daily page views.

And you can see that it goes into the range of maybe between 2 and 4 million page views per day. That means it is somewhere in the range of about five to ten thousand top websites in the world. For daily users, it's around 200,000, currently reflecting the ongoing league. Additionally, you can see that "Path of Exile" is a game that features this league system, where every three or four months, they release a new version. People tend to come back during these times, and then interest may decrease before they return again. This pattern has continued, even through some unpopular leagues in 2022, as you can see. However, they managed to recover from that in 2023. Now, we’re getting into the story part about what I actually learned from doing this project over the last eight years. The first lesson I'm going to discuss is one called "Embrace Constraints."

And when you're starting a side project, you're going to be constrained in some way, right? You are going to be constrained financially; you won't have the money to go out and buy a huge server. There's no revenue, no resources, and you can't purchase an expensive database license.

You will be constrained on time for sure. This is a side project; it's not your main occupation. A lot of your time and energy will go into your main job, and then you'll be working on this on the side.

But you'll also be constrained by the number of people, right? Most of the time, especially when you start up, you're just one person or maybe a few. This also means that there's a limit to how much code you can write. You can't write as much code as a team, but on the other hand, you also don't have to coordinate with a team, so you can make decisions a lot faster. There are pros and cons, but you are definitely constrained in many ways. One of the constraints that I picked up recently and that became a design goal for me in this project was that, as it happened, I was deploying this site to an Azure App Service, and due to financial reasons, it had one core.

And so I thought, you know, Path of Exile is a big game, and there's a large audience. Let's build this site so it can run on a single CPU core. That was kind of my design goal. What does it mean to run on a single core? Well, it means that if you have a request that takes a thousand milliseconds, then you're going to be serving one request per second, right? If you exceed that one request per second, bad things are going to happen.

Your service is going to struggle significantly. If you can reduce the response time to 10 milliseconds per request, you could potentially handle 100 requests per second. This means you can actually serve a large volume on a single core if you design it correctly. What you see at the bottom is from last week, taken from my monitoring software. This shows the response times from my backend, which were measured by Nginx on the same server. You can observe that the 50th and 75th percentiles are also reflected in the data measured by Nginx on that server.

But you can see that, in fact, the 50th and 75th percentiles are displayed, and the 99th percentile is 0.011. This is because Nginx doesn't report anything less than 1 millisecond. Currently, the 75th percentile of all requests takes less than 1 millisecond, which makes me very happy because it means I can still run it on very few resources. Initially, I started out with a single core, but I later expanded this to a single server. Most of the complexity arises when you move to multiple servers and distributed systems.

I do that at work. But if you're a single person, keeping everything on one server really helps. So I'm fighting pretty hard to maintain that. This went fine for about the first two years, but then I realized, oh, I've covered most of the economy in this game.

Maybe I should try something new. I realized that I've covered most of the economy in this game. It's always challenging with these characters. I would love to explore different options, but I always have trouble figuring out exactly how to build.

Maybe if I could build something where I would be able to explore data on how other people are building their characters. If I could do that, it would be helpful to add. The economy part was pretty easy, right? I was collecting data, aggregating it, and serving it. It's quite simple to do with a single call.

But then, you know, this build site, this is what it looked like back then when I released it. It looks very different now, but, oh, at least somewhat different. But how do you build faceted search, sorting, and search on a single core, right? That's going to be hard with a big audience. And so I remembered a library I had seen that was called Crossfilter.

It was created by Mike Bostock, who also developed D3.js. The approach they took was essentially to serve all the data to the front end and then perform all the filtering on the front end. That sounded very appealing; given my constraints, it seemed like a great solution. I faced two challenges: there were 15,000 characters in the ladder of the game that I needed to download, and I had to ensure that I could reduce this dataset to a manageable size for reasonable frontend serving.

And then, of course, I needed to create some logic for actually filtering this. It turned out that I could reduce it to maybe 5 megabytes of JSON. If I gzip that, it was around 500 kilobytes. That was acceptable. I could serve that.

Crossfilter was utilizing bit sets, and if you haven't explored bit sets much, they're quite remarkable, as computers are really fast with them. The filtering provided by these bit sets in JavaScript was also highly efficient. Essentially, there was a brief loading period followed by nearly instantaneous filtering, even on phones. This was fantastic, and everything worked out well. Let me show you exactly what it looks like today, just to give you an idea if you haven't seen the site yet.

So this is basically a list of all the cameras. You can see that the UI is a bit nicer now. Additionally, you can see here, for instance, the passive tree. This is a heat map of what the 97,000 characters are building. If we say, "Okay, I want to look at all the Slayers," you can see that the heat map changes accordingly.

And we can dive in. So everything is basically set for us to explore this dataset, right? We can drill into the data, and it should be nice and fast and fun to explore. You can also press the heat map and specify that you want to see everyone who is actually using this node.

And then it was actually filled with the heat map. I don't think a lot of people know that, but now at least it's shared with the world. That was great; it was a fun experience.

And something, again, I wouldn't have done at work, right? I wouldn't have pushed all the data and done the filtering on the front end. But because I had this constraint and leaned into it, it became more of a creative challenge to find a solution to this problem. Unfortunately, the problems didn't stop there.

It turned out that 15,000 characters were not enough because people were reaching the maximum level faster, and my site kind of got stale after that. So, Grand Gear Games released another API. I increased my goal to be able to serve 100,000 characters. However, this also meant that the gzipped JSON suddenly became about 5 megabytes. We were entering a space where it might not be as enjoyable to serve anymore.

I'm also entering a space where phones are becoming less effective due to the excess of data. It’s getting overwhelming. So, I started exploring ways to improve the situation while still maintaining my constraints. I researched various column databases, as they are typically suitable for analytical queries, and I tried out several of them.

Um, they were all, not fast enough for what I was trying to do on a single server. I had the idea that I could try to build a database myself, but whenever you think you should build a database yourself, you should stop. You should not do it unless you're in a database startup; then you can pursue it. Otherwise, don't go there. So I kind of put it off, thinking that it was crazy. However, a few months later, I decided to create a prototype to see if it would work, and it did work. So I continued building it.

And so I did build a database, but it was a very specialized database designed specifically for this use case. It is also optimized to be highly cacheable by Cloudflare, which is in front of my site. Overall, I would say it consists of maybe 1,000 to 2,000 lines of C# code. So it's not a huge project, but I finished it and successfully launched it. That was the one you saw earlier, and it remains very fast. It typically responds to queries in about one millisecond, but in some cases, such as when complex sorting is involved, it may take up to five milliseconds.

So that was quite fun. And again, I'll repeat: bit sets are magic. It's really the main driving force behind this database. Just to give you an idea, there are around three thousand dimensions. You can see all the ones on the left; we need to count each of them across a hundred thousand characters. That's more than 300 million operations—right, 300 million comparisons. But because it's using bit sets, that's just very fast and can be done in one millisecond, which is amazing. However, I almost gave up on this back in 2021 because I was on a virtual server with 16 cores at that point.

And I had been watching my CPU usage go up and up as more users began to use the site. It had been very gradual, but this was what I was observing in 2021, before I engaged in building a database. Looking at this graph, I realized that maybe now was the time to optimize my code.

Maybe now is the time to start designing a distributed system or at least figuring out how to offload some of this. I tried profiling, and it’s pretty hard to know exactly what your production server is doing. I couldn't really find the root cause. However, one day while profiling something else—the application startup time—I noticed that there was a bit of code collecting some data and serializing JSON into the HTML of the page.

So basically, since I'm caching a lot in front of my site, I also include some IDs in the HTML to break the caching, allowing users to see the new data. What clicked for me in that moment was that, since this is embedded into the HTML, it means that this code will run on all the unique URLs that people visit. That's actually quite a significant amount of traffic coming to my site. At that point, I didn't really think that was the issue, but I thought, you know, what the heck.

Let me try and see. I'll just cache it. It doesn't change very frequently, so I mean, I could just cache it for one minute, and that would kind of remove that load. And I did.

And then I deployed it. My server load fell to four cores. This is also a note on whatever you think your production server is doing. I had no idea that my server was spending ten calls serializing JSON for the HTML. That was crazy to me.

So, I mean, whenever you're guessing anything, this is not actually about side projects; this is about profiling in general. Whenever you have an idea of what a performance problem is, you need to measure and find the issue because it's usually not what you think. That was about leaning into constraints and using them not as a bad thing but actually as a creative challenge. It's also because I enjoy solving these problems, and so that's what works well for me.

In a side project, incremental learning and innovation are important. I mentioned earlier that you should avoid introducing too much technology when you're trying to create something. However, it's perfectly fine to do so over time. Once you've worked on a side project for a while and have a viable foundation, you can start bringing in new technologies and experimenting with them. Since you already have a working base, any migrations that may take some time will be easier to manage.

And now is the time when you can start bringing in new technologies and experimenting with them. These migrations, which may take a long time at work—mostly because you want to avoid any downtime—are actually quite fast when you’re just one person. You can simply make the changes without needing to train others or hold discussions.

You can basically just do it. These migrations that we're accustomed to are often very expensive, but they may not be as costly in a side project. It makes a lot of sense to gradually lean into this approach over time. Additionally, when you choose the same tech stack as your work, you suddenly become responsible for everything in your side project. You take on the entire stack, including the server, the infrastructure, the pipelines, the backend code, and the frontend code—everything.

And it means that if you use the same stack as work, you're going to encounter many other areas that perhaps other teams or team members are responsible for at work. This certainly means that you will know a lot that others might not, which is very beneficial. At least, this is one of the significant advantages I have found in my career: usually when people discuss a particular topic, I have played around with it in some random side project, and I tend to know more about it than just what is in the documentation. The story I want to share here is actually about the evolution of the infrastructure, illustrating how it developed over time and how I contributed to that growth. Of course, it’s natural for infrastructure to grow over time, but you can also make relatively big shifts, especially when you are just one person.

Um, especially if you also take some knowledge that you already have and apply it. So this was the initial version that I mentioned before, right? I had an Azure App Service, Azure SQL, and the Path of Exile API. I was deploying code directly from my machine because, you know, Azure App Service makes that pretty easy. So that was kind of the lowest friction I could achieve to just get something out.

And so, this is what it looks like today. Don't be scared by the whole Kubernetes thing; I'll get something out.

I'll get to that. Basically, what happened is that in 2017, I moved off of Azure. I could see that remaining on Azure would not be realistic in terms of egress, traffic, and everything else. Additionally, I needed to scale up the server a bit more. This was also around the time that ASP.NET Core was becoming really solid on Linux.

So I moved to a Linux server. I switched to PostgreSQL, which I can definitely recommend. I put Nginx in front of it and added Cloudflare as well. Then, in 2018, I added Pipeline. I also added Humio for monitoring because we were using it at work, and it was really nice to use.

I also, by this time, had moved servers a few times, and we had used Ansible at a previous job, so I introduced some Ansible scripts for provisioning my machines. Whenever I moved to a new server or needed to scale up further, I would apply my Ansible scripts. Then, in 2019, I added another community tool from the Path of Exile community called Path of Building to handle a lot of the calculations for me, and I put this in a Docker container. In 2022, because I was looking for better ways to implement zero downtime deployments, I added Kubernetes. However, this is the most basic Kubernetes installation you've ever seen.

It's a single-node Kubernetes cluster that mounts resources directly from disk. It's not a complicated version of Kubernetes; it's basically just the bare minimum. I already had a good understanding of Kubernetes from work, but I definitely learned more when setting this up. The main purpose of having Kubernetes here is to manage the process of building Docker instances and to facilitate zero-downtime deployments for my main Docker image. It's the simplest Kubernetes setup you can create. This setup is just to give you an idea of how it can evolve over time, taking what you already know from work and gradually building it out. None of these migrations have been very difficult; I think the worst one was probably the Postgres migration because changing databases always involves some challenges.

And lately, I've been exploring Cloudflare R2, which is an alternative to S3 and blob storage, basically for additional file storage. Additionally, I've integrated a few more APIs. The next thing to consider is that when you have a side project, it’s important to remember that it is, in fact, a side project. I think, especially if you have a long-running side project like this one, you may sometimes get lost in the details. You might start thinking about how you can grow the site and make improvements, even if those changes aren’t what you truly want to pursue. Instead, they may just seem like the logical next step for the project.

It's the right thing to do for the product, and I know that I'm at least actively working on it. I remind myself of this: this is my side project. This is something that's supposed to give me energy, something I'm supposed to be passionate about. Why else would I be doing it? So, I have to actively step back from thinking about how to grow the site and consider, "Okay, what do I think is fun?"

What do I want to do next? Is it to dive into machine learning, or is it something else? There are always a ton of options and a lot of things you can do. However, I think it's important to remember that if you are actually looking to create something, you should optimize for fun. Otherwise, you might just stop doing it.

And so one of the things that I've decided is that it's not open source. There have been a number of people over the years who have asked whether or not they could contribute. Of course, I'm happy that they would, and it would be great to have some more development power on the site. However, I realized early on that this is my playground. This is where I get my energy.

And especially as my career has progressed and I've moved further away from code, having this project to return to, where I can play around with code and actually engage with the technical details, really helps me keep my programming skills up to date. Additionally, when I need to make technology choices at my job, my hands-on experience is beneficial. I don’t like reading about things; I prefer using them. That’s how I learn.

So I need this project to: I want to write code. I don't want to manage pull requests. I don't want to manage issues. I'll take the occasional bug report. I respond to people who write to me.

But I don't want all of the unnecessary details. I just want to write some code. That's basically the goal of the project. The other story here is a bit of a sad one, at least for me. In 2020, after I had released this build site, I suddenly received an email from someone, and you can see that everything looks a bit funky up here. It's because it's in Korean.

But I was contacted by a company called Kakao Games. They are a Korean company that runs the Korean realm in Path of Exile. They had a problem: a lot of users really wanted to use my site, but there was a significant language barrier for them.

The English was not good for them, so they reached out to say, "Can we figure something out? We're happy to help with translations and any other things that you need. But can you provide the site in Korean?" I thought, "Sure."

Let's give it a try. I built a small translation system for the front end. I spent some time extracting language data from the game files. It took a few weeks, but it wasn't a huge task.

And actually, I reached out to Grinding Year Games. They were nice enough to use their translators to translate all the UI strings I had. So, I launched this with seven languages, which was super nice. Then, everything kind of slowed down a little bit.

Because what I hadn't thought about is that when I added this translation system, it actually meant that I suddenly couldn't make changes anymore. I didn't have a team supporting me. I didn't have someone who could go out and create new translations. So if I wanted to change the UI and break something up, I would now need to find these translations somewhere, and I would need to create new translation strings and everything. That adds a lot of friction when you only have one person's time.

And it was also work that I didn't think was fun. So, I kind of slowed down my pace. I didn't really know why at the time, but I stopped working on the site as much. Then, in 2023, I realized what it was.

And I took the somewhat painful decision to rip out the entire translation system and delete it because it meant that I wasn't building the site. If I have to optimize for fun, it also needs to be something that keeps me building, and this was draining all the energy out of the work for me. Yes, I was sad to remove it because I know it helped some people, but it was the only path forward for me. Of course, if I had had a team, if it had been a company, or if something else had been the case, then it might have been a different story, but it's just me.

So that was the decision, and it really helped a lot. That was the good part. I really picked up the development, and you can also see that this is in 2023.

This is just before I did the whole "build your own database" idea thing. The last section here is about unexpected events. If you build something like this and it takes off, you're also going to have some unusual occurrences. Of course, this spans over eight years, right? I think the Kakao game situation was also a bit unexpected.

The first experience I want to share happened back in 2018 after I launched the build site. A major Chinese company called Tencent acquired a significant stake in Path of Exile and Grinding Gear Games. I received some emails from them, but there was a considerable language barrier. They were discussing a project that I didn't fully comprehend. I attempted to ask for clarification, and while they did respond, I still didn't understand what they were trying to convey.

I wished them luck with the project, and the conversation kind of fizzled out. Then a few months went by, and I didn't think any more of it. One morning, I woke up to find my Reddit inbox full of messages. I also had an email from Tencent. They were discussing the launch of their project.

I'm a bit confused. I don't know exactly what they're talking about, but I click the link and see this. If you remember what you saw before, this looks familiar. Do you recall that I mentioned I built the site in a way that only requires data to be supplied from the backend?

Well, that's what they did. They copied my front-end code and implemented a new backend, and then they launched that in the Chinese market. I was a bit surprised. I wrote back to them, and we discussed it back and forth. I also talked to Grinding Gear Games, and they were helpful.

And it turned out it was a misunderstanding. They thought that I had okayed this. We talked back and forth, discussing how we could proceed. I knew for a fact that I wouldn’t venture into the Chinese realm.

The Chinese API is in Chinese, and we discussed how we could address this issue. I knew for a fact that I wouldn't venture into the Chinese realm. Let's refrain from revisiting the language topic, even though this has occurred before.

But I knew that I didn't want to enter the Chinese realm. So, in the end, I said I was actually okay with them running this on the Chinese realm because I knew that it would help people. I said, "You know, just don't bring it to the international realm." Also, could you add a link back to my site? They added this thing at the top.

You can see the green one there. It's a small button that says something I can't read, "Ninja," and something else I can't read, "Rasmus." I believe it says, "Special thanks to the Ninja author, Rasmus." To this day, this site is still running, by the way, and this button up here is actually one of my largest direct traffic sources. This wasn't unexpected, so I can cross off the whole running on a large Chinese company website off my list. The second one was "Torchlight Infinite," which launched in 2022. It's also an ARPG and is heavily inspired by "Path of Exile." They reached out to me, and probably to other community builders in "Path of Exile," and asked, "Do you want to build a site for this new game that we're launching?"

And I thought, you know, it looks kind of like Path of Exile; there were a lot of systems that were the same. I thought, I have all this tech for my build site; I could probably build one, and that would probably be fun. So, I said, "Yeah, sure, I can build a build site."

I would give that a shot. And they were like, "No, no, they're not really looking for a build site." They were thinking more along the lines of a wiki or something else. Okay, that's fine. But I decided to try out the game, and I played it for a while. Then I found out why they didn't want the build site.

And that's because, inside the game, there was this feature. If you look at it, you can see that there is the main skill five link plus. If you visit my site, you will find my main skill five link plus. So, I mean, it's not a complete copy, but it's very similar. It turns out that the reason they didn't want the build site is that they had already incorporated it into the game, essentially.

And the last of these stories here is ExileCon. Path of Exile, or Grinding Gear Games, held a conference last year called ExileCon. Because of this event, I decided to buy a ticket and travel to New Zealand. Again, I cannot recommend traveling to New Zealand for just six days, but the trip was worthwhile, and it was truly amazing.

I got to talk to a lot of players who are much more hardcore at Path of Exile than I am, for sure. More importantly, I had the opportunity to connect with many of the other community tool creators at the event. I also had a chance to talk to several people from Grinding Gear Games, especially the web team, which is the group I collaborate with most at Grinding Gear Games and the ones responsible for all of the API work. This was a significant experience that I definitely wouldn't have had if it hadn't been for this project. I think that kind of concludes the war story part of this.

So, to recap, I would encourage you to go and build something. It can be a small thing. As I mentioned, I think I've done 100 side projects, and this is the biggest one. It could also just be a small task that you automate today that will help you tomorrow.

Or maybe you'll discard it, but you'll probably learn something. When you do, pick an appropriate approach for what you're trying to do. Just reflect and think, "What am I trying to accomplish here? Am I creating or learning?" Be especially mindful of your constraints. Use them as a force instead of seeing them as a limitation.

Find creative ways around these building blocks. Again, you'll learn a lot as you go. You'll also engage in activities that you would never do at work. Some of these may seem like terrible ideas, but they might actually work. Incorporate innovation and new technology as you progress.

Try to keep it concise at the beginning, and most importantly, have fun. I believe that brings us to the conclusion. Please remember to evaluate the discussion as well.

If you have any questions, I'm happy to answer them. If you would like a ninja sticker, you can come up here to get one. Do you have to put any money into this? Thank you.

I actually also pay for Cloudflare, one of the plans. However, I do have ads on the site, so it generates some revenue. Because of this, I no longer pay for hosting. I have had people wanting to sell the website or inquire about it in other ways. I have had both people wanting to sell R&T.

There are ads on my site, and there have been some people interested in buying it. Yes, I've had some conversations, but I decided to keep it. Any other questions? What do you want to know? Actually, I was thinking about one of the things that exists in many other games. This is very power user-oriented, right? You really need to know what you're doing. I want to look at juggernauts, and I want to see a specific skill. You still need to know a lot about this.

It's hard to see exactly what's popular. So one thing I've considered is to actually create a more condensed overview of the current meta and the popular builds. Another thing I recently started is saving some packet files whenever this overview is created. I'm going to try and look at these files in DocDB to see if I can generate useful information over time on which ascendancies and skills have been popular. Additionally, I would love to explore the possibility of applying machine learning to some of this data, although that area is a bit challenging.

But there are a lot of different options you can consider. If you have any ideas, feel free to write to me, and I'll take a look at them. I have a very long list of things I want to do, but usually, some people also come up with bad ideas. Yes, I'm playing right now; I still play, and I usually participate every league. In the years when it slowed down, I didn't play as much, but I still play for each league, mostly to keep the site up to date. However, with the current league, I've been having some fun as well. I don't know if I'm doing a lot of grinding in their games to expose their data for the community to use because, for example, I'm also interested in other people's experiences with Trackmania. The developers don't provide any official APIs, and they don’t really engage with the community. They kind of data mine the internal APIs, but they don't guarantee that small changes won't occur or that everything is fairly transparent.

When it comes to aggregating layer numbers or any other data, I think they are fairly protective, but they are also collaborating with community sites for sure. I believe the main limitation is probably people. Grinding Gear Games is primarily a game design studio; they build games.

They don't build APIs; that's not their main focus. However, they do have a web team that creates APIs, although it's not the primary activity at Grand Thee Games.

So, I think that's also why sometimes things might be lagging behind. I believe it's mainly due to a lack of manpower, especially since they are currently working on Path of Exile 2, which is a significant undertaking. I don't think they are unaware of privacy issues; there are definitely limitations when using their APIs regarding that. However, they have made a lot of data available, at least. I think what they also emphasize is that you shouldn't monetize access to their API.

"Don't, please don't do that. Right. But other than that, I think they are relatively open. Yes. Uh, how much time, like a week?"

I'm sure it might have some flow. It varies a lot. I think just like leagues in Path of Exile, my effort also goes up and down. Whenever I have a need to build something, it fluctuates. I would say that most weeks, I probably spend around one to five hours on it. However, if I'm building a database or working on something else, it might take more time. I find those tasks enjoyable.

I usually have too many things active that I want to do at once in this project. There’s a lot of ongoing work that remains unreleased. Yes, that has happened, but it wasn’t my fault. There was a new version that caused some issues, and I accidentally deleted the database once. However, since none of my APIs actually called the database, the site was not down. There was one instance where the site was down due to an ASP.NET upgrade at some point. They had an API that was supposed to be asynchronous, but in that version— I think it was 2.2; that was a long time ago—there was a bug that caused it to suddenly become synchronous instead of asynchronous.

And because I am handling a lot of traffic on one server, that single issue caused my CPU to max out and fail. It was actually quite difficult to deploy a new version because the server was completely overwhelmed. But other than that, I don't think so. I don't think so.

No? Any more questions? Otherwise, I'm here for the entire conference, so just come and find me or reach out to me on Discord. One more question: How helpful was that initial constraint of solving just one core?

I don't know. I think I just made the decision that I would serve everything from either memory or a static file. That was kind of the initial thought I had. I just knew that I had very limited resources, and I was aware that the community was large.

So I was thinking that I needed to do something about it. If you consider it, having economic data is really just about what you're doing: collecting all this data and then running some aggregations over it. Producing an overview in a JSON file, instead of creating that overview for every request, makes a lot of sense. It naturally emerged from the problem as well.

But I did think about it, you know, this is going to be a problem if anyone uses it. The other thing I have is that if you are working with data, it has to be fast, right? If you're exploring data, you need to be able to click and get a response quickly. Otherwise, it's not going to be enjoyable. If you're waiting three seconds every time you click one of these filters, you're not going to be exploring or looking at the data for very long. All right, it looks like the questions have stopped. Thank you. Vielen Dank.