0:02I don't think it would be that surprising for anybody in the room if I said that AI has radically changed what it means and how much work it takes to be the maintainer of an open source project today, especially if it's a popular open source project. On the one hand, it's made it much easier to contribute to your favorite project, which is great for open source in general. But on the other hand, this in volume of contributions that we're getting as maintainers makes it much more difficult to keep up and generally is is it causes a lot of problems if you don't have the right tooling in place so what I'd like to do today is talk a little bit about how we're addressing some of these challenges in our own project which is called effect as well as some of the tooling that we've put in place with a specific focus on the foundational architecture for how we're building these tools.

0:59So a little bit about me, my name is Max. I'm one of the founding engineers at Effectful Technologies, which is the company that's kind of driving the open source development of Effect and all of its ecosystem packages. My background is actually in DevOps and systems administration, which I think is kind of how I got my interest in building robust, reliable software, just kind of a general interest in making sure things are as reliable as possible. own brand of anxiety is thinking about all the possible ways things that fail. And I think that's what drew me to Effect about six or seven years ago when I first became a maintainer, because this is what Effect is all about. And I'm actually curious how many folks in the audience are familiar with Effect? All right. All my Effect enjoyers out there, welcome.

1:47But for those of you who are not familiar with the project, Effect is a toolkit for- building production-grade applications in TypeScript. But all that kind of sounds very jargony and ephemeral. What do I mean by production-grade applications? Well, I consider any application that needs to be production -grade an application that has to handle all of the possible cases that can happen when your app has actual users, right? And those users can be humans or agents. You want to make sure you're handling all of the possible scenarios, whether it's operations that can fail, whether it's operations that can fail, whether it's It's resources that need to be acquired and then cleaned up later to prevent memory leaks, whether it's coordinating concurrent tasks or even introspecting your application at runtime. And effect is a set of tools that's designed to give you all of this out of the box.

2:37And more than that, effect, all of the pieces of effect are designed to compose very well with one another. So that instead of installing 30 NPM packages to build all of the production grade... features into your application, you just have a single install that you need to deal with, and you get all of that for free with Effect. But I mentioned that Effect is also an open source project. So I have another query for you all. How many of you all are open source maintainers yourself? Cool. All right. So then in the last six months, how many of you have seen a pull request that looks like this? I don't know. Yeah, I mean, there's a pretty significant dichotomy here between the... of time it took to generate this PR, this contribution, right, the amount of human time it took, this could have just been a single prompt, right? And there's nothing inherently wrong with that. There really isn't.

3:25Every PR or contribution, there's a signal behind it, whether it's slop or not. The problem here is that on the one hand, the amount of human time took to generate this is vastly different from the amount of maintainer time it's going to take to properly review this and decide whether it makes sense to. include in the library at the end of the day. And so our initial approach when AI started kind of taking off and we were being bombarded with you know a thousand PRs and it was it was really getting untenable was to try to put processes in place to kind of like filter out the sloppy PRs or contributions or kind of like block things and we see projects doing that now right with trusted contributors or some projects have even blocked pull requests.

4:12altogether. But over time, and especially as models started getting better, we decided to take a slightly different approach and really lean into both accepting these contributions as well as using AI very heavily to kind of deal with the influx of contributions we were getting. And this is an example of a case we had when we first started trying to figure out what tooling worked for us. where uh we had 200 pull requests opened over the course of i think it was like 50 hours or something like that it took down all our github actions like our whole project was was a mess but at the end of the day it gave us a lot of insight into the types of tools that we we um found useful in order to kind of manage all of this and really like solidified for us that leaning into ai was the right thing because like i said every single contribution we get we view is a signal, something that didn't work for the user, something that we could have done better, a feature that was missing.

5:13And everything is in the service of trying to improve the library.

5:21So we did start off by evaluating existing tools that were out there, right? We, you know, looked at code review tools. We looked at AI auditing tools. We looked at like all of the different things we could possibly leverage to kind of fit into that. workflow that we found was working for us. But one of the things about automation that I'm particularly, I don't know what the word is, it's a sensitive topic for me, is if automation introduces friction, then it's worthless, right? If it introduces friction into your workflows, it's not serving you. So ultimately, we did the thing that every engineer does, and we had to make the decision between using some existing solution or building it ourselves, and I think you can sort of guess what we decided to do. We decided to build it ourselves, of course, because we're a bunch of engineers.

6:10And the application is called the Janitor. And it's really quite simple. It's just designed to put automations in place that help us deal with repetitive maintenance tasks. It's nothing complicated. It's a relatively simple app in what we want it to do. But I think the interesting piece here is the architecture we put in place to serve these automations, these workflows that we have. So... we started evaluating all of the different tasks we wanted to automate, and they all kind of had a similar theme. It was a sequence of steps that needed to happen, sometimes calling out to external systems to get information, sometimes running AI agents ourself in order to do different things. And if for some reason that sequence of steps failed, we didn't want to repeat any of the work we already did.

6:56We wanted to pick up right where we left off on another machine or whatever and keep going. so that we could remain as efficient as possible, right? Because I mentioned that, you know, with the influx of contributions we're seeing, we don't want to repeat work if we don't have to. And this kind of process of, like, picking up where you left off with a sequence of steps that you're doing, it has a name, and it's called a workflow, right? Pretty simple, it's a workflow. And Effect has a system for defining durable workflows built into it. So we wanted to use our own software for defining... many hour workflows. And Effect also has an actor system built into it called Effect Cluster. So we were very interested in using Effect Cluster and Effect Workflow, kind of dogfooding our own stack in order to build this application. But we were also really interested in Cloudflare as the infrastructure provider, because we really like the fact that Cloudflare has very composable primitives on the infrastructure level that you can pick and choose from for the different tasks which again fit our brains very nicely because that's what effect on the software level is all about, composable primitives.

8:01The problem is that our actor system is designed to run on long -running servers. It's not designed for serverless or edge compute. But our community, which tends to be very vocal at times, had been pressing us to kind of figure out a solution to let them run effect clusters on top of cloud flares primitives on top of workers and durable objects so we thought what the heck the editor is a great opportunity for us to try to build that out and see if it works and so that's what we did so instead of focusing on the actual automations we put in place for our own repository because it's nothing exciting right pull request reviews auto labeling like kind of all the things that you would expect of a normal maintenance tasks I'd rather focus on the architecture we put in place the foundation we put in place to build these workflows in a very reliable and durable fashion.

8:58So we'll start with talking about effect cluster in the traditional sense, running on long running servers, and then I want to compare and contrast that with how we designed the system to run on top of Cloudflare and durable objects. In a traditional effect cluster, you've got one or more nodes, and these can be physical machines, these can be virtual machines, they can be containers. whatever but this is the place where your os level process will run your node .js process your bun process whatever and that operating system process the application itself we call a runner and the reason for this is because the runner is the thing that's actually going to run the code that you define for your actors right when you build your actor the runner is the thing that actually coordinates getting the messages to your actor and running them

9:51in effect cluster terminology we don't call them actors we call them entities because we want it to be fancy and the entity when you define an entity it's it's something that you you specify the protocol for it right you specify the messages that you want to accept for your entity and you also specify the behavior that each one of those messages should trigger just like in a standard actor system right

10:45when you wanna enable message persistence. So, excuse me. Oh, I just realized my microphone is right here. In an effect cluster, messages are volatile by default, which means that if you send a message and it fails to be delivered, the message is gone. But you can turn on what's called at least once delivery of your messages just by enabling message persistence. And so then you get durability on the message level. You'll always get the message. end up getting it more than once right and that's kind of at least one's delivery in a nutshell and persistent messages are stored in our shared storage as well

11:24runners entities can message between themselves so on the same runner or between different runners on different nodes you can send messages to different entities and you can also have clients that send messages into your cluster you can even have clients that join your cluster as like a client-only entity And getting back to these shard things that are on the screen, shards are collections of entities that we use internally as a routing optimization. It allows us to do a lot to make sure that routing a message around the cluster is as optimal as possible. But one of the things it gives you as the user is the ability to address your entities without caring where the entity actually lives, right? All you specify is the entity type and the ID and the message.

12:09make it to that entity in the cluster and so by with this sharding framework we have in place you get location transparency by default which should sound similar to folks who have used cloudflare and durable objects before because addressing a durable object is much the same

12:28in an effect cluster when a node goes down we're able to deterministically reassign shards to other nodes so if you're scaling up or scale down, and we can move nodes around in the cluster to try to accommodate that. And

12:47there's one other thing I wanted to mention here, which is that when you have, you'll notice that there's no central coordinator here either, right? So in an effect cluster, you don't need a central coordinator. Runners know deterministically, using a consistent hash ring, where shards should go, which is pretty nice. have a central uh like coordinator process that you need to deal with but what if we didn't have servers right well i mentioned to you we wanted to build a cluster that worked with cloudflare and we had two options we could either build a whole like generic protocol that let us route messages around in cloudflare but also on other serverless platforms or we could lean into cloud flare as kind of like a platform and use its native like routing primitives to do this and ultimately we opted for the second approach So if you define an effect cluster with our Cloudflare integration, instead of every entity living in a shard on a node and a runner, an entity is just a durable object.

13:47And instead of having shared storage, message persistence uses a durable object SQLite database as its mailbox. You can still send messages into the cluster from clients. This is showing like a worker sending a message to a durable object. object, but you can also do it straight to the DO if you want to. And you can send messages between durable objects as well with the cluster. And the nice thing here is that from the user perspective, all the code looks the same for the protocol and the handlers. The only thing that changes is what infrastructure primitive you plug in at the very end. We call them layers in effect. So we have a special layer for Cloudflare. We have a special layer for running on like a server on a long-running server.

14:35So if we compare and contrast the two here, you'll see that with both of these things we get location transparency. We just do it in slightly different ways. In our long-lived runners we use shards to get location transparency and the Cloudflare implementation we lean into durable object routing. And the other thing that we get here is durability on the message level, but the handler for that message is not durable by default. Remember I said for our handlers, we want to be able to pick up right where we left off. So we also have effect workflows that you can use to define stepwise workflows where every step is persisted after it's completed. And the workflow engine will resume where it left off if anything fails.

15:20So if you want to map a workflow into the effect cluster, when you define an effect, workflow and you run your workflows on top of effect cluster the workflow itself every execution becomes a new entity in the cluster and when you define workflows you define individual activities for that workflow which are kind of like you can think of them like save points they're the idempotent steps of your workflow that you're interested in persisting over time so we like look at this from a practical perspective as we run through our like auto labeling workflow. We check if all our labels are still current. We persist the result. We record our plan for what we're going to do with labels, and then we record the results. And if we fail, eventually this workflow will boot up again elsewhere, and we'll read back the previous results, and then maybe it will complete successfully the next time, right?

16:08But the point here is that we don't duplicate the work that we've already done for each activity. So it's very similar to maybe other workflow platforms that you've used, but the definition of the the same no matter where you run it. So you may be asking yourself now, why would I use effect workflows if Cloudflare already has them? Why would I use effect cluster if Cloudflare already has durable objects and workers? Well, one reason is to avoid locking yourself into infrastructure. I mentioned that your code will look the same whether you are running on a Cloudflare cluster, an effect cluster, or a serverful cluster. If you're already using effect, you can lean in. into the rest of the ecosystem for telemetry, for any other effect services that you may use, any other platform packages that we have, everything will kind of just work out of the box because again, effect is designed to compose well together.

17:06And because the code looks the same no matter where you run it and you're just plugging in a layer to tell the workflows and cluster how to run at the end of the day, you can test your cluster and workflows in memory or in containers just by plugging in. different layers depending on where you're actually running. So testing becomes much like pretty simple with effect. So I want to close with some of the future directions we're thinking of taking this system. Right now it's doing very simple things for us, right? We have auto labeling, we have code reviews built into it, and a few other things. But it's doing nothing fancy, it's just kind of taking work off our plate. But if any of you have pop open source projects, you'll know that the community is also a really big part of open source, right?

17:53And especially if you have a large project, the discussions that you have about your project can often be quite scattered across many different platforms, across many different arenas, I'll call them. And we have the same problem. There's a lot of esoteric knowledge that gets lost kind of instantaneously between GitHub, Discord, Slack, whatever. has to remember where it happened. So one of the things we're interested in potentially exploring with the janitor is using it as also kind of like a memory layer for ourselves and our community. So kind of funneling information in to the janitor from different systems and reusing this information when questions get asked in our discord for example or if we're trying to figure out why we made a particular decision about an api and instead of combing through.

18:43discord and slack and github maybe we can use the janitor for this effect and in particular we're very interested in optimizing this to make sure that we're always giving the most current advice because again if you have a popular open-source project you know that it's constantly evolving the commute with especially with AI contributions are helping this project evolve and so we always want to make sure we're using the most current information this is not a problem we've solved yet ourselves. It's just an interesting direction we're thinking about taking the project. But one of the takeaways I hope you'll have from this presentation is that, especially in the era of AI, personal software, custom software that fits your own workflows is very much achievable. You don't have to use off-the-shelf solutions if you don't want to or if they don't fit your workflow.

19:31And Effect can often help with

19:39keeping things highly composable and flexible for yourself and avoid locking you into different decisions that you've made over time. So with that, I'll close. Thank you all very much.