0:03So let's start the talk, a couple of prompts, carpet is a little bit odd here, so if I fall down, we will turn this into a stand -up very, very quickly, right? And you will just forget everything that happened, right? Okay, so today I'm going to be talking about CommandCode. I am Ahmed Ovest, founder and CEO of CommandCode. As long as I can... remember I have been a developer I've been writing code and it's been a while since I have written code uh by hands right um when I was like nine years old I started writing code in when I was in my 20s I had my code running in NASA Mars Ingenuity helicopter mission so I've seen the entire stack I'm an electrical engineer uh with majors in computer science so I love to double between both software and hardware and that is sort of what we have done here but uh I've written, like, 300 best open source software, hundreds of millions of downloads.

1:00And in my last role, I was VP of developer relations and developer experience at Rapid API. We got acquired by Nokia for some reason, beyond my understanding. But, yeah, we've done a lot of things. I'm mostly known as master of CLIs. How many people here have written a CLI in their life? Oh, awesome. My kind of crowd. So the story actually starts in 2020. And these are, by the way, our investors, awesome founders that I look up to. Founder of GitHub, PwV, led our seed round. We raised $5 million. And that in the blue corner, you see Dean, CTO of Cloudflare, is one of our angels as well, right? So this story starts in about six years ago.

1:45I, because I'm me, had written a CLI to track COVID. I was traveling a lot. I think I had. like 44 talks in 2020 and I knew that this thing COVID keeps coming up so I before everybody you know before WHO was like you know there's this thing I was tracking this because you know refunds right and I got a message from Sam Altman I didn't know who that guy was at that time right if pretty much ignored it then we got a message from Greg Bruckman hey dude we are trying out your CLI this is really weird we connected it to this thing called GPT -3 you wanna get access to it So GPT -3 was like 27 days old, mid -July 2020, when I got access to it. And the first thing I did was I built CLAI, so a CLAI with AI in it.

2:33And that has now, six years later, become command code. Last year, around the same time, at Cloudflare Connect, I presented the idea of Taste 1. And it was really... hard to explain what this model was. It was not an LLM. Now it's running at 100 trillion tokens a month scale. It's a decine model very similar to Jeff. We trained it about a year ago. What it does is it learns what our developer prefers to do, and it does not keep any type of data in it. Also, it has a really nice scale divergence loop where the reward signal it has is it's not going to learn things that and LLM already knows, which is the hardest part to get right.

3:21In October, just a couple of weeks ago, we launched Agar. So I was born and raised in Pakistan. Agar is an Urdu word. It means ifs. So Agar is basically very similar to Jeff. It's a non-deterministic if of sorts, right? It's an open-wake model. We open-sourced it a couple of weeks ago. When we did open-source it, it was number... one on the CN index beating Jeff on tool calls. Right now, the taste thing that we did, the taste model that we launched was for humans. And something really exciting happened last year. We stopped writing code, right? And as a founder of now three years old startup, I was like, you know, I built this amazing thing for developers who are maybe writing 10 to 20% of their code.

4:12Now they're not writing any code. So what do I do with this, right? And I ended up writing a lot about this publicly. Five months ago, we started learning taste of models. Most models are writing code, right? It makes sense. This is the entity that now writes code. And we started discovering some really, really interesting facts about different models. We saw that when you're writing Rust code, GLM doesn't really like a particular... type routines and it repeats that same mistake around 6 p .m. every day talking in PST which was very odd and we figured out maybe it's the peak R start of peak R when it has to make a lot of decisions something weird is going on in the kernels but it's an open model so it's not just one provider that is making this mistake the open model itself is making this mistake like it knows when I am under a lot of pressure I'm going to not do this right?

5:08It was like a new form of swap, right? So we wrote about it, and it became a thing.

5:20It became such a big deal that we ended up pivoting our company. Command code became the main product. In five months, we now have over 50 ,000 customers. We have broken into a space that is extremely saturated, according to Asian space. On paper, it looks like we are coding harness. part of that, right? But what we are doing is we are repairing models. We have the ability to repair models without fine-tuning them. And we do this three to five million times a day right now. We are processing anywhere from 100 to 150 trillion tokens. It's a massive scale. On a yearly basis, we are at a quadrillion token scale. I think in five months, we are one of the largest buyers. of open models tokens like DeepSeq and GLM in the world.

6:09We have 46 providers and we are running anywhere from, I don't know, maybe 1,300 to 1,800 B300 workloads, which is, if you had to buy it today, maybe $200 million worth of hardware, right? This is a young startup within five months in extremely busy space of coding agents. What we have done is we have created a space for us. And now... what i think about it is i think we are building costco of ai and i'm very excited about this because okay open open source guy been building open source software do not like that all of the ai is being held by entropic and open ai right so outside of entropic and open ai everything entropic is like amazon you build something really here they will come back to you with amazon basics product and undercut you uh cursor cloud code, right?

7:01Similar thing, open -ended, co -pilot, and then codex, right? So the thing that they will discover that you do really well with their models, they will actually do that themselves as well on every layer, app layer, cloud layer, server layer, hardware layer, compute layer, everything. They're going to come and eat your margins, right? So as a developer, as a founder, I want you to own a little bit more of a moat. on every single one of these layers. So outside of these three companies or the companies that depend on them, like, I don't know, a bunch of coding agents that are built on top of Claude and GPT margins, right? We are building Costco of AI. We want you to, we want to make tokens so, so cheap that you feel like, you know, instead of feeling like, you know, right now, I think when you're consuming a lot of token, it feels like you are in a super car and tokens are few.

7:57and you're burning them, right? It feels good at a cost, right? And I personally want to turn that feeling away from that supercar burning fuel to you turning on, you know, your light in your room. Yeah, we want to turn that token into such an abundant entity that it feels like electricity. We all pay for it. It's super useful, and we're going to make so many different... use cases out of it. So again, Costco of AI is what we are doing. And we're going after Anthropik and OpenAI. We are not actually competing with them at all. We are right now probably the only coding agent that was purpose -based for open models, not for Cloud, not for GPT. In fact, 6 % of our users use Cloud and GPT.

8:4394, 92 % of our users on any given day. scale like on our self -serve we are bigger than together fireworks based and put together all of their workload on open router for self -serve open model tokens we are bigger than them so yeah how is this happening this again i'm not going to bore you with a lot of details but this is the tweet the infamous tweet got like two million people who read this uh founder of deep sea tweeted about it he's like oh we should build harness, and they are built in harness, and we still beat that harness on independent public benchmarks. I was able to use model repairs to beat, use NAPSEC plus model repairs to beat Opus.

9:37Five months ago, this felt like a joke. A lot of people were like, are you crazy? What kind of slop is this? Are you a thread boy who's just writing content on Twitter? And now it's very much believable, right? In fact, two nights ago, I was reading that on value index, one of the biggest security index, the best model to do security work is right now DeepSeq. It is meeting GPT-6 extra, right? Again, not gonna go into the details of why that is happening. I'm sure, you know, GPT and Claude are really good. They're premium, at premium rate.

10:11So, how is this happening? How are we actually doing this? Our thesis was there will be so many open models in the future. In fact, you guys in the near future will be able to turn all of your workflows and talk to your agent and get it to train a model that is just for your workflows. It will be your model. The only struggle you will have is getting it to work as good as Claude and GPD. That is what we are building here, right? It's a hardness and inference problem every time you hear somebody say things like, oh, I know, open models take too long. they spend too much tokens, they waste a lot of time. Yeah, I know they are cheap, but they don't really work. What does this sound like? It sounds like crude oil to me.

10:57Like you put, instead of fuel, you put crude oil in your car, and now you're complaining, oh, my car doesn't sound right. The engine is making weird noises, staking to them. The thing is, open models are being trained by so many labs. distillation or not, these labs lack a lot of data to make these models work really, really well. If you read the leaked S1 of Anthropic, you would see that they claim that every single Anthropic model is profitable.

11:34They have, I think, 44 times subsidy on their subscription, and they're saying, we are profitable here. It's not. really a subsidy, it's an engineering innovation of sorts. When you know what your models are going to do, when you can predict what type of workload you're going to have, you can make so many optimizations that you cannot do on the API side, which is why API is so expensive. Nobody was doing this for open models five months ago. We built this company outside of Enthropic and OpenAI that does the same thing that Enthropic and OpenAI are doing for their models, but we are doing it for basically the top 10 open mods and we have like you know stats to prove it it's very silly things like these like if you can hopefully read well when you're expecting timeout to be null in a tool call a tool that anybody could you of you could have created deep seek would randomly just send null or an empty object and your harness would be like what is this and the moment you tell up If you think that you're wrong, it's not going to listen to you.

12:41For some reason, it has some weird alpha male energy. It's like, I'm right, you're wrong. And it generally takes like 50 back and forth in a session with 1 million tokens to get it to do what you want to do. I often take an analogy like, you know, think of it like you're training your younger brother how to drive a car. And you see he's about to hit a truck. You have two options. you can push the picks for him or you can tell him after the accident you know this is what you should have done. So what we are doing right now is in Claude code for example if you use DeepSeq in Claude code you're doing the letter. DeepSeq is a different model than Claude. Claude code is not built for DeepSeq.

13:26If Claude code tells DeepSeq you're doing this wrong it's an afterthought. the error has already happened you probably have already broken the cache anything that runs in your cache is 10 times cheaper one of the biggest you know anything that runs in cache is like your smooth drive of a car you know it's not making weird sounds like so that's like just to get you to understand what this is you know in these shapes of these errors change per model in on the time in the type of opacity and concurrency that GPU is actually running on. So if you go home and you try and you're like, no, he was wrong, it's not gonna do that. But when you see like hundreds of trillions of tokens, at that workload, you wonder like, what is happening here?

14:16Why aren't these 300, why is this tool failing 300 times every single hour? And sometimes it's not, right? So these models are actually very undeterministic. It's hard to reiterate why this is happening. And when this started to happen, I was like, you know, is this a thing? Is this where we are headed? So we reached out to Glock, a SpaceX team. And we tried to give them exactly the same source. We're like, you know, can you guys fix this for yourself? They are in our Slack. There's this guy called Elon Musk. He posted about our work with them, right? Which was good. He actually posted about us and he went to bed. So we were like, you know, 200 million people saw us. throughout the night. And at that moment, we were like, oh, shit, this is real.

15:05Grok, as a lab, did not know about this because obviously they see a lot less data compared to Enthropic and OpenAI. Then MuseSpark came around. I tweeted to Alex, your cache is shit. Hey, sorry, 17% cache is a bad idea. We're like, no, we have a lot of compute. Yeah, I know you don't care. but can we do this, this, this, and this and see what happens and their cash went back to about 77% not great but you know we were able to prove what he says that this exists they're going to discover this in a couple of months and they're going to improve this right we again prepared it we started doing repairs in flight so this is not a fine tune this is not something like download an open model, you cannot download this engine. This is what we have built, right?

15:54And this exists on so many different layers. It started in our client, which is the hardness. A very clear outcome for that was we became number one hardness on independent benchmarks, especially for two things, the read tool and the shell tool. The read tool is the context engine. It's going to read everything. So you want to make that it is the most token efficient tool, no matter what. And I shared very publicly all of our research on that. Founder of Hermes actually posted about this. They adopted the same thing, and they are now, I think, doing less, 79% less tokens. So it did really help, right? And then the shell tool. Shell tool is the tool that takes a lot of actions.

16:40One of the most important tool. You know, it's going to read your log file, or it's going to do something with your log file. So part of this exists on our coding agent. the hardness part of it exists in the cloud which is mostly built on cloudflare which is why i'm here in a huge part of this exists in kernels so gpu kernels depends on the type of gpu you are using the type of kernel the type of model that is running in the time of day the type of traffic you have right again you cannot really do all of this if you don't have the most amount of data in the way so The main code is not open source, by the way. I am an open source guy. I was like, I was hell bent on like, is this a real thing?

17:25Because as an engineer, it's very hard to figure out what is the real thing you go and build nowadays, right? Where is the mode? You know, what do I do that OpenAI or Intropic is not gonna come and just eat my cake, right? So I was like, I'm not gonna open source this, just like CloudCode is not open source. I'm gonna make sure that this works because this thing is actually helping people. It's creating a lot of accuracy and it's giving people a lot of good code. We actually got blamed, like, are you burning VC money and you're sending Claude when we select DeepSeek for tokens? Because it feels like this cannot be DeepSeek. It's doing really, really well. Anywho, so we do these repairs. Part of our engine is actually built on workers, DOs in all of the, you know, stack.

18:14We are now roughly repairing a model one million times per trillion tokens that we serve of that model. That could be roughly five to eight hours of our workload. Again, number one, on most amount of tasks, hard tasks that we are able to pass in an independent benchmark, this is our coding agent. We exist as a CLI desktop, all the shebang, everything. You know, you use CloudCock 4. So this is sort of what we are doing. We want you to enjoy a Cloud-like performance with open models, with Costco-like pricing, right? And this is how we are actually, I know it's very small, but we're able to give you $100 worth of inference for just $10.

19:04It doesn't actually cost us to do $100 of inference, right? That's the engineering. and inference engineering that I'm talking about here, right? It's only going to improve, I can assure you, probably hopefully next year I'm here, and I'll be talking about, remember, now it's like electricity, right? So hopefully we are headed that way anyway. And at scale, the numbers are unbelievably impressive. Like a trillion tokens of Claude would cost you about $5 million. It costs about $50,000 or less to do the same amount of work. on our production workload with top 10 open models in $50,000 or less rate. I know.

19:49Yeah, so we are offering 3 trillion tokens a day. This is all paid tokens. We have a huge amount of free tokens as well. 1 million repairs per trillion tokens. Best read and shell tool. And one of our North Star is 98 to 99% of our workload. in cash, which is 10 times cheaper, which is how you get 100 bucks of inference for 10 bucks without us burning a lot of money. This has amassed a huge community within five months. 200,000 developers use command code now. I know a bunch of you might be hearing about it now. That means you are drinking the Kool -Aid of Claude so far. So I would like you to try out open worlds. A bunch of companies users as well. Our coding agent has been downloaded a million times.

20:35People think we are doing something good. And all of this is we are trying to turn AI from a privilege that it is today into a right. That was the goal anyway, you know, six months ago. But I recently just read one of the most authentic resource, A16Z's state -of -market report that came out last... week that 98% of U.S. households are not playing for Claude or GPD, which is mind-boggling for me. So I think if AI is super, super cheap and good, we will end up using it for so many use cases. Yeah, that's pretty much it. This is CommandCode.

21:21Let me know if you guys have any