0:03A short introduction. So my name is Nikola Novoselic. I'm the technical director of the Zero Trust program and agentic engineering at SwissPost and the finder of Grammatter, which is a consultancy mostly for big and complex enterprises, supporting them in the full scale of Zero Trust integration and agentic engineering transition. So let's see if we can get my tablet up there. Okay. So this topic is something that we kind of danced around for two days now. We've seen glimpses of the problem in the keynote speech from Grant. We've seen glimpses of it during Peter Steinberger. Problems with infrastructure didn't catch up with agentic AI engineering and so on.

0:52And now here, basically, it was very...

0:58startup vibe coding focused lots of the topics and or for enterprises that are kind of have a higher risk appetite the uncomfortable truth is that outside of this bubble of startups or enterprises who have a higher risk appetite it's basically impossible to adopt a big AI engineering. So let me make the case for it. Let's take business domains that are not heck bubbles and AI. Let's take traditional stuff like logistics, transportation, finance, healthcare, public sector, critical infrastructure, all of these enterprises, which is a vast majority of companies that we are working.

1:52with or trying to sell things with or work at are all trying to adopt ai to optimize internal processes and workflows and everything and the truth is and this i see that from swiss post and the companies i work with no matter how much money they are throwing at solutions like github copilot and keros of the world and thousand different things and no matter how much they burn on tokens, millions per year, they see zero efficiency gains. Faster coding results in zero percentage of actual faster delivery of a solution.

2:37So let's try to understand why. Why is the problem? Because what I'm saying kind of makes no sense. If you're reading, you're only reading about AI, the anthropics and open AI. the solution so you would think this is a solved problem why are we even talking about it and the opposite is true it's not a solved problem not at all so um uh basically let's jump here and let me first explain why why we have this problem i will explain that on a real use case at swiss post so let's take a swiss post sorting center so this is a huge installation size two football fields full with IOT and OT technology right so let me put it on the whiteboard let's say that this line here I don't know if this is active is a sorting center so this will be probably a WTF moment next two minutes bear with me this sorting center runs an edge of a campus network The backend where it's processing everything runs on a cloud service provider.

3:50So let's sketch the way from that installation to where the actual backend is run. So let's start. We'll start from the sorting center. First, we hit a local firewall. Then we go through intrusion detection, intrusion prevention. Then we hit an MPLS edge router or an ST-WAN edge router. Then we go through the campus. we hit the landing zone which is normally in the data center then we hit the core firewall stack after which we have another nice set of ids and ips and normally we have a reverse proxy integration pattern towards the external network and cloud service provider on the way to which we hit a network address translation so now we will hit our external firewall stack that is standing a direct connect or an express route connection to AWS and Azure in this example.

4:47So now we take this connectivity through the interconnect and land on a direct connect gateway or an express route gateway. We go through a transit gateway and land on the VPC. Then we go through a cloud firewall. Then we hit the load balancer which normally will not the goddamn connection. and after that we go through network security groups until we hit the service mesh Cilium or Istio mostly so after that we are in Kubernetes where our pod is running and here is our back end so I hope the WTF phase is done so now tell me which part of this chain does agentic coding solve because that's the part we're coding we're using clods we're using codecs and whatever we're shipping faster than ever what are we accelerating and here's the whole problem what we are accelerating is this

5:56which means we are moving the bottleneck down the line you're just hitting the bottleneck faster so in the end two actually configured and roll out this full chain left to right you need about seven teams firewall proxies dns pki certificates cloud guys and network security groups and kubernetes teams and everything until it hits a developer who will plot code something deploy it and it's going to land and work so this means for agentic engineering to be adoptable for traditional enterprise which is or the enterprises we want to adopt our tech we need to find a way to serve this entire chain of solution delivery lifecycle right okay so let's try to assemble this so let me scroll a bit down and let's see how we can build now a platform using all the primitives we've seen today that solved this problem for us.

7:07So let's start with an agent now actually one moment we're gonna run out of space like this let me zoom in a little

7:29There we go. Let's start with an agent. What is an agent? It's a model in a harness, right? And this harness, basically what it provides to the agent, tools, skills and rules, memory, so it can remember and build context, a connection to some kind of... inference endpoint extensions which are deterministic expression of corporate guidelines processes and workflows so that we can steer a probabilistic system which is an agent through a gate that is deterministically coded and defined running within the harness and we need external tools we need MCP how so it's we supposed to do this harness right here is Pi.

8:25The reason why it is Pi, because it's open source, because it's not opinionated, because it ships with all its four tools, read, update, write, bash, actually now code mode and MCP came to that, and it's fully extendable. Because it's fully extendable, that means that we can implement our own extension to come to to capture our own in processes, our own internal guidelines and gates and whatever is needed, we can keep them on our version control, we can we can build them, compile them and we can scaffold them to the harness as it's starting up. So this is literally scratching the surface because this harness is absolutely packed with a persistent memory system that consistently captures the entire experience of an engineer working within the harness continuously and builds a knowledge graph of this cumulative experience which means that everything that the engineer has ever done the agent can address and find within milliseconds spending next to no tokens yeah so do you think it's safe to deploy this harness on a traditional workplace device

9:51absolute disaster i mean that's how productive databases get dropped right so the problem is the traditional workspaces workplace windows mac whatever they have evolved over 20 years they are always over privileged often running with with administrative rights on the machine which is just crazy they are geared towards a person using it has a contract and a consciousness and a sense of responsibility and accountability, none of which an agent has. So the last thing within an enterprise you would want to run a harness is a workplace device that has network access, that can access SaaS applications under an identity that, like I said, even if you have zero trust, the sad truth is, is mostly overprivileged.

10:45So we also heard that Peter Steinberger said that the workplace and the aging harness and work in agentic engineering should move towards fully cloud -delivered. So let's do that. So we're going to encapsulate this harness into a completely ephemeral container.

11:15why is it ephemeral because the accumulation of stuff that an agent will download and install and configure and everything on the device the credentials the files they are a supply risk just waiting to happen they will cause agent to drift down the line because it has garbage in its accessible space that it that doesn't need, shouldn't consume, and will just lead to a drift. But now the problem is I describe the persistent memory system and an intricate system for delivering enterprise resources in terms of skills and rules to the harness, to scaffold it with the full knowledge an enterprise wants and needs to distribute to its workforce.

12:02So how does that work if the entire harness is ephemeral, which means it gets detonated every time a session stops? The harness is stateless. So now we need to build something to make it stateful and persistent. So let's introduce among these the four components that we need here around it.

12:32Let's call them gatekeepers. Within the solution that I am architecting you right now they're actually called interceptors because i watched mad max when i was doing this but uh since they are very very architecturally similar more like a super powered version of cloudflare os gatekeepers let's just call them gatekeepers so you understand what i'm talking about so these four gatekeepers let's put them here the first one his egress controller second one a gateway to consume external tools. The third one enables access to infrastructure to access, monitor, change and manage infrastructure but also to consume infrastructure so we can make this ephemeral environment state.

13:23And the last one is inference.

13:34And now the important thing to understand is

13:40all of this go through zero trust primitives.

13:48This means that everything that leaves this boundary carries an identity attached, passes the boundary based on a policy, and everything and every bit and by that is... egressing and ingressing gets inspected, including inference and the traffic itself. Now why would we want that? Well, what does an agent need to work? It needs to access external resources, needs to fetch documentation, github documentation nodes, release nodes, debug stuff, whatever. It needs to access internal resources, API, web applications, endpoints, Kafka topics, databases, whatever. So, this egress needs to be controlled. How is it controlled? Well, we're pushing this traffic through the secure web gateway that is for us doing data loss prevention.

14:42And we're doing that so we can access the world wide web. You've heard several times today, if you took part in the session from Nikita, he's talking about the purest approach to zero trust, which means there is no internal network. every internal resources you publish on the internet and protect by zero trust access policies and you inspect all the traffic this is a beautiful approach to zero trust is what we're doing at swissbos because we equalize everything and treat everything the same there is nothing that bypasses an inspection and nothing is implicitly being allowed to consume a resource so now that we built this we can do this and the agent can consume internet malware inspection and antivirus scanning and everything else that you would want the gateway is doing filtering on dns network and layer 7 but this is not enough because there are companies on this planet that are somehow for some reason really good at bot protection botnet like cloudflare so you have a problem now that you're doing something on cloudflare and you're trying to consume something from a different company like maybe documentation, which you cannot because it's protected by Cloudflare bot protection.

16:00So how are we going to solve that problem? And this is not the only problem. We want our agent to be able to navigate a web application like a human to programmatically render, scale for different orientation, device sizes, mobile, foldable, tablet, desktop. It should be able to inspect that. that web application using a console, network traffic logs. It should basically do everything an engineer that can using Chrome web tools, scaling, and emulation. So we need to give this agent remote browser isolation, which means basically we extend the secure web gateway to consume internet in this way. So now if the agent is blocked, it can literally just spin up a browser.

16:48It can scrape a website or... it can literally extract markdown files from a javascript heavy website compressing it from 2 megabytes into 10 kilobytes which it then ingests into context. So with this we have basically solved the whole egress from this agentic engineering environment. So let's proceed. One thing that I said but let me write it down. So what this arrow right here is caring. the user's identity who is the user right now the user is an engineer a person who is driving this agent i call this directed agentic execution so he is controlling this session he's doing planning delivering and improving and so on and so forth okay let's proceed now before i go on one thing that i didn't mention why is it also important that we are running this in an ephemeral

17:47How many of you or how long did you use a coding agent until you lost your mind and you typed slash permission full access? Do you love clicking those prompts approve, approve, approve? I mean, after you click it 10 times, that's it, you're done. I mean, I literally forked cloud code and hacked it so I can run it in root mode on my laptop before it was possible. to configure. So we want the agent to have administrative privileges. We want to give it an unlimited super-powered environment where we can install everything, configure anything, download anything, because if it breaks itself doesn't matter. Session self -destructs, you start a new one, you continue exactly where you're left off.

18:35Okay, so how do we deal with tools? Why do we need something here in the first place. Now we're coming here to the area of Kenny Johnson. Here what we need is an MCP gateway. Why do we need it? I mean we can just connect MCP servers here. One, two, three, four. Let's take a few examples. Github MCP, Atlassian MCP, ServiceNow and SAP. You connect these four servers you have about 100 tools. Each tool comes with description, front matter and the schema that describes how to use it. You are spending 50 ,000 tokens without writing a single prompt, which means you will be compacting context all the time, losing vital information and it will cost much more.

19:32This is just one reason that optimizes for context consumption.

19:39Another reason is why you would want this is because you reduce complexity because you don't have to integrate with every MCP server, you connect them on the gateway. So we have GitHub here, we have ServiceNow, SAP, whatever, doesn't matter, custom MCPs that you will want. Now each of these exposes many, many tools. This is why the MCP gate is not the same, it's not the same, it's the same, it's the same. basically what it does it collapses all that into two tools search and execute so the agent doesn't need to load hundreds of tools burning away context and costing money it can use the gateway to fetch the tools find the ones it needs and then execute that MCP call and of course the added benefit add that you need for this MCP on the right side some form of authentication open id connect propagation of the identity or whatever custom authentication you might have there which is completely abstracted and handled by the mcp gateway right okay so we've solved that problem now let's continue infrastructure is an interesting one what do we want to accomplish here first of all we've it's like literally we've talked these two days about agentic engineering in terms of coding but

21:09all those vertical lines i drew in the beginning we need to solve the agent needs a programmatic network layer access to those infrastructure resources so it can manage migrate configure update vms containers servers firewalls routers switches whatever else we want the agent to be able to manage that for us. So what we want to enable is infrastructure operation and we want to do that again using single sign -on so what we want is for our identity once again to be attached to this call just like we do here when we access the MCP Gateway so that we can model a gateway policy, allow, deny, block, inspect, whatever, and we can consume infrastructure targets directly from the agentic environment.

22:09We need that to cover the entire scope of agentic engineering. But we need something else. So up until this point, our agentic environment was ephemeral, stateless. So let's make it stateful. How do we do that? Well, for every single user that logs in into this platform, when he first time logs in, what we create is a storage bucket. His personal encrypted bucket that only that engineer can access and read. And in this bucket, we hold everything that the agent is scaffolded with for it to...

22:55be able to useful within that particular enterprise so I wrote your skills what I wanted to write is storage there we go and basically what we hold here we hold skills rules extensions and configuration but we hold something else and now this is a huge debate that is very very that we're having right now at SwissPost, what we also hold here is the entire collective memory of the entire experience, work and domain knowledge of an engineer who used this agent. So the interesting discussion we are having, wait a moment, we are literally outsourcing experience and domain knowledge to an agent, right?

23:51So after one year of work, the cumulative experience of 20 years of a firewall engineer is captured in this bucket. Why do I need this person anymore? I mean, literally, the agent has absorbed 20 years of experience of managing, configuring, deploying, playing with whatever firewall or whatever other kind of infrastructure. I don't have an answer for this. But I feel that we're leaning towards, we are not allowing this particular bucket to be decrypted. It's literally encrypted with the keys that are scoped to the user for who it was created. Because that way we're basically protecting his experience and identity that he put into the work that was captured automatically, mapped to a knowledge graph, and is basically why.

24:47accessible is being captured in his private bucket is that the right call i have no idea we will learn that in the next year or two but this bucket is not enough because what an enterprise needs is a controlled way to curate resources that are distributed to the energetic harness during the process of scaffolding right so the enterprise needs to have a curation and immune control over skills, rules, extensions, instructions, configuration that it's distributing to its employees. Which means that we have an additional bucket, this is a managed storage bucket, which is holding again skills, rules, extensions and configuration.

25:44But this bucket is receiving its data from a Git repository and is injecting this scaffolding into the user storage bucket, making them immutable.

26:01This is how we turn an ephemeral environment. We want it in ephemeral. It is still ephemeral, but everything that's worth protecting and keeping, everything that's useful to keep the agent... on track, stop the drift, and to have all the information it needs to work is protected in a user -encrypted storage bucket managed by the enterprise, right?

26:25Okay, what we still have is inference. So without delivering inference into this environment this, of course, nothing will ever work, right? So we're putting here Again, you've heard it in a session from Kenny, an AAI gateway. We are consuming this inference again with an identity based on an access policy, right? And now this AAI gateway is, let me summarize, has three main functionalities. First one is dynamic routing. Now, why is this important? This is important. because you can basically take your own AWS Bedrock, Azure, AI Foundry, Cloudflare, Worker's AI or anything else and connect them to the gateway and then you can model dynamic routes and now here's the interesting part today what happens in a normal enterprise like in SwissPost everybody's freaking using Opus 5.5 secretary translating an email from german to english and then getting pissed off why does it take two minutes but she spent 30 000 tokens on reasoning from a developer from an intern everybody uses the same it's a huge spend what dynamic routing enables us is to deliver the right kind

27:55of inference to the right cunts of the organization so secretaries are getting a sonnet 5 developers are getting opus files but the code review process that they are running out of band is using Opus 5.5 or Fable something, so you can distribute the right kind of inference to your enterprise, including fallbacks. AWS has a problem, bedrock is down, the dynamic route automatically falls back to workers AI or whatever else is configured, right? Next part is AI firewall, protecting and prompt injection, and all those kind of stuff and data loss prevention. Now don't mistake this data loss prevention to the data loss prevention up here. On the AI gateway we're talking about inference.

28:43We are talking about secrets, personally identifiable information or intellectual property that is being sent to the third -party inference provider. The secure web gateway data loss prevention is literally somebody up sensitive information to their own Dropbox yeah so they do come even though the name is the same and they reuse the same profiles it's a work on a completely different layers right and the last important part or logs and analytics like you can literally see every single person in the organization what prompt they sent and what response they received you have analytics about each team each member what models they are using how many tokens they spent and how much they cost and thanks to this analytics you can model dynamic routes where you can allow a team member two hundred dollars of spend per week and after that they're downgrading to to haiku 4 .5 and I think it's better to work manually right okay now this basically enables us extreme depth of control in FinOps, in the entire spectrum of a Gentic AI engineer.

30:03I have the finest possible control to deliver the right kind of inference to the right kind of people. You have fallbacks rooting between them and budget limits, but we need something else. Hands up. Who thinks here that tokens will get cheaper in the future?

30:23I really hope you... I really hope that's true. Like, guys, I really hope that's true. I think once Entropic and other frontiers go public. Anyway, I hope that's true. But here's the thing. I said earlier, basically, we are outsourcing our knowledge, experience, domain knowledge to the AI model, provider, agent, whatever. So what happens when that provider says, you know what, from tomorrow... we are raising the price 1000%. What are we saying?

30:59Nothing works without it anymore. I mean, you literally build a complete dependency on that provider. Sure, you can have here alternatives. That is a huge, huge help and security. But in the end, you have a dependency, right? So what we've built is our own local inference mesh. way it works is like this. We have here inference mesh and basically it has two parts. One part down here are basically inference node agents. Compare them to Zero Trust, maybe Cloudflare 1 agent or an EDR agent.

31:49you deploy it on a node using device management. When it's deployed, it's automatically pushed with the configuration, so it's automatically functional. What the node agent does, it brings with it the capability to deploy many different inference engines. VLLM, Mesh LLM, LAMA CPP, whatever. And you can build as many meshes as you want. and you basically join your devices in it so what are we doing with it we're literally deploying this on our workplace windows mac linux devices or workstations in our postal offices that are plugged into into a power outlet and lan cable 24 7 they do nothing they're 99 .9 percent idle so we're basically using our empty compute capacity to create our own inference provider.

32:45So now you're thinking, how the hell are you doing that on a stupid Windows machine with 8 gigs of RAM? Well, this is why we create several different groups of meshes. On these really kind of low -powered devices, we deploy a runtime that can shard a large LLM model across different devices. So you don't have one device, one GP.

33:15You have 20 devices, each responsible for certain shards of that mod. They communicate to each other in a peer -to -peer connection, using the Zero Trust Agents activity as the backbone, and expose this inference through a centralized inference router. Where is this reference router? It's right here.

33:44Now, those nodes, MacBooks with 64 gigs of unified memory, Mac studios, bigger machines, they simply form a different mesh where they get deployed a runtime that is targeted for a single node execution, including prefix caching and everything you would want, and they are more suited for developers actually working with them. The sharded models are very suitable. for autonomous agentic execution where you don't need a long -running context that needs to be reused. Agent spins up, does something, shuts down. That context is unnecessary and can be nuked after it's done. Now the nice thing about this is that it connects here.

34:29It becomes a custom inference provider on the AI gateway which means it can be modeled within dynamic routes. which means we can now configure every dynamic route all the inference is consumed from our inference mesh costing nothing only some electricity right the inference router is an open ai compatible endpoints with a bearer token so it can be connected normally to an ai gateway as a custom llm provider and this is how we're basically using empty compute capacity to have our own infer running on hardware, just sitting there, deprecating over time, doing nothing, right? Okay, so this on architectural level is the solution that we defined on the beginning.

35:19This is how you solve the entire end-to-end process of solution delivery lifecycle for agentic AI engineering end-to-end. Now, This is a blueprint. I actually want you to be able to deploy this tomorrow. And to do that, we have to map this now to technology, right? Now, I literally tried to implement this several times.

35:51And let me tell you, there is only one platform on the planet where this is implementable. And that's Cloudflare. And the reason... that is is not because Cloudflare does better compute than AWS we can argue it does or Azure or anything else the reason for that is because Cloudflare does zero trust and the Cloudflare compute workers platform natively integrates with all these zero trust primitives workers VPC and different platform level bindings access to the Cloudflare mesh integration with work let's map all of these primitives so you see how do we solve all of this problem because what we want in the end a company they don't want primitives they need a fully fledged solution that's assembled not parts that you have to assemble yourself so once they spin this up they want to start solving problems and not designing a solution that can solve problems so let's see what happens let's map it let's start up here what is this in the middle so I think this is quite clear this or I think I have to make my pens a little bit smaller there you go so these are CloudFrey containers and durable objects this is CloudFrey Gateway and CloudFrey DLP right here.

37:25This is browser run, there we go. This here is MCP portal using code mode right here. Down here to access and operate that infrastructure we need several things. So imagine you want to just tell the agent SSH into sorting center one and run diagnostic, draft a plan, whatever needs to happen, and create a pull request to orchestrate that using infrastructure scope.

38:11So we can write a Cloudflare access for infrastructure policy where we can allow this agent. to basically connect with single sign-on to an SSH target using a principle that we define in the policy. So we give the agent a read-only principle so it can see anything, it can diagnose everything, and then it basically drafts an implementation plan which executes out of band through an infrastructure as code delivery pipeline. Of course, what do you do on an infrastructure that does not support infrastructure as code? you grant the agent a principal that has execution rights on the target. So in that case, the plan needs to be different.

38:58You need a rollback, backup, and so on and so forth. But basically, you can completely automate all of these use cases. But what else we need here is we need Cloud3 Mesh or Cloud3 Tunnel

39:15in order to connect and also authenticate. Storage. This one is simple. This is R2 for both of these buckets but with completely scoped keys to actual users operating this saved in the secrets vault. This one is of course CloudFrey AI Gateway. How is the inference mesh built? This is a worker.

39:50and we're using D1 because the worker is stated so it's pinning traffic to the node with which it establishes initial context for prefix reuse for caching we're caching here in this case content on two places on the AI gateway itself and on the execution engine it runs inference on the node itself so this will be this part and then we have a node agents so i said earlier that all these nodes need to see each other peer -to -peer connectivity now imagine how do you make nodes in a peer -to -peer connectivity networking mode if they are separated through 1 000 different locations would have been a nightmare if we wouldn't have the cloud for one agent or

40:44So basically we have a device to warp to warp connectivity, so all these nodes see each other as if they would be literally plugged into the same switch. So they elect the leader among them for inference between sharded models. And there you go, this is our mapping to technology. okay now this here solves the problem of directed agentic execution so here in the middle where we have pi is a person sitting the beautiful thing is this works anywhere because it's cloud deliberate it works in any browser it's heavily optimized for mobile foldables tablet work on any laptop works on a freaking TV if it has a browser you just plug a Bluetooth keyboard on it and you can you can go which enables us completely new capabilities of how does the workplace of the future look right okay so but here's the thing after an engineer has perfected a pipeline we have a mature rules and skills for execution do we still want to do the same thing over and over come age and connect again to the no diagnosis we need to patch.

42:06This is an established process that's in mature execution, the agent did it 30 times perfectly. We're transitioning this entire process to fully autonomous agentic execution. So what does that mean? We have to rebuild all of these guardrails, everything we talked about for every agent we deploy. Or maybe the system is designed from ground up so the entire middle layer can be simply slotted with something else and go figure that's the case

42:52so the same environment supports fully autonomous agentic system built with cloud for agent sdk and flu So any process that needs to transition in fully automatic execution simply runs next to the fully directed user environment, inheriting the same context, the same identity. And that's the crazy part. How does an agent inherit an identity without giving it an identity? Like literally this environment has no secrets. There is nothing to exfiltrate. There is no password. There's no secret. There's no user name. Nothing exists because it's injected at the gatekeeper boundary. But how does an autonomous agent work in this environment?

43:39And that's the beauty of Cloudflare, worker loaders and dynamic workers. Basically, it's the closest thing in the digital world to a Schrödinger's cat. A worker exists and doesn't exist at the same time. And it springs into existence in very few milliseconds the moment it... invoked and needed it attaches as a child process to the worker that invoked it which in the case of a user executes within a user identity and cloud free access which is simply inherited on a platform binding level to the worker that executes a fully autonomous action right okay

44:20this would be uh this would be it now last thing I told you that you can deploy this yourself, right? So what do you need to do it? You need a Cloudflare account with a $5 worker's plan. You also need to design the entire container lifecycle and all the alarms keeping it up. You need to bridge all the bindings and connect all the MCP things. You need to manage and encrypt the storage and also configure and... manage access for infrastructure you need to completely implement the ai gateway routing and distribution so i would say only like a year and you're done or i mean like you you can literally go right now to codeflare .ch yes the name is deliberately confusing and you can deploy it in 20 minutes the entire environment including the ColdFlare Inference Mesh.

45:30Yeah, and that's it, we're done.