Notes provided by Gemini
Oh, yeah. I do actually read it. I actually don’t read it. What I do is I read it. No, I don’t. I let it create a PR and then I read the PR. Oh, that’s amazing. It’s all It’s all there. It’s all there. All the tips. Well, it’s a it’s a it’s a view on YouTube, right? And it’s the best view. Like, you know, I like it like like ringing from a terminal. like bringing it from IDs. Well, I give I give it um I have it as its own token that can like read and write to MR comments. I can like in GitLab because we have our own self host of GitLab. I can like say I don’t like this. I don’t like this. And like it can pull those specific things which I like that. I mean it just makes intuitive sense to me that UI and then I’m learning what it is, right?
Yeah. Like Yeah. So So I usually write issues. I write issues and then I’m like I do this too. Yeah. Give it the link and say implement and then the link right and if I leave a comment you server gh I use gab github like the gitlab github has a command called gh let it run it. So it’s trained it’s trained on how to use gh. Okay. So you don’t have to like add anything extra to the model. You don’t even have to tell it. Okay. You don’t have to like tell it just knows very simple and then I’ll bring it up like it’s really helpful. Being so far ahead can lead to kind of convoluted craps showing up. Sometimes it’s better to not know and not Yeah. And this and this and this and this and you’re like, you mean I legitimately have to write all my code? Yes. Yeah, but then wide open.
You can go talk to it. It’s Yeah. Uh, come on. Come on. Hello. There we go. Okay. I know that works. Okay, good deal. Of course, I did the thing again um where I highlighted the link in the email and I put the right text in. Um, and then I highlighted the actual link and it’s still the old link from last week. So, uh, yay. Um, so now my protocol is when I screw that up, um, I log my phone into the right one and this into the wrong one. And that way anybody that shows up on this one, I can never go over here. So, ah, anyway, let me share my screen and then we will get started. All right. And then see if I can figure out how to make this go away. So I don’t think we got anybody here that hasn’t been here before. So welcome back. Uh for those first time equation force, um this is where I work, Charlie works, Josh works.
Um, not like physically. Usually, uh, usually we’re dispersed through at least three different buildings or two. I don’t know. Are you both in 406 when you’re over there? Uh, yeah. Okay. Um, so there’s that. Um, and I think that’s it. We’ll get started. Um, what I was going to cover initially was, uh, and of course I had to shut down to get my keyboard thing started. So bear with me for one. Actually, now this is live. I can show this this way. Um, it’s not the latest and greatest up to date, but this will take a minute. Uh, because I’m using the cheapest thing I can possibly find for hosting because I don’t have any revenue or anything coming through here yet. So soon as that happens, we can flip it over and do something different. Um so initially uh the thought process here actually I’m going to no I can delete this one and start another one. Um let me go delete this field real quick and add a new one.
So the well actually before that the concept really is as a farmer you pick your field and do some things uh you click through uh see reports of uh things like your this is actually a measure of vigor over time uh with three different satellite inputs. uh you got temperature and then you’ve got precipitation at the bottom. Uh and then there’s also uh some events of things like uh frost damage that could have occurred on February 13th. And these different the whole point of the thing is the events themselves. The graph is pretty and all, but most farmers kind of have an idea of what their fields look like. Um, what they can’t tell a lot of times I haven’t actually got a view of the actual zone system that it does. It can take a full up area and then figure out what parts of that field had better uh what what’s called figure than others and it can actually draw you a map of that. You can see where the where the rough spots are and stuff.
Um, it can it can do some things like uh detect what’s called lodging, which is if you got corn and wind blows and you got, you know, thin stocks, it’ll actually lay it down. Um, so you need to call your insurance company immediately doing things like that, whatever. Not like they have like normal insurance companies. Um, so there’s that kind of thing. But the whole the thing I was going to get into initially is that um drawing something out like uh you can actually see um on here there’s actually a set of buildings that you have to bypass. You know, there’s uh there’s some spots in the middle of the field um that I didn’t want included. Things like that. Doing that by hand sucks pretty bad. Um, so what I decided to do, this is something I talked about a while back, actually. Sorry, I forgot that I put this in. Uh, like everybody else, deleting stuff on Amazon, you copy the name, you paste it in, it goes bye-bye. Um, so what we’re doing with the actual input now, uh, we as in me, Uh, zooming through, uh, was, let’s see if we can go find a spot that’s a little more interesting than that.
I know of one over here near my house. Sure. Uh, hold on. Let me go back over what? Browns fairy. Oh, yeah. Is it this somewhere over here? Guide me in. That’s 72 going across. I’m below 72, right? There’s Browns. There’s Browns Ferry towards what? Bies. Bies. Okay. Over here. That’s Bies. Now go east. Now go west. Is this you? Right here. Okay. Go. Sorry. West. East. See that green patch? This one. left or up. Yeah. Guide me in. Okay. Move it that way. Okay. All of there. So, all of there. This one may cause a little bit of a problem. We’ll see what it’ll do. I’m about to push another update where right now you just pick the the concept is you get the field as big as it can get in your actual area in your view and then you click inside the middle of it somewhere.
It ships it out over to something called modal that we’re going to talk about in a minute. Um and then it generates a boundary uh based on what you clicked and it comes back. Um, initially it decided to pick this one, which there was a kind of a road across here, so I could see why it would do that. Um, we’re about to push another update that lets you pick five points within the field and then three outside the field and see what kind of uh addition additional thing we can get there. Um, so pretty pretty easy. Um, Whoops. Shouldn’t have done that. Control Z doesn’t work. We’ll do an undo. Maybe that’ll work. Oh, wants me to click again. Let’s see what happens if I pick something like this. One up here. I’m I’m checking to see if it takes out this little spot. Is it Jones Farm? No. Is this on? And this is out towards at this is Limestone County going out uh shoot Tanner Tanner.
Okay. Yeah, I can confirm the boundary and it’ll go set it. Uh no, so I can give it. Right now I’m just doing soybeans, cotton or corn because that’s what I’ve been playing around with. And this is uh not really acorn. Uh and then at the moment I’ve got a full stack of Amazon stuff running in a giant step function going pulling data over three years from three different satellites along with climate data and other types of stuff to stitch together. Um, but all of the code that is doing that, let me then flip over. Uh, oops, wrong one. Uh, GitHub. Come on, go to GitHub. Right now, I’ve got it in a private repo and we’re in a package called modal. And so it’s pulling SAM 2 uh model to segment and all of that. Oh, don’t tell me. Okay, there we go. The entirety of that code is 113 lines. Um, so there’s a package called modal um that you get.
Let me actually go back one and look at the read me because I also uh wrote up some stuff on what it takes to deploy. So uh modal is al also comes with like an SDK CLI kind of thing uh that installs locally. Um, if I’ve got this particular method, it’s modal deploy. Give it the method, then it shoves it out there. And uh, it does a pretty good job of uh, error reporting. So, the way I would have done it in the past, I’ve done this u with RunPod where I’ve created my own image, shoved it out there to runpod. I’ve done this with AWS Fargo, Fargate, Fargate, Fargate, whatever. Huh? Fargate, Fargate. Um, and dealt with that. Um, with Fargate, I also had to deal with how do I get access to this thing and does it have a public API or not a public API, you know, all this kind of stuff. Um, runpod.io was a lot easier, but still I’m dealing with uh what I ran into there was you can you can put a credit card on file and tell it to automatically re-up every time it gets lower than some amount.
What you can’t do is put a top level budget on that. So that was a no. uh with modal you can do a budget you can do all the things but um when I’m deploying this function we’ll look at it in a second but uh the the python file itself has a list of all the dependencies it needs so it then on its side goes and builds the image so I’m not building an image and posting an image I’m giving it the function that has the information it needs for its dependencies then it goes and builds the function spins it up and all of that and you you saw about how long it takes to spin up an image, call an actual inference, and then get the data back to me. Um, it’s not like directly interactive, but it’s good enough for what I’m doing. Um, and it for me it turns 5 to 10 minutes of fairly meticulous plotting around the edge of a field into a 30 second activity um that I could expect pretty much anybody to follow.
Um, also if you have secrets uh that you need to pass along. For my case, uh, my modal thing, it actually grabs some stuff off of my S3 bucket. Um, so I’ve got access keys, things like that I can pass across. Um I had it. You can uh if you have your own Python code um and you want to call modal um basically you you import it um and then actually you call the function and then you remote this will dispatch to that modal instance spin it up run your inference and give you your data back. So, I’ve got it right now hooked up where I’m hitting it from a React web page going across to a REST endpoint that it then converts and calls this thing. If you wanted to have part of your code, if you’re writing it in Python, but I’ve got this one piece, I need an inference engine running for this piece. you could just push that one class over to modal and then every time I hit that point it’s it’s basically dispatching to another um you know container somewhere running the inference giving me my data back and I didn’t even have to code any of the networking stuff um so that’s fun um and of course I built all this with cloud code so I had it also I’ I’d like to run it locally to make sure because I don’t want to spend all the money on
inference if I already have a GPU to do whatever. Um, so let’s go. First off, well, before we do that, we’ll look and see here’s the code itself. And I tell it here, I need an image and all of these are classes from modal. So they’ve got a bunch of different kinds of images you can use. I need Python 312. I need torch torch vision, uh, CUDA 12.1. Also grab fast API hugging face and these kinds of things. uh somewhere up here. I don’t know if it’s in this file or somewhere else. See there we there we go. Okay. So I am telling it uh when you deploy uh put this on an AT so they’ve got them along with you know 16 uh gig of memory things like that. Um, also grabbing secrets from what I had set it. So if you’ve got a bunch of different functions out there that you need different things, you can split those up by name. Uh, and then underneath this code is basically as if you had SAM 2 installed in your local environment.
Um, so when this is running, it imports Torch. It pulls in uh these items are actually from the SAM 2 codebase. uh you know pulling models from hugging face. Uh I believe that I have it pulling the small model. Uh now this is on load models. This is actually when it enters. Um you can speed it up a little bit by actually pulling the weights when you build the image instead of when you’re doing the part of it, but it wasn’t I haven’t cared enough yet. Um, if I go over and I log into modal, gives me a dashboard. Of course, it’s AI infrastructure developers love. Uh, it’ll show me my function. Um, it gives me uh right now five free credits a month. Credits I’m guessing they call dollars. Um, see how many times I’ve hit this thing. as as far as I can tell. So startup this is not bad. So startup was about 9 seconds. Execution time was about 2 seconds.
Um for this uh not quite sure metrics considering I am the only person right now hitting it. Uh you’re not going to see a ton. Uh, but so far from what I could tell initially is that it was costing me about 4 cents per run. Have no idea if that’s expensive or not expensive at the moment. Expensive, huh? That’s expensive. Expensive. Okay. Um, so that’s what that is. So you’re probably paying more for the infrastructure and the development setup and stuff. I wonder if there’s a a flat cost for like the startup and stuff like that because A10s are should be super cheap, right? I was trying to see how many times I had actually hit it because I’ve spent out of the $5 I’m at 21 cents and I’ve probably hit this thing more than 20 30 times. So, I may be wrong on that calculation, but no, it may as well not be dollars. It goes dollars, right? credits, but $100 dues Canadian.
Oh, yeah. So, what I’m what I liked about it um was how stupid easy it was to spin up and I’m not worried about what containers go where and all this kind of stuff. Um I could see getting to a point where using this to do some quick iteration stuff because I can go change that file, hit mode deploy, and it shves out a new a new version of that. So if I’m trying to work on my inference or whatever, you know, it’s about as fast, the deploy was about as fast as just building local. I mean, it was if you were building an image anyway. Um, so that was that was pretty quick, dirty, and easy. Um, I could see doing that iteratively pretty quick to get to what you want and then later on it cost as an issue shoving it somewhere else that’s a little more uh cost-effective. mobile like on your own server. I don’t know. I doubt it. I think the I think most of what they are is the infrastructure and making it easy for developers to shove stuff over to and I’m not I don’t know why other things haven’t done this because I mean they make money by running inference and charging you for the inference you run.
So the easier they can make it for you to deploy your stuff on their framework or their infrastru the developer and the checkbook are normally not in the same hand you know so you may have developers making all kinds of cool decisions over here and the modal make it bank because whoever it is didn’t realize that there’s a cheaper way to go um and then maybe by the time I’ll let you know if I ever have to move this somewhere or else how hard that is to unravel from. It didn’t seem like it was too sticky. Um because I can take the same function and pick that up and run it as if I were you know in my own container itself. Um do you have to set up the imports like that like the entry point the local entry? Uh I let cloud code write all of this. So I didn’t write a single letter in here. Yeah. Um I missed the beginning of the pitch. So I might have missed something.
But it looks like the model is self package where you where you’re able to put all the piv packages etc. But are you able to define the buffer and then uh like put the entire container in with the model to do the same thing? I don’t know if I could put it in my own container and then have it host my container or my image. Um, it seems like the way it’s built is it has a set of uh, you know, images that it wants to that it’s got in its collection and then from there you pick what you want installed from a pip install. I haven’t checked to see if there’s any other ways to install other packages like if you needed ffmpe impeg or something else that may be a you know Debian package or something like that rather than a you know pip install I haven’t checked I mean that specific one you definitely can’t do a pipe because I mean I mean so modal I would think of this like a competitor for data bricks and like snowflake and like all that sort of stuff, right?
So, where they get money is uh like one of their big partners is Dagster. You mentioned them a few times. Yeah. Because you can just take in a single node of your DAG and have that go out to do inference on this sort of thing with your code location, right? And so they’re they’re looking that’s why they give five bucks away for free because they’re their big bills are people who’s like running their entire big data pipeline, right? Or you know, their inference or training data pipelines on this, right? Uh, so I’m looking at using I’ve got another piece that I haven’t implemented yet that’s actually doing prediction of vigor moving out a couple of days because I got prediction of uh climate and those two things kind of go together. Um, unless somebody went and harvested then it’s sorry there’s not much I can predict on that one. Um, so that’s uh that will also be running you know on modal. Um, I haven’t checked to see uh the the algorithm I’ve got that’s taking the radar data and then um estimating what the op optical would be from.
So I’ve got one piece running a transform from radar over to optical using about a year’s worth of of data. Um that right now is all CPU bounce. I don’t I don’t know if there is a variant of that that would that would be faster because right now it takes I mean it’s probably still turnurning on the field that the not a real corn field. Um not sure if it’s finished yet. We can go take a quick No, it’s still turning. Uh but let’s see now. It’s picked up at least some pieces by now. Uh which is not going to see a whole lot because you’ve got pretty much um it wasn’t a real thing with crops and cycles and such. Um, it’s also got a quick uh thing right now that goes and uses another project uh that comes out of uh I think it’s USDA um where they actually estimate what crops were grown in what place um over time. And so it thinks based on that estimate that it was corn last year, soybeans before, and then corn the year before that based on what USDA thinks was in the field behind uh Todd’s stock farm or former sod farm.
Um I know you said the farmers know their own lots obviously, but I do think the new graph looks a lot better. You’ve been advancing that. Not really. I had it go add points where you can actually tell the difference between LANCSAT and a sentinel. I’m probably going to convert the terminology there for good, better, and best. Uh because sentinel is 10 meter uh resolution, LANCAT is 30 meter resolution and the what is called SNAP over here is actually the estimate based on radar data. So that’s probably 5% plus or minus accuracy. Um, what’s interesting is everywhere you see them is another thing I haven’t backed out yet. Uh, LANCAT uses different wavelengths than Sentinel, which is why your boxes are always exactly a certain amount higher because the there’s a there’s a linear conversion I have to do there to bring them back into the same uh whatever the word for that is. Um, but yeah. And then Jay, what does cheeky uh uh growing degree dates. So after you put a a seed in the ground, based on what kind of plant it is, you need x number of degree days before you expect that to break ground and emerge.
Um you see where it’s got right now, this is just corn stages as it moves through each after a certain number of days it’s needs to it’s expected to be at a certain stage. Um, so for me, there are certain things to watch out for when I’m in VPN versus VT. You know, nobody cares about lodging until you actually have ears on the corner. It’s not necessarily a a thing. Um, so I mean, there’s a there’s a lot there. Um, so yeah, that’s what I’ve got for that. I’m I’m looking at another graph. This one is focused on how well the plants are growing. Um there’s another graph I’m going to drop underneath that is more about how much moisture do I have in this field. So you have evaporation and you’ve got temperature and a lot of stuff. And if I’m an irrigation farmer, um there’s actually a calculation you can make for how much moisture you need to reflect uh based on that and what the forecast is. You can do a lot there.
It’s actually should be a helpful thing. Um something they already track and do um a lot or pay somebody to do. Yeah, I just signed up. I only got a dollar in free credits. What? That’s same as yours. It says I can get up to 30 if I add payment and do other things. Well, I haven’t added a payment or anything and it’s telling me I can get 30, but I That’s weird. I wonder if it’s some random we could I wonder if they’re running a model to try to figure out. They’re like, “Oh, no. This dude, this dude’s not gonna use it.” They’re like that. They’re like, “No, he didn’t come in with flaw to set it up, so he probably knows things.” Um, oh yeah. Um, that’s what I’ve got. You want to flip over and grab this thing? I need it. If you want Google meet or Yeah. If you want to join the Google meet, click the link in the uh in the invite.
And you may need to mute your laptop. Yeah, I got it. Can I see my screen then? Nope. You haven’t plugged this in yet? Oh, well, not that. Oh, I wonder if I can plug this one in. You have like a smaller one? Uh, don’t know. Oh, wait, wait, wait. This one. Uh, no. I don’t have a what? No, I haven’t. Okay. Well, if you want to share your screen, I will if you share to Google Meet, I’ll plug into there and we’ll get that’ll show it up on the uh projector. If I can figure out how to share Oh, there we go. There we go. All right, you’re there. Let me stop my camera. Let me bar this then I believe back in wherever that was the side. Come on, you can do it. I’m just not going to look at that because it’s gonna bug me. It is so delayed.
I’m just gonna set this here. So, not to brag or anything, but with Enterprise, you get Enterprise dollars. So, you get the $5, I get the 140. Okay, I’ll switch. The downside is you don’t get it every month. You have to you get it one time and then after you do it, they search hard to your credit card. So, that’s how you But yeah, you can you can see I’ve I’ve used I think a whole guest have my own LLC. I’ll just use that to sign up for Yeah. So I’ve used 100 like $123 in this whole like experiment, but because I signed up on this like Friday or whatever. Anyway, not to brag. All right, so Amazon Bedrock agent. But I don’t know who told me to I got to say this again. I don’t know who told me to use Claude to generate my slides, but I just put an outline like like I normally do. I just outlined my speech and then gave it to Claude and it generate I was like generate in the style of HSV AI.
You got the logo. I’ve got stars in the background. You can’t see the stars here. I got stars in the background. It like did all this fancy stuff. I can tell you those colors come from the color scheme that I had set up. Oh, it said it it did the deep dive of the HSV AI and said these are the this is the color palette and uh presentation. Yeah. And it’s it’s just awesome. So like if you haven’t done this yet, you know, just just play around with it and do it. Um Did you use cloud code? Yeah, clock code. Not even code word, just cloud code. It’s an HTML file. So, it generated JavaScript and HTML and just created my slide deck and it’s got like uh you know I can go forward, backwards, whatever. Right. Yeah. So, kind of going to talk about two things here. Uh just going to go over like Amazon Bedrock and then B bedrock agent core.
These are just two different tools uh Amazon kind of has when you’re talking about LLM agent. actually probably the same model too. Um just models in general. Bedrock could probably do the same model probably. Yeah. Yeah. Um so first thing I’m going to go over is Bedrock. What it is. Um and it’s it’s basically just one unified API for generative AI. That’s what they say. Um like I said, it probably can do the same. I know you can do speech uh you can do speech with it as well because they have Alexa, right? Uh that you can do video. So it’s not just generative AI. I think that’s just tagline. Um but but it’s basically it’s an abstraction layer for for AI, right? So you can uh uh basically just use to access any of the foundational models. I got anthropic, open AI, meta, minstral, moonshot, which is Kimmy. Um so any of those are available to you when you when you kind of use it.
And you can also uh AWS has their own models as well. They’re called Nova. They’re actually kind of they’re really good actually. But you can use those models to fine-tune anything. So if you’re making like an app like that, you know, like a production app and and instead of you know having skill files and stuff that you program, you can go fine-tune it and it’ll just know your path like the backend tools and stuff and how to interact with them. And so that’s kind of I think what a lot of people do these days when they’re when they’re embedding like the chat client in there and stuff. So uh kind of kind of kind of show you what is um so you just kind of log into AWS and then you just type Ed Rock up here and it’ll take it take you to it. Wait. Um the I guess the only thing I can really show is the model catalog and and it’s got like here’s the most popular one. It’s got CL I like OpenAI wasn’t showing yesterday.
They just um they just got the partnership this week. So Open AI is supposed to be there but I haven’t been able to find it. Um but you can see there’s there’s Meta uh there’s Opus and they all have different rates. You have to go look up your plan. If you’ve ever dealt with AWS, you know how complex it is to figure out the pricing on anything and God help you. Use dollars though instead of credits. They’ll use whatever currency you want. I mean, we have bills and and uh euros apparently too. So, I don’t I don’t know it works that depending on where you execute it, whatever. Um, you know, so these are kind of some of them. I think just goes on and on and on and on and on. Um, there’s Gemma. Uh, there’s also a marketplace. So, if you wanted to host, if you want to create a model, and if you’re not, if you created a model and you haven’t done this, you need to go do it.
I mean, you know, if you put ever put up anything, I mean, why not? I mean, you can search. I don’t know what was that thing called, Sam. SAM. SA M2. S A M2. Not No, but but you can go and you can filter it. You can say, uh, what was I don’t know that they’d spell it out. I mean, you can get segment anything. segment anything or just type in segment see if it finds something. Well, it’s got like the providers like there’s one for like there’s hugging face. So, hugging face has got uh some I haven’t actually looked at the the prov um Sam is meta. Yeah. Oh, is it meta? Yeah. Metapare. Well, meta would be on probably the serverless one. So you talking meta anything, Sam? Yep. I don’t know. Yeah, they got Oh, boy. The metal models are actually really good. Um, the scout ones, the the little small ones.
I like the small ones. Um, especially for chat bots and stuff like that because it’s really fast, right? So, you know, I like to use those. Um, especially when you need a dumb model that doesn’t do anything. Um, yeah. So it it’s really let me uh try to show you the code. I’ve got like a little demo here just kind of show you how it works. This is just a lane chain app. I don’t know if it’s just a lane chain is just a har that’s not even a harness. I don’t know what it’s like below the harness whatever it just talks to the model. And it’s a framework for building like a agents and stuff like that. And uh you can kind of see literally uh I just have like here’s my system prompt and then uh what is the weather in Huntsville, Alabama, right? So and for this one uh mostly it’s just invoke I think like in in Langchain there’s got like a link chain AWS is the import and you invoke it somehow.
51 line 51. Yeah. This is where you invoke it and so you send the message and the roll and stuff. So let me just kind of run it. So I think this is rock. No. Oh wait, wait. I forget how to run stuff. UV run. And this may fail because like it’s live, right? Can y’all see that? It’s good. Yeah. And you see it giving me the weather, right? And so so that’s using the the old standby the the um sonnet and I can you can see I can just come in here and like switch out the model. So I want to like use the maverick. I think somebody said something about that one, right? So you can just run that and you can see gives you more de more things. So just like you can just go and change out the model whatever. It’s way different. Yeah, the models give you different responses depending on their training data and stuff like that.
Yeah. Not 58. It is true. Which one’s right though? That’s the first one. It’s not I don’t even know if it’s 76. Somebody step outside. Well, so the interesting thing the interesting thing that Oh, it is. I’m getting 80 on my watch. So apparently what I think happened is it just ignored instructions and went and fetched it because I’ve got a tool in here that’s supposed to like make it the same. Right. So I wrote a tool for weather and it’s supposed to come in here. Maybe something Maverick has it’s it’s actual attention how they train the attention for the law models is brokenally. We’ll see. At least it didn’t say as of my last training data. Let’s see what the m This is uh this is the no the nova the no AWS nova. So I guess that one’s kind of similar to the first one, right? So who knows what it did. I don’t know. So it probably just either made stuff up or um it went and uh fetched it a different way with the web browser, right?
And who knows? But but yeah, we’re supposed to use a tool anyway. So that’s Bedrock. That’s pretty much pretty basic. I mean, makes things if you just want to play around with models, you know, it makes things really really easy for that. If you’re I Yeah, we move on. This is cool. Huh? This is cool. In fact, got any questions about bedrock? Yeah. So, you didn’t set up an MCP server to fetch that data from a source? Yeah, I just wrote a tool. It’s just coming directly from the model. Uh, no. Uh, so I wrote a tool, right? So it’s a lower level like MCP MCP is like a a tool right? Yeah. And then it’s a framework that you can go and connect. Yeah. So I just want a lower level just tool. So so if you so what it what the tool does I can kind of go over that.
I mean there’s is it just goes and fetches it from here. So the model says hey so when it boots up it gets a list of tools. it knows what tools it’s got installed and um it’ll go get current weather. So if you ask it for the weather, it’ll go it’ll go do it, right? This is just a lower level way of doing it. This is what you did before MCP, right? Yeah. This is where you got to be really good at your tool descriptions and things so that you’re hoping the model picks your tool. Yeah. Instead of trying to make its own thing. So this is what it uses. Okay. But hang on versus the same thing. It’s just once remote, once local. Thank you. Yeah. I mean, so like right you basically take a tool and wrap it in a in a web web call, right? And then that’s MCP. This is literally what it The other thing with MCP is all that info had to come through tokens and context and such.
Yeah. Because it’s coming remote. It’s kind of like you’re passing stuff across a lot. Yeah. The only thing that’s difference the MCP is a remote procedure call. This is just procedure call, right? So it has its own interpreter basically that sits there and you can execute on your current system. There are some advantages of using local tools like secure if you’re really care if you really care about security you’re like worried about man in the middle attacks because like MCP can be hijacked and then bump injected. So like if you’re really really concerned about security, you can write through local tools and it enforces it to always be that the it force it always to use that tool, right? And the agent can’t change this code. It it’s forced to use it and and so there’s other things with MCP other problems with MCP called rug pools where the prompts get pulled from you or changed. So while the model’s running, right? So like if you got something that’s a long running model, you have the risk, especially if they just change the API in the back end.
So So there’s a there’s some advantages and disadvantages. I I prefer tools over MCP, but you know, uh MCP is a great tool still. Yeah. If you’re if you’re going to write thirdparty apps, you want to write MCP, right? So if you’re if you want to provide a service to to customers, MCP is the way to go. want to write it yourself then ah I’m torn right it depends on how enterprise you want to be right so it’s like what do you want to support do you want to support code or do you want to support like a service right so um you can go either way experiment is providing a like so right now scraping up the data from weather right and then you’re asking question about weather but if you give a completely different website say scrape up data from the food website and ask the question about weather I want off the hazard bike. It pro, you know, it would probably So, so the the the smarter models will probably reject it and then go find a different way.
Um, something like uh the uh the uh what was it? The meta one just didn’t even use the tool at all. So, you know, like sometimes it’s hard to just get them to use the tools that you like to even use the tools like use the right one. It’s a very hard problem. And it’s the same problem with MCP. Like you’re not gonna that’s it’s I don’t know. There’s there’s so uh the there’s a lot of like observability you got to put in those things to make sure they’re how they’re like understand how they’re calling things and then and then optimizing it so it’s more friendly for the element. Does that make sense? Yeah. So and uh ironically a lot of these tools are in agent core. So um although they’re broken in my AWS account because I deleted something I know how to fix it but so I can’t demo that stuff. So, it’s not as nice as and easy to use as modal, but but it’s got all that stuff if you want to spend the time and effort.
Um, so let’s go on to agent core. So, agent core is just a way to build, deploy, and operate AI agents at scale, right? Um, so basically it’s enterprise grade. That’s the only thing I can say. Call it enterprise. Uh, it’s a code force uh platform. You can build, deploy agents securely. It’s got all the things that you would want, right? Um like if you’re trying to deploy an agent in in your infrastructure and do it securely and all that kind of stuff. It it supports any framework that you want. So it’ll support uh Amazon Strands, Lingchain, uh Crew AI, I don’t know if there’s another one, but it’ll support all of them, right? Uh, so it’s not it’s not dependent on that. It’s more dependent on does it have a library that’ll interface into it. So like Python’s got libraries obviously. I don’t know about Java, right? So with Java, you’re just limited whatever interfaces into it. I know that it’ll it’ll work with any node framework and it’ll work with any Python just out of the box.
Uh, and then also you can use any model, right? So pretty flexible. Um, so it handles all uh all the things that you’d want it to do like memory. So you if you want a rag, it’s got got hooks for that gateways like HTTP, SQS. Um, so you can send it information however you want to deliver it. Um, it integrates deeply into the AWS ecosystem. So like like if you wanted to send a a request from Lambda, then you could use it to to execute it. Takes care of off. So uh you can use IM to to set up all the things. So it’s passwordless and then any if you have users that log in and you authenticate with AWS and they have access to it and uh it also can do MCP. So if you want to wrap an API that you have already in AWS like use API gateway you can then wrap it with MCP and now you’ve got MCP for it. Yeah I know it’s the coolest thing.
Uh it also takes care of policy and policy and uh just that’s just guardrails at the harness level, right? Um so what is it for small projects? No, just use spot, right? Like if you want to do one one thing, you know, don’t don’t use this. This is probably not the right thing. Well, but for really large agent problems where you need to like launch a swarm or handle lots of traffic or massive research task where you need lots of agents, right? This is where it really kind of excels. This is where this is where you want to use it, right? Um, so a little bit about the gateways, MCP frame like so once again framework for how agents communicate with external sources. Uh, you can hook up lambdas, APIs and more turn them into so you can also turn a lambda into MCP. Uh, you can interact with thirdparty tools such system securely in a way that scales with your team. um identity. So you can control who who is accessing it and then what what it act what it’s allowed to access.
So you’ve got really deep identity things where you can say you know put policies in place of like where what it can communicate to and why. Just makes it really easy to do that kind of stuff. Question about that. So is that is the identity different than I am or are they related? You can do either. So IM is an identity, right? And then then there’s all sorts of other different ways you can do it. So it doesn’t just support IM. It also supports OOTHQ and okay uh I don’t know off the top of my head what all this supports. I literally just started looking at this Friday. Um it’s got observability built into it. So it’s got uh open telemetry framework already embedded. So when all the calls uh they get logged to Cloudatch, you can see all the inputs that were made, see all the outputs that were made, the conclusions, uh you can track latency, how long it took to track it. Um and then you can log all the stream activity.
Um and then port policy basically policy is doesn’t care what the model thinks, it’s going to force it. So no matter what the model thinks it can do, it’s going to either allow or deny it. And this is at the harness level. It’s not like, you know, it’s not going to let the the agent do anything that you don’t want us to do. So then McDonald’s, so there are, if you’re talking about the McDonald’s where they store your prompts, there are I think you have to have at this point like an LM that sits in front of your LLM that it’s a guard. It basically says is this content like really? So, I mean there uh Microsoft sells a service. I can’t I think it’s called Sunuard or something. I don’t know. And then uh I don’t know if AWS has a service yet. I know Digital Ocean does. Um and Digital Ocean has like three or four of them where one of them’s like moderation where like if is this person an actual like forum user, right?
Stuff like that. So, there’s all sorts of like things out there that you probably want to run on your inputs when people are asking questions these days since we’re kind of like surpassing the I guess the honeymoon phase or whatever. So, um okay, we’ll do a demo. So, agent core, I mean it’s the same thing. Just like type agent core up here and it’s got bedrock agent core. Uh this is an Amazon product. Yeah. Yeah. Okay. Got it. So, so I’m just gonna kind of start with the easiest way to get started. And it’s going to create a harness. And what this is is the harness is actually in the cloud. So, instead of it running locally in lane chain or something like that, you can actually launch a virtual harness. And so, like it’s pretty simple. You just go here and say click create. And now it’s creating the harness. And then this is everything you need to get your agent started.
Um, I don’t know how long it’s going to take. Uh, maybe I thought this through. Wait, it went there. And you’re out of money. Yeah. Okay. So, it’s done, right? So, once you’re here, uh, it’s kind of like the same thing. You can select your source like Bedrock, OpenAI. So, you can also do, it’s not limited to Bedrock. You can also do like if you want to connect with OpenAI or Gemini or something like that. In this case, it’s got um Red Rock selected and like URL helpful system. That’s your system prompt. And I’m going to ask it basically the same thing. What is the weather in Huntsville, right? Uh and then we’ll see Alabama because there are other Huntsvilles. Maybe it was thinking it was Huntsville, Texas. And this one will probably deny me because I didn’t enable it, right? So, I think it’s secure by default. um very fast. So, so what this is doing is it’s actually launching a container like a lambda container and then that host the model and all that stuff and so the spin there is a like with lambda you have a spin up time and it’s the same thing with this will it uh have you played around with relaunch of something you’ve already run is it faster after the first time things like that yeah and it told me I was sorry so it may have just been trying
to search and this may have just been the trying to figure out how I can help me right um so I’m going to give it what I’m going to do is come in here give it like access to a web browser. So, there’s built-in tools like uh the web browser and I’m going to come in here and actually it’s in the harness is where I got to put it in. So, I’m going to edit this and I’m going to come down here where it says tools and I’m going to able the browser. Put save and come in here. Go back to the harness where I was launching it. Wait, wait, wait, wait. It was updated. Is that what it says? Okay. Yeah. So, it had to make a change. So, it’s just like lambda or layered. Yeah. So, I’m going to go here, test the harness, and I’m going to come back here and say what is weather hle, right? What is the weather console?
And then do that and then it should maybe be faster this time because it has a tool or I don’t know. Dumb question. But since I’m always curious for the system prompt you put in that’s just a standard system prompt interface. They’re not adding any you can change special enforcement to it like uh if you tell it you know you are only a weather assistant that’s all you can answer. Yeah, you can do that. Um you can see it’s actually it actually shows that this is like accessing it. This is what it put in. Um and there’s it generating the weather. So that was pretty simple I guess. Um so I can take this in the harness. there’s like a way I can invoke it. So, what I’m going to do is take this code if I want to. I’m just going to invoke it like locally and then cloud code will help me make it work because whatever make directory uh oh uh let’s see new file demo.py Pi and then I think I have to like tell Claude to like make it install packages or something.
Let me see Claude and then yes make the script work capital in there right? Oh wait that’s no. Yes. And then so it’ll come in here like do it. There’s um I don’t know what else like there’s uh some things to know. Let’s see. Yeah, this is this one’s actually that if you use the harness stuff, it’s a little bit different than the other stuff I’ll show you. But um this is just the invoke harness. This is what you would do if you’re using the the har the virtual harness in the cloud. Um the session ID is an ID that you give it. So if you’re tracking a session across wherever uh like you you you basically give it the session and you you can follow you can follow it or whatever. Uh let’s see. I just make it ship cabinet. Here we go. I think there’s like um Wait, wait, wait, wait. Where is it? That’s not it. Um, so this needs to be a UID.
Let me like get a random UU ID. Uh, let’s see. It’ll let me. So you can currently so you can call it from code. You can hook it up to where it gets called from a lambda or from an SQS or from a Q source something like that. Mhm. Is there is there a timer thing where you could just set and say just kick it off with no no input at all every minute, two minutes, 10 minutes. Um, so you can use SQS. So like we can create events with those ghost, you know, I don’t know like how would you could uh how would you generate events and you you’d probably do a lambda with a cur or something to that. Yeah. Yeah. You you schedule a lambda to run on a schedule or something to just nudge it. Yeah. Um yeah, it’ll take events from anything, right? So SQS like it can do HTTP, you can do all that. Uh I’m not going to have time to show you all of it, but um at this point it’s 37, but um I do want to show you I do have a test project, but um that it’s kind of goes above and beyond what this one does.
Um it so this is generating the that and then me quit this. What do I have down here? So it this just says, “Hello, how can I help you?” So let me just do a UV run. Let me run uh was it harness demo? Yeah. And this is just using this is using the virtual harness. And so it’s just telling me what I what I can do, right? So that’s that’s pretty much it. This is a command line app. Um that’s not as cool. I So I do have another demo. And this kind of like what what I have is I didn’t use what I did is I wanted to set bring my own harness, right? So, what I have set up here is I’ve got under runtime, I’ve got uh basically a resource there. This is this is uh what I would hook in. So, I’ve got my harness that’s in Wangchain and I’m going to hook into this resource here. Um and let me get rid of this.
Hold on. I explain what this does. Um, so I’ve got this thing called inventory app because I I was trying to like kind of be cyber securityish or whatever. Um, the main things about this is when you’re using your own harness, the the way that you hook it up, you got two things in here. You got uh bedrock agent core app. And so this is one of the includes that you put in here. And then there’s a decorator on the end entry point which indicates which function is going to be called. This is just like lambda, right? Very similar. So with the two calls, you can basically turn any Python script into a harness, right? And uh so can I tell you what this app does? This app uh I have a virtual machine that is running or EC2 instance. It’s running in the cloud right here. Um, and it is if I SSH to it. Um, let me just like SSH right here. I I kind of do like a Docker run or I’ll just Docker LSPS, right?
Docker. Yes, I think. No. Oh, see. Oh, yeah. It’s lagging. Okay. So, Docker PS. So, it’s got like a Docker container that’s running in it, right? And so, I wrote this app that basically comes in here. I gave it the system uh prompt. You’re a middle-aged system man that’s very sarcastic and sassy. Do as right. And in here, it I’m just going to ask for the IP address. And I’ve got a tool that will SSH to it and give me so SSH to IP gather inventory of the system and answer the following questions. What is what is it running in Docker? How is SSH configured? What ports are open? Um finally give me a short rant about the lack of security. Right. So, so, so essentially like if you’re if you say you had a thousand servers, right? Uh, you could just give it a spreadsheet and then run a run like put it in a queue and then just let it go, right?
So, if you if you wanted to and it would just launch a thousand agents and then go and do it, right? And so, that’s kind of like the use case. That’s really the use case for this, right? um when you’re trying to do stuff at scale and I don’t know if you had a lot of targets to go check right um in inventory apparently. Um so what I’ve what I’ve got here is I’ve got a script. Um so this this right here is the is the code. This is already hosted on a service like a Lambda, right? And so I deployed it and it’s really easy. It’s it’s actually probably about the same. You basically do like agent core create deploy and it deploys it. So, uh it’s got like I can just kind of show you the commands really quick, I guess. Let’s see. Inventory app agent core. So, it’s got all these commands. I don’t know if you can read it. You gota wait for the net to catch up.
I didn’t realize it was bad. Um, is the network bad or I don’t know. I’m assuming has it been like this the whole time? I just haven’t noticed. No, it’s been mostly good. So, it’s running on your machine. Yeah, it’s already done. It’s not breached the internet yet. No. Oh, wow. This is This is my afternoon. Yeah. You can see it’s like done here. Oh, wow. Yeah. So, for those on the team’s call, it looks excellent. Yeah. Oh, yeah. That’s That’s all connected. Right. I am so confused right now. Like, it’s like stopped. Can we just like un Can I like stop it and reshare? You can stop and reshare. See if that might do something. That’s That’s how I fix it at work with the customer. It’s great. It’s the same thing as mine. They’re used to it every day.
Yeah. No, that’s exactly. It’s like this is normal. I’m like, “No, wait. This is not normal.” Paul’s like, “This is what I have to deal with every day.” It’s normal for us. Oh my gosh. It’s your new normal. We have one. Yeah. So, something is disconnect from the meeting. Yeah. See if Well, did you disconnect and reconnect? Totally. I can do that. We’ve got a conference call every week with about 80 to 100 people on it where 15 minutes into the call the conference room just disappeared from the call. They don’t know it because they know there’s no indication in the room and everybody else like well it happened again. somebody. Um, and it’s it’s one of those where you have to find the phone of somebody near the room who can then go knock on the door and let them know that hey, the thing is shot or they’ll eventually figure out, hey, we haven’t heard from anybody else in 30 minutes.
That is normal. Wow. Uh, so yeah, we are somewhat broke. did enjoy your prompt. Oh, it’s just wait till it I just want to Yeah, just wait. I was thinking you were complaining about, you know, short ramps. Well, that’s where you could work in your timing thing. Just work in every five minutes. Send a reminder about security in a new way. Uh, I can try to No, I think everybody else online is probably okay. Is it just like bad here? Yeah, it looks like it just it start a little faster now. Is it just yours? I don’t know. Truth. Oh, and my speaker’s muted, too, so I couldn’t hear anything. Anybody else in a I’m unmuted now. Um, the folks online, did that actually come through at all? Yeah. You have an adapter? Oh, your car. Never mind. I don’t have it. Note to self. Uh, what kind of adapter?
HDMI UBC. Yeah. Uh, one moment everybody online. Technical difficulties will be right back with you. Yes. Tom has like here’s my demo. Right. An hour on this. So, I won’t go into details, but I have the only wired connection at my office, which is which is nice sometimes. We ran into something a couple of months ago where one of our developers was able to just stream through all I mean everything worked great for him and then come to find out his VM was hosted with all of the other Windows VMs that nobody ever touches. Yeah. On one he had a whole server to himself pretty much while everybody else was like loading 10 people on one server trying to do things. Yeah, you never know. See if it works. Oh, yep. Okay, I got it. RJ, I can always bring that to death. Oh gosh, I don’t want to see that thing again. Okay. And with no delay lag.
So So there’s this agent core. Let me just do it again. So there’s this agent core app kind of like you had with the Well, I don’t know if this is even going to work if you try to share it. If you try to share your screen. Oh, okay. Well, sorry for those online. Right. Anyway, there’s a command like basically to create a project, you just go agent core create and it walks you through and it’ll ask you if you want to create a new um agent uh agent core app or if you want to create like an MCP. So, it actually has like wizards for MCP and all that stuff. Let you map it to the API and all that kind of stuff. And then when you’re done, you just do like deploy and then it goes and deploys it. Um so, what I’ve got is so it’s it’s deployed in the cloud. Um, I’ve got it set up on in this runtime. Here’s my runtime for it.
And then what I’m going to do is I am going to come in here and execute my script that is basically um just a glorified HTTP client. So, it’s it’s just going to hit it with HTTP. Um, and this could be anything. This could be like a SQS SQS thing or wait, that’s not it. inventory app scripts. Yeah, this is it. Okay. Um, so it just basically goes and connects to invokes the agent runtime and then runs it. And so this is the basically the lambda in the sky of the AI AI lambda. Uh, and then let’s see cd scripts. And then, oh, I got to wait. UV run. Yeah, it’s ask me for my IP address. Uh, I’m going to give it the IP address of this, which is the the thing in here. And then, uh, this should, if I’m lucky. And so what it’s going to do is it’s going to go connect and try to connect to it via SSH and log in and then it’s going to just gather information about you know and it should give me a lot of stuff when it’s done.
It takes about like a minute usually in a real model. Is it uh I use cloud use cloud to do it. Um just because claude’s a little bit better at this kind of stuff. Um you use too is really good. Um, uh, yeah, IQ is going to be about a tenth of a cost. So, keep in mind if you do this on like a thousand machines, you’re looking at maybe a dime a machine if you’re using Sonic. And then, uh, probably with HighQ, you’re probably looking at like a tenth of that. So, you’re like a penny. So, like for a thousand probably be like, you know, $10, right? Uh, so it is kind of expensive. I don’t know why it’s taking so long. So the main point really I think would be this is more I see this almost as a way instead of writing a lambda that you could hardcode a script to go do some of these things. Now you’ve got the ability to nearly run an agent like as easy as it is to run a lambda which also gives you the flexibility.
So, if an agent logs into something and it’s not set up exactly like your script would have had it, it’s able to, you know, it’s got some flexibility and some um I don’t know what the right word to say there. Um I think the fact that it has the identity stuff hooked in so closely and it has OTL out of the box. I mean, you get a lot of your not your ML ops level stuff, but something approaching that if you wanted to maintain this as a service. Yeah. Deep side. Deep side. short but that looked pretty long inventory for it. So what’s running in docker just a simple engine xd container because why not expose it in the lap of the world. Um how sassy it is. At least it’s not running privilege so there’s a microscopic win. How is SSH configured? At least you’re not completely insane. Let me walk you through this mix bag. The good permit login. No. So great. I didn’t use root login allow that.
Um password authentication is actually disabled in the config but overrides in main config. Uh password authentication yes in main config. Thank goodness. Uh PD interactive authentication yes still enabled. exit of Ford because apparently me and Ford doies on a server in 2026 went over to the internet. So great. Um
Anyway, um yeah, this is so what you could do with this is you could dump this to S3. So if you wanted it in like a JSON format, you say give it a JSON format, do it. Um you can uh you know dump it to a database with a tool. So you could you could program a tool to say do an insert query and then that would you know you don’t give it direct access but it’ll you do it you could also do it to like a like a Dynamo DB or something like that. And so you just use it to collect data um or research project something like that just at scale, right?
And um you could also use it to create a chatbot. So if that’s what you’re going for and you need thousands of users, you know, it’s got all the plumbing, it’s got the session IDs and all that kind of stuff that let you build that. Um what else? I mean pretty much I don’t know the question the question I have like with the the cloud orchestrating stuff you know it’s kind of questionable you know you’re looking at you got claude that’s got its framework got codeex that’s got its framework so they they both have orchestration frameworks they’re kind of like this right and they’re they’re both building them and so AWS is just another ecosystem that like the only the only advantage of doing this now is instead of cloud or instead of codec is that you can switch the models out, right? So potentially you could save money, I guess, in a way. So it use a cheaper model. Uh if a task doesn’t need a expensive model, you can go switch it. Um but then you’re locked in the AWS ecosystem, so I don’t know which is worse, right?
Um anyway, that’s pretty much all I got. Um there’s it also comes with like a I’m sorry, it also comes with a dev environment. It’s pretty sweet. So um what what this lets you do is uh you can develop your app locally and then before you send it off to the lambda. So the the same kind of thing where it’s like test prompt do that and then it’ll go and do it wait oh oh I forgot to set the I got to set the region apparently. Yeah. What was that IP address? I forgot it. But yeah, you can do the same thing with this. You just do it in your local prompt and this this does it and it actually gives you a trace of everything that’s happening. Uh you can see what it’s doing. Um you can see um all of the stuff like this is a really nice developer. This would be a really good way if you’re teaching people how to build and work with agents and stuff as a quick here’s a thing.
It’s cheap enough if you’re using something you know throw haiku at it or something just to play around. Yeah, you get 100 you get 150 bucks for just signing up for AWS right go play. Um yeah that little thing has the resources traces and memories right there. That’s cool. It’s got the same stuff in AWS too. It’s got the same stuff in AWS. Oh, sure. But I haven’t seen, you know, something that’s not AWS with a UI like this. Right. So, if you I was trying to teach somebody like here’s why you want to trace agents. Yeah. Here’s, you know, instead of popping it over to lane views or whatever it is and this app and that app, there’s enough that you guys about that I still haven’t done that signing up for this just to So, this this actually said 11 tool calls. You can expand it. You can see it ran SSH command. This is what it sent over. So, docker ps- a.
This is what I got back. This is the data I got back. Um, it ran uh it could connect. It could connect could connect to Docker. So, it’s pseudoed. That’s fun. Um, it looked at the SSHD config. Um, so you can it, you know, that’s what one of the things that it did. Um, here’s the output from that. So, you can kind of see. So you can use this like if you knew you had a pseudo you could go put in in your prompt you could go fix that and save yourself some money because at scale if you have to do a cool call that’s going to be extra cost right if you’re looking at millions of calls that could add up real quick and so you could kind of see in here okay you need a pseudo so then you go tell it in your prompt go pseudo right and then it would save a call which is if you know thousands of agents I could add up real quick but yeah so it’s yeah this is just the observability Anyway, that’s all I got.
Sorry for going over. Yeah. Anybody else have questions or anything? Thanks for the extra pickiness in response for us. Yeah, I Yeah, I built this in an hour. So, um I’ve done Sparky, but I haven’t done Snap Sask. Yeah, I actually write all of my design. So actually actually all my design works I do at work actually say pretend you’re a middle-aged like system in and that’s how I start everything like and it usually gives you the best like like it looks it’s like it’s like actually wrote it instead of like you know being able to tell that wrote it. Well, the the trace part. So, let’s say you had a chatbot that’s open and all that kind of stuff and you got a user that just complained about getting obscene stuff back in a chat, you know, and you can actually have the trace of what happened, what actually was the input, what what guard rails weren’t there or who knows. So yeah, I would say it’s more about looking at user input and then seeing what people are asking um to see like how people are using your bot and understand um like what one of the things we found is like people were asking our chatbot for features and so we put in a tool to detect when somebody was asking for a feature and then now when somebody asks for a feature that it doesn’t have it goes and
subs the chat Cool. Nice. Well, I’ve just got another email that my API experienced an error. So, right now I’ve got Cloudatch. I’ve got a filter on there that’s routing errors to my email address. I could just as easily spin up a code agent built on the same model that I’ve got in in Claw Code and route it to there and have it go ahead fix create a pull request. not yolo enough to just have it go in without me looking at it. You can trigger it off a log, right? I mean, that’s what it’s it’s already triggering an email to me. You could easily trigger an SQ I could route it to an SQS to an agent. Yeah. And then you just wait, you exit, do the S deep dive, and then Right. Yeah. And you can sh Yeah. hybrid situations where you want some local GPU that you have access to, right? And the cloud at the same time. So like, hey, I don’t have a lot of hits, so I’m just going to hit it a local, but I I need to scale suddenly and now I need to redirect the cloud.
Is that So I might do that with mine. So you can if you have a router. Yep. So So you have, you know, your VLM or whatever it is. Yep. And then you have a router that sits in front of it that serves as a load balancer. And so actually I so I run a cluster with models to different models and sometimes I’ll have like a node not doing anything and I’ll just say you run this model that I want extra of. And so once it saturates you know whatever my concurrency limit is it’ll start routing things over to this one. Maybe just as easily also have that route to cloud model you know whatever it is. I don’t think in bedrock you can do that. Um oh well I mean if you you own engine X you own how that local and then completely so I can have you know there’s 38 you know 100 whatever it is that is my router entrance and whatever is doing that secondary route it can be whatever you want you know it could be a different model entirely it could be a different a harness it could be something that returns it would be it would be interesting to see you can do that with it I mean at that point you’re just using it for preservability and load balancing
yeah Um, yeah. You know, I think the coolest thing about AWS is that if you’ve ever if you’ve ever written something or used a library to do something, AWS has that capability somewhere. You know what I mean? Like for example, load balancing in AWS itself is totally possible. The tricky things with AWS is figuring out what parts are free and what parts they charge you for. Um, because it’s like the stuff I was doing with Fargate. Well, I had to have a VPN. That’s great. Well, I need an external internet gateway. Great. Cool. Add that in. Oh, crap. That’s $20. That’s a flat thing. Oh, I need a load balancing thing. Well, that’s a You know, there’s C. And it’s it’s weird because half of it is just pure free stuff that you create, do whatever you want and where they charge sometimes isn’t where you think they would. It’s very weird. It’ll be like a feature off of like the thing that’s free.
It’s like, oh, you want a firewall on that? Okay, here’s Right. And the answer is no, I don’t. I’m just put in an image that I’ve already built a firewall. You know what I mean? I can do that, but it’s it’s so weird sometimes on the uh if you hit your uh where I found it the most was looking at my bill at the and it’ll break out exactly where you know here’s the 12 services you’re using that you didn’t realize you were paying whatever and looking at it like wait a minute I have an $8 charge this month for what? Yeah. And then you figure how to back that out. But but it is after you hit the scale that you need where AWS is required. You know what I mean? It’s I do think it’s cost effective after you hit a certain level. No, it’s not. It’s the opposite. It’s cost effective until you hit a certain level. Careful with those EC2 servers by the way because they run constant and you you know it’s an operating system, right?
So even it’s always running. Even something like agent core at some point will cost There’s there’s a point in time where it makes sense to go out and then go I think it’s a mid we have we actually have this problem that’s why right I you can do like millions of calls like a minute right I understand that but before you got to that it was cost effective somewhere and then you keep getting lower then it’s not cost you know I mean it’s it’s cheaper to spin your own thing initially until you get a bunch of users and then you figure oh crap I need AWS and you go there and then that scales up until oh crap I could there’s hoping they’ve got you locked in right that time and all of our all of our dev is in AWS because it’s more cost effective to do it that way yes then to have like waste people’s time with VMs and stuff right and so it’s more and then anything new that we’re doing but AWS and then we always try to make sure that if we do hit that threshold that we can data so but yeah at scale is the there’s a point like and the scale that we run it’s our bill is actually cheap I believe but it’s not even it’s like I think I think we end up paying like $250 a month so yeah it blows my mind but yeah but we have 12 data but you’re also doing a lot we have 12 data centers and all the networks that go to that don’t even cost that right yeah cool well I’m gonna end the meeting online so for those that hopped on. Thanks for sticking with us through some network fun and whatnot. Um and we will see you next time.