AI-powered-Discord-Chatbot

Building Artemis (our AI-powered Discord chatbot): Choosing an Agent Harness

Sorry – there was no video captured for this meetup. The notes and transcript from Gemini are below.

Aug 19, 2026

Building Artemis (our AI-powered Discord chatbot): Choosing an Agent Harness

Invited J. Langley showalter.david@gmail.com

Attachments Building Artemis (our AI-powered Discord chatbot): Choosing an Agent Harness

Meeting records Transcript 

Summary

Project Artemis initialization focused on framework selection and security architecture with strategies for robust AI deployment.

Project Artemis Architectural Planning
The project will prioritize a harness-based development approach for chatbot management. Decisions finalized the use of SQLite for logging and committed to implementing transcription support for documentation.

Security and Infrastructure Standards
Security protocols will mandate Docker sandboxes or microVMs for isolation, avoiding local desktop execution. Team members agreed on prioritizing cost-effective local models like Llama and DeepSeek.

Quality Assurance and Documentation
Development will utilize test-driven workflows to mitigate AI-generated code issues. Future phases include using automated tools to generate technical specifications from meeting transcripts for improved knowledge retention.

Decisions

Aligned

  • Initial Artemis chatbot configuration set The project will operate on a single Discord server, restricted to Matt’s user account initially, with separate conversation sessions maintained for DMs and guild chats.
  • Bot security and data storage strategy All secrets will be kept outside the repository, and the bot will be deployed in a secure, isolated environment (excluding personal desktops) with logs stored in a SQLite database.
  • Technical stack and harness adoption The development team will use an AI harness rather than a framework, leveraging DeepSeek Flash and Llama to facilitate initial development.
  • Project documentation process established The team decided to prioritize the creation of a design document as the first step of the project.

We’ve updated the Decisions section using your feedback.

Let us know what you think: Helpful or Not Helpful

Next steps

  • unchecked[Matt Brooks] Write Code: Develop the initial Artemis chatbot code with a Discord interface.
  • unchecked[The group] Select Harness: Identify and select an appropriate agent harness or framework for the project.
  • unchecked[The group] Implement Permissions: Implement user-based permissions for the bot to control access within Discord channels.
  • unchecked[The group] Secure Runtime: Establish a secure runtime environment using either a dedicated virtual machine or Docker sandbox for the bot.
  • unchecked[Matt Brooks] Create Design Doc: Draft a technical specification documenting how the system works and include mermaid diagrams to visualize interactions.
  • unchecked[The group] Add Transcription: Implement support for meeting transcription to facilitate the documentation of session logs.
  • unchecked[Matt Brooks] Provide Llama Access: Share the container and access key required for the initial Llama implementation.
  • unchecked[The group] Join Discord: Join the project Discord channel to coordinate and follow the project building process.

Details

Did the screenshots in this section make your notes better or worse?

  • Project Artemis Overview: Matt Brooks introduced the “Artemis” chatbot, a community-driven project for Discord designed to serve as an assistant, noting that the primary goal is experimentation and learning rather than immediate practical utility (00:04:59). The initial requirements for the bot include connecting to a single Discord server, supporting both direct messages and guild group chats with isolated sessions, and implementing a basic “ping” command (00:05:59). Matt Brooks intends to be the sole user during the initial phase to maintain control over token usage and security, with future plans to expand access to other users (00:06:57) (00:13:10).
  • Technical Architecture and Logging: To maintain security, the team agreed that no secrets will be kept in the code repository. Conversations will be stored in an SQLite database, and all logs, including private messages, will be accessible to authorized operators for diagnostic purposes (00:09:48). The project plans to utilize Llama and DeepSeek models initially due to their cost-effectiveness and performance (00:08:51) (00:14:21).
  • Project Roadmap: The project will follow a sequence where requirements and documentation are established before any code is written. The development steps include assigning a “home” for the bot, performing a status check, ensuring the bot can remember conversation sessions across reboots, and establishing a “human-in-the-loop” mechanism for updates to ensure safe modifications (00:10:59).
  • Access Control and Hosting: Matt Brooks emphasized that once the bot is functional, they will work on removing the initial restriction that limits usage to one person. The implementation of user-based permissions will be a priority after the initial setup to manage bot interactions within channels (00:13:10) (00:16:24). Regarding hosting, the group discussed options like AWS but stressed the need for a solution that allows the community to run the bot locally for testing purposes (00:14:21) (00:51:33).
  • Development Approach – Framework vs. Harness: Matt Brooks outlined the two main approaches to building the agent: a custom framework (a “box of parts”) which offers maximum control but requires manual management of memory, context, and error handling, or a pre-built harness (a fully assembled “loop”) which includes built-in execution engines, context handling, and retries (00:17:33). The group leaned toward using a harness to reduce the effort required for manual loop construction (00:19:29).
  • Harness Trade-offs: The group discussed that while harnesses are “slower” and more resource-intensive due to multi-step agentic reasoning compared to the robotic, single-response nature of a basic framework, they are better suited for complex tasks like research and parallel sub-agent workflows (00:22:12). Participants suggested implementing web-based links to provide transparency, allowing users to observe the bot’s reasoning process and traces (00:24:54).
  • Evaluation of Harness Options: Matt Brooks reviewed several harness contenders, noting that Pi is an “entry-level” option with many plugins but carries security risks regarding file system access (00:29:17). Other options included Codeex, which is tightly aligned with the Open AI ecosystem; OpenCode, which is open-source; Deep Agents, which provides strong capabilities for human-in-the-loop pauses; and Cortis, which is a modular, new, and web-focused option (00:30:14).
  • Sandboxing and Security: Given the potential for unauthorized actions, Matt Brooks argued against running agents on local desktops and proposed using dedicated Virtual Machines or Docker sandboxes for isolation (00:39:20). The discussion highlighted the effectiveness of microVMs, such as Firecracker—the infrastructure behind AWS Lambda—due to their security, speed, and ability to enforce network restrictions, although they are limited to Linux environments (00:45:51).
  • Transcription Support: RCD and Matt Brooks discussed the need for better documentation of the meetings and project development. They formally decided to pursue transcription support as “issue number two” to ensure that sessions are recorded and archived for future reference (00:54:36) (00:57:41).
  • Community Knowledge Base Construction: Matt Brooks proposed implementing a system similar to a Discord community they observed, which focuses on stable diffusion and Comfy UI. They suggested aggregating transcript data into a searchable database that allows group members to use a cloud code skill to query specific technical advice and historical discussions, creating a centralized “hive mind” of their meeting content (00:59:13).
  • Target Audience and Networking: Matt Brooks discussed the evolution of their meetings since 2018, noting that while the sessions began with a focus on niche, technical topics, they have become more accessible over time. The group compared their format to business incubator environments and discussed the value of lowering the barrier to entry for new participants. Matt Brooks also mentioned Microsoft Copilot as a preferred agent for business-centric applications (01:00:27).
  • AI Model Costs and Hardware Infrastructure: Matt Brooks emphasized the importance of understanding the underlying costs of AI projects, noting that cloud-based services like Cloud Code and Codeex are often subsidized by approximately 10x their actual costs. To move beyond these limitations, they encouraged the group to experiment with local models like Llama, DeepSeek, and Qwen 3.5. Matt Brooks highlighted that deploying large models, such as Fable, requires significant hardware investments, estimating a cost of roughly $1.2 million for a two-node system, while smaller models like Gemma can operate on consumer-grade hardware (01:04:33).
  • Local AI Deployment and Security: Matt Brooks demonstrated a local model running on their laptop, noting the constraints of using shared memory for execution. Regarding security, they recommended using a Docker Sandbox environment for managing local deployments. Following the demonstration, the group transitioned to personal introductions, with Matt Brooks identifying as a software engineer and Melissa, a data scientist, sharing their experience working on projects for the Air Force and Space Force (01:09:23).
  • Participant Introductions: During the introduction segment, several attendees described their professional backgrounds: John Bman detailed their work with automation and system improvements; Charlie noted their experience in cyber security and robotics; Josh mentioned their role in managing internet infrastructure; and Taylor discussed their transition from an IT role at a health insurance organization to a new position focused on AI integration (01:12:58).
  • Business AI Adoption and Regional Dynamics: Taylor described their company’s initiative to roll out an AI advisory practice, focusing on foundational structures and future agent deployment for business processes. Taylor also expressed an interest in balancing technical knowledge with business-centric perspectives often found at Birmingham networking events. Matt Brooks contrasted the cultural dynamic between Birmingham and Huntsville, describing Huntsville as a hub for “doers” who prioritize implementation over simple discussion (01:15:40).
  • Multi-Agent Workflows and Project Documentation: The group discussed the levels of AI adoption, distinguishing between simple tools like ChatGPT and more complex, multi-agent workflows. Matt Brooks outlined a plan to utilize Codeex to generate a design document and mermaid documentation from their meeting transcripts, establishing formal technical specifications for future projects. They also briefly discussed the importance of file provenance and the use of SHA-256 for validation (01:19:06).
  • Managing AI Output Quality and Defensive Behaviors: Matt Brooks shared strategies for mitigating “slop” in AI-generated code, specifically by enforcing strict test-driven development to ensure the model focuses on passing defined tests rather than generating unnecessary content. They explained that AI models often exhibit defensive, “traumatized” behaviors when scrutinized by a second verification model, which is a necessary step for ensuring code validity. To improve outputs, Matt Brooks recommended using static analysis to remove extraneous test configurations and using secondary models to reword AI output for better quality (01:22:41).

Aug 19, 2026

Building Artemis (our AI-powered Discord chatbot): Choosing an Agent Harness – Transcript

00:00:00

Matt Brooks: Hello. All right, we’re it’s probably time now. All right, so g start with this.

00:04:59

Matt Brooks: I mean, so kind of so I guess we’re I guess I’m going to give an introduction, I guess. So this came was like an idea where we’re like, oh, we got a new chat room or a new Discord server and we’re like, you know, uh can’t remember who said, hey, let’s put a chatbot in there. I wanted I wrote a chatbot like a long time ago and then there’s like issues with me bring it in. So I was just like, well, why don’t we just do one? Why don’t we just write our own? And there was like some discussion around names and then some other stuff and we came up with like just naming it Artemis and then doing like a presentation and kind of get bootstraps and then kind of make it a joint project cuz there’s a whole site of like I know if if you’ve seen case there’s a whole set of case you guys don’t know and you’re about to find out once the stuff gets here that about like and and I think I don’t know if any of of you guys have used any of the Claude Slack stuff but it’s kind of like kind of like this similar to like how it’ll work.

00:05:59

Matt Brooks: But but basically once we get the idea is once we get in the chat room it’ll build itself, right? So, um, so kind of the main thing here is just trying to get like the basics down to like say, okay, what’s the bare minimum we need to get this thing in Discord and then so we can start building or or whatever, you know, just to get it on, right? And so some of the things I kind of came up with is uh yeah, also we’re building it because it’s fun and we just want to learn stuff, right? Um, it actually being useful is not the point, you know? So it’s it’s going to be something for us to experiment with uh and just kind of do whatever, right? Um so first just got to figure out what we’re going to build yet. No code, just requirements. Uh I I’ve made a an attempt to just kind of initially write some like initially operate one Discord server. We’re just going to do one. No more uh support conversations both direct messages and guild group chats.

00:06:57

Matt Brooks: Like the group chats are called guild chats. I I think that’s from the the beginnings of Discord. So it was like I guess guild chat or whatever cuz it’s a gamer centric thing. Um anyway uh basically provide a ping command. Discord has commands that you can do slash commands and so we’ll just make one called ping and it’ll pong, right? Um yeah, exactly. Pong. Uh initially only allow Discord user me because like I don’t want you guys to spend my tokens unless I want, you know, until I put some scapeguards around it. And I’m kind of gonna be a dictator for a while until I can get this thing boot strapped to like it to a place where it’s safe because it’s gonna be crazy at first. Um, silign normal chat messages from another user for now. Um, generate conversation response using an AI model. Maintain a separate conversation for each direct message and group chat. So this means every guild chat will be a session. I guess you can think of that.

00:07:54

Matt Brooks: And then every message will be a session and they will not mix. Got it. So, so Artemis will not be aware of anything you directly message it, but we can add some of that stuff later, but gets kind of complicated, but each individual direct message is a session. Each user has their own session. Yeah. So, group chat will have a session. So, if you put it in multiple chat rooms, it will have different personalities. It actually will get a different personality based upon who’s interacting with it. And it’s it’s it’s very weird. There’s actually like places I like to like with case there’s like other places I like to interact with it because it just has a better personality. And and also uh like I can set the personality in case which is another thing we’ll probably end up doing where I can say you’re a um like you’re a cowboy or something and it’ll go yee-haw after everything. Um but then you can it’ll randomly select them too and so like it won’t tell you if you until you guess what he is.

00:08:51

Matt Brooks: It’s kind of interesting. Um, allow administrators to change the AM model and other runtime settings without modifying application code. So, this will allow us to like switch out if we want to test models. Um, for now, we’re going to use a llama. That’s the only technical implementation detail we’re going to have because I’m cheap and and we’ll start with uh Deep Seek Flash because it’s also cheap and and it’s pretty good and and for other complex stuff like we can sub it out. We can we can find other interesting ways for you guys to spend money outside of my bank account. So, Yeah. Yeah. I imagine that we’ll have other people pull their bots up and then it’d be interesting to get them to talk to each other. Uh like this is not going to be a coding bot. It’s obviously not going to be coding, but it can initiate other bots to help it out, right? So, um it’s going to keep all its credentials and secret outside the repository.

00:09:48

Matt Brooks: So, no secrets in the repository. Um it will not be aware of the secrets. So, and we’ll and this is kind of like the security part of it where you know how do you how do you secure how do you secure the bot uh without sharing all the with all the you know without it sending your details to open AI or whatever right um if discord disconnects we want to reconnect um the court application activity errors and any model provided reasoning for diagnostic so log everything conversations so conversations going to be dug so everything will be logged Right? Even private conversations, it’s going to be so y’all know and uh make the logs available to operators. So anybody that’s an operator will see the logs. We will probably end up storing this in SQL light. So it’ll be stored in a database which is going to be interesting. Um so following there will be interesting things. Yeah, if we allow it to join other channels, you can also log the conversations everybody has and then do interesting stuff.

00:10:59

Matt Brooks: And and actually, so speaking of log chat logs and stuff like that, I took this and I took our conversations in Artemis and I merged it together and told me to pull Claude to like make me a hype deck. So, so it started saying, okay, so it started with Artemis the lore. Um I I don’t so so I I think it’s the story of Artemis like the creation story. Um she was born in a group chat for real and the chat could not agree on what she was. Uh then J. and Langley made the repo anyway. David Showalter wanted a ride or die. I think you said you wanted your personal assistant, right? So, apparently that’s you want to ride or die. Um, I said, “No, let’s keep our community going. Uh, we just want to book. So, uh, receipts first, code later. So, I think she’s talking about like the or talking about the requirements. We’re going to do that, then we’re going to code later. Um, step one, give her a home, then a pulse check.

00:12:14

Matt Brooks: Uh, and then only talks to one person, obviously me for now, and then everyone else gets led on red. Uh, and then she starts remembering. So I think this is where it’s talking about logging and then uh surviving the reboot. So when you log like when you reboot it the session or restart it, it’ll pull the session from memory and stuff like that. Uh and she does not gossip. So that’s keeping the session separate and new model who uh and she gets back up on. So I guess it’s restarts and she’s up and answering. She listens. She remembers. She keeps secrets. She has receipts. She’s secure isn’t a vibe. So, it’s taking the chat to say whatever. So, secure isn’t a vibe, but it’s a test suite. So, focusing on security. She can glow up. So, be able to make changes um but not unsupervised. So, there will be some kind of human in the loop type of thing.

00:13:10

Matt Brooks: And then this is the bare minimum respectfully. And so, pull up to the issue and cook with us. So, that’s that’s the story of Artemis. So, uh, I didn’t really know what to present, so that’s what I present. Um, so some of the other things. Yeah. When do you want? Yeah, I just it it’ll probably be today or tomorrow. Um, we just got to get it on Discord first. So, I haven’t written a single line of code or anything. So, we’ll we’ll probably start the the I I’ll probably start with codeex and then just write like write it out and then get it on Discord and then and get it to work and then once we once I get it to work then we’ll kind of see about removing that safeguard where it’s only me. Um because I’ve got to because uh when when it probably gets depending on the harness and I don’t know which harnesses are out there. We’re going to talk about them. Um, so, uh, and we’ll make a decision kind of like which one we want to experiment with, which one we’re going to use.

00:14:21

Matt Brooks: Um, and I’m not experienced with some of the others. So, I just want to make sure that it’s not going to eat all my tokens and do stuff, right? So, pretty what’s going to be critical is making it so it’s easy for other people to work on it. Oh, yeah. Yeah. A lot of us have stupid money. I could, you know, I throw 25 bucks, you know, if I’m playing with, you know, something like that. You know, that’s a lot of that stuff, too. you’re talking about, you know, logs, key management, you know, we can throw it on GCS. Uh, I think Jay has AWS stuff already for Unbel AI. I don’t care you know what it is, but you know, one of those and we might be able to use that rock, you know. I think DeepC 4 is a great idea for start. That’s what I mean. It’s so even better, too. Yeah. Yeah. So, I think it’s one of it’s the best model for chat in my opinion.

00:15:20

Matt Brooks: It’s the best if you’re banking a chat bot. It’s the best model out there. um if you don’t need coding or anything like that. It can code but it’s not good at it um at all. But but it’s for like just a general purpose like I don’t want to I want to use something cheap and um I don’t know why everybody’s not using it. People are still using Open AI and stuff like that for their chat bots and it’s just like you’re insane because it’s what like a quarter of conversation or something. Yeah. Yeah. making sure when it is open. Oh. Oh, okay. Okay. So, opening up for editing, it’s opening open now, right? Um there’s a there’s a there is a public GitHub repository. So, PRs are welcome. What I mean by opening it up is meaning when I put it in the the Discord chat room like a lot of people can do some a lot of evil stuff with it and I just want to make sure that that’s set up first.

00:16:24

Matt Brooks: It’s only going to be temporary when it’s just responding to me. And we could if you want to we could either add other users as well um to start with. That’s too much right now. We’re not even talking about rulebased permissions. It’s only on a channel. You can still direct message it. And realize there’s a bot on this Discord channel over here that has access to go find the Discord channel. Go in it. You’re really putting a big We can do things in Discord like uh this bot center is recruit. We can set you have to be a quote unquote member with these privileges to use any bot. Yeah. I don’t I don’t know how we’re going to set up. I think userbased permissions are going to happen and that might be the first the first thing that we implement after like after we can all do it something, right? So like once it’s up and working then it’s the first the first step is remove me from that being this whole thing, right?

00:17:33

Matt Brooks: Um yeah, so uh so kind of about like kind of the whole reason we’re here, right? Um using an ANA harness. Uh I once again I just use cloud code to make this because I don’t have time. But uh so kind of the question is framework or harness? We can start either way. There’s they’re both kind of viable ways to get there. Um uh so I think it’s just kind of like showing uh saying again she answers questions from the community remembers what said reaches for a tool or plain answer is enough everything below is downstream one decision what runs the lose uh loop so it says surface is discord needs memory and tools context that survives across turns plus real actions not just chat completion so you can tell the element that it’s a lot of words um so always pursuit just use a framework right it’s like I think where most people kind of start. Uh, ask anyone. If you ask anybody, you know, you’ll land here. This is just this is kind of like where everybody kind of says this is where you should go.

00:18:36

Matt Brooks: Um, framework is kind of like a box of parts. So, if we’re looking into like just starting with a bunch of parts and not having anything, I mean, we could spend a lot of time just doing memory management, context management, stuff like that. So it’s really like do we want to spend a lot of effort in that or um you know because we’re going have to manually configure memory context management the loop itself u maximum control but but you know you pay for it all up front you get maximum control so we can tweak it any way we want stuff like that with a with a framework everything’s kind of plug-andplay uh framework a harness comes fully assembled so if you use a harness uh you basically get the loop you get uh execution engine runtime time. Um, you know, it’s end to end kind of stuff. Uh, you get tool calls, context handling, so you get all the memory management. You don’t have to worry about that, especially if we’re just going to have a chat that just keeps feeding into the context.

00:19:29

Matt Brooks: Um, then you know, you don’t have to worry about managing it. It’s just going to manage itself. Uh, it also, yeah, you bring the tools, personality, brings the machinery. So, whatever. Um, so it’s basically two ways. uh write the agent loop yourself. Design memory context add retries error handling by hand. Sub aents are are a project of their own. Um harnesses the loop is the product context management is handled. Retried error handling is included. Sub aents are kind of a first class feature. So uh this is something to to kind of remember. Um and we are talking about coding harnesses. So most of the things that we’re we’re kind of going over today are coding harnesses. I think with the Deep Seek harness that just came out this week, you’re starting to see a pivot away from that, which is just building a general purpose harness that kind of does it, which is kind of interesting to kind of educate. Yeah, we want to have the ability to test about, you know, I’m doing a paper review on trajectory memory sort of saving and compiling with all the agents in September.

00:20:34

Matt Brooks: Wouldn’t it be great had something that we could easily plug and play and say like, “Hey, here’s a here’s a this paper for Artemis.” Yeah. And we can plug it in and say, “Okay, toggle it on here and this is what this looks like with this thing. We’ll leave it on for a week and then if we like it, if you” But I mean, something like that, you know, it’s not a business. This is this is like a tinkerers maker club sort of thing. set of every couple months they can live. Yeah. Be able to test out. Here’s Yeah. What pie looks like, here’s what, you know, using it in deep sea harness looks like. Well, I think if you choose I don’t know. It’s going to be hard to switch out harnesses. I think once you pick one, you got to go. If we’re going to do another one, maybe we can just send code at clot at the current one and say make this work in a different harness.

00:21:24

Matt Brooks: That’s probably our best bet. Then we just have a new bot. Maybe we have uh well you mentioned you thinking bots in a channel or something like that. I think that’s that’s kind of getting past the you know obviously the art of that’s where we’re starting but you know do we how do we make it easy to do you know different things and not get stuck like this is the art of mold you know we are the you know so it’s really easy for somebody to that just wants to do something different to bring that and then show it to the group and be all learning. Mhm. Uh, so what you stop building when you use a harness? I think this kind of repeating itself like skill support, agent loop, retry logic. So, so it’s kind of repeating the same thing, but skill support for they didn’t say that until now. Um, you’re going to have if you use a framework, you’re going have to write that in, right? And there’s not a module for it.

00:22:12

Matt Brooks: Um, so harness ships, most harnesses ship some SDKs like Pi, Codeex, Open Code, Deep Agent, Deep See, Cortis, they all have SDKs. Um, Claude sucks. It is awful. Yeah. So, it’s not that’s why it’s not even on here. Um, but, uh, everything else has kind of like an SDK and they’re different kind of levels. Uh, and we’ll kind of go a little bit into that. Uh, also it changes the vibe when you just have like a a framework. It’s kind of a robotic feel. Uh, when you use a harness, it’s definitely more like cloud code or something else. And if you’re annoyed by cloud code, then maybe you want a framework. But uh, but in my opinion, it just sounds better, I think. Uh, so it’s kind of blasting fat at a framework. So, you know, you kind of wrote it, so it’s kind of predictable what what it’s going to you’re kind of like programming what it’s going to say and how to handle every situation.

00:23:08

Matt Brooks: Uh, answers running quickly, usually in one model call because it’s not a loop. So, so kind it comes out robotic uh templated. Um, you know, if you go to any website, that’s the feel you’ll get, right? So, if you if they have a chatbot, that’s generally the feel you’ll get. Yeah. It’s kind of that lame grappy, you know, sort of area. And it’s just it’s good for the data pipelines. It was good when I needed structured outputs on small models and like I didn’t trust to do stuff, but it just gives such crap. You know, you can’t get it to like keep using tools and you want to do it in a good way. Uh it’s like I’m spending a lot of my time getting things just run with like an open code run attached. Go view stuff. I don’t get paid with view. then bring me back that thing and if I need it to go structured output, I’ll then postprocess that. It just doesn’t feel good.

00:24:00

Matt Brooks: Yeah. I mean, it’s it’s definitely a different feeling. And that’s uh the the the other the con of it is the harness will be slower and more alive. Uh so loop things, it checks itself, and then tries again, right? So a lot you’re not just doing one LM call, you’re doing multiple, and that’s why it’s slower, right? Uh answers take longer, sometimes noticeably longer. Uh tone Toad reads like cloud code in a chat window. um when right when questions need actual work. So like if you wanted to research something you’d probably want to use a loop like so and if we wanted to do like daily tell give us all the new articles and stuff like that and then and then think about what it it’s already told us and and and then you know don’t don’t tell us things we don’t already know you know have some kind of um go find all the news. Yeah. And then and then and then tinker with that and then do alerts like that.

00:24:54

Matt Brooks: Um you know that’s is going to be better. I’m always on response. Yeah, but it is it is a trade-off, right? One thing, you know, especially if we’re doing a thing, we might have to think about how to make that that feel good. That’s that’s what I’ve been trying to figure out is, you know, I have this thing. I send send it off in this UI, which is Discord. Do you know do you send them a little link so you can watch Artemis work or you know whatever it is and like see a little because that’s what you kind of really want to see that stream of stuff but I don’t want to you know discord so you know but have something to like be able to go watch it with those traces or whatever they are then yeah something people do that are the web UI guys that aren’t agent focus um and if nobody wants to be that person then also can do that. Um, no, that wouldn’t do it. No, no, no.

00:26:09

Matt Brooks: I think the link ideas is a great idea. Um, if we could extend Discord to do stuff, it would be great, but we can’t. Maybe we could put a feature request in and see. I mean, you never know. like was like, “Hey, we want to add an LM capability to Discord chat and you know, have it do all the like show us the thinking and reasoning and yeah, you can probably tracing, you know, being able to see the laws of what it was doing with the live feedback.” That’s something I’m playing a lot a lot with with the agents of like what do you show? Like what do I see the reasoning? How do I see it? How do I see that in a do all and you know we’re going to log it all. So it’ll be in a SQL database and then you just, you know, if you want to make it a website, we just ask it to make it a website and then it makes it a website and then You just display it wherever it’s displayed, right?

00:27:01

Matt Brooks: Um, you have the same problem in GitHub, by the way. If you make bots for GitHub, by the way, you make bots for GitHub, you still you don’t you don’t have a like Microsoft doesn’t want you making they want to make the LOM, they don’t want you to make it. So, you don’t have any hooks to make things nice and you know, and you don’t want to piss everybody off with, you know, leaving comments on everything. So, um, you know, the links, the link idea is actually kind of nice and I’ve actually used it. I got a GitHub bot that I wrote and well I didn’t write it. Something else did it wrote itself but uh it did. Um but uh you know it and I used the link and you can go click it and you see the status and there’s like other options and stuff that like if it gets stuck you can go retry or or whatever because they they do get stuck. Um but it it is important to do this stuff because like I didn’t know like when when I was working on case and I was working on uh we call tars which is now called yellow wait yellow um you know having structured output on the end is actually kind of interesting because you can enforce the LM because as time goes on the LM forgets what it’s doing and then it the the context gets wiped or whatever especially you’ve got longunning agent that’s you know thinking and doing and

00:28:18

Matt Brooks: it’ll forget it’s as instructions that you originally gave it. And if you force it to do a structured output, you can read the output and say it’s and basically kind of judge if it wrote the right thing and then tell it it if it didn’t say you wrote the wrong thing, go rewrite it. This is what we’re expecting. And so you basically give it the original instructions that you originally wrote it and then tell it what it’s doing wrong and it fixes it so that it actually kind of does it in a deterministic way. And uh and so just interesting things I would have never known if I hadn’t like experimented with this stuff. Yeah. Um so what we get with the slowness task run long run and then reply uh you get multi-step research uh sub agents in parallel work that outlives a turn. So yeah it’s pretty much it. There’s probably more to kind of recommending harness first then drop down later. So we just ship one on a harness and see what happens.

00:29:17

Matt Brooks: Um this is just to get it up as fast as pos possible. The harnesses are opinionated. That’s one of the things. So a frame the differences really between a framework and a harness is is a framework is less opinionated. Bunch of tools you go make whatever. A harness is going to be opinionated. So picking the right harness actually matters because all they all have different strengths. They all have different weaknesses and and so now it’s you know and you’ll see in a minute. So the contenders right now I’ve got Pi. So Pi’s kind of I don’t know has everybody heard of Pi? Maybe just you. Everybody heard of it. It’s fun. Um so it Pi is basically a coding agent that basically feels like a framework. Uh, so it has a lot of plugins that you use kind of make it work. It’s basically runs in yellow mode all the time. Uh there is no guardrails. It just it just runs and so you just you uh yeah so so safety is a concern because it’s like there is no safeguards.

00:30:14

Matt Brooks: Um but you just get it it’ll do some interesting stuff. Um openclaw is built on this. This is the the thing that makes openclaw work. So that the bare at the if you if you take all the UI away from open claw at the end of the day it’s just pi and and so that’s that’s a that’s a thing to keep in mind um codeex so it’s polishable document um it is aligned tightly to the ecosystem so this is kind of a trade-off like pi basically supports every model known to man uh codeex really only supports open AI there’s some people who have gotten it to work with other things but typically nobody wants to um the file system is always on. So, we can’t turn it off. So, it’s always going to have access to the file system. I don’t know if that’s a good thing or a bad thing for a bot to have access to. Um, so that is a that is a thing. Um, yeah. So, only thing you can do is make it read only.

00:31:08

Matt Brooks: Open code. Uh, it’s open source, I guess. So the runtimes uh it it’s kind of like it’s it’s kind of like pi where it supports every model but um it’s typescript only. Pi’s also type scope only. I’ll take the strength of it’s open source pretty much it. I don’t know if I’d use that in code. Um, yeah. Yeah. So, I would start it this week where the main thing that they’re doing with V2 is changing how plugins and the SDK. Basically, anything that we can be typing into works to be better. Uh, so, but I would suggest this one as being a pretty good option to build in for the sort of things that we’re talking about. It’s also got a really good headless mode, but I think Pi does too. Yeah, Pi doesn’t have that. I guess it does, but server that you can kind of tag it to and work as a web app in front of it. Okay. Uh, deep agents.

00:32:13

Matt Brooks: I haven’t really experimented with this one. Um, this is Lang Chain’s version of a harness. Um, it has actual real safeguards. Uh I would say out of all of them, this one like has the best kind of framework for pausing and waiting for an approval. So if you need like and it doesn’t have to be an a human approval either. It can be an agent approval. Um but but you can basically pause at any point and it’s got an API that hooks into it. Whereas codeex you you know it’s kind of like you either you either run in automatic mode or you you know you can’t really use the SDK to approve something happen. So if you want a human in the loop kind of thing, this is kind of like where I would say is the only the only thing is is Python only. So Yeah, Cordis is the new the new kit on the block. Uh this is we were talking about deep sea caris. Yeah. Uh TypeScript only.

00:33:07

Matt Brooks: There’s a Python SDK, but it’s not really like if you’re writing if you’re writing code for it, you’re writing TypeScript and then the SDK kind of triggers it to do stuff in Python. So it’s not really a not a one for one. You don’t write modules in Python, you write modules in Typescript. So they uh I think it they really are focusing on the web UI. Um it runs headless too, but they’re really focused on the web. I would say it’s to me it I don’t know. It kind of kind of reminds me of their kind of v view on uh claude co-work something like that. But you don’t have to use that. You don’t have to enable the web parts of it or anything like that. Um it’s configured via YAML. So if you like YAML, it’s your thing. Yeah. Anything on anything on what? Not today. Not today. We could write our own. I mean, we could try. Rust.

00:34:08

Matt Brooks: I don’t know. I couldn’t find anything actually. I didn’t actually look. So, uh, Rust is probably a good choice. Rust is a good choice for uh to write any kind of if you’re if you’re starting from scratch. and you want to uh write a project with AI, I think it’s a good choice because of some of the safeguards it has in the language and then kind of forces the uh forces the AI into doing things correctly and stuff like that you’re making tools for the AI to call. only static analysis tools. I don’t know how much it gets from there’s not a lot of good UI for breaks for crust at least not last time but that’d be a good paint but I guess why why are you crying right okay um back to back to Cortis um it’s very modular I don’t know how else how else to put it it’s very very very basic building blocks that you kind of put together to make a a bigger app. It is very I think it’s too new.

00:35:17

Matt Brooks: Oh s***. I think it’s too new to really like I mean maybe want to use the new thing, but I think it’s too new to like really like say today that’s a good choice. Yeah. Um I think give it a give it a couple give it a month or two. I think it’ll be great. Um so what I came with was kind of a list of tables. is kind of like the ones I mean PI’s kind of weakness there is safeguards codeex’s models uh yeah I still said I said probably either pi or deep agents I don’t know recommending one of those but if you want pi if you want javascript you’re saying open code so maybe we consider that it’s fine yeah So With the the PI YOLO mode, we’re gonna want to sandbox the crap out of it. Well, either way you got to sandbox it, right? Yeah. But you can disable most of the stuff in Pi. So like in Pi, you can disable the file system mode.

00:36:11

Matt Brooks: You can disable you pretty much just strip it completely out and then you can also change the system prompt. So really wanting it to be truly pentless. I think you lose I use open code like transitionally both active coding agent and you know other stuff but you don’t really want that with this as much. No you do you make up for it with tools. So so in a chatbot you don’t have the file system access but you make up for it in tools. So, so instead of think of it instead of allowing to write to the file system, you make it write to tools which are very deterministic and have very uh so like this probably the purest way to do it is you know you can only have these inputs you can only input this and you can only output this, right? So it’s not like it’s so all your inputs and outputs are going through a tool. Does that make sense? Yeah. Okay. I think because of that comes out because like you said are engineated.

00:37:10

Matt Brooks: Yeah. Part of an open opinion is how to be a good active coding part which is totally just noise. Yeah. Which this is not need, right? Yeah. So it’s like so so like an example would be um so you still want to give it access to his code, right? You still want to see it to see his code so it can figure out what’s going wrong with it, right? Um so you would give it MCP to GitHub, right? And then it could go pull his code and then create issues and and you know and you do it. But you’re still doing it through kind of like a agent gateway instead of like a uh well actually we’re use tools. We’re not use a agent. I guess it could be an agent gateway. A tool I guess a tool is an agent gateway. We’ll just say it is. So huh. Uh, I I have they’re just like I think they had like 750 pull requests last week.

00:38:08

Matt Brooks: Read a read read a version history for 55 I love those guys. But their their smashes open was great. And that was their benefit was that they weren’t breaking it like the openfall guys do. But I don’t I don’t know that now that they’re moving so fast there’s that whole thing is I mean it’s still running but when they break it it’s not like disaster kind of breaking. Open claw and Hermes is a good choice when it’s only you. Yeah. But we’re throwing in a chat room to the world, right? And so having imagine having Hermes with anybody in the world being able to tell it what to do, right? It’s going to be you’re going to be able to make it do very bad things. Uh and and and so That’s just a security concern. I’m not ready. I’m not ready to do it. Maybe if we could get it one day to where like I think open like like uh uh cloud code and uh and codeex have like a a judge for input and it does a pretty good job and knows when it’s trying to do stuff that it shouldn’t.

00:39:20

Matt Brooks: And uh I’m still wondering if anybody’s managed to break it. say they say that cloud code has not been broken yet been tricked into doing anything but I I just don’t know it is more uh but there is research out there that says AI is a better judge than we are than humans. So we get it right 33% of the time and they get it right 90% of the time. So depends on what you’re doing. Step one is to put it in a safe. Yeah. Environment that can be blown away. It doesn’t have access to anything. And then there’s I think for me it’s there there’s two options. I mean you could put it in a dedicated VM. That’s one. I anything off your desktop. Don’t put it on your desktop. That’s that’s a pretty much a bad no no. Even Open Claw would not suggest it. Um so get it off your desktop. You could put it in a v it own VM where that’s the only thing.

00:40:17

Matt Brooks: Um you could use something like Docker which I think is like the worst of the options. Um because it can break out of Docker really easy and it has root to things. So it has root to the kernel and all sorts of crazy stuff it can do and rewrite memory. Huh. Desktop’s the worst. Docker is better. It is better marginally. Uh I like I like Docker Sandbox though. Docker sandbox has a lot of it’s a microVM so it is in its own like virtual space. It’s hard for it’s harder for it to break out. Uh with Docker sandbox you can control what what networks it can access. So you can say you can only access this site, this site and this site known more and it can’t access them. You can also uh give it credentials. So like say you have a uh GitHub credentials um you give it a credential and it changes it and gives it to the agent and then it keeps the credentials up to date and then like you can use that as a proxy and so docker sandbox has got a lot of good features.

00:41:20

Matt Brooks: It does cost money though like if you do it in the enterprise but um but it’s kind of like a good middle ground is essentially just like yeah I mean that’s so I I always use Kubernetes for but it has all that stuff stacked up so you have like your portfol are are you know connected through a secondary agent so your agent can’t go up the screen to get to them. It only gives exactly, you know, what it has. And you can have it saying, “I give you access to read this repository.” Okay. Hey, you want to make a pull request for the next 10 minutes, you can make a pull request. Okay, so I mean that’s, you know, one of the things you can do. You can obviously you’re talking about logging, so you have things watching on the monitor. So if you have a whole bunch of weird requests coming in, you know, you can all of it credentials, you get shut down or you do whatever it is that you want to do. And Kubernetes is helpful with that, but it’s complex.

00:42:24

Matt Brooks: Yeah, I think but the agents are actually touching real credentials there, right? Okay. So, they’re not actually they’re touching their real ephemeral. Okay. And then it gets converted. Yeah. So, it’s exposed for 10 minutes. Okay. Um, if you’re looking for like a secure door, it’s like you’re looking how do you use secure bot or agents, right? How do you secure it? Um, Door Dash is probably I would say the one of the better places to kind of follow and see what they’re doing. There’s actually, let me see if I can find it. Huh? H, no, it’s called, what was it called? Flux. Um, so Door Dash Flux. Um, the agent platform. You can’t download it. Um, see if I can find it. I don’t know if this is it. No. Um, I’m trying to find the diagram that kind of explains it. Uh, but I’m This is something I’m really kind of focusing on lately as far as like how to do this, right?

00:43:32

Matt Brooks: Uh, where is it? Door Dash Flux. Uh, agent. There’s a blog article. There’s a blog article actually that they wrote. Um they they so so this is how it works. Um you’ve got events. You got events. So So imagine it being able to run on your desktop, right? And so you from your terminal you can you can run it. It launches from that like everything’s a playbook. So like if you’re familiar with Ansible like you write a playbook and you like you just run a command and it launches your agent. In the command it can be like uh the prompts and then maybe telling it what to do but also it you define like what networks it can go to and what uh what applications and the infrastructure it can access. And so the playbook kind of defines here’s what you have access to and then you put it in and then the it controls the agent gateway via MCP servers and then enables basically on um like so this will basically say okay it can only access this this and this gives it a token and then that’s how it accesses the rest of the corporate network.

00:44:51

Matt Brooks: And because it’s in a playbook it’s um deterministic and it’s the agent doesn’t have any control of it. So it then will launch like a sandbox. box they use my uh firecracker but but you could do it in docker sandbox as well but then the most of the work here is in the gateway and then trying to secure it from other things. Does it make sense? So it’s like you just give it like a secure environment and then say you can only get to these networks and you can only get to this these so everything’s done through MCP like uh knowledge bases logs metrics repos so GitHub everything goes through MCP uh their deployments all their PRs and stuff like that and then inside updates reports and then logging gets put back to the to the through the playbook and so it’s something I think interesting to to kind of follow. I hope they release something, but it’s really interesting to read how they’re doing it. Um, and what it does is it’s basically just three primitives where you have a playbook that defines what it what it’s allowed to run, uh, a sandbox and a gateway, right?

00:45:51

Matt Brooks: And you can use that pattern for a lot of different things. You could even uh launch, for example, you could launch cloud code and a sandbox and say, okay, only these networks that my project only needs access to these networks and these MCPs. And then you just do actually use cloud code inside of it. Linux, Mac, Windows. Does it run in all three? Security, you know. I don’t know. But I almost want to say Windows at this point might be more secure because of all the vulnerabilities had and they’ve had to fix. So right now several actually make it autoatically escape. Yeah. So so micro VM I don’t think an agent escaped a microVM yet. So um a lot of people are kind of banking on that as kind of the tech. Um they’re saying containers aren’t good enough. Uh microVMs are the future. Have you all heard of firecracker? Okay, so um firecracker VMs, it’s uh written by Amazon. Oh, there it is right there.

00:47:20

Matt Brooks: Um fasture microVM. So it’s very very tiny VMs like an actual virtual machine and it takes six seconds to boot up. That is the infrastructure that drives Lambda. And so whenever you’re building a lambda, you’re creating something ba b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b based on firecracker and and so they use it for lambda, they use it for uh ECM and they use it for uh what’s the other the other technology that uses it. I cannot remember right now. Fargate. Yep. Yeah. So hey, the only downside to firecracker is it’s Linux only. I think also 64-bit. Oh, but it supports arms. now. So that’s good. But um yeah, written by Amazon. It’s really good. It’s actually really really good software. So but it it lets you do all the stuff like defining network restrictions and um it doesn’t do application restrictions, but definitely network networking stuff.

00:48:16

Matt Brooks: Charging start. I mean they’ve been doing this forever. Yeah. So they’ve been uh firecrackers. Yeah. So, however long Lambda has been Lambda’s been around a long time. So, so this is probably a 15 year old 15 16 year old project at this point. Yeah, I know. They contributed something. It’s great. They’re usually This usually doesn’t how it works, right? Not with them. Um, they actually have done a lot with AI recently with Curo and uh uh what’s the they’re a harness too. I can’t remember. Strands. Uh I don’t strand is kind of like I don’t know it’s mid. So but you can do a lot with a lot with I don’t know I don’t think there’s really any any advantage to use strands over everything else that’s out there but but they have it. It’s out there. The Yeah. So that’s pretty much all I’ve got for that. I don’t know if there’s anything else.

00:49:16

Matt Brooks: Let me look. Um is that it? That is it. So So obviously we don’t have to choose one of these. Um, so I would like to decide, you know, what things are kind of important. Well, just kind of write it down here. What I thought, so what what are you guys wanting want wanting to learn? How about that? Like you wanting to learn. So we could we could try to do something on pi, you could do something on open code, you could do something on deep agents, you could do something on uh Cortis, we could try like what or we could do a framework. It’s like what is does anybody kind of have any opinions on this or Ask for those that have already done this. What do you find? Entry level would be pi for real.

RCD: David,

Matt Brooks: Yeah.

RCD: I can hardly hear you.

Matt Brooks: So um so it’s like use pi to explain all that it’s open and anybody who’s gonna take this will know what that means.

00:50:23

Matt Brooks: No, they wrote their own. Um I think they use microVMs inside of Hermes. Huh? I can’t remember what they but I do think that I think it does. Let’s see. Hermes uh use microVMs. I think they do. No, not micro uses isolated microVMs. Yeah. Oh, wait, wait. Yeah. So they use that kind of like as their that’s how they make security like if you if you uh like whenever uh uh Nvidia was talking about Nemo claw or whatever it was they’re using microVMs which is basically firecracker um so everything’s everybody’s kind of like centralized on microVM kind of being like the secure way to do things um like the most secure way to do things um Docker Docker’s also secure um this really depends on if you guys like if you want to learn microVMs or if you want to learn uh all that kind of stuff. I can So, I’m going to I can host it on my I’ve got a I’ve got Linux boxes. I’ll probably host on mine.

00:51:33

Matt Brooks: Doesn’t really have to be I mean I mean we could we could try it in AWS. We could host in AWS. It kind of gets expensive in AWS, but we can do that. They give out credits to like educational sort of things from their giant sports. Okay, maybe we start simpler than that. Um, so you know being able to things would legitimately be able to take take questions like like today I was like where are we meeting again? I don’t know is this at Deote or that would be great. Yeah, Case kind of does it but it forgot the it did forget the location today. And he did it, but he did not say where it was at. And I was like, you stupid bot. I think that would be such a So I think I think even simpler than that. Yeah. Yeah. So I I would say a different constraint. Uh community must be able to run locally. So if we’re going to use something like AWS or GCP, whatever we build it in must be able to run locally so people can play around with it.

00:53:21

Matt Brooks: Yeah. So that does kind of limit what we select as far as providers. And then it also you kind of wanna Yeah, I can take this and run it on or or open one which is the open AI battle API instead. instead of mo like like why would we need the to the clock. Well, we can excuse me, you could extend it, right? So, like we can make it pluggable so that um we can support multiple models and if somebody wants that feature, then we can just go build it and then they can run whatever they want on it. Like clot cloud is probably the worst because they just don’t want you doing it. So, unless you use the API, they want you to use the API, right? But it’s 10 times more expensive to go do it that way. like and I’ve hooked I’ve hooked case up to it and it was just sucking down your money like like crazy. So I was like, “Okay, let me put $20 in it.” I ask it three questions and it’s like through 15 bucks and I’m just like,

00:54:36

Matt Brooks: “This is dumb.” instructions like you have set up in cloud code or just however this is set up. Yeah. Yeah. Yeah, we could do that. Um but I would say okay to start with we’ll just use a llama and then that’ll be simple and I can provide that key and then and that container and all that and then we can kind of go from there. Um anything else document process I don’t know if you guys transcribe for the calls or anything. Oh, you mean how it was built? Like show the actual building of it or Yeah, just like having notes from these sessions. So like the session dump of the thing. If we use Pi, we could actually probably just dump it to the session and then share it, but like getting pi like getting pi and llama and glm to do this from scratch is probably not going to be fun. Um, we could try. Why not? Um, so we did I missed the question that’s kind of all we just talked about.

00:55:46

Matt Brooks: question on does this play nice isolically or uh a llama you do not so a llama allows you to use a local model um you have to use one of the ones they support uh yeah yeah and and yeah and it’s actually very nice it is very nice so Gemma the only the only problem is if you run it on your laptop like me um it’s very slow if you have the hardware to do. It’s probably very fast and fun. Uh, I’ve tried everything out on it. Um, I think I’ve tried I I think the worst is like anything Facebook has made. Um, it seems to not like follow any instructions, but everything else is pretty good. The only lava lava lava is lava is actually some of the to your point on documentation have a documentation where somebody can say hey if I want to recreate you what’s what’s 32 again okay so create design doc and we could do the design doc first so the first thing we do is just like that’s actually probably a good idea um yeah Um, as far as do we care about the Well, we’re going to be storing all of Well, I mean, so there’s there’s building the bot and then there’s building there there’s there’s the bot itself with the logs, but then there’s also the logs that you use to build it.

00:57:41

Matt Brooks: Are those important? Like keeping the logs that like the session logs from from whatever you use to build it, are those important? And do you want to keep it with it? That’s kind of like maybe the question you’re asking. Or are you want were you just wanting a video or So I I live in Birmingham. Okay. Here for work today and some tomorrow. So I probably wouldn’t be able to come to all of these. I know y’all have the virtual invite, but like if I can you could ask the bot like hey wouldn’t they come over to the Oh yeah, that’s a good idea. Let’s create issue two. Issue number two. Add support. Transcription support. Yeah, we’re gonna put that down as issue number two. So, oh, what am I doing? Uh, okay. What What am I doing? You want to type? I don’t know what you’re talking, but just come over here. You want to talk?

00:59:13

Matt Brooks: No worries. So, there’s this uh um uh Discord group that’s really focused on stable diffusion and they’re like they’re pushing all this sort of stuff like in Comfy UI. Um and the leader of that group named Palm um he stores all the logs for that entire Discord uh network and he loads it up to a Superbase instance uh each day. And so he has it so you can basically ping the hive mind of this discord community and I mean they do thousands of messages you know every half a day on like here’s how I tune this kernel on this sort of thing. Um, and basically he has it so that he has this little sort of thing that he can you can load a skill up into your cloud codeb and say what does everybody say is the way to interact with Miniax 3 to do with it this you know and so we could do something like this we don’t have thousands of messages going you know even something like this where we just got this little you know stupid database and whatever Jay can afford uh that has all of our transcripts and a skill for somebody to come in and load that in. I mean something like that could be super fun and we all we have to do is say hey do this but for us you know that’s Yeah.

01:00:27

Matt Brooks: Okay. Um, so back to the issue. So as so as a as a person who lives in Vanir Yeah. That’s Yeah. They do these like breakout sessions interesting consideration for building. So discuss one of our next topics about signific used to object. Yeah, I I definitely think people will like more businesscentric people they showed up sat down. Yeah, it’s like that. One of we do like being a little bit more towards the nerdy side. I mean, we’ve been doing we’ve been doing this since 2018. So, we were doing this whenever it was super nerdy. Now, it’s just kind of nerdy. the IPC one is which that for the task force. Yeah, the task force. Oh yeah, this is kind of that niche is covered and there’s definitely things for that. Right. You want to be friendly to that but you know we’re happy to go that one level. So the other there’s another group that I I just see it’s a business incubator.

01:02:54

Matt Brooks: Yeah. And so they’re they uh they basically have a bunch of companies and they that they fund and they to go build stuff. Yeah. It’d be like Innovation Depot, right? Yeah. That’s not This is This is the low. Well, it’s it’s going to be a lot easier than you think. Yeah. Like if you get on the Discord, we have a Discord, right? So, join that. Um, and then watch us build it cuz you’ll see. Oh, people get so laid out, right? And the barrier is a lot lower than I think it is and by the you know year later when it’s covered here and I’m doing it still hasn’t the whole site guys oh we can all do it. So even even with that delay you still end up like talking about like here’s how you do Microsoft pilot which sure there is value to you know you know I hate to say All right. So, Copilot is my second favorite agent. Y’all can hate on me, but I’d really like it.

01:04:33

Matt Brooks: I’m interested in building this because I came from IT looked at a construction company. It’s like, hey, I want to be able to reference my historical assets on job site. So, having an agent, it’s great to look at a data lobone stuff to then be like, what’s the money? There there is a need for people who understand how to who understand how to price out a AI models like how the best for the we actually have this problem where I work is some of the projects don’t even get off the ground because they can’t make it cheap enough and um it’s a so that’s that’s kind of the hidden thing that nobody really that’s the thing that nobody really talks about is how much the stuff actually costs and when it’s going to cost that much because the like stuff like cloud code and codeex are being subsidized 10x their costs are so every dollar that you spend they’re spending $10 to make it happen without profit is that what 10x it’s 10x without profit yeah somebody can say hey this and pull up these three prior sessions we had but but once you go down the lama rabbit hole which I suggest you do um you will see kind of you get a better idea of what all models are out there what they’re capable of doing like like get out of the

01:06:20

Matt Brooks: codeex and the cloud code box get out of that and then experiment with something like a llama pi and then see what they’re capable of doing Um because so download Yeah. Yes. literally you download the button and it installs it on your computer. Um and then that’s completely it can be. So some of these things are like cloud models. So you can say uh you can like uh like you can say these are only cloud ones like deepseat that’s only cloud. You can probably get that one on cloud. Is it big? But it’s but it’s cheap. But it’s it’s big but it’s cheap, right? Yeah. So yeah. Uh but some of these you can like like Quinn 35 I think has got local models. So you can run this one locally if you have some. Yeah. But it so you’re talking about but like you can’t like I I I don’t know how many servers like like Fable would be probably like three, right?

01:07:28

Matt Brooks: Like three mot like fully modulated. So like 1.8 trillion. So you’re looking at probably two nodes two nodes just to make just to run the model. So you’re So that’s like a $100,000 each, right? So $200,000 system just to 200 node is about 600,000 600,000 so two so it’s a $1.2 2 million to run Fable and people and they’re giving it away basically. I mean it’s just No, that’s the that’s the upfront cost. That’s the sunk cost. Yeah, that’s the hardware. I mean, you’re never going to recover that. Let’s see what I got. Let’s see what I got. Actually, that 590, which is $4,000 like pretty sonet. I got Jimma. So, I’ve got Jimm on theirs locally. I can just show them. I mean, so Yeah. Clawies main check. set up a virtual machine but not security out there especially because it’s a local it’s a local model. So I like I said tell me the weather.

01:09:23

Matt Brooks: Uh see I live in Hustle Hu N Yeah, I uh I would I would start with Docker Sandbox is probably the easier one. If you want to if you’re if you care about security, you don’t care about security, then just yellow everything. But uh Oh, he does. He can’t. This is a gig. GP, but when I’m working with it on this, I don’t there’s a bunch of options set up. Well, this is cool. Uh, we should probably like um well, I would say like what I mean this this is a local model by the way. So, this is this is pi um running a local model. Um I think and I’m asking it to tell the weather, so maybe it’ll be able to do it. And I don’t it’s kind of like locking my system up, but but uh apparently it’s thinking real hard. But uh yeah, but this is running on my laptop like without a I think it has a GPU, but it’s shared memory. Um so the memory and it’s kind of slow.

01:11:14

Matt Brooks: Um for max can run local models pretty well. Uh, this is an air, so it’s can only run certain ones, but it does a good job like with Gemma, but you can’t really you have to kind of limit what you put it at. But yeah, so fetched the weather for me. It told me that it kind of give you an idea of what what it can do. So although it thinks it’s Saturday, There’s not many of us here. Start us off. Yeah, I’m Matt Brooks. So, I’m soft for engineer, I guess. Uh Uh, so I forget Sangoma, but it’s pretty much it. I’m actually probably one of the non non-government people. Only I may be the only one, right? Mostly commercial. Okay. Yeah. Hi, Melissa. Um I got to this room was mentioned earlier um but I am a data scientist and I’m the reason we’re here too. So um when the meetings are here um but yeah I mostly worked on consultants air force space force all of those kinds of things um the more fun projects we’re applying AI to those problems the less fun projects are just like being in charge of sis admin so it’s a matter of things thank you All you do is just

01:12:58

Matt Brooks: tell people I’m John Bman and I work mostly stuff but that’s mostly automated and making things easier to do things like that. I do that on the side which is mostly just not really shower space. Um so I get a whole integration first and a lot of my work in this because what other people make. Yes, use this outside one is withd So, a lot of basic projects like this might build also builducks. This is a little baby. Charlie, I uh I do it cyber security jobs, but uh I’m mostly doing some uh automation and a little bit of robotics. Josh in this box. I make sure there’s hot and cold running internet. What is hot and cold? What does hot and cold mean? Internet. I’ve never heard that. Oh, got my pants. My name is Taylor. So I work at I just moved over. So I was in IT at Blue Cross operations project. So I was in that main I didn’t want to do that anymore.

01:15:40

Matt Brooks: I don’t know like is a CPA firm by trade. So EMS would have the same business model. So they ruled out AI advisor practice. This is mainly like right now foundational structure prep another client like hey we need training co-pilot that sort of I’m looking at bigger picture of like eventually we’ll be deploying agents in these businesses to help them bank business process whatever I’m trying to learn the foundational concepts around this. We have a sister company under the organization Adus. They’re MSP, but they’re kind of moving into this data AI world helping with cyber security side. I’ll just be more like looking at the business processing potential data. a at this partner like here’s what I’m seeing here. So I go to Burton Mai events, they’re more showing like you know coordin but it’s more like businesscentric audience not tech. So looking forward to trying to keep up with you all virtually. Yeah. Do you do do you work with website that I found? Do do you do you work with Will much?

01:17:36

Matt Brooks: Will Blackidge or you know he’s the guy started Birmingham? Yeah. Uh he’s got he’s got a company called Death Clarity. So I’ve seen him at the sessions but be honest it’s gotten so massive that like Hundreds of people at sessions sponsored. Yeah, I saw a picture of them. I was like that’s that’s I think this is great for they try to do this. Yeah. We get a baby. Yeah. Really? Yeah. We we’ve we’ve always said and this has been a thing that’s going been going on from Birmingham for like in Huntsville for like 20 years like like Birmingham likes to talk about doing things and Huntsville likes doing them. So it’s like we are we are the doers and they are the talkers and it’s nothing to f like has groups that are become members. Yeah. He doesn’t want to serve pizza because then people just you know that we could grow a lot. Yeah. I think it’s turned more into like it’s always been like that though.

01:19:06

Matt Brooks: It’s always been like that. Like Yeah. But the screen type and then J goes out to businesses and here’s here’s the adoption here like I can’t really get to this you’re supposed to pay everybody. But I totally understand where you’re coming from with chat GBT. That’s where my company is right now. Like that’s there’s like levels to this stuff. Like like there’s chat GBT is like level one. That’s where everybody starts. when and the executives of my company cannot think past they they think AI is chatbt and we’re like on level three which is like multi- aent workflows and all this other kind of stuff yeah like yeah they have to charge something but just getting your understanding context having some semblance of the fold structure mark so it knows what’s going on why isn’t it building this excel exactly as I need to I have never I’ve never been to these meetings where I said like a lot of stuff is beyond There’s always the release funding and I’m like, “Oh, So we have like we’ve had where it’s like hey make it generate and and probably the first thing I’m going to do your suggestion is take this I’m literally going to take this link right here dump it into codeex because I’m cheap and say implement this as a design doc and then so the first commit will be a design doc and it’ll kind of design out the technical specifications how

01:21:31

Matt Brooks: everything work and I’ll probably say add a mermaid documentation so it’ll show you how everything interacts and then we’ll use that as a reference going forward. I used to be very very big on open um spec but I don’t think you need it anymore. I think you just tell it to do the design doc and it does it so and it keeps it up to date now. So what it seems like specifically seems to add to Shaw 256 file providence swap every single place every single I’ve never seen it do that checking on what I don’t know it’s got something in it that’s just like this is the file that I I is valid okay you know you know I but I actually have I actually have I mean my agent prompts are pretty pretty crazy though. I mean so maybe I don’t see them because I actually and I don’t know if it’s just like a I don’t know like let me find one super repositories the first time you see you’re like this is pretty good. Yeah. Yeah.

01:22:41

Matt Brooks: Okay. All right. Chill out. So so like a lot of things like like things that like make it follow are strict test driven development. So like I don’t get a lot of the slop because it’s creating tests first and then the only goal is to make them pass, right? So I don’t see any of that stuff I think and then like I don’t know but I’m like basically saying like do the bare minimum to make it work right and so because I it does drive me crazy when it slops it like like that is the worst part about AI. It tries to be extra about everything when you just want to do this one thing. those reporting the verifiable reports they do or basically because that model is is is scrutinizing every single thing it does and so it’s super defensive like they’re trained to pre like defend their code like no no no this is valid this is a sh because if they don’t do that and the verifier says you’re dead I’m going to jol your brain again there’s so much yeah Yeah, James.

01:23:49

Matt Brooks: Yeah, they have a second model. Basically, imagine if while you’re doing your work, every single piece of code that you wrote, you had an entire data center of GPUs. who’s watching you and say, “Hey, is that right? Is that right?” Because it does act very traumatized. But also, it’s like generative, right? So, the whole process is generative, too. So, it’s it’s basically like take take the slop that comes out of catch and it’s just generating that stuff and then it has to have something judging whether or not that’s a good thing. Yeah. That’s what One of my best instruction doesn’t work all the time. It’s still a steel literally what I said the second model down. Now the second model the second model it’s childhood trauma. Yeah. So it’s like mama is in the room but she’s been there for 20 years. That’s the second model that we’re fighting against. So yeah I I have I do a bunch of things. I have actually static analysis things that it’ll read the code and if the test is just testing a config file basically it’s testing that this JSON says that stop. Yeah. Delete that test. I have I have a deepseek 4 if I’m running with codeex I have it read the the thing and just say reword everything so it doesn’t sound like you know yeah my favorite like sound like a middle-aged system then yeah was that what was that part of it oh no yeah I got out to meet some people for dinner. I appreciate youall having me. I’m definitely gonna follow along and join the channel. Yeah, sure. Yeah, reach out anytime. Come over and thanks for joining. Helps you out in I think it I’ll have it in the uh in something. Okay, perfect. Yeah, I’m uh because we’re like just exploring this um within our organization. I think they they started like crazy sandbox. Cross one we go.

Transcription ended after 01:26:40