Transcription provided by Huntsville AI Transcribe
So we covered a bunch of stuff that I forgot to start the recording if you’re watching this later, so apologies. We’re jumping down to April, around the April time frame. Christopher Coleman, who is actually on the call, so this is a little interesting, filed a trademark application for Huntsville AI, and then the same day the filing went in and sent a cease and desist email to myself, and I believe he also hit the AI Innovators of Huntsville group and some others with threats of lawsuits, threats of a lot of other interesting things. We actually had a lawyer as part of our group put that together and give a reply. We haven’t gotten anything from his legal team after that, so typically every time we post something on LinkedIn, there’s a comment that I then have to go delete, usually it’s some kind of a harassment or something like that, so normally I just have to clean up after.
So that started around April. The last interaction I got was a message from Christopher saying that he was abandoning the trademark application and had forwarded this meeting to some other folks. So again, anybody’s welcome to join, this is open for everybody. So if you want to join and you want to, you know, if you want to participate and do all the fun things and be a good community partner, that’s fine. We did a good series on Retrieval Augmented Generation, or RAG, if you will.
We probably spent 10 sessions going over that. It is more of a, gosh, what’s the right word, a naive RAG. It is just straight, you know, grab the stuff, store your pieces in your token and do the search and then provide that item to the, you know, that list to your large language model and, you know, ask questions, get answers, things like that. There’s a lot of other approaches that we skipped, like re-ranking and other stuff like that.
But if you go back and watch those, it will actually, we went through, like, what kind of vector store database you could use, what kind of tokenizer you should pick and how to pick the right tokenizer, how to get it all connected.
Went through Llama CPP, Python, as far as how to put together a full stack.
We ended it with, if you did want to put this all together and host it yourself, what kind of cost are you looking at?
And that was pretty interesting. How much of this is not really the fun parts of the LLM, but I’ve got to find a place to put a vector database, I’ve got to find a place to put a UI, I have to find a place, you know, and then putting all the piping together.
So it turned into, I’m trying to remember what the breakover point was.
We did something to show if you just wanted to use OpenAI’s API and go straight to that, it’s pretty good, it’s really easy, it’s really good to start from testing perspective because it’s, you can do a lot for free on there. And then you got to the point where your breakover point was pretty far in.
You had to have a lot of users making a lot of queries every day.
Before the cost breakover to where you would even want to host it yourself. Unless you work in a lab like me where you don’t have access to other stuff, then you kind of are forced that direction. So that was interesting. We got a lot of good material out of that. We also had Huntsville Business Journal came out and did a story on us for, you know, what we’ve been doing and what kind of impact we’ve had on community. That was really good. So that story is out there. We had a really, really good session with Andrew and I can’t remember, I can’t remember the other attorney from Wobble, Dickinson & Bont.
They came out and talked to us about law and AI. They were talking more about the impact of law on AI, on things like, well, you can’t patent that or you can patent this or you can’t trademark that or you can trademark this, especially the generative era of if it is purely generated, sorry, you’re not going to patent and trademark it, whatever the thing is. But if you take something generated and then you put a physical effort into either changing it or arranging it a certain way, then you can do that. I want next year, we’re probably going to do something that’s actually inverted. I want to know what the impact of AI on law is because we talked to, he was interested in the RAG system we were developing because there’s a couple of domains.
I run into it in DOD.
You definitely run into it in medical. He runs into it in law, as far as I can’t co-locate this data with this data with this data. You don’t want your client’s data for this client to be anywhere near data for that client because these two companies might actually be, they may be your clients now. Later on, they may actually be suing each other or something. So that was interesting. But figuring out what the impact of law on AI, or AI on law is, especially now that you have models that can pass the bar exam, you know, things like it, it just makes things really, really interesting. Also late in August, we’ll probably do this again next year. This was a really interesting thing for those that aren’t in the DOD sphere.
There’s a conference, actually it’s a symposium.
They had to call it a symposium because they can’t call it a conference because that affects people’s travel budgets. So every year in August, there’s a space and missile defense symposium that happens at the Blount Braun Center. Yes, sir. So we normally go, I’m usually there as a, you know, for my job.
This year was the first year they actually had an AI track as part of the papers that they did for that symposium.
Part of what I was able to do as part of the AI International Task Force, they actually had us review all of the submissions to figure out which ones were valid and which ones were just a… One of the things we had to do was basically call anything that’s just, you know, a sales pitch for a product or something like, you know, it’s… Yes, I mean, they have… You can go buy your own booth and set up your own thing and give demonstration for your product. That’s where that, you know, that’s where that goes. They didn’t want all of that being in the middle of all the, you know, paper presentations or things. If you wanted to show how your product works, that might work, you know. So as part of that, we actually got a group from Huntsville AI and met up, you know, I think it was six o’clock, something like that on a Wednesday. And then I had gone through most of the booths that are out there that claim to have anything related to AI, which was about half of them. And then I figured out which ones actually did.
It was interesting to find the flyers that you could tell were definitely created from a generative AI, you know.
I try to remember the key phrase, in a world of blank. So that was fun. So we were able to walk through and actually go stop off at several of the booths there and actually get them to show us what they had and talk to some folks. It was fun. We did a couple of socials. I don’t think we had as many socials this year, just probably because I’m not very good at socials.
So if anybody wants to remind me next year that, hey, we probably ought to do it once a quarter or so, because when we do, we have a lot of people show up.
And it’s a lot, it’s really interesting sitting down and talking to folks and learning what they do and what their interests are, because it’s hard to do that here.
Also this year, we had, Sora was the one thing that dropped this year that I had to basically stop whatever I was doing. And we had to go talk about that because, oh my gosh, there was, up to that point, it was very limited as far as what you can do for generating video and things like that.
And it just blew the top off of it. So everything else this year has been somewhat planned out.
So it’s been a lot better from that perspective. So Sora is the text to video model where you explain what you want in your video, and you can throw that out there and get it cranking. It was interesting to see, they’re finally still trying to figure out how, it was one of the things they put out there from OpenAI, make a splash. It’s great, but you can’t have it, you know? Which they’ve opened up some access to it. I think you can now go create six second videos. No, it’s out. 20 seconds, 1080p on their biggest plan. And then five seconds, I think, on the smaller one.
Is it the $200 a month plan?
Yeah, yeah, for sure, yeah, that’s crazy. Yeah, it’s out. The six second, and there’s, the cool thing about some of those, they make a big, I think sometimes they push things out a little early to make a splash because they may know something’s coming behind them. So they got to hit market first. So it’s kind of interesting.
There’s been a lot of other tools that have come out since then.
So that’s pretty interesting. The other thing we hit, which was Pigstrel, that was also basically the reverse. Hey, let’s take an image and shove it in here. You tell me what’s in the image. And I mean, we went through a lot of examples there.
One of the cool ones was actually looking at a graph of GDP for different European countries and stuff. And actually, you know, based on this graph, give me a list of the top five countries ordered by GDP.
And so they actually had to go figure out, so the countries on the graph were color-coded.
So it had to know, European was green because there was a table up here.
So it went European, green, okay, now find the green boxes.
And in each green box, there’s text in the green box that has the GDP. And it was pretty interesting to rip through that.
There’s been a lot of interesting work going on in image-to-text over the last six months to a year. There’s some things you can do now that you definitely couldn’t do before. We had a, God, Yagiri’s not here. We did part of a session on Hurricane Helene, looking through the response of that and the problems they had run into with getting access to data, access to communications, access, you know, because everything up there was extremely localized. So that was an interesting conversation. And then we did voice-to-text, text-to-voice. We’ve done voice-to-voice, fun with whisper, fun with faster whisper, fun with, I mean, there’s been a lot of interesting things that we’ve talked about there. Another interesting thing that pops towards the end of the year, if you’re involved in Department of Defense stuff with their SBIR topics, they’ve changed the way they do that. They used to drop for, well, it was three sessions a year, and they’ve changed that up now. So now the new topics drop on the first Wednesday of every month. Where before I would put together, I knew when they were gonna drop, they’re all scheduled, I go ahead and schedule a Hunt’s Way Out session, we’ll cover the SBIR topics, we do it three times a year, good deal. Now we have to look at every month and see what’s going on.
So that’s interesting.
Last month, I think there was one that was interesting from an AI perspective, but it was a direct to phase two. So another one of those, if you don’t already have a prototype, that’s, you know, somewhat in use, you can’t even try to go after it. Then the other thing, this was kind of geared up and set up through our interaction with AI Huntsville. They actually had me on a smart place podcast and I got to yammer about what Huntsville AI has been doing for a little bit. That was really nice. Clark Dunn, I don’t know if y’all remember him. He was actually one of the folks that helped organize co-working night when we were meeting downtown at the, what was that? It was near below the radar, maybe next to it. It might’ve been below, below the radar. I don’t know. Do what?
Times building.
No, it was, this was before, this was after that, I think. It was off the homes, kind of in the basement. It was hard to find. You parked next to the basement of the Times building. Oh, was it? You’re right. I was confused. At the Times building, the first place co-working night met was actually the AL.com building. So I’ve got the Times all mixed up.
And that was the Smart Place Podcast. Quick kind of walkthrough on this stuff. Is there anything on the list from the technical perspective, we got the rag part, we got Sora, PixTroll, text to voice, voice to text, you know, stuff like that.
Is there anything on there that you think has a shelf life of less than a year that next year we’ll be doing something totally different?
I know rag is changing by the minute.
It seems like everybody’s either got a notebook or a whatever, and now instead of ChatGPT, I can create my own little place and I can upload these files and I can add that to my session and it’s very similar to a rag. I know other services are doing similar things and they call them different things, but it feels like they’re doing the same thing. Text to speech, that one, there was a lot of interesting stuff that we covered there. I don’t know if anything is blowing the lid off next year where we won’t be using, I mean, you can already generate some maybe context links. Yeah, it’ll just get better, I think.
And there’s also a lot of stuff around, you know, generation for other audio modalities. So like there’s not a lot of good open source music stuff. There’s some basic stuff that’s true speech to speech where it has, it’s like interspersing all the modalities interleaved. So it’s outputting true tokens instead of doing a pipeline of, you know, speech to text to text to text to speech.
So I think that that specific element is gonna get a lot better.
And maybe we don’t need the pipeline so much anymore.
Which one, fixed latency?
Yeah, I do think smaller, smaller, smaller models that are more powerful is something that has been really, really cranking up.
Oh, David was trying to get in and couldn’t make it.
Um. I asked GPT for a picture of something and a remedial came back and said, I can’t do pictures. Yeah. Is that true?
With that one, you’d have to, I don’t know if it’s a different level that you’d have to pay for or, I’m not sure, I don’t know. They’ve got a DALI that you can talk to. I don’t remember if one of the, I don’t know if 4.0 can do it, but you can switch to their DALI model to do images.
Yeah, some of them are not. And it’s really picky about phrasing. Really, really picky. It’s very hard to be like, I can’t draw that.
You’re like, well, I wonder what I said has triggered your, so you change synonyms around it until it draws it.
And you’re like, oh, you didn’t like the word sorb or something. Oh yeah, hold on, let me, oh. It won’t tell you what it doesn’t like. I can’t draw this because of this feature.
It’ll just be like, nope.
So the image I use on this is a generated image from DALI. And my gosh, at how long it took me to get the right prompts, to get the right thing to generate that. Yeah.
Because one of the, I don’t know if they figured out how to put words into images. That’s all that if, oh my gosh. Cause I was trying to have it actually put like words on the page, like it’s transcribing the audio to, and oh my gosh, it was, it’s kind of like generating images with hands.
It’s putting that up was nuts. But yeah, so smaller powerful models are something that we’ve seen.
We’ve been working through with like quantized models for LLMs and getting smaller and smaller things you can run on a CPU.
Not super fast, but possible.
Trying to what, Pixtrol seems to be fairly solid.
I don’t know what, yeah, I don’t really.
The big thing with Pixtrol was it was the first model that it has the flexible sort of encoder.
So it can do resolution because of how it’s splitting up the tokens.
But it’s also the first model that had the self-taught reasoner sort of chain of thought training in it.
Plus the vision model before it was like, they would throw it in with like, you know, llama, llama doesn’t have self-taught reasoning chains and they didn’t have that yet.
So it was kind of the first one that did that. That was really cool. It did some cool stuff like their Siglip encoder and all that. That was cool. But in general, like what you’re saying about smaller models being the thing, that’s the 2025 story for LLMs because of all the test time compute stuff where we’ve kind of unlocked the second level of scaling where you’re not just scaling pre-training where you’re doing all this stuff building these giant data centers, but you can actually just give it more time at compute, let it think for a minute and you will essentially get another 10X times compute that way. And so that incentivizes the smaller models. You know, the 3B, Meta just came out with some research this week where they’ve been able to, with test time compute and llama, be able to get their 3B model to perform over their 7B model because of how they’re doing it.
And so that’s good because it gives us lots more capability to scale.
It also is beneficial for people who have lesser hardware because then maybe I can’t afford an A100 but if I just run my query for 10 minutes, you know, it can go summarize this research paper and do all this fancy stuff. So it’s cool. It’s nice to have levers.
Yeah, it’s a lever thing. Yeah. I know it was last year we were going after a competition for segmenting audio tracks into their various pieces, you know, drums, guitar, vocals, things like that. It wasn’t very long before we figured out that we had a model we were training and we could publish our results like every day to see how closer we are and the board of all these people. And we come across the folks that are in like the top 10, they’re submitting like 20 submissions a day. That was like, well, they are obviously on some kind of academic force power rack or something playing around on the site.
It was like, well, okay, we have no chance. So that was interesting. But with that, the other thing that I think is gonna be big and I really messed up your last name. That’s okay. So Paul is on Discord as P-Bay.
So I’m gonna figure out what the heck.
One of the things for next year, I think is also gonna be big is agents. Agentic AI, it feels really weird to say, just, I don’t know. One of the first things I ever did was autonomous agents back in, this was 2006 timeframe. Autonomous agents, things like that.
But that’s definitely not what we’re talking about here.
A little bit of interest. So do you wanna come up and I will throw the presenter thing over to you. There, let’s see. Figure out how to do things again.
My ghost, yes.
Okay, and feel free to talk about, you know, whether AI or anything else you need to. All right, yeah, so my name is Paul Bayston, not Paul Bay. I joined a P-Bay as a play on eBay. Like, so I just made my username P-Bay. So yeah, I work for, I’m not a sales person, but I work for Boomi AI. If you wanna check out Boomi and see what they do. It’s an integration platform. Integrate’s basically anything with anything. It’s very low code, very easy to use.
We used to be owned by Dell and then got sold to a private investor about four years ago.
And then like, so it started as an integration platform. Now it does APIM, it does data management. It does data management.
Data management, AI does a lot of things. So pretty cool little niche software company.
I’ve worked in IT since I was 12.
So I have my first computer, my grandfather taught me to build computers.
My first computer I ever built was like, I still had a 56K modem, had the old ATI video cards. So I’ve been around a long time.
I’m definitely not the smartest person in this room. I’d say most of you are probably smarter than me, but I know a little bit about a lot of stuff. So I’m currently the AI domain lead at Boomi.
So you would think that means I would be an expert at AI. I’m far, far from it.
I just know all the pieces work together. I know how to, people have ideas. They come to me and say, hey, Paul, can you make this happen? And I will generally figure out a way to make it happen.
So that’s a little bit on me.
So today I’m gonna talk about agents. I said, yeah, I’m not an expert. Some of the stuff I’m gonna present could be super simple. And you guys are gonna be like, wow, this guy’s an idiot. But that’s okay. I’m gonna get more complex. I’m gonna start off easy, get harder as we go because I have five kids. And if you have kids, you know that you don’t explain a concept that’s as big as agents at the toughest level first. You start, take baby steps to kind of get there. So that’s kind of how I teach.
So hopefully that’s okay. Hopefully it doesn’t bore anyone. So I built this on our Boomi platform just because it’s easier to display. I was gonna do this all on Python and then as I was writing it all out, I was like, this is making this look really, really complicated. And honestly, agents are not that complicated at the beginning. They get very complicated, but when you’re just trying to get one started there, they’re not hard at all. Let me refresh just to make sure I’m logged in.
All right, so I wanna start with something that this completely not an agent, but kind of build into it.
So the kind of first program that everybody ever builds is the old hello world.
So I can hide that.
Yeah, there we go. The old hello world thing.
So this is a, like I said, I’m just using Boomi because it’s slow coding so it’s easy to see.
Basically all we’re doing is having a message that displays hello world at the end, right?
So if I run that and it’ll take a few seconds, literally it’s not doing anything else but telling me hello world.
There’s nothing intelligent about this. It doesn’t make any decisions.
It’s not autonomous. It requires me to click something. This is the most traditional way that you get anything.
So like by the output document, we literally just got a response of hello world.
So what we started there, like when we started programming, that was probably like the first iteration of programming.
It was like, you need a direct input to get a direct output.
From there, I’d say like a little bit more modern is now we have what’s called an API, which most probably everyone here is very familiar with.
So an API, you can either call it from an outside source, you can have a listener, you can have a lot of different ways to set up an API.
So this allows a little bit more interactiveness. So like basically this one is just a simple website that I have set up on my web server.
So literally I just go to my local host, to the port, to where the API lives and it enter and it’s like hello world.
But still it’s not anything intelligent, right?
It’s just saying hello world. Like I’m just saying, go to this place and it’s responding with one thing, not anything spectacular. So we can keep kind of building on that, right?
So now I created a greeting API and this would be similar to like your heartfelt statements or case statements being what programming languages you like using.
But this one I just had it saying, we’re gonna send a single sentence or we can send anything to this really. But if you say good night to the API, it’s gonna return good night. If you say anything else, it’s gonna say good day. So this adds some slight intelligence, but not that much. So it’s like if I say, so if I say good night, but it’s gotta be case sensitive, it’s gotta match everything’s gotta be perfect, right? Say good night. Oh, I did it wrong, I added a space again, see?
You’re wrong, respond wrong. It’s gotta be perfect. All right, so just have a great night. I say anything else after that, it doesn’t match anymore, right? So now it’s just gonna say have a good day or a wonderful day. So again, that is a very minimal intelligence. And I believe one of the first ones I came to, you presented like an old school, like it felt statement thing where somebody like tried to make something you could communicate with, it was just tons of if and else statements. Like if you say this, say that. If you like, and to do that is just a mundane task, right, that would take forever. And so that’s where AI finally comes in and does something a little bit useful, right? So now this I’ve taken an API and combined it with an AI to just create like a simple AI greeting.
So now you can say anything to this.
And notice I don’t have a decision shape anymore. It’s not deciding anything at all. Instead, I have a chat GPT prompt.
And it’s just a, if you’ve not done this before, this is it accepts a simple JSON and you have a user system message and a system message.
So the system message is like the instructions you’re giving the GPT of how you want it to operate.
If you’re not familiar, I think most people here are, but I’m just saying, hey, you’re a greeter and you respond to any greeting positively and you try to encourage further conversation. And actually let me change this. I want to change it to that way. It can just adjust to whatever I do. Anyone saying, I’m here to pull it up.
All right. So now if I go into Postman, I can say, yeah, I can say anything and say, you know, what is your name? Actually, I doesn’t have a name, but you can send that over to chat GPT and it will, I’m going to have a bad request because I did something. All right. Well, I’ll tell you what, we’ll do it in here just because I almost spent a lot of time troubleshooting.
Change that to no data.
And we’ll just put it in here. So let’s change this to a static. So you can ask it any old question, how are you today?
Save and so calling from API, I’m just going to manually run.
And it’ll run through, create, take the prompts, send it to chat GPT and then return us a document here.
And the main part, I didn’t parse it out because I didn’t want to make this super complicated. You can parse this JSON out to only return the content, which is really what you’re interested in, in this response, but it says, I’m doing great. Thank you for asking. How about you?
What’s been on your mind lately?
Like, so it gives you like this whole response to encourage conversation. So this is when artificial intelligence actually kind of enters the picture in programming.
Everything else before we’ve been doing is all been if-else statements and nesting them or doing decision trees and doing all these things that take a lot of time.
But now with AI, we don’t have to do that anymore. So something, I’m going to show you something I’ve been working on that at my company that’s getting ready to be released. And this is a support agent. And so right now it’s still got human in the loop.
Whenever we develop AI, if you’re not familiar with that term, basically when you first build an AI, you don’t make it autonomous right away.
You need to put a human in the loop, have it take an action, then have a human review the results several times, several thousand times, really, before you’re like, okay, I’m going to give you the keys and let you do this. So what this does is our support team, I think it’s like 25,000 tickets a year or something like that.
And they’re like, okay, what can we do to get more time for our support employees to do training, to do other work, or help them troubleshoot tickets? Like what can we do to make our life easier?
So we made a support agent.
And basically how I designed this is you can, right now this is human in the loop.
Eventually this will be able to just, when a new ticket comes in, it can automatically do all this.
Right now I have human in the loop, so I have to manually put in the ticket number.
And then like, if I click triage ticket, it’s going to run through, and not do it. I might need to be logged in again.
Might have logged me out worse than there. That’s going to work. Fun with live business. I don’t know.
I thought I logged into everything before I put the computer up here, but I hope it’s opening it down again. Check. It takes about 10 to 15 seconds to stay a little longer than I should.
Oh, there we go.
It worked.
Okay, it’s just being all slow. Okay, so what it does is, like I said, right now it’s all human in the loop.
Eventually it will actually update our Salesforce instance and do all this automatically.
But it looks at the case and it determines what domain or what team it goes to.
So like this case that I put in here, this case number was an integration issue.
So it automatically, eventually once we make it autonomous, we’ll automatically take that case out of the queue and put it in the integration queue so that the right team has the ticket to look at it. And then it has a suggested first response for right now.
We’ll clean this up and I can actually put the username and the support agent’s name and all that in here eventually.
But right now it gives them a thank you for contacting us. We understand this is what the issue. Here’s some steps you can do to fix it. And it does this through like a rag interface. It goes through, looks at our documentation, sends it over to the LLM, says, okay, this is probably how you fix it.
And if it has any further questions, we’ve given it a troubleshooting guide to ask further questions, to try to see if there’s not enough information.
And then it’ll respond.
And so what it does is it immediately, by immediately triaging a ticket, we now meet SLA guaranteed every time when you run forward about missing SLA again.
It’s already in the right queue.
Most likely those, I’d say it’s about 90% accurate, which is actually better than most of our actual engineers do. So it’s not bad at getting the right queue.
And then this is something we’re working on right now, but it also is starting to do a high tap or a basically an initial technical analysis. We’re still working on a few things with this to where it can pull the logs from their local runtime and actually analyze the logs, as well as take the information they provided in the ticket to give the support engineer saying, this is the big picture of their environment. This is what’s going on.
Here’s what we found in the logs.
And like, go ahead and give them a good starting point.
So this has basically become tier one support, like is what it’s doing. So that’s the first step we took. And then I, so to go a little farther, I can also say, let’s say this case is like solved and we don’t have a KB article.
That’s like a big, I think a lot of companies struggle with this is getting knowledge articles created for issues.
So now you don’t have, like once you resolve this case, this particular case has already been solved.
So like I can just click sharing KB article.
It goes through both all the case information, all the conversations that we’ve had with the customer, finds out what the steps were that we used to resolve it and then creates, I gave it a template of how to, how it should look.
And so it creates it with issue cause and the steps to resolution and all that. So like now, if I’m a support user, I can just literally copy and paste this, format it up and post an article 10 seconds.
I don’t have to sit here and like review the case, try to figure out what happened in there.
So I said, all of this right now though, is human in the loop. So this isn’t quite truly an agent, but what will make this an agent is when it does all of this autonomously.
And so what that would look like is when a person opens a Salesforce ticket week, Salesforce has a product called platform events.
So you can set up an application.
It’ll listen, a ticket will come in.
As soon as it comes in, that ticket number will go into this API, which will then feed it all that information to the AI and then reply to the customer, to the initial technical analysis and assign it to the right category.
So now you have literally a support employee as an AI.
So that is, this is a, it’s a very overall, it sounds like a simple agent, but it’s a very fundamental use case for an agent.
A lot of people are doing the same sort of things like chat bots and things like that right now.
So just to kind of give you, and I’ll give you an overview.
So now the ones that we’re showing you is very simple.
This is the support agent and you can see there’s a lot more steps in here of what it’s gonna do, but it basically takes that initial request, passes in a case number.
We go to Salesforce and get all the information related to that case number.
And then like, we do actually have to do a decision because we have rules where AI cannot be used on federal customer data.
So I match if it’s a federal customer, I filtered out return back saying, hey, this is a federal ticket.
I’m not gonna give you any information.
It doesn’t even bother going to the AI. And then if it passes, it comes out here, it gets the rest of the information from the ticket. I map it all down to just the data I actually care about.
I remove PII from it and if it matches normal PII standards, sometimes people put passwords in crap.
And then basically I have like different, so we have our first response, initialized habitat, like special messages for each one.
And then I process that data, that’s where I kind of do like my magic stuff there to get it to work.
And we send it over to open AI to clean it up and send it back. And then we put all the documents together and return it back to the user as a final thing. So that’s the one that we have finished right now that we’ve been working on. Then I think for the next one, I wanna just, I was gonna like draw it out, but I think since we have a whiteboard, I’m gonna do it on there if you don’t mind. It’s okay. Yeah. Can I erase all that?
Ah, thank you sir. All right. I don’t like it, it’s right there. You’re right. All right, so the cool thing we’re working on right now, I think several companies are working on this. It’s kind of like a race to the finish line to see who’s gonna get into market first. But we’re actually working on something. They wanna switch it to the camera. Yeah, let me see if I can do that. Do I need to stop sharing here? I think you stopped sharing. Yeah, I think. And then I’m gonna make the camera. He’s doing it on his own. Yeah, it’s gonna follow you. It’s gonna follow you. Yeah, so if you draw stuff. Draw stuff. Or let me see if I can’t go back. Let me try going back to saying it on something. Yeah, I’ll just like start checking your eyes. Nope, I think I’ll start right here. Okay. All right, here we go.
Awesome, all right.
So we’re building a agent builder and this is an AI agent that will build AI agents.
So that now we’ve even removed the necessity of having an engineer design it.
And that’s what a huge customer has cause you mentioned 2025 agents are a big thing. That’s what pretty much every call I have with a customer they’re like I want AI to just do stuff for me. I don’t wanna figure it out. I don’t wanna have to try to learn how to write models. I don’t have to train stuff. I don’t wanna do all this. I just wanted to say, hey, create a meeting invite at 4.30 and send it to these people.
And it just doesn’t, you know, it’s like, okay.
So it’s like, it sounds easy, but that’s not easy. And there’s a million other use cases, right?
Of things that people want AI to do.
So the basic architecture I’ve started with and we’re still about five, six months out on this.
So this is not a finished product.
So I know, like I said, some of you guys are smarter than me.
After we’re done, if you have ideas, feel free to share.
Cause I would love to hear. All right. So basically, you know, you have a, you have a request, right? I need to write that bigger.
Nobody know. We’re gonna read that.
I don’t have good handwriting.
I’m sorry. All right. So you have a request, right?
For an agent and that can be in any form, right?
People are just gonna type whatever they want.
We can’t really, we could narrow it down and say, click some buttons. Like, what do you want it to do?
That’s probably not gonna, in real world, that’s not the best way to do it, right? So we accept basically any requests. We take that and we immediately take this request and we have a requirements file.
So that’s gonna go to, right?
So requirements are like, what do we need for an agent to work?
Like, for example, do I need credentials?
So like if you say, send an email to johnsmithatgmail.com.
So it’s gonna go and say, okay, what are the requirements that I need to make that happen?
So it’s gonna say, okay, I need a, I need an email account.
So that’s the first requirement, right? It’s gonna say, I need your username and password to that email account. If there’s a port that needs to be open for your SMTP server, which probably not required, but you know, we can just say security, right? Security or web traffic, whatever. So it’s gonna go through and generate requirements. So we have like a predefined list of requirements that it should check for based on whatever the request is.
All right, so then it has to then, once it determines this, it has to determine if we already have an agent or not, right?
So the first one, obviously we don’t have an agent.
So there’s no, we have like a, I’ll call it like a repository basically, a repo.
And it’s full of, it’s gonna be JSON format.
So we’ll have call like top level will be agents.
And we’ll say, we have an email.
Well, yeah, we’ll say email agents or email services that are available. So like we could say you have Gmail, you could have Yahoo, like we could have several, right? It’s Accenture, and then like, if we wanna set up like the Zoom meetings, you could have Zoom, whatever.
It’s gonna be like a whole JSON full of all your agents that are created. They don’t exist if you can’t find one that matches the tools you need. So it’s like Gmail would have mail services, maybe meeting, documents.
Google’s got a few services already built into Gmail.
So if they can’t find based on the request you sent and the requirements needed to make the request happen, it’s gonna say, okay, I need a new agent.
So at that point, we’re gonna go out to chat GPT or the internet, whatever. We’re gonna go search for this. So it’s like, what do I need to, or it already knows what I need, how am I required to solve?
So it’s like, I need these. So it’s gonna create a request that says, chat GPT. I need all of these things.
And what I want you to do is build this into a Docker file.
So we’re gonna use a base Linux.
We always use base Linux.
Then we’re gonna use Docker and compose. And we’re gonna use Docker to create the requirements that we need. And then we’re gonna use Python libraries, a Python file that we’re going to include in our Docker compose file to do this. And this is just to keep it because every single agent could have unique things that it needs. So we wanna compartmentalize that to inside of a Docker image that it’s contained.
We don’t really want, you might need different libraries, different models, different things for different use cases.
So we wanna keep that separate and spin up every agent in a Docker file so that it’s separate from other agents. And that way also you can have multiple agents running at the same time.
Like you could have two users asked to send an email at the same time.
You want one for Bob, one for John. So you want two different images. You don’t wanna get those mixed up. So a lot of different reasons to segment out that. So once this is created, you then wanna also, you wanna update your repository here with this file and say, okay, I’ve created this new agent here.
New agent. And you know, here’s all its skills, right?
Here’s what it can do.
Then also if it needs any of these credentials, there’s also a credentials file.
So if it needs a username, password, whatever, it’ll have a credentials file.
It will have the service.
And then if it’s used, it’ll have the type of auth.
And then depending on if it’s username, password, key, certificate, whatever, we’ll store it in a set format under credentials that will match whatever the agent is.
It’ll have that same name to it so that it can find it.
So as it writes to repo, writes to credentials, then it has everything it needs to create this Docker compose file and create the agent.
Then once it does that, we can do a Docker run command.
Or we’ll use, I think we’re actually gonna use Kubernetes, but you can use Docker run or Kubernetes if you want.
Create a YAML file, Docker run this thing.
And then it should get the output of what the customer needs and have that agent created.
And then it can fulfill the request.
And then anytime after that, that they need to do that again, this now it already exists every time. So we can just keep going to our repo and rerunning the same agent every single time.
That’s a very simplified look at it.
There’s a lot of coding that goes into here that makes all that happen.
But that’s one of the cool things we’re working on now is agents that create agents. So we’re already replacing AI’s jobs, not just people. I said, if anybody has any ideas, feel free to pitch them. Cause I am not the expert, but I just worked with it a lot and know how to make things work. So I’m cool. Did I go too fast to lock it in? Now we’re good. Let me switch back, I’ll share real quick. We’ll wrap it up and I can take a point for it. No, it’s fine. You don’t want to know inversion on the bails, death by PowerPoint, I would set it up.
Yo, all right, share screen.
Let’s do that.
So again, thanks Paul for coming and talking about stuff. One of the neat things we’ve always had in this group are if there are a lot of different people, folks that are in the middle of building new models, folks that are in the middle of just using models. And we’ve got some that are actually putting together products that the general public uses. And that’s always interesting when you see real people do things to the products that you built and then you try to figure out, wait, why did they do that? So to close out, I’m looking for any kind of thoughts for 2025. We’ve got going into next year, we’ve got three sessions that I’m doing for Learning Quest. That’s going to be more of an overall intro to AI. I think you have to be part Learning Quest to go to that, but that’s something that I’ll be putting together. And I’ll also publish that on our GitHub as well. More of a very high level overview of AI, a lot of background info.
I will probably capture all of the fun questions I’m going to get, because that’s always an interesting thing.
You get people that are interested and they come from different backgrounds. They’re like, well, what about this? I’m like, crap, I never thought of that.
You know, so got to go write that down. February, the end of February and first weekend in March is the Hudson Alpha Tech Challenge. I know there’s some activity on the Discord channel about forming a team. We’ve actually had folks from here that were winners before.
I think that was Phil Warding and a team.
And I think he, oh, you were on that team too? I wasn’t on that team, but I won at second place. Okay. I don’t know, second place is winning in my book.
Right, yeah. So we’ll put together some stuff for that. It’s always fun to get to interact with kids, high schoolers, some college, some professional level, you know, coming and doing fun stuff. After that, I’m pretty open as far as what topics to talk about, what you guys might want to cover. So if you got thoughts, go ahead and throw them out there as well as those on the line. Let me go check and see who we got on the people list. Nobody’s in prompt sharing night.
Oh, prompt sharing night, that would be good. Let’s see, off the chat. Relevance AI, prompt sharing night.
All right, the other, one of the other things I thought about doing, we got several different kinds of services and models that you can plug in to Visual Studio Code or other kind of code editors.
I was thinking about trying to farm out some different, you know, anybody here use Claw for that?
Okay, maybe 15 minutes you walking through how you use Claw as part of your technique.
Or if I use co-pilot, what does that look like for me?
You know, maybe three or four different approaches because they’re all a little different in how you interact with them.
I’ve got one that’s more of a, I’ve got a chat DPT window built into my editor.
I’m like, okay, I could have just used my browser, but that’s fine. Some of them do, you know, full code completion, other stuff like that.
Some of them, you can actually highlight a segment of code or a couple of functions and say, build me some unit tests for this.
And it’ll go create unit tests for you.
Thought about doing some of that. You guys ever done like a paper reading group sort of thing for like decoding, you know, there’s all these papers that are coming out.
There’s usually like one or two each month that are just big.
It might be interesting to decode, you know, like the one that you’re talking about, the bite one from this time, you know, whenever there are police Sora and then the new video models, like saying like, okay, what’s actually happening here?
What does all the math actually say here?
That might be an interesting thing to do.
Okay. If you needed somebody to do that, I read a lot of pages.
Yes, I would, I do, when we go deep, I take, I typically put a, I can’t remember if it’s a nerd alert or a what the, you know, we’re gonna dive into some stuff, you know what I mean, versus some of the other pieces.
But yeah, that’s definitely a thing.
Maybe like a second track or something like that or something like that. Yeah, yeah. We can either do a second call, only, you know, that sort of thing.
Yeah, that is something that we’ve done before where on the alternate weeks, we just did a virtual online. Cause it’s a lot easier for me actually, to dial in from home. You know, there’s every other week I have to be in town anyway.
So I’m here working and then, you know, it’s six o’clock also gives me a reason to leave the arsenal earlier than normal.
So yeah, get out of that.
Are AI service services or potential services? Is that something you guys want to look into or talk about? Like providing services or what services are available? Right, the biggest challenge I see coming up for our society is what is true and what is false? Okay. What are the real facts?
Okay, so you got all these news media, they sorely need some help in sorting weed from the chaff.
Right. Now that’s kind of dependent on the language model you got though, right?
How good is that?
How big is it?
I’m sorry, these are neophyte questions. No, it’s not a neophyte. It is kind of what we do.
We do swing that direction sometimes and we talk about culture and AI and the impacts. And interesting things. I didn’t really, I was kind of watching during the election cycle to see what might pop out of that I didn’t really, it wasn’t anything close to what we’ve seen before with tons and tons of misinformation shut down on social media that was back when it was poorly generated and it was kind of easy to tell for most, but we can definitely jump into that. How do you know what’s true? One of the things we’ve done before is looked at the different models that get built and you’ve got a lot of interesting, since AI is more of a global thing, different cultures have way different viewpoints on certain subjects. It gets interesting because you actually have to say, this is a thing and it can be contentious and we’re just gonna talk about it from a objective point of view. This culture thinks this is true. This culture thinks this is true.
What happens when AI gets in the middle of it?
Yeah, things like that.
Synthetic data and train models with it. Yeah. Oh, okay, yeah.
I didn’t even think of looking at other countries’ election.
Oh, that might be interesting.
Some are very simple.
Yeah, I’ve been tracking AI use in some of the other conflicts that are going around.
That information is a little harder to get and even if you do get it, you can’t share some of it, but it’s still a very interesting thing. I haven’t, yeah. I would be curious to hear more about how AI is impacting different industries. Okay. China is starting to use something called Squirrel.ai for individualized learning for K through 12.
Okay.
So I’d be curious to see how that might be impacting other countries’ educational system, especially education seems to be a hot point here in the US. Yeah. I myself am in manufacturing, which in the US is the industry that uses AI the least throughout the entire world, the US manufacturing industry specifically. So my company is trying to kind of leapfrog over several AI developments in order to start implementing it. In fact, the National Institutes of Standards and Technology, they’re finalizing a proposal now for a brand new institute specifically to bring AI to manufacturing in the US. Yes, are you involved in that? In that?
Okay, do you want to be?
Yes.
There are people that are looking for experts in that kind of field that have a manufacturing background, which we talk after.
I’m interested in the AI, I’m really pulling for them. I’ve got the manufacturing background, so you and I could talk, but I would be curious to see how AI is impacting K-12 education, even the medical diagnosis. We’re gonna have agents for the medical field, manufacturing, all these different industries. Yeah. We’ve done some with, if you got thoughts on different types of domains to cover, we’ve done, especially here, we’ve done a couple of sessions on AI and genomics.
You know, where is that, you know, kind of thing.
It’s interesting.
It’s a little hard sometimes to get folks from Hudson Alpha kind of involved because they’re here every day for work.
Five o’clock, six o’clock, they’re ready to leave, you know, they’re like, you know, go talk about AI. So yeah, medical field is very interesting. We actually had a surgeon that’s been here a couple of times. This first time we’ve actually had like an actual doctor, doctor, medical doctor puts in. I don’t know if you’re gonna say something or not. No, no, no, no, no. I read an article, it was on the internet, and it said the GPT-4 was now obsolete because of this new introduction of ability.
AI was going to be able to take a look at its own output and correct it or improve it. Right. And it also had the ability to talk or had some visual properties.
Now they were kind of vague.
Right. Have you heard of any such things going on?
Yes, I think what they’re talking about is this test time.
Yeah, O1 is the self-correcting version for sure.
So yeah, some of that is out there and it’s really good. Obsolete’s a weird word, I don’t think that’s really right, because it’s not always, you don’t always want the biggest model. If you have a smaller model that does the job and it does it correctly, you know, you don’t hire a PhD to flip birds at McDonald’s. You’re fine with that person, you know.
Even the input and output sort of stuff, that’s all 4.0, which is the old quote unquote model.
But it sounds nice to say, ah, we’re in a new age, this new thing is awesome and everybody go get this thing.
That’s really expensive, but don’t worry about that. Right. That’s the, we could almost do a session on how to remove the exaggeration from some of the stuff you see.
I mean, the same thing, like every, like you can walk in the supermarket today and go look at the bleach aisle and there will be things on there for bleach that will say new and improved. Really? You know, it’s so figuring out, cause at the same time, you’ve got AI helping marketers, right?
New things that tie in with people. How do I make this stand out?
And yeah, I mean, it’s.
Which 4.0 is better.
Like the old models are better at creative tasks and sounding like a human and all that sort of stuff. Like these new models that they’re talking about, they’re really, really, really good at one thing, which is working with symbolic logic and hard requirements where you have to constrain yourself. I have these rules that I, my answer, I have to constrain with all of these.
I’m asking it to write me a poem about Shrek. I’ll need that. It speaks like an engineer almost.
It’s very matter of fact, very, you know.
I like the small model because my son is a vice-president in a medical community in Tennessee, Kentucky. They have medical clinics, 26 clinics. They are trying to incorporate AI now. I can lay you out a graph about all the things we’re looking at, but you got so many people that know nothing about it.
And so you’re trying to find what’s going to do the best. And you really need a small model that addresses only the kind of medical information that you want.
You’ve got diagnosis and you’ve got new drugs. You’ve got papers being written on new procedures. So you need all those things.
How do you begin to find a language model that’s going to have those elements?
How do you do that?
That’s also the thing of, you know, a lot of people are getting crazy because AI is hot right now.
So you want to throw AI at everything.
But to me, it seems almost like the really awesome use case is using AI kind of like at the edges to bridge the gap in your programs and your things.
So we had all these things.
We’re going to break it up into models.
Yeah, yeah. But how do I query this thing to go get me all this stuff and then I’m back in a normal program. But you know, there’s something that for me to write the program to make that jump would be that if-else statement that looks like 9,000 different, you know, blocks.
If I can have something, I put something in and it generates something reasonable, that’s fine.
But then maybe I don’t try and get it to design the whole car with CAD coordinates and stuff like that, which some people try and do that.
It’s like, okay, you know, let’s zoom out a little bit, you know, and look at where this thing is.
I mean, your example of Clorox.
Yeah.
Do you know in the- By the way, I said bleach. Yeah. Yeah. Yeah. You don’t even want to laugh at me. Yeah. Yeah. Yeah. Definitely.
I don’t want to.
I don’t want to get sick. I can’t remember what- They’ve left a lot of bad habits in 84 years. Oh, yeah. But, it seems like the one.
Bleach, Clorox.
Oh, yes.
You have these new drugs that are coming out, like you say, new and improved. But my son tells me they aren’t new and improved. They’ve taken a decongestant and they’ve added it to an allergenic product and then it extends their patent rights and then they put it out there and there’s nothing new about it. So, he oftentimes goes back to old drugs because they’re just as effective, right? Yeah. But, how does a doctor have time to analyze all that about the medicines that are coming out? Especially when that guy sells people beating down their door, trying to sell them the one thing versus the other thing. That’s why he needs artificial intelligence. There’s a whole bunch of stuff. Oh, I take that as a use case.
That is no one use case. Right.
I’ll say there’s a couple biotechs that we work with and I don’t think I can say which one is probably, but there’s one in particular that does a lot of AI for their research and they said what they’re really excited about is quantum coming into play because of how much faster it can go through scenarios.
Yeah, we could definitely do a kick on quantum here. There’s actually a quantum computer over at Davidson right now.
That was, I mean, for real. You could probably depending on what your use case is and what your connections are, find a way to get time on their view. Okay. I’m interested in having them still online for free.
Yes. Yeah, this is actually a local, I mean, this was one of the things they were talking about at the SME symposium. They had just come online with this. I can’t remember what company they’ve partnered with. Davidson?
Was it Davidson?
And whoever knows the quantum stuff.
Yeah, maybe original, I can’t remember. Maybe.
But I do need to start wrapping this up, but, oh yeah, bioRxiv.
I forgot that was a thing. Let’s close that with that.
I’ll capture all the comments and try to get everything in the, after I close up the Zoom, actually, let me go ahead and stop the recording part of it. Stop sharing. So everybody that hopped in online, I appreciate it. If you’ve got anything from, any comments or anything you want to drop into the end of the link, you know, we’ll hop on that. I’ll get that in as well.
Let me do it. Thank you. Yeah, hop stop. Yeah, thanks for doing this as always. I appreciate it. Yeah, thanks, Dan. Thank you very much.