Huntsville AI

AI Challenges and Competitions
2024-02-07 18:00 - 19:00 (America/Chicago) | HudsonAlpha, 601 Genome Way Northwest, Huntsville, AL, 35806
AI Challenges and Competitions will be the subject of our next meetup on Feb 7th. We switched our meetups to the 1st and 3rd Wednesdays for the month of February. We will likely stick to this through May.
When we started Huntsville AI back in 2018, the primary avenue for challenges and competitions was Kaggle. This brought notoriety to several people (e.g. Jeremy Howard) and led to an approach for quickly iterating through several approaches to find the model or data manipulation that best fit the problem.
The landscape for challenges and competitions is now much different, with many requiring a significant resource investment in hardware to be able to compete. We’ll take a look at several organizations that manage these competitions and walk through a few open competitions to discuss.
Links:

Challenge.gov
Kaggle
Numer.ai
Driven Data
AI Crowd
ML Contests

Details:

Date – 02/07/2024
Time – 6-7pm
Location – HudsonAlpha
Address – 601 Genome Way Northwest, Huntsville, AL 35806
Zoom –https://us02web.zoom.us/j/88417365505?pwd=[REDACTED]

HudsonAlpha Tech Challenge (HATCH) will be March 1-3 this year. They are not doing sessions specific to HATCH leading up to the event, so we will stick to our normal AI focused programming. If you would like to sign up as a tech mentor, there are still a lot of slots available. Sign up on their Sign-up Genius Form.
Huntsville AI Task Force is an effort by the City of Huntsville to put together a group of people to evaluate the impact of AI on Huntsville’s workforce and provide guidance to shape future efforts from the city. I was invited to participate and will be flowing information and questions to you. So far, there are members of this task force from the usual RSA folks (Army, FBI, NASA) along with HudsonAlpha, UAH, AAMU, Huntsville Hospital, Mazda Toyota, IBM, and Meta. There was a kickoff meeting on January 26, with the next meeting to follow at the end of this month.
Space and Rocket Center AI Symposium is happening on February 20-23. This might be of interest of you have the $450 to sign up for both days. If some of our folks can attend, maybe we can put together on overview to present afterward to help with our vision: “Our vision is a group of individuals and organizations in the metro Huntsville area who collaboratively advance the knowledge and application of artificial intelligence in ways that make it available to everyone and improve our quality of life.”
Topics for 2024 will be the end or our discussion on Wednesday. From the end of 2022 and all the way through 2023, there were ground-breaking announcements on a monthly basis that made it difficult to plan ahead. I feel like this may be settling down for a minute, so we should be able to get back to something more sensible. Take a look at what we have planned and completed in the past to see what’s already on the board. I have some scheduled based on SBIR topic releases, but I need your feedback for what other topics you would like to cover. You can reach the board on Github here: https://github.com/HSV-AI/presentations/projects/1

AI Breakthroughs in Video
2024-02-21 18:00 - 19:00 | HudsonAlpha, 601 Genome Way Northwest, Huntsville, AL, 35806
This week we will be talking about some exciting breakthroughs in video generation and understanding. The main splash was with OpenAI’s announcement of Sora last week, but there have also been similar releases and updates from Google, Apple, and UC Berkeley. There’s a description of each below if you want to read ahead. It will be impossible to cover all of these papers in an hour, so let me know if you want to do a deeper dive into these at a later meetup.
February 15 – OpenAI Sora – “Sora can generate videos up to a minute long while maintaining visual quality and adherence to the user’s prompt.” There is no paper linked to the release, but the post on OpenAI that covers the technical details is really good.

https://openai.com/sora
https://openai.com/research/video-generation-models-as-world-simulators

February 13 – UC Berkely – Large World Model – this is a multi-model (text and video) model capable of processing 1M tokens. It can generate images and videos as well as describe the content of videos or answer questions about what happened in a video.

https://largeworldmodel.github.io/
https://huggingface.co/LargeWorldModel
https://arxiv.org/abs/2402.08268

February 8 – Apple KeyFramer – “A design tool for animating static images (SVGs) with natural language”. This isn’t necessarily a video tool, but generates CSS code to animate an SVG.

https://arxiv.org/abs/2402.06071
https://www.theverge.com/2024/2/14/24072917/apple-keyframer-ai-animation-tool-2d-images-text-descriptions

January 23 – Google Lumiere – Text to Video, Image to Video, and Stylized Generation.

https://lumiere-video.github.io
https://arxiv.org/abs/2401.12945

October 31, 2023 – SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction – I’m not sure how to attribute this to a particular organization, since it appears that several Chinese universities contributed. While not quite as impressive as the other releases above, it shows a broader move to video generation.

https://vchitect.github.io/SEINE-project/
https://arxiv.org/abs/2310.20700
https://github.com/Vchitect/SEINE

Details:

Date – 02/21/2024
Time – 6-7pm
Location – HudsonAlpha
Address – 601 Genome Way Northwest, Huntsville, AL 35806
Zoom –https://us02web.zoom.us/j/81284789520?pwd=[REDACTED]

HudsonAlpha Tech Challenge
2024-03-01 18:00 - 2024-03-03 18:00 | HudsonAlpha, 601 Genome Way Northwest, Huntsville, AL, 35806
This is a placeholder for the HudsonAlpha Tech Challenge. We usually support this event with mentors and/or teams to compete.

Retrieval Augmented Generation
2024-03-06 18:00 - 19:00 | AL
This week we will be talking about Retrieval Augmented Generation – also known as RAG. The basic premise is to use an existing LLM to generate content based on an existing collection of documents. This removes the need to fine-tune an LLM, which is cost prohibitive and hardware constrained.
Over the next several sessions, we will be diving deeper into separate components needed for RAG – hopefully resulting in a chat-based Q&A service for the NASA Technical Report Server. We were introduced to this data during our submission for the 2022 NASA SpaceApps Challenge – where we placed 2nd. Our submission was a semantic search based on the abstracts for the NSTR dataset of 10,000 papers.
For this session, we will cover an overview of RAG as well as walk through the submission from the challenge. I have recently updated it to work with current versions of the dependent libraries – so we can demonstrate the starting point for this project.
Also – if you missed last week’s session on AI & Video – you can find the recording here – https://hsv.ai/2024/02/22/ai-breakthroughs-in-video/
Links:

RAG Paper – https://arxiv.org/pdf/2005.11401.pdf
Huntsville AI 2022 SpaceApps Submission – https://github.com/HSV-AI/spaceapps2022

Details:

Date – 03/06/2024
Time – 6-7pm
Location – HudsonAlpha
Address – 601 Genome Way Northwest, Huntsville, AL 35806
Zoom –https://us02web.zoom.us/j/82845221441?pwd=[REDACTED]

Hope to see you soon!
-J.

Vector Store with ChromaDB
2024-03-20 18:00 - 19:00 | HudsonAlpha, 601 Genome Way Northwest, Huntsville, AL, 35806
Continuing our discussion about Retrieval Augmented Generation (RAG), this week we will incorporate ChromaDB as the storage and query mechanism for the embeddings. Our initial SpaceApps submission held all of the embeddings in memory and used a cosine similarity metric for queries. We will go through the ChromaDB API and design decisions required to replace the custom approach with a more robust solution.
The initial work for converting this is posted in the SpaceApps repo linked below if you want to check it out early. Also – I double-checked our availability at HudsonAlpha, so we are set for an in-person meetup if you can make it.
Series:
Over the next several sessions, we will be diving deeper into separate components needed for RAG – hopefully resulting in a chat-based Q&A service for the NASA Technical Report Server. We were introduced to this data during our submission for the 2022 NASA SpaceApps Challenge – where we placed 2nd. Our submission was a semantic search based on the abstracts for the NSTR dataset of 10,000 papers.
I hope to have the video from our last session posted today. You can check for updates at https://hsv.ai/videos
Links:

Chroma DB – https://github.com/chroma-core/chroma
RAG Paper – https://arxiv.org/pdf/2005.11401.pdf
Huntsville AI 2022 SpaceApps Submission – https://github.com/HSV-AI/spaceapps2022

Details:

Date – 03/20/2024
Time – 6-7pm
Location – HudsonAlpha
Address – 601 Genome Way Northwest, Huntsville, AL 35806
Zoom –https://us02web.zoom.us/j/81139026249?pwd=[REDACTED]

Document Chunks with LLM Sherpa
2024-04-03 18:00 - 19:00 | HudsonAlpha, 601 Genome Way Northwest, Huntsville, AL, 35806
Continuing our discussion about Retrieval Augmented Generation (RAG), this week we will incorporate LLM Sherpa to provide chunks of text from PDF documents that have been retrieved from the NASA archive.
Our initial attempt used PyPDF2 to read text from the PDF documents. It was very slow and provided limited strings of text that did not match the paragraphs in the documents. We’ll take a look back at what was available at the time, and then look through the LLM Sherpa API and see what it looks like with that piece incorporated.
As we get further into this project update, it has become apparent for the need to split the monolithic application into components that can be hosted and updated separately. We will go through what has been done so far to containerize both the ChromaDB vector database and the LLM Sherpa for chunking.

Choosing an Embedding Model
2024-04-17 18:00 - 19:00 | HudsonAlpha, 601 Genome Way Northwest, Huntsville, AL, 35806
We have Josh Phillips presenting this week, so you don’t want to miss this!
Join us as we explore the crucial task of selecting optimal embedding models to enhance AI performance across a variety of applications. This meetup will delve into the Multilingual Transferable Embedding Benchmark (MTEB), a pivotal resource providing a comprehensive framework to evaluate embedding models over diverse task categories and numerous languages. The selection of the right embedding model is vital, yet challenging due to the myriad of options and their inherent trade-offs.
This presentation will not only introduce you to MTEB’s holistic approach across eight core NLP tasks but will also guide you through the practical steps of identifying, shortlisting, and benchmarking models to find the best fit for your specific needs.
Agenda:

Introduction to Embedding Models – Gain insights into why choosing the right model is critical for AI tasks.
Overview of MTEB – Understand the framework of the Multilingual Transferable Embedding Benchmark and its application across 100+ languages.
Deep Dive into MTEB Tasks – Explore the eight fundamental tasks within MTEB, including bitext mining, classification, clustering, and more.
Case Studies – Walk through real-world use cases, demonstrating how to apply MTEB in selecting models for tasks such as walking path recommendations, form filling automation, and building a documentation assistant.

Series:
Over the next several sessions, we will be diving deeper into separate components needed for RAG – hopefully resulting in a chat-based Q&A service for the NASA Technical Report Server. We were introduced to this data during our submission for the 2022 NASA SpaceApps Challenge – where we placed 2nd. Our submission was a semantic search based on the abstracts for the NSTR dataset of 10,000 papers.
I hope to have the video from our last session posted today. You can check for updates at https://hsv.ai/videos
Links:

Multilingual Transferable Embedding Benchmark (MTEB)
MTEB Github
MTEB Paper
Huntsville AI 2022 SpaceApps Submission – https://github.com/HSV-AI/spaceapps2022

Details:

Date – 04/17/2024
Time – 6-7pm
Location – HudsonAlpha
Address – 601 Genome Way Northwest, Huntsville, AL 35806
Zoom –https://us02web.zoom.us/j/81626946368?pwd=[REDACTED]

2024 SBIR Topics Round 2
2024-04-24 18:00 - 19:00 | HudsonAlpha, 601 Genome Way Northwest, Huntsville, AL, 35806
SBIR/STTR Topics
The second round of SBIR Topics for DoD were released last week. As usual, we will go through any that are labeled as Machine Learning or Artificial Intelligence – which looks like 13 SBIR and 9 STTR out of 176 total topics.
You can find the full list below in the Links, just search for Learning or Artificial.
2024 AI Symposium
The sessions from the Space and Rocket Center AI Symposium were recorded and just released on YouTube! Based on my notes from the sessions that I was able to attend, they range from high level overviews to in-depth use of AI. I believe that all of them would be considered practical rather than theoretical. Again – look for the link below.

Series (picking back up next week):
Over the next several sessions, we will be diving deeper into separate components needed for RAG – hopefully resulting in a chat-based Q&A service for the NASA Technical Report Server. We were introduced to this data during our submission for the 2022 NASA SpaceApps Challenge – where we placed 2nd. Our submission was a semantic search based on the abstracts for the NSTR dataset of 10,000 papers.
Sessions from this series so far are being posted at https://hsv.ai/videos
Links:

Defense SBIR/STTR Portal – https://www.dodsbirsttr.mil/topics-app/
Space & Rocket Center AI Symposium Videos – https://www.youtube.com/playlist?list=PLvvHQqQynqmtkQuLsvfFtmy0OohBcA5H4
Huntsville AI 2022 SpaceApps Submission – https://github.com/HSV-AI/spaceapps2022

Details:

Date – 04/24/2024
Time – 6-7pm
Location – Zoom
Zoom –https://us02web.zoom.us/j/88996968876?pwd=[REDACTED]

Deep Dive with Llama-Cpp-Python
2024-05-01 18:00 - 19:00 | HudsonAlpha, 601 Genome Way Northwest, Huntsville, AL, 35806
This week we will take another look at llama-ccp, this time using the python wrapper available in llama-cpp-python. This library can be installed and imported directly, or run as a separate service available through the OpenAI API.
We covered the basics of llama-cpp last year and looked through the different quantization approaches available. It provides a great way to run a lower level model using only CPU resources and RAM. There is some quality loss due to the quantization, but the results are good enough for prototyping at a much lower cost.
Agenda:

Introduction to Llama-cpp-python
Installation and configuration
Python API
OpenAI API

Series:
Over the next several sessions, we will be diving deeper into separate components needed for RAG – hopefully resulting in a chat-based Q&A service for the NASA Technical Report Server. We were introduced to this data during our submission for the 2022 NASA SpaceApps Challenge – where we placed 2nd. Our submission was a semantic search based on the abstracts for the NSTR dataset of 10,000 papers.
I hope to have the video from our last session posted today. You can check for updates at https://hsv.ai/videos
Links:

https://github.com/abetlen/llama-cpp-python
https://llama-cpp-python.readthedocs.io/en/latest/
Huntsville AI 2022 SpaceApps Submission – https://github.com/HSV-AI/spaceapps2022

Details:

Date – 05/01/2024
Time – 6-7pm
Location – HudsonAlpha
Address – 601 Genome Way Northwest, Huntsville, AL 35806
Zoom –https://us02web.zoom.us/j/82295484520?pwd=[REDACTED]

OpenAI API
2024-05-15 18:00 - 19:00 | HudsonAlpha, 601 Genome Way Northwest, Huntsville, AL, 35806
This week we will cover the use of the OpenAI API with both llama-cpp-python and OpenAI itself. After this session, you will have a solid understanding of how to use the chat completion API along with an approach to run the same code locally against llama-cpp-python.
We will cover account setup, billing & budgets, as well as the python code needed to make function calls. Next we will show the same python code running against a locally hosted model.
Updates:

The videos page at https://hsv.ai/videos/ is updated through our last meetup, so if you want to catch up on previous sessions – give that a shot.
The meetups through the summer months are all planned – check the schedule toward the end of this email, as well as https://hsv.ai/calendar/

Series:
Over the next several sessions, we will be diving deeper into separate components needed for RAG – hopefully resulting in a chat-based Q&A service for the NASA Technical Report Server. We were introduced to this data during our submission for the 2022 NASA SpaceApps Challenge – where we placed 2nd. Our submission was a semantic search based on the abstracts for the NSTR dataset of 10,000 papers.
Links:

https://platform.openai.com/docs/api-reference
https://github.com/abetlen/llama-cpp-python
https://llama-cpp-python.readthedocs.io/en/latest/
Huntsville AI 2022 SpaceApps Submission – https://github.com/HSV-AI/spaceapps2022

Details:

Date – 05/15/2024
Time – 6-7pm
Location – HudsonAlpha
Address – 601 Genome Way Northwest, Huntsville, AL 35806
Zoom –https://us02web.zoom.us/j/81984208192?pwd=[REDACTED]

Summer Social
2024-05-29 18:00 - 20:00 | Stovehouse, 3414 Governors Dr SW, Huntsville, AL, 35805
We are going to take a break from the formal sessions this week to get together at Stovehouse and hang out for a bit. This is a great opportunity to relax and meet others involved in Huntsville AI. We have a wide variety of backgrounds, so I always learn something interesting. This officially starts at 6pm, but I will be there early if anyone wants to help hold a table.
We’re suspending the RAG series for the next two sessions, but will resume at the end of June.

Series:
Over the next several sessions, we will be diving deeper into separate components needed for RAG – hopefully resulting in a chat-based Q&A service for the NASA Technical Report Server. We were introduced to this data during our submission for the 2022 NASA SpaceApps Challenge – where we placed 2nd. Our submission was a semantic search based on the abstracts for the NSTR dataset of 10,000 papers.
Links:

https://www.stovehouse.com/
Huntsville AI 2022 SpaceApps Submission – https://github.com/HSV-AI/spaceapps2022

Details:

Date – 05/29/2024
Time – 6-8pm
Location – Stovehouse – most likely near the Pourhouse

Law and AI
2024-06-12 18:00 - 19:00 | HudsonAlpha, 601 Genome Way Northwest, Huntsville, AL, 35806
This week we have Andrew Tuggle from Womble Bond Dickinson (US) LLP joining us to talk about Law and AI. He has been involved in our meetups for years and always has an interesting perspective from an attorney’s point of view.
As AI becomes more advanced and ubiquitous, it is raising new legal issues across many domains. In this session, we will explore key legal considerations surrounding AI through the following topics:

Personhood and agency of AI systems
Intellectual property issues including authorship, derivative works, and open source
AI decisioning and regulatory oversight
Regulatory schemes – Biden executive order and the AI Act in the European Union

Also – we will resume the RAG series again following this week:

Series:
Over the next several sessions, we will be diving deeper into separate components needed for RAG – hopefully resulting in a chat-based Q&A service for the NASA Technical Report Server. We were introduced to this data during our submission for the 2022 NASA SpaceApps Challenge – where we placed 2nd. Our submission was a semantic search based on the abstracts for the NSTR dataset of 10,000 papers.
Links:

Huntsville AI 2022 SpaceApps Submission – https://github.com/HSV-AI/spaceapps2022

Details:

Date – 06/12/2024
Time – 6-7pm
Location – HudsonAlpha
Address – 601 Genome Way Northwest, Huntsville, AL 35806
Zoom –https://us02web.zoom.us/j/84283975695?pwd=[REDACTED]

Hope to see you soon!
-J.

Vector Storage with Weaviate
2024-06-26 18:00 - 19:00 | HudsonAlpha, 601 Genome Way Northwest, Huntsville, AL, 35806
We’re going to add in a new topic to the NASA RAG series this week, so check below for some rearranging of the schedule. As always, if there is something that you would like to talk about (or have someone cover) – please let me know!
After a few rounds of iteration using ChromaDB for vector storage, it has become apparent that there are better options. While it is possible to run ChromaDB locally, it has too large of a footprint to run alongside the other components to develop a RAG prototype. I’ve spent a few hours replacing the ChromaDB integration with Weaviate, and I’m quite happy with the results.
In this session, we’ll cover everything you need to know to get started with Weaviate. There are several options that I’m not using for the NASA RAG, but may be relevant for other types of products that you may want to build.

Series:
Over the next several sessions, we will be diving deeper into separate components needed for RAG – hopefully resulting in a chat-based Q&A service for the NASA Technical Report Server. We were introduced to this data during our submission for the 2022 NASA SpaceApps Challenge – where we placed 2nd. Our submission was a semantic search based on the abstracts for the NSTR dataset of 10,000 papers.
Links:

Huntsville AI 2022 SpaceApps Submission – https://github.com/HSV-AI/spaceapps2022
https://weaviate.io/platform
https://github.com/weaviate/weaviate

Details:

Date – 06/26/2024
Time – 6-7pm
Location – HudsonAlpha
Address – 601 Genome Way Northwest, Huntsville, AL 35806
Zoom –https://us02web.zoom.us/j/89078627184?pwd=[REDACTED]

RAG Prompt Engineering
2024-07-10 18:00 - 19:00 | HudsonAlpha, 601 Genome Way Northwest, Huntsville, AL, 35806
If you have ever tried to prompt an LLM to give you what you want, you have probably run into some frustrating challenges. This frustration can range from trying to write what you want in pseudo-code to arguing with a child about putting their toys away.
This week we will focus on some guidelines for building successful prompts for a RAG system. There is a lot of art and craft associated, and the answer to most questions is “it depends”. To help understand this better, we will pull back the curtain and talk about WHY certain prompts are effective and what types of situations they may apply to.
I am learning more about this subject every day, so if you have prior experience or tips that I should include – please reply and I’ll add them to the talk.

Series:
Over the next several sessions, we will be diving deeper into separate components needed for RAG – hopefully resulting in a chat-based Q&A service for the NASA Technical Report Server. We were introduced to this data during our submission for the 2022 NASA SpaceApps Challenge – where we placed 2nd. Our submission was a semantic search based on the abstracts for the NSTR dataset of 10,000 papers.
Links:

Huntsville AI 2022 SpaceApps Submission – https://github.com/HSV-AI/spaceapps2022
OpenAI Discussion – https://community.openai.com/t/prompt-engineering-for-rag
Prompt Engineering Guide – https://www.promptingguide.ai/
OpenAI Cookbook – https://cookbook.openai.com/

Details:

Date – 07/10/2024
Time – 6-7pm
Location – HudsonAlpha
Address – 601 Genome Way Northwest, Huntsville, AL 35806
Zoom –https://us02web.zoom.us/j/82515370839?pwd=[REDACTED]

Private RAG Deployment & Cost
2024-07-24 18:00 - 19:00 | HudsonAlpha, 601 Genome Way Northwest, Huntsville, AL, 35806
We have been covering a series this year based on how a RAG works, and how to build one. Now it’s time to look at what it will cost to deploy it!
BLUF: If you have constraints that require you to host your own LLM instead of using services such as OpenAI, Anthropic, or Google – this is going to get expensive.
The baseline that we will be using to derive requirements and prices will be the a subset documents that we have been using along the way from NASA. To start, I have pulled 500 of these documents which total 109,516 paragraphs and 4,254,032 words.
We will start with a basic use case with using OpenAI GPT4o and a Weaviate hosted vector store. We will move from there to a self hosted solution for all of the components and add up the month cost associated.

Series:
Over the next several sessions, we will be diving deeper into separate components needed for RAG – hopefully resulting in a chat-based Q&A service for the NASA Technical Report Server. We were introduced to this data during our submission for the 2022 NASA SpaceApps Challenge – where we placed 2nd. Our submission was a semantic search based on the abstracts for the NSTR dataset of 10,000 papers.
Links:

Huntsville AI 2022 SpaceApps Submission – https://github.com/HSV-AI/spaceapps2022

Details:

Date – 07/24/2024
Time – 6-7pm
Location – HudsonAlpha
Address – 601 Genome Way Northwest, Huntsville, AL 35806
Zoom –https://us02web.zoom.us/j/84452278503?pwd=[REDACTED]

SMD Symposium Roundup
2024-08-07 18:00 - 19:00 | HudsonAlpha, 601 Genome Way Northwest, Huntsville, AL, 35806

2024 SBIR Topics Round 3
2024-08-21 18:00 - 19:00 | AL
This week we will cover the last round of DoD SBIR Topics that are scheduled to be released on Wednesday. This is always a good checkpoint to see what areas of AI that DoD organizations are interested in.
The last time we covered SBIR topics, there were several on the call that were new to this approach of funding research. If you are also in this boat – here’s a good resource from SBIR.gov that covers the basics: https://www.sbir.gov/about
You can think of it as seed funding, with the initial round being somewhere around $100k for a 6 month effort, followed by a potential $2M project for 2 years. Each agency and topic may have a different funding allotment, so be sure and check the details of any topic that you are interested in.
Our normal approach is to use the topic search provided here – https://www.dodsbirsttr.mil/topics-app/ but this time we will also see what happens if we ask a LLM to analyze the topics for us.

Links:

SBIR Topics and Topic Search – https://www.dodsbirsttr.mil/topics-app/
Previous SBIR Awards – https://www.sbir.gov/awards

Details:

Date – 08/21/2024
Time – 6-7pm
Location – HudsonAlpha
Address – 601 Genome Way Northwest, Huntsville, AL 35806
Zoom –https://us02web.zoom.us/j/86939079628?pwd=[REDACTED]

PRE-RELEASE: August 21, 2024
OPEN: September 18, 2024
CLOSE: October 16, 2024

Faster Whisper for Audio Transcription
2024-09-11 18:00 - 19:00 | HudsonAlpha, 601 Genome Way Northwest, Huntsville, AL, 35806
This week we will cover the Faster Whisper library. It is similar to the approach used by llama.cpp and provides a much smaller footprint and faster execution times than the OpenAI Whisper library.
We will also walk through the approach for using this library within an AWS Lambda function, along with tips and tricks for deploying in a docker container.
Also – I’ll be joining the Hsv.py meetup on September 19 to talk about Llama-cpp-python if you would like to join us next week as well. More information in the links below.

Links:

Faster Whisper – https://github.com/SYSTRAN/faster-whisper
AWS Lambda Limitations – https://blog.awsfundamentals.com/lambda-limitations
HSV.py meetup – https://www.meetup.com/hsv-py/events/302429808/

Details:

Date – 09/11/2024
Time – 6-7pm
Location – HudsonAlpha
Address – 601 Genome Way Northwest, Huntsville, AL 35806
Zoom –https://us02web.zoom.us/j/86564069248?pwd=[REDACTED]

Pixtral Model by Mistral AI
2024-09-25 18:00 - 19:00 | HudsonAlpha, 601 Genome Way Northwest, Huntsville, AL, 35806
We will cover the Pixtral multimodal model released by Mistral AI last week. Hopefully we can get it loaded up in Le Chat (French ChatGPT?) and kick the tires on it a bit. From what I am hearing from some of our group, it’s definitely something to look into.
From the Mistral announcement:

Natively multimodal, trained with interleaved image and text data
Strong performance on multimodal tasks, excels in instruction following
Maintains state-of-the-art performance on text-only benchmarks
Architecture:

New 400M parameter vision encoder trained from scratch
12B parameter multimodal decoder based on Mistral Nemo
Supports variable image sizes and aspect ratios
Supports multiple images in the long context window of 128k tokens

Use:

License: Apache 2.0
Try it on La Plateforme or on Le Chat

Links:

Pixtral – https://mistral.ai/news/pixtral-12b/
Le Chat – https://mistral.ai/news/le-chat-mistral/
HuggingFace – https://huggingface.co/mistralai/Pixtral-12B-2409

Details:

Date – 09/25/2024
Time – 6-7pm
Location – HudsonAlpha
Address – 601 Genome Way Northwest, Huntsville, AL 35806
Zoom –https://us02web.zoom.us/j/81091066071?pwd=[REDACTED]

vLLM, New SBIR Topics, Helene Response
2024-10-09 18:00 - 19:00 | HudsonAlpha, 601 Genome Way Northwest, Huntsville, AL, 35806
We have a few topics that we will cover this week, led by Josh Phillips (vLLM) and Gary Turner (Helene Response). We will also cover the list of AI related SBIR topics that were released last week.
vLLM
In this session, we will demonstrate how easy it is to deploy vLLM on RunPod using an A40 GPU instance (approximately $0.40 per hour for 48GB of VRAM). We will be deploying the new LLama 3.2 11B vision model to showcase vLLM’s deployment simplicity and performance capabilities in a cost-effective environment. This hands-on example will provide practical insights into getting started with vLLM and show the real-world benefits of using a platform like RunPod for efficient model serving.
Hurricane Helene Response
Gary and I have been having a running conversation about the chaotic response in the Eastern TN and Western NC mountains. This will be a short brainstorming session to see if there’s anything we can do to assist.
SBIR/STTR
According to the announcement on the DoD SBIR/STTR website: “New DoD SBIR & STTR topics will be released on the first Wednesday of every month.” To align with this schedule, we will stop planning our normal sessions specific to SBIR and cover new topics as they come in. From the list of topics that dropped last week, the AI related topics are:

AF244-0001: Interactive Knowledge Graphs for Situational Awareness
AF244-0004: Securing Legacies: Developing an AI-Enhanced Privacy Architecture for Military Family Knowledge Systems
AF244-D002: Automated Correction of Optical Distortions in Canopy Transparencies
AF244-D006: Artificial Intelligence-Enabled Design Rule Checker
AF244-D016: Develop next-generation autonomous tracking
OSD254-P001: Generative AI BAA Open Topic

Links:

vLLM – https://docs.vllm.ai/en/latest/
RunPod – https://www.runpod.io/

Details:

Date – 10/09/2024
Time – 6-7pm
Location – HudsonAlpha
Address – 601 Genome Way Northwest, Huntsville, AL 35806
Zoom –https://us02web.zoom.us/j/89843812772?pwd=[REDACTED]

Fall Social
2024-10-23 18:00 - 19:30 | Stovehouse, 3414 Governors Dr SW, Huntsville, AL, 35805
Our next meetup with be a social at Stovehouse, where we can get to know each other a bit better. We had a large group last week and it’s difficult to catch up and learn what everyone is doing with AI in such a short period of time.
If you’re new to the group, or haven’t been to a meetup, this is a great way to get involved and see who is doing fun things with AI.
From last week – the video is posted along with a transcript – see the Links below. It’s always interesting to see what the transcription engine does with terms in a southern accent. This time it’s “BLLM” instead of “vLLM”.
In other news – there is a “Huntsville AI and Machine Learning Technology Exchange and Expo” coming up on November 13 at the Westin Hotel. While it’s not affiliated with this group, it is a really good conference with good speakers. It’s free to go, and if you would like to register as part of Huntsville AI, let me know and I can get you the name & address to use. You should be able to register personally for free as well.
Links:

vLLM, SBIR Topics, Helene Response – https://hsv.ai/2024/10/09/vllm-sbir-topics-helene-response/
Huntsville AI and Machine Learning Technology Exchange and Expo – https://meetingsevents.reg.ext.hpe.com/event/0144e3b2-0951-4eea-b9a9-21be49da3441/summary

Structured Outputs with Finite State Machines & Open Source Speech to Speech
2024-11-06 18:00 - 19:00 | HudsonAlpha, 601 Genome Way Northwest, Huntsville, AL, 35806
We have two guest speakers this week, Josh Phillips & Charlie Rogers! Josh has covered a few topics for us in the past, but this will be Charlie’s first time presenting for us, so please come and engage.
Topic 1 (Josh Phillips) : Structured Outputs with Finite State Machines
Discover how the integration of finite state machines with guided decoding can significantly enhance the stability and reliability of Large Language Models’ inference processes. In this session, you will delve into the intricacies of structured generation, learning how it not only ensures strict adherence to desired formats but also accelerates processing speeds by up to five times.
Experience firsthand demonstrations of various pipelines transitioning from unstructured to structured outputs, illustrating the ease and effectiveness of implementation. Join us to unlock advanced techniques that optimize both the performance and consistency of your language models.
Topic 2 (Charlie Rogers) : Let’s Talk It Out : Open Source Speech to Speech (S2S)
Dive into the world of open-source speech-to-speech systems and discover how to build efficient pipelines from scratch. Learn about the modular approach, explore user-friendly tools, and learn from Charlie’s experience with automating client-server architectures using RunPod. Ideal for developers and researchers looking to advance their speech processing projects.
2025 AI Symposium:
We should also have representatives from the Space & Rocket Center AI Symposium stop by to give an introduction to the symposium for next year. Last year was a lot of fun, and this year appears to be even bigger. I’ll drop a link to last years sessions below.

Links & Other Events:

2024 AI Symposium Recorded Sessions – https://www.youtube.com/playlist?list=PLvvHQqQynqmtkQuLsvfFtmy0OohBcA5H4
2025 AI Symposium – https://www.rocketcenter.com/institute
Huntsville AI and Machine Learning Technology Exchange and Expo – https://meetingsevents.reg.ext.hpe.com/event/0144e3b2-0951-4eea-b9a9-21be49da3441/summary
AI Innovators of Huntsville (Andrey’s group) at GigaParts –https://events.gigaparts.com/events/gigapartshuntsville/1422131

Details:

Date – 11/06/2024
Time – 6-7:30pm
Location – HudsonAlpha
Address – 601 Genome Way Northwest, Huntsville, AL 35806
Zoom –https://us02web.zoom.us/j/85119272393?pwd=[REDACTED]

We had a fantastic turnout for the social, and it was great to meet a lot of new people coming from different backgrounds. Thanks to Phillip Lee with ISSA for boosting our exposure with the local cyber community!
As always, I really appreciate the support and replies to these emails. You can also help by following, sharing, liking, and dropping comments on my posts on LinkedIn and Facebook – especially the ones directly for the Huntsville AI page on LinkedIn – https://www.linkedin.com/company/huntsville-ai
Hope to see you soon!
-J.

Overview of Text to Speech Approaches
2024-11-20 18:00 - 19:00 | AL
Thanks to Josh Phillips for hosting last week while I was out! I should have the recording up soon at https://hsv.ai/videos/
This week we will take a look at Speech to Text models in three different categories:

Products that create audio from text in an offline mode
APIs that can be integrated into a product
Open source models that you can host

Each of these presents different challenges that we’ll cover such as latency, realism, and hallucination. Here’s the list of products and models so far, so if you don’t see your favorite in the list, let me know and we’ll check it out as well:

Parler
Coqui
Bark
OpenAI (6 models)
BASE TTS (Amazon)
MetaVoice
MeloTTS
ElevenLabs
Facebook MMS

Also – a few of us went to the Huntsville AI and Machine Learning Technology Exchange and Expo last week, so we might do an overview of those topics if time permits.

Links & Other Events:

2024 AI Symposium Recorded Sessions – https://www.youtube.com/playlist?list=PLvvHQqQynqmtkQuLsvfFtmy0OohBcA5H4
2025 AI Symposium – https://www.rocketcenter.com/institute
Hugging Face Text to Speech – https://huggingface.co/tasks/text-to-speech

Details:

Date – 11/20/2024
Time – 6-7:30pm
Zoom –https://us02web.zoom.us/j/89971705398?pwd=[REDACTED]

As always, I really appreciate the support and replies to these emails. You can also help by following, sharing, liking, and dropping comments on my posts on LinkedIn and Facebook – especially the ones directly for the Huntsville AI page on LinkedIn – https://www.linkedin.com/company/huntsville-ai

AI Application Scaffolding
2024-12-04 18:00 - 19:00 | HudsonAlpha, 601 Genome Way Northwest, Huntsville, AL, 35806
This week we will walk through the components used in the transcription service along with how they are integrated and deployed. This has been updated over the year to incorporate new features and infrastructure as it became available.
This session will be a little light on AI, but should give you an idea of what is involved in making a product that is consumable by the general public. There is likely too much to cover in an hour, so we’ll start with the basic framework and dive in wherever folks want. Also – if you join us for the meetup, I’ll share the code behind all of this.
Basic Technologies:

Docker – separate containers for UI and transcription service
Faster Whisper – does the transcription
Paragraph Detection – finds the best breaks
Summarization – does it hallucinate, you bet!
Flask – web server behind the UI
React – Typescript based UI
SocketIO – live updates between web server and UI
Pytest – making sure stuff works
Terraform – couldn’t do AWS without this
Stripe – how to get paid

Amazon Web Service Components:

AWS Container Registry
AWS CloudWatch
AWS Lambda
AWS S3
AWS Fargate
AWS SQS
AWS SES
AWS DynamoDB
AWS AIM

Also – the recordings for the last two sessions are now posted at https://hsv.ai/videos/

Links & Other Events:

2024 AI Symposium Recorded Sessions – https://www.youtube.com/playlist?list=PLvvHQqQynqmtkQuLsvfFtmy0OohBcA5H4
2025 AI Symposium – https://www.rocketcenter.com/institute

Details:

Date – 12/04/2024
Time – 6-7:30pm
Zoom –https://us02web.zoom.us/j/81815701970?pwd=[REDACTED]

2024 Year in Review
2024-12-18 18:00 - 19:00 | HudsonAlpha, 601 Genome Way Northwest, Huntsville, AL, 35806
We made it through 2024! Well, close enough for this group anyway 🙂 This week we will get together and go through the list of topics and other items that we covered over the year. There are 3 questions that I want to ask for each topic:

Did it stick, or was it overcome by other technology or approaches?
What is the “shelf life” of the topic?
What direction is the topic headed? Advancing, decay, community interest?

In addition to the review of 2024, we also have Paul Bay coming to talk about what he has been doing with AI Agents. Based on what I am reading, Agentic AI could be the next popular movement in the field.
We’ll end the session by talking about plans for 2025. The kickoff is scheduled for January 8th. I have placeholders for other meetups & associated events on the calendar through February if you want to take a look here: https://hsv.ai/calendar/

Links & Other Events:

2024 AI Symposium Recorded Sessions – https://www.youtube.com/playlist?list=PLvvHQqQynqmtkQuLsvfFtmy0OohBcA5H4
2025 AI Symposium – https://www.rocketcenter.com/institute
2025 HATCH – https://hudsonalpha.org/techchallenge/

Details:

Date – 12/18/2024
Time – 6-7:30pm
Zoom –https://us02web.zoom.us/j/83751396321?pwd=[REDACTED]

2025 Kickoff
2025-01-08 18:00 - 19:00 | HudsonAlpha, 601 Genome Way Northwest, Huntsville, AL, 35806

Intro to AI @ LearningQuest
2025-01-16 13:00 - 14:30, 2025-01-23 13:00 - 14:30 | Huntsville Madison County Library, 915 Monroe St, Huntsville, AL, 35801, United States

AI for Code Development
2025-01-22 18:00 - 19:00 | HudsonAlpha, 601 Genome Way Northwest, Huntsville, AL, 35806
It has certainly been a busy few weeks! The first session for LearningQuest (slides linked below) had about 150 people in attendance and went very well. I had several great questions come up that I will have to research before the next session on Thursday.
We don’t know exactly what the weather will do this week as far as closures. We have the big conference room at HudsonAlpha reserved, but please check for further updates before coming in person in case we switch to a virtual only meetup for this week.
The main subject of the night will be AI for code development. We looked at this some time in the past with the StarCoder model, but there are many options available now. Some are free (and limited), while others are inexpensive. We will try to cover Copilot, Cursor, and a self hosted option. Please let me know if you have experience/opinions on these and we can get that in the discussion as well!
There is also an NVIDIA meetup at Torch earlier on Wednesday. I believe that several of us might be able to make it, so we should be able to share any info we pick up. I’ll drop a link to their signup in case you want to go.
Links & Other Events:

Introduction to AI Slides – https://github.com/HSV-AI/presentations/blob/master/2025/250116_Introduction_to_AI_LearningQuest.pdf
NVIDIA Meetup at Torch – https://torchtechnologies.surveymonkey.com/r/CSHQHBQ
2025 AI Symposium – https://www.rocketcenter.com/institute
2025 HATCH – https://hudsonalpha.org/techchallenge/
2025 BSides – https://nac-issa.org/events/bsides/

Details:

Date – 1/22/2025
Time – 6-7:30pm
Location – HudsonAlpha
Address – 601 Genome Way Northwest, Huntsville, AL 35806
Zoom –https://us02web.zoom.us/j/89635091670?pwd=[REDACTED]

2025 AI Symposium: Automation Vs HITL
2025-01-28 00:00 - 2025-01-30 23:59 | Space & Rocket Center, 1 Tranquility Base, Huntsville, AL, 35805, United States
