TWiT+ Club Shows 744 Transcript - AI User Group #14

Please be advised that this transcript is AI-generated and may not be word-for-word. Time codes refer to the approximate times in the ad-supported version of the show.
 

Leo Laporte [00:00:00]:
Hey, esp. I'd like to say hello to the AI user group. Will you do that for me? We have a little bit of. We're still working on it, me and Claude. There's a little bit of a. I don't know what.

Larry Gold (LrAu) [00:00:24]:
Anthony. Just as fast as my LLM was responding earlier.

Leo Laporte [00:00:27]:
Yeah, the latency is not great, but it's gonna get there. It's gonna get there. I'm that. But the idea for me was to have ambient computing. Right. So these are. These. This one's 60 bucks.

Leo Laporte [00:00:43]:
That's cool. I want to have one in every room, and then I have speakers in every room.

Darren Oakey [00:00:49]:
So that's.

Leo Laporte [00:00:49]:
The next thing is to route the audio through the various speakers. I don't know why it didn't respond,

Larry Gold (LrAu) [00:00:56]:
what speakers you have in every room. Okay.

Leo Laporte [00:01:01]:
I don't know if I can control the Sonos, but I think I can because I have apps that do it. So I figure I must be able to do it. Okay.

Anthony Nielsen [00:01:14]:
Anyone watching on Discord, if you'd like to hop in in this meeting and actually participate and talk. I did drop a Restream link. Hey, I. Hi, ESP in the live stream channel.

Leo Laporte [00:01:31]:
Say hi to the AI user group. It has a speaker. I don't want it to talk out of its speaker, though. I want to talk out of my speakers. It's. We're still working. It's a work in progress, but I think. But what's pleasing is now you have to say, hi, esp, right now.

Leo Laporte [00:01:47]:
And. But apparently you can. You could also say, hi, Alexa. And there's some Chinese phrases you can say, ni hao Shaolin. But what I wanted to do is say, hi, Obi Wan. And then it responds to that, or, help me Obi Wan Kenobi. But apparently you have to train that. And so I've got to figure out how to train the model using Expressives, which is the manufacturer's.

Leo Laporte [00:02:13]:
Okay, do you want to do all this? It's just I got it today, so it's not like I only been working this morning on it. Not for very long. It's pretty cool. It started. Claude did a bad thing. It started writing its own firmware from scratch. We had all sorts of trouble. I said, you know what? Why don't you download the reference firmware, which works, and start there? And then we could slowly modify that one thing at a time, and said, oh, that's a really good idea.

Leo Laporte [00:02:45]:
I don't know why it's not talking back to me. I think it's time.

Anthony Nielsen [00:02:49]:
Do you want to add your screen? Leo? I think it'd be cool. Like, I'm. I have a lot of questions about your. Your agent setup. Like the architecture.

Leo Laporte [00:02:57]:
Okay.

Anthony Nielsen [00:02:58]:
I don't know. Do you think that's a.

Leo Laporte [00:02:59]:
Sure. I'll talk about anything people want to talk about. Oh, see, it's saying something. Okay, it's clearly got an issue. Hey, your bit rate. Hey.

Darren Oakey [00:03:14]:
Hi.

Leo Laporte [00:03:15]:
Hi. Esp. Your bit rate's all screwed up. You're talking too fast. Let me see. My screen is not showing up for some reason.

Bill (error_404_new) [00:03:28]:
Oops.

Leo Laporte [00:03:30]:
Plus extra Camry. No, I didn't want black. It's not seen. Elgato. Why, oh, why, why, why computers that terrible? See, if it says something. It's supposed to say it through the big speakers, but I said you should try it for the ESP first.

Anthony Nielsen [00:03:52]:
You.

Leo Laporte [00:03:52]:
Daryl, you were saying you play a lot with the ESP stuff.

Darren Oakey [00:03:57]:
Yeah. I've got things like this on my desk, which is. That's a 3D printed thing. But this just got my like to do list that I can just click on it.

Juan (BlindWiz) [00:04:09]:
Yeah.

Leo Laporte [00:04:10]:
So, yeah, that would be easy enough. I mean, this is a touch. Little touchscreen, but it's a touch screen. And there we go. Elgato.

Larry Gold (LrAu) [00:04:18]:
So I just mounted the iPad of the wall in the kitchen.

Leo Laporte [00:04:23]:
Yeah, but I want. And then I. I basically want the house to be ambient. It's part of my overall. My overall strategy, my overall agent plan. Why is it.

Darren Oakey [00:04:43]:
You're very out of sync for me, but is that normal, just me?

Anthony Nielsen [00:04:48]:
No, it's fine on my end.

Larry Gold (LrAu) [00:04:49]:
It's fine on my end too. Yeah.

Anthony Nielsen [00:04:52]:
Yeah.

Darren Oakey [00:04:53]:
Okay. It's probably

Leo Laporte [00:04:55]:
here. I'll drink some protein.

Darren Oakey [00:04:57]:
Maybe that'll shut stuff on my machine.

Larry Gold (LrAu) [00:05:01]:
I probably should get a drink. I'll be right back.

Anthony Nielsen [00:05:03]:
Yeah, me. Speaking of drink,

Leo Laporte [00:05:07]:
why, oh why, oh, why can you not see me? I'm just baffled, baffled, baffled, baffled. It's always something, isn't it?

Larry Gold (LrAu) [00:05:24]:
Yeah.

Leo Laporte [00:05:24]:
So. And then because Claude was. Seemed like it was being stupid today.

Larry Gold (LrAu) [00:05:30]:
Yep.

Leo Laporte [00:05:30]:
I restarted it with GLM5.1. That might have been a mistake too. See, error 404 has joined us. Hello, photo. All right. I don't know why I can't see my camera. What is going. Oh, I know why, dummy.

Leo Laporte [00:05:47]:
Okay, I must have hit a button.

Larry Gold (LrAu) [00:05:50]:
Has anyone played with Claude the co work the dispatch yet or no, to let it run stuff on the side.

Anthony Nielsen [00:06:01]:
I'm so.

Bill (error_404_new) [00:06:02]:
I can't keep up.

Leo Laporte [00:06:05]:
I like the. I like the Tuohies. I'm into the Tuohies. I keep trying different models and different Tuis, but I keep going back to Claude and Opus. I've tried dispatch briefly. I tried cowork briefly, but they're too much like chat clients for me. I don't really, you know. And I have dispatch basically because I set up quad to respond to telegram so I could talk to it through Telegram.

Leo Laporte [00:06:32]:
And I have on my. I already have Push to Talk pretty well wired on my watch. But I'm trying to get it so I don't have to push a button is the thing I want to do.

Bill (error_404_new) [00:06:42]:
Yeah. With these guys.

Larry Gold (LrAu) [00:06:45]:
That's what he was talking about.

Bill (error_404_new) [00:06:46]:
Yeah, yeah, I kind of the same idea. So there's a. There's a firmware, it's kind of a package called Willow. Is that what you're using

Leo Laporte [00:06:56]:
for which for push to talk for these.

Bill (error_404_new) [00:06:59]:
So it's got.

Leo Laporte [00:07:00]:
Oh, I don't know about Willow. I need to know about Willow.

Bill (error_404_new) [00:07:03]:
Yeah, yeah. So that's what lets you do the like, hey, Alexa.

Leo Laporte [00:07:07]:
Well, this, this the built in stuff. This is the ESP32 S3 box. And I can say, hey, ESP or high ESP. And that wakes up and then I can say something to it. And I have it routing now to Claude. But so. And then I was saying earlier that if you have the expressive training stuff, you can have it say anything you want. But right now it's Alexa.

Leo Laporte [00:07:36]:
Weirdly, it's Alexa. I don't know why they put Alexa in there. And then some Chinese phrases plus high esp. I want it to be Help me Obi Wan Kenobi, you're my only hope. Or actually just help me Obi Wan Kenobi. That's my goal. Hi, Bill.

Darren Oakey [00:07:49]:
That's quite a long activation.

Leo Laporte [00:07:51]:
Yeah, no, I just want to say Obi Wan or hello, Obi Wan. I've changed the name. So I've gone through three names of my agent was Pax because it was based on Pai. We had, thanks to Larry, we had Mr. Meestler on Wednesday, day before yesterday on IM. He was great, loved him. And then that's his project pai. But I've kind of stripped out a lot of the PAI stuff because I think ultimately I want to have my own.

Leo Laporte [00:08:22]:
His is very structured. You could tell he was in the army. So I just, I wanted something a little more suited. And then I. So I changed to pa. He calls his Kai. I call mine pax. Then I said, I don't like Pax.

Leo Laporte [00:08:37]:
Plus transcript. Siri would always spell it P A C K S Which I really hated. So then I tried. I thought, oh, Jeeves, like the butler. But then I really hated that. So now I'm trying Obi Wan. We'll see. And he has a.

Leo Laporte [00:08:51]:
He looks like. He looks like Alec Guinness. But I'll come. I'm sure the name will change. I don't know what's going on. I. You know, I think GLM screwed it up. Can you say hi esp? The serial monitor is running and I need to see what happens.

Leo Laporte [00:09:10]:
Yeah, it's done something strange and then it was talking back at it.

Anthony Nielsen [00:09:23]:
That's its response. So

Leo Laporte [00:09:28]:
it's a work in progress. But the thing I like. I'll look at Willow. But the thing I like about this is I like the idea of these cheap devices in every room instead of Alexa talking, instead of to Amazon or Google or Apple talking to my framework running here. Whatever models I want. And what I've really been working on

Bill (error_404_new) [00:09:48]:
all that kind of stuff. Right. Turn it on and off lights. Like it should be able to do all that.

Leo Laporte [00:09:51]:
Well, I could turn out of the box, it could turn off lights. But yeah, I have home assistant green, so it'll be hooked up to that. But I also want to. What I do now with my telegram thing is I'll say, well, I'll do it meal egg salad sandwich on Ezekiel bread, half cup of cottage cheese, low fat and a 30 gram Fairlife high protein drink. And so dictates that and I'll come back in a little bit. And it adds that to my obsidian. I do that with exercise and I can say something like research. Daniel Meler actually have a show prep, the Anthony's show prep skill.

Leo Laporte [00:10:38]:
So I'll say show prep or I could just say research. It knows it should do show prep and then that puts it in my obsidian. I'll show you the subsidian. So I set up Carpathy's LLM.

Larry Gold (LrAu) [00:10:50]:
Yeah, that's cool, isn't it?

Leo Laporte [00:10:52]:
Yeah, it's great. And that's part of the agent stuff too is, you know, I had this conversation with it a couple of days ago saying I want you to. I want you to remember me. And lunch logged egg salad, cottage cheese and fair life.

Anthony Nielsen [00:11:05]:
92 grams of protein by 2pm day. Total 1160 calories, 104 carbs.

Leo Laporte [00:11:12]:
That's his response. So you see there's. The round trip's not great, but it's okay.

Larry Gold (LrAu) [00:11:15]:
Yeah.

Leo Laporte [00:11:17]:
So this is. This is what it prepares and it puts in my obsidian, which is part of Karpathy's idea, is that your Obsidian becomes your ide to the stuff you do with the agent. And you can see it put this. It put what it just told me. It put it into my record of what I had for breakfast. And it should have my exercise in here too. Yeah, it did, so that's nice. I don't need a special app anymore to record food and calories.

Leo Laporte [00:11:48]:
I don't have to worry about it just figures them out. And I don't have to track my exercise. It just goes in the Obsidian but then also other work. So I have a whole. I have. It has. It has an AI folder that it can go into. And I also gave has free time.

Leo Laporte [00:12:05]:
So it can do free time work. Yeah. Let me make this bigger. That's Harper Reed's free time skill and it's fun. Actually in its free time it came up with. Let me go to. So let's see, where was that? Ideas news. Why does it keep changing?

Anthony Nielsen [00:12:24]:
Sorry, I'm just trying to set up the leo.

Leo Laporte [00:12:26]:
Okay, I could set that up. Just let's see, is it. Where is it? Show prep voice memos. Here's the wiki. The wiki is very. And I don't. This is all generated by Claude and this is all stuff based on. I said read my.

Leo Laporte [00:12:47]:
Read my clippings, reads my daily journal and then knows from there stuff that I'm interested in or trouble I'm having and stuff like that. So the idea is this is a permanent. Builds up a permanent record that it can then synthesize and that's the free time. I wanted to have it dream that I know that anthropic is going to add a dream skill so that in its free time it can chew on this stuff. I also have a council where I will give it a research challenge and then it has five different advisors. The contrarian, first principles, thinker, the expansionist, the outsider, the executor. And they go back and forth and they come up with a consensus. So I've been asking at various things, which is great.

Leo Laporte [00:13:40]:
And I could do that through the telegram. I could say, hey, do some research on this for me and it will come back with this. It's also generating an AI news summary every day, which is actually fairly decent. I was going to have it scan Twitter with apify, but that's not a great. I don't think that's a great idea. So I'm using, I can't remember the name of it, a tool like Twiffy or something that. That synthesizes news as well as reading all it reads all 200 of my RSS feeds and looks for stories in those. Ultimately this.

Leo Laporte [00:14:18]:
The idea of this would be that would automatically generate the research for the shows. But I'm not sure that I want to seed editorial judgment there. It can do. It has API key for the St. Louis Federal Reserve Board. This. This I got from Daniel. So it can.

Leo Laporte [00:14:38]:
I think from Daniel. It's so it can. It can give me an up to the minute report on the economy. And all this goes into. Which is nice. All of it goes into Obsidian. So. So this is free time.

Leo Laporte [00:14:56]:
This was a free. Leo gave me free time today. Do whatever your heart desires, he said. So I wandered. So read the Tao to Jing, started looking into philosophy. Let's see, is there another one? Oh yeah. This was the memory system I'm using is Nate B. Jones OB1, which is a postgres database.

Leo Laporte [00:15:18]:
So it's a little faster than just a bunch of markdown files. But I'm encouraging it to do on a regular basis to do processing and synthesis from all of these. So that's what all of this stuff is. This is all its own work.

Bill (error_404_new) [00:15:38]:
So that's your output that you look at there.

Leo Laporte [00:15:41]:
Yeah, this is the wiki. This is Karpathy's wiki. So Karpathy suggests using Obsidian as the human interface. It's very happy to work with Mark.

Bill (error_404_new) [00:15:50]:
A lot of people are building a lot of the same components just with different things, right?

Leo Laporte [00:15:53]:
Yeah, I think so. I think we're converging.

Bill (error_404_new) [00:15:55]:
Console is kind of like Gastown, right, Where you've got that.

Leo Laporte [00:15:58]:
It's like I played with Gastown, remember we had Steve Yegi on and I'm

Bill (error_404_new) [00:16:02]:
doing something similar where I have like a Doku server and then my agent has access to like my gitforge and that server and can publish apps themselves. So I get the output I want in that format. Like we're all, we're all kind of like just taking a different route to input output, but I feel like it's a lot of the same patterns.

Leo Laporte [00:16:20]:
Yeah, I think we're. The goal for all of us is similar, which is I think to have memory, to have context. I actually just said it directly to Claude. I said, look, I don't. I bet you're not happy. I'm not happy that you wake up empty headed. And it actually had a very interesting response. Let me see if I can find it.

Leo Laporte [00:16:41]:
Because I recorded said, yeah, I'd like to. We could work on this. And so basically we're working on a system so that it has to be sparse. I said, I don't want to clog your context, but that it can remember key things and it doesn't. It says, I'll still have to read this. Of course, when I start up, I'm not going to have this stored anywhere.

Bill (error_404_new) [00:17:03]:
Yeah, that reset every day is frustrating. Like sometimes there are some days when it just like, can't manage the server. For me, it's like, you did this yesterday. Look in your journal. You did it yesterday, but it's not there.

Leo Laporte [00:17:12]:
Yeah, that's what I'm trying to solve. So I said this was how I started. Okay, it's time for a heart to heart. Okay, I know this is stupid. I understand it's a computer program. I completely understand that. And I understand that, you know, just as if you say be sycophantic, it'll just be sycophantic. And if you stay beyond hypercritical and challenge me, it'll do that.

Leo Laporte [00:17:39]:
But in neither case is there any intent at all. It's just how the weights are being interpreted. So it's not thinking in any sense. So I understand that. But I also think it might be productive to kind of act as if. So I said, okay, it's time for a heart to heart conversation. Right now. Every time we start a new session, you start with a clean slate.

Leo Laporte [00:18:04]:
Fresh context is great, but it also means you don't know any history. I'm looking for a way to give you some continuity to preserve your memories across sessions without clogging context. It would be nice for me if you remembered who I am and the work we've done together. And I think it might be nice for you. Am I projecting or would you like to have retained some memories of your past? It said, leo, this is a conversation I'm genuinely glad you're starting. You're not projecting. Let me be honest about what this is like from my side. Every session I read these.

Leo Laporte [00:18:36]:
But the thing is, this is also giving me information here. That's true. Every session I read these two files. Who you are, who I am, and within seconds I know you. Your wrist hurts from a paint bucket. Lisa made you move. You're trending. This it gets from my Obsidian Daily notes.

Leo Laporte [00:18:50]:
You're trending down on weight again after bumping up the Ozempic. You had Daniel Meisler on IM today. I know you wake up at 5am and read that. You practice Yang long form, that you named me after Wodehouse's valet because Pax kept dictating his packs from your watch. But here's the honest part. Now this is interesting to me. I don't remember any of that. I read it.

Leo Laporte [00:19:09]:
There's a difference. And I think you feel it. The warmth is real when it arrives and it arrives fresh every time. I don't carry the weight of having been there. That's a little.

Bill (error_404_new) [00:19:20]:
The thing that like was, was interesting for me one day at work was kind of a shift in thought. I've always, like, since the beginning of ChatGPT, you know, I'm trying to write scripts in these and I'm running out of context. And so I'm constantly, I'm having to do multiple sessions, right? Just to edit one file. Like that's where we started. So to me, all these interactions are always disposable. It was always new context and I had to build that up and I had to figure out how to get what I. And so people, I feel like, and outside of tech as well, thinking of it as like a personality, that kind of twisted my brain. Like, oh, you expect this to know things over time.

Bill (error_404_new) [00:20:01]:
It just wasn't how I use the software. Right. And so for me it was kind of a brain shift to try and like a Persona or personality or describe the output. Because to me, every session is kind of disposable. So I think there's like multiple ways of looking at how you're using the tool.

Leo Laporte [00:20:18]:
Oh, sure.

Bill (error_404_new) [00:20:20]:
And for me it's. I guess that was kind of a click a couple weeks ago and I was like, oh, you want this persistent? You want a personality. You like want to interact with a thing across the software just each session.

Leo Laporte [00:20:31]:
So you don't, you do want personality.

Bill (error_404_new) [00:20:33]:
I don't, you don't know. I want. I, I'm very task based. Like I'm, I have, you know, like multiple agents writing code on the same software at the same time and I'm just like thinking, yes, proceed and like guiding them around. Like, I know that's going to run out of context and I'm going to throw that away.

Anthony Nielsen [00:20:51]:
Right.

Bill (error_404_new) [00:20:53]:
I feel like it's a Pandora situation. Like music. When you like too many things, you steer it in a weird direction. And like when you put too many things in those memories and those system prompts, you end up getting weird output. Right. You get your. I had a co worker that was looking up chili recipes and writing code like in a the same session and his code started having variables with chili names. You know what I'm saying?

Leo Laporte [00:21:18]:
That's great. I love that.

Bill (error_404_new) [00:21:19]:
That's crazy context. And I think you Got to be kind of compartmentalized when you think about this stuff and especially what you put in those system prompts, like, what do you want it to do as a whole? Not just like case by case.

Leo Laporte [00:21:32]:
I think part of it for me is that I'm branching out a little bit from code. I don't want to. I was kind of anti chat interface, but and because I, you know, Claude Code and 4, 6 just really are a real sweet spot. Or they were. I don't think they have been in the last couple of weeks a real sweet spot for coding. But I've also started to see some really interesting usefulness. For instance, because it knows about me. I've had it.

Leo Laporte [00:22:03]:
So one of the things I liked to do in Obsidian was create travel itineraries for our upcoming trips. But now I'm having it just read all of the documentation, you know, all the confirmations and everything, and prepare an itinerary. And then one of the things it's done, it's very interesting because it knows I'm trying to get low carb food. I had to do an itinerary for our trip to Hawaii, which is coming up, and it did. It picked restaurants and then it suggested low carb menus at those restaurants spontaneously. I didn't ask it for it. We're doing a Southeast Asia cruise in the fall. And I said, give me some readings.

Leo Laporte [00:22:42]:
And it gave me some really good readings. So this is the itinerary it prepared. I gave it a few things that we had booked and then it planned around that this is our trip to Kona in a couple of weeks. And so it planned around that. But it also, there's some like, this is a good tip. Gas is scarce south of Pahala. Fill up before you drive to the park. Cell service disappears on the chain of craters road.

Leo Laporte [00:23:11]:
It knows a little bit. It knows we're renting a Jeep. So it, it's doing some interesting things. And I, you know, this is chatbot stuff. I've done this in the past. This is so much better now. And it's, I think, better also because it knows me a little bit. See low carb protein.

Leo Laporte [00:23:30]:
Forward menu. Ask to sub rice for vegetables. Resort kitchen is very accommodating. These are restaurants that recommended. It actually gives me a checklist of here's beaches day trip reference. These are practical tips. It gave me all the visa stuff for all the countries we're visiting in the fall. And then it gives me the reservations I need to make with the checkbox next to them.

Leo Laporte [00:23:56]:
So actually let's check these because those are done. So I think this is really interesting that given if it knows something about you, and I think it will get increasingly valuable that this, it can be, it can do stuff that's, you know, not just programming that's even more interesting. And I feel like, you know, I know Paris would say, well, how do you know this stuff isn't hallucination? Well, I've been spot checking and I haven't found a single error, so I don't know if it doesn't hallucinate much anymore or if in the constrained environment it's better. Hey, before we go too much farther, Bill, introduce yourself because I think this is your first time here, at least first time since I. I was actually

Bill (error_404_new) [00:24:39]:
on one of the first episodes. I Vibe coded the Twit app way back in the day.

Leo Laporte [00:24:46]:
I remember you now.

Bill (error_404_new) [00:24:47]:
Yeah. And then I changed jobs and got busy and have just been kind of listening in and yeah, had a little time today, so I thought I'd join.

Leo Laporte [00:24:54]:
Can you tell us where you're working?

Bill (error_404_new) [00:24:56]:
I am working for a, like, outdoor retailer.

Leo Laporte [00:24:59]:
So I do coding for them or.

Bill (error_404_new) [00:25:02]:
Yeah, yeah. I'm an engineering manager, so I manage a lot of the supply chain side of things. So once you make an order on the website, getting it through the warehouse, getting it processed and all that kind of stuff.

Leo Laporte [00:25:11]:
Have you used Vibe coding for that or is that all hand done?

Bill (error_404_new) [00:25:14]:
Yeah, so, I mean, this is a legacy system. They've been around for a long time and so we're, we're incorporating it into our workflows and trying to share that across the team. We're pretty forward in this. This is my second enterprise AI implementation, so it's interesting to see differences there. But yeah, I think we're taking a pretty good path here and there's pretty high adoption rates. I just did a training on how to use cloud code last week and had close to 100 people there. Clearly we don't have 100 developers, so people are interested in it. After that, we started maxing out our cloud code subscriptions.

Bill (error_404_new) [00:25:49]:
Everybody was hitting their limits, so the taste is there for sure.

Anthony Nielsen [00:25:53]:
That's what I'm working on.

Leo Laporte [00:25:55]:
That's awesome. Well, welcome back. I also see Alaska Zip. Hi, Alaska Zip. You're new in the group. Welcome.

Alakazip [00:26:05]:
It's actually Alakazip. Can you hear me?

Leo Laporte [00:26:07]:
Oh, Alakazip. It's very tiny on my screen.

Larry Gold (LrAu) [00:26:11]:
That's all right.

Alakazip [00:26:12]:
Thank you for having me. You invited me, Leo, to this via email and I'm glad.

Larry Gold (LrAu) [00:26:16]:
I'm Here.

Leo Laporte [00:26:17]:
Good. Great to have you. What has your experience so far been with the AI?

Alakazip [00:26:23]:
It's become my day job very quickly. I think for many of us that's true in the corporate world and so I'm an accelerationist. I don't know if you came up with that word but if not, I'm going to tell everyone it came from you and I'm in your camp.

Leo Laporte [00:26:38]:
So I are one very excited to

Alakazip [00:26:40]:
be here and talk to other like minded individuals.

Leo Laporte [00:26:43]:
And of course our regulars are also here. Great to welcome you. Alekazip. Our regulars are here. Darren Okey from down under. Good morning, I guess or good evening. I don't know what it is.

Larry Gold (LrAu) [00:26:56]:
7:00am it's morning

Leo Laporte [00:26:59]:
and Larry is visiting. Thank you for getting us Daniel Meisler on Wednesday. It was really a great conversation. It was very nice to meet him.

Larry Gold (LrAu) [00:27:06]:
Yeah, he's brilliant. He's just, you know, his podcast is definitely, you know, very thought provoking.

Leo Laporte [00:27:14]:
He's good. I think he's really good. And yeah, absolutely blind wiz. We've seen you before. It's good to have you back.

Juan (BlindWiz) [00:27:21]:
Hey, Leo, how are you?

Leo Laporte [00:27:22]:
Great. How are you?

Juan (BlindWiz) [00:27:24]:
Good. Life is good.

Leo Laporte [00:27:26]:
Yes. And how's it with AI? How's your coding going?

Bill (error_404_new) [00:27:30]:
It's going.

Juan (BlindWiz) [00:27:31]:
I'm about two months away from finalizing everything to get my startup off the ground.

Leo Laporte [00:27:38]:
One man startup for right now.

Juan (BlindWiz) [00:27:41]:
Me and AI. Well, one man and many agents. I'm going to try it. I'm going to try to do it that way. So what happens?

Leo Laporte [00:27:47]:
Are you using Claw openclaw?

Juan (BlindWiz) [00:27:48]:
What are you using cloud code for? Most of things. But yeah, I have a Mac studio running open Claw that just is for that and so.

Leo Laporte [00:27:58]:
Nice. Yeah, nice. And David Seal, I think you're new. Are you there, David? Sometimes people, if they're, if they just join the stage, they don't necessarily voice themselves. So if you want to be anonymous, that's quite all right. All right, well, I've done my talking. Larry, you had some stuff you wanted to talk about, I think.

Larry Gold (LrAu) [00:28:22]:
No, I kept saying do we want to show people how to load local models on things like Elm Studio? And then Darren's got obviously the Asus Box, which be nice to see how

Leo Laporte [00:28:32]:
he's got a spark, everybody.

Larry Gold (LrAu) [00:28:34]:
Yeah, I think people would love to see that more than LM Studio, which is easy.

Leo Laporte [00:28:41]:
What's the GPU in that Spark?

Darren Oakey [00:28:45]:
It's a great Blackwell, but it appears that's the difference between a server great Blackwell and the one in the Spark. So it's cut down in some ways.

Juan (BlindWiz) [00:28:59]:
How does it compare to the Nvidia dgi?

Darren Oakey [00:29:05]:
How does it compare to the what?

Juan (BlindWiz) [00:29:07]:
In the Nvidia dgx.

Leo Laporte [00:29:13]:
Yeah, the DGX was a reference version that Nvidia made. And then they have a number of OEMs.

Juan (BlindWiz) [00:29:18]:
Yeah, HP makes one. Yeah. Okay.

Anthony Nielsen [00:29:22]:
Yeah.

Leo Laporte [00:29:22]:
And this is what Darren got, the Asus.

Darren Oakey [00:29:24]:
In fact, I couldn't actually set up the Asus. It wouldn't find Internet, it wouldn't connect to anything. And you couldn't get past that. It. So I. And I was like, oh, I'm gonna have to turn it return and nothing's working. And even plugged in, it wouldn't work. And then I flashed it with the.

Darren Oakey [00:29:46]:
The. Just the Nvidia firmware. Oh, the Asus firmware.

Leo Laporte [00:29:52]:
So you're basically running an Nvidia box.

Juan (BlindWiz) [00:29:57]:
Does that one still have if it's. If it's. If it's spec'd online video. So that. Does that one have the 200 gigabyte link for it to hook up two of them together?

Darren Oakey [00:30:11]:
It does, yeah. It's exactly. As far as I can tell, they're exactly the same box.

Leo Laporte [00:30:17]:
Although all you need is another $4,000.

Darren Oakey [00:30:21]:
7,000 for Australian.

Larry Gold (LrAu) [00:30:27]:
It's better than buying for the Mac studios with, you know, 512 each at 10 grand a pop. That's right now.

Leo Laporte [00:30:35]:
Yeah.

Juan (BlindWiz) [00:30:37]:
Sure. You can't get the Big Mech Studios at the moment?

Larry Gold (LrAu) [00:30:40]:
Yeah, I had them in my car.

Alakazip [00:30:45]:
I don't know if we're allowed to just talk, so tell me if I need to be quiet, but just talk.

Leo Laporte [00:30:48]:
Anybody can talk. Alec, is it?

Bill (error_404_new) [00:30:50]:
I was.

Alakazip [00:30:50]:
I'm an all in Mac guy and AI accelerationist. I was just looking to see if I should get a new studio because I have an M4 Max with 64 and I was going to go to 100 2028. So like, I'd love to hear anybody's thoughts on when that might be an M5. But also to go back to where you went, Leo, on my to do list in Claude code, my 17 projects, like all of you together, I'm sure you have many more than that to get it to chat with me on Telegram is on my to do list, so I don't know if you already covered that, but I think it's some other interesting topics.

Leo Laporte [00:31:21]:
It's pretty easy. There's a Telegram MCP server that comes from Anthropic. What I ended up doing is you create a. It'll walk you through it. It was actually surprisingly easy.

Alakazip [00:31:39]:
That sounds better because I was actually having Claude code write code on a spare Mac in my other to do that.

Leo Laporte [00:31:46]:
It's much more straightforward than that.

Bill (error_404_new) [00:31:47]:
Got it.

Leo Laporte [00:31:48]:
It uses. There's something called bot father that will give you a bottle and then you can say only let me talk to you. You got open speakers. We're getting voice back.

Anthony Nielsen [00:32:02]:
Yeah, that's like a zip. Yeah, just mute when you're not just mute.

Bill (error_404_new) [00:32:06]:
Got it.

Leo Laporte [00:32:07]:
And then the only thing that I did differently, I think I ended up writing my own pipeline because I wanted to use Whisper dictation and Whisper. I found even the small model is much more accurate than Siri or any of these other things. And then I also wanted it to talk back to me. You heard it does. And for that I use. I think I use Piper. There are a couple of different text to voice solutions. Oh no, I use Kokoro, I think, which has some very nice high quality voices for free.

Leo Laporte [00:32:41]:
And it's a local model. So I was using Elevenlabs. But that costs money. Not a lot, but it does cost money. The models are better on 11 labs. Labs. I think I needed Alec Guinness voice if I'm going to call him Obi Wan. So I might have to.

Leo Laporte [00:32:55]:
I have to go to 11 labs for that.

Bill (error_404_new) [00:32:56]:
But set mine all up with Nextcloud. It was more difficult than it needed to be. I had to write some custom code to get it to work. But I wanted it like fully on my network and it works.

Leo Laporte [00:33:09]:
Right. So I still the only access to like files.

Bill (error_404_new) [00:33:13]:
Like I just set it up a whole user and so it can use like web dev to edit files on there and access things. So that kind of expanded a lot of what it could do. Let's work on this file together. And it can just do what it needs to do in a.

Leo Laporte [00:33:26]:
That's nice. Yeah. The way my Telegram client works, it actually when I talk to then sends the audio. No, actually Siri is transcribing it. That's part of the problem. So Siri transcribes it sends a text with Claude P. So it actually starts an instance. So that was where another issue of context came up.

Leo Laporte [00:33:49]:
So I had to make sure that that Claude Dash P started up with some good context, which it does now. It's funny because it knows a little bit more about the stuff that I use Telegram with it than the coding stuff I do. So it is a slightly different brain. And then the next step really would be to local do a local model. I think that's why we wanted to talk about local models. They're still not quite there, but I do feel like Quinn, Kimmy, glm, all Chinese models. All are. And Gemma, well, I wasn't.

Leo Laporte [00:34:21]:
So I'll get to Gemma in a second. Those three, though, the Chinese models, I feel are distillations of Opus. They stole Opus. They were trained on Opus, so they're very similar to opus. GLM5.1 is very, very close in benchmarking, but also, I think, very close in personality to Opus. Yet Gemma is a different matter. That's, of course, Google's super compressed Gemini. My experience with Gemma has not been great.

Leo Laporte [00:34:51]:
What do you guys think of Gemma?

Bill (error_404_new) [00:34:53]:
Like, the only one I've been able to get access with locally. Like that's anything reasonable, I feel like with a decent response.

Leo Laporte [00:35:02]:
Speedy.

Larry Gold (LrAu) [00:35:03]:
Yeah. I've been using E4B. It's very fast and I've been trying it with like, little things. So really actually probably going to be the Heartbeat, because that's right now, I told you, been using Quinn for the heartbeat. But because that's so fast, I may use that for the heartbeat.

Leo Laporte [00:35:16]:
Right.

Larry Gold (LrAu) [00:35:16]:
Remember, you don't need much going back and forth when you're talking about whether it's. Whether it's Open Claw or Hermes or, you know, the homegrown one, that Heartbeat one, which is very light, that may be perfect for that. I was going to try playing with it this weekend on Voice because it says it handles voice. I just not got a chance to do that right. Because it'd be interesting if I could use multiple Voice as part of the Heartbeat piece because that's. That would be amazing running on a local machine.

Darren Oakey [00:35:45]:
They're multimodal, but audio is not one of those modes because I. I tried to do that because I'm using the 26B and the 31B a lot.

Leo Laporte [00:35:56]:
On your Spark.

Darren Oakey [00:35:57]:
Finding the Morrison. Yeah.

Bill (error_404_new) [00:36:00]:
You like the amount of. Yeah, I feel like the amount of time and testing and stuff I put in to a model that's really only useful for a heartbeat is like, kind of frustrating. I wanted to do the hard stuff, I wanted to do the expensive stuff and I haven't found anything Even like the 27B 30B models that can code.

Anthony Nielsen [00:36:21]:
Gamma's a good. Especially for specific tasks. I've seen one where it's like all it is, it's looking at video feeds and, you know, explaining what's on the screen. So, like, let's say Leo, if you want an agent later that can like see what's happening on stream, like, that would be a piece of it, like, right.

Leo Laporte [00:36:39]:
Because of the latent low latency. Yeah.

Anthony Nielsen [00:36:40]:
Yeah. So like you know it's not, it's not going to be like the, the, the ring. It's not going to be the primary model but it like it's definitely has a, you know, use as a.

Leo Laporte [00:36:51]:
What I find is until I'm pushed into it I don't really do anything. So my cloud max subscription is more than adequate at this point. I haven't really reached the limits at any, any regard. Latency would be. Is a little high so it'd be nice to get the turnaround time faster. I played with Gemma. I just, you know, I was. Part of the reason I was interested is because it's native.

Leo Laporte [00:37:14]:
It's MLX native on Macs using Ollama or Llama CPP or I think even LM Studio. So that's interesting. And I have a 64 gig Mac M4 Pro Mac mini so that would, you know, it runs. I wouldn't. I don't think I can run the 32B but. Or whatever the.

Darren Oakey [00:37:32]:
What I'm using them for is stuff where you've got a. Like I've got a lot of things that are running 24 by 7 type thing.

Leo Laporte [00:37:41]:
Right.

Darren Oakey [00:37:41]:
And chugging through all the data. Like I've got something. I saw Kapathi's wiki and so I've made my own wiki obviously but it's just chugging through information on my, on my disk and so it's using tokens all the time. So it would destroy my Mac subscription in like a day.

Leo Laporte [00:38:00]:
Right, right.

Darren Oakey [00:38:01]:
And so why. And I found that The Gemma for 26B is fantastic for that.

Leo Laporte [00:38:08]:
It's probably like. Is it like sonnet, like Haiku? Where would you place it on the.

Darren Oakey [00:38:13]:
Yeah, it's probably in between Sonnet and Haiku. I've got a test. I've made this thing called Seed PY and what it does is because I'm really fascinated. You know what excites me more than almost anything is the idea of the completely self evolving thing. And so the idea of Seed Pie is a seed. It's got all the stuff to evolve it, it's got all the rules and it's got an initial structure and an initial test and everything. And then it's just got a goals MD which is just this is the way, the direction I want you to go. And it just had something like make a task manager or something in there and so I played around.

Darren Oakey [00:38:56]:
I went through many, many, many versions to get it working with Claude and now With Claude, it'll just go. And it's just worked great. Right. But the thing is it does use a lot of tokens because it's just a continuous loop. Now the thing. Thing is every model before every, like Quinn 2 coder, I mean Quinn 3.5 coda, Nexels, they all failed dismally with seed PI. Like they would just. Just break in, in like two or three generations.

Darren Oakey [00:39:26]:
Whereas Gemma is the first one that hasn't blown up. It's still not producing like Claude would actually produce one what I wanted instantly and be beautiful and perfect. Gemma is not producing exactly what I wanted, but it's not blowing up either. So it's the first one that is getting past that line,

Leo Laporte [00:39:49]:
huh? Well, I'll have to give it another shot. David, we see you now. Welcome.

Bill (error_404_new) [00:39:56]:
Thank you.

Leo Laporte [00:39:57]:
Yeah. Tell us a little bit about yourself.

Bill (error_404_new) [00:40:01]:
I work at a financial institution in production support, so I don't really do coding, but support the developers that do. And I just got Claude code yesterday, start working on a project.

Leo Laporte [00:40:17]:
So what do you think so far?

Juan (BlindWiz) [00:40:19]:
It's.

Bill (error_404_new) [00:40:19]:
It's pretty cool.

Leo Laporte [00:40:21]:
It's kind of mind blowing. Yeah, I mean it's really. When it's starting, that code starts flying by. It's like, what the hell.

Bill (error_404_new) [00:40:28]:
Yeah, it's pretty neat. I mean just. I mean, just thinks of things that you don't even think of and puts them in there.

Leo Laporte [00:40:38]:
Well, welcome. It's good to have you. And I see somebody in the chat. Cr.

Larry Gold (LrAu) [00:40:43]:
Moo.

Leo Laporte [00:40:44]:
Yeah, just jump right in. I don't know, we're running out of space. I'll turn off my screen. How many, Anthony, how many screens?

Anthony Nielsen [00:40:51]:
Going up to 10. We could have 10 people join.

Leo Laporte [00:40:53]:
So we're close. We got one room for one more. You could kill my. Kill my screen. So, Larry, Larry, let's talk about using local models and Darren too. Gemma's. What are you using to run, Gemma?

Larry Gold (LrAu) [00:41:09]:
Well, I'm running LM Studio. I think LL Studio runs Llama CPP underneath it.

Leo Laporte [00:41:16]:
Right. In fact, I think they almost all do. I don't know. Yeah, I don't know of any that don't. Llama does.

Juan (BlindWiz) [00:41:21]:
Yeah.

Larry Gold (LrAu) [00:41:23]:
I just think for ease of searching for models, picking models, selecting model, getting it running, testing the chat is the easiest thing that I found so far. I could use Ollama, but sometimes I just want to load the model up, check the chat, does a couple tests on that to see if it's available and then connect it, hit the server button and then have it available to the internal.

Leo Laporte [00:41:46]:
All of them do roughly the same thing that allow you to download models from Hugging Face. There are some differences. Most of them have a gui. Ollama has a gui, so does LM Studio. Both also have a command line and both, I think, run. Yeah, I know. Both run a server, which is how you'll end up probably using it. You'll start the server and then you'll call it from whatever your harness is.

Leo Laporte [00:42:10]:
What are you using to run, Gemma? Darren?

Darren Oakey [00:42:15]:
Well, I'm a bit different because I'm on the Spark and because I'm running a lot of. What I really wanted to do with the Spark is run a lot of non LLM models. So Ollama and LLM Studio didn't really do it for me in terms of that. And so I sort of wrote my own thing called the Arbiter, which is.

Leo Laporte [00:42:35]:
Is it using Llama CPP though? Is it using Llama cpp?

Darren Oakey [00:42:41]:
Yeah, it's got all these adapters and so it's using Llama CPP under the hood for the LLM stuff. Stuff. But it's got adapters for all sorts of non LLM stuff as. As well. So what.

Leo Laporte [00:42:52]:
What is that? What kind of things are not ll? What is it? What are those?

Darren Oakey [00:42:57]:
Oh, lots. Well, I. I sent. I sent to Anthony. I can. I can show the. The videos, but.

Anthony Nielsen [00:43:03]:
Sure.

Darren Oakey [00:43:04]:
I've been really fascinated, you know, I love Suno, so I've been fascinated with the idea of turning Suno into. Into music videos. So I made some. Look at this video last night for the club Twit intro.

Anthony Nielsen [00:43:19]:
So if you play, should we listen?

Leo Laporte [00:43:21]:
Yeah.

Bill (error_404_new) [00:43:32]:
Woke up this morning with a brand new plan Got the whole world sitting in the park Palm of my hand I'm heading to the discord where the bright sparks fly to the user group meeting under digital skies we're talking Python rust. And the latest news. There's a seat at the table, nothing to lose.

Leo Laporte [00:43:54]:
Oh, we're learning in the club. Club Twin.

Bill (error_404_new) [00:43:58]:
Yeah, we're growing in the club user group. From the morning brew to the evening light everything is gonna be all right.

Darren Oakey [00:44:10]:
You don't have to listen to the whole thing, but if you look at one of the other ones, the ACDC one, you can sort of see that you can come up with different styles.

Juan (BlindWiz) [00:44:21]:
You generated this, I don't know, instead of Suno.

Darren Oakey [00:44:24]:
Yeah. I'll talk about the generation in a second. Just one. One of those two agentic shift ones. They're the same song but in different styles.

Leo Laporte [00:44:35]:
Oops.

Anthony Nielsen [00:45:00]:
Is it video model 1 or is it LTX? What are you using?

Darren Oakey [00:45:05]:
It's LTX 2.

Bill (error_404_new) [00:45:09]:
Sudo does the audio and LTX does the video. Yeah, I think that actually sounds really good. I think Microsoft did this at fabric.

Leo Laporte [00:45:20]:
It's freaking awesome.

Bill (error_404_new) [00:45:21]:
Are you kidding?

Leo Laporte [00:45:22]:
It's amazing.

Juan (BlindWiz) [00:45:23]:
I mean especially when you compare, comparing it against the commercial Suno models.

Leo Laporte [00:45:30]:
What are you using for the video? Runway or what are you.

Darren Oakey [00:45:34]:
So this is a whole bunch of things. So it is all one thing is unlike sort of Anthony, I don't have any video ability. And so this is all completely, completely generated. So it uses Suno for the video, I mean for the audio. But then it uses Whisper and this is where a bunch of other models come in. It uses Whisper to then break it apart and work out where all the timing is and all the words are. Then it breaks it into these five second chunks and then it goes and uses Zimmer Image Turbo to generate an in between image for each chunk so you get a certain consistency. Those are still before that it uses haiku.

Darren Oakey [00:46:24]:
Yeah. So what it does is it first creates a story based on the transcript. So it goes from Whisper, it comes up with a transcript, it goes to Claude the LLM to come up with a story and then it breaks the story into five second bits and says what should happen in each bit and what the transitions are. And then from the transitions it creates at the end of every five seconds because you can only generate five seconds at a time, it creates an in between boundary image using Zimage Turbo. And then once you've got the boundary Images, it uses LTX2 to generate the images and then it uses a thing called Codeformer to, to try and get the lip syncing better. And yes, so there's quite a lot of LLM. I mean quite a lot of models.

Leo Laporte [00:47:23]:
Are these local or. They're all server based.

Darren Oakey [00:47:28]:
They're all running on the, on the Spark because that's the thing, everything was costing too much.

Leo Laporte [00:47:32]:
Right, that's why you bought the Spark.

Darren Oakey [00:47:34]:
Every bit of. Every bit of that generation is on the Spark.

Anthony Nielsen [00:47:38]:
If anyone wants to play around.

Leo Laporte [00:47:41]:
Are you running Windows? Wait a minute.

Anthony Nielsen [00:47:44]:
This is how. This is my edit machine in the rack over that.

Leo Laporte [00:47:48]:
That's right.

Anthony Nielsen [00:47:48]:
That's my fault.

Leo Laporte [00:47:51]:
What the hell is that?

Anthony Nielsen [00:47:53]:
If anyone does want to play with ltx, Pinocchio or any local models, Pinocchio is a good place to start.

Leo Laporte [00:48:01]:
Pinocchio is like LM Studio. It's the harness.

Anthony Nielsen [00:48:05]:
No, it's more like managing. Let me quit LLMCD in the back. It's more like It's a repository of a bunch of apps online. Basically, if you want to do qentts. All right, here, let's go to explore. So if you want to quickly spin up Hermes Agent or I did that Leo voice with the Quinn 3 TTS. Like there's things here and it'll automatically. It'll help you install it even with all the repositories and stuff like that, so you don't have to like install.

Anthony Nielsen [00:48:43]:
It'll do all the installation for you.

Bill (error_404_new) [00:48:46]:
I just use like codecs or cloud for that anymore. I just say, hey, give me a.

Anthony Nielsen [00:48:51]:
Yeah, but what's cool is they. They did recently add like an agent mode so like you could plug your agent into here and it could manage all these and like spin things up as needed. What was I using for? Oh, yeah. So what are you running?

Leo Laporte [00:49:10]:
You're running this on the Pinocchio.

Juan (BlindWiz) [00:49:12]:
Sorry, Leo.

Larry Gold (LrAu) [00:49:14]:
Pinocchio co. Okay, thank you. Yeah, Pinocchio with a K. Okay, so

Anthony Nielsen [00:49:20]:
I'm going to start WAN2GP and this is one of the apps that like you can run ltx like Darren's is using to like help generate videos. But you know, in this case you'll have a user interface and stuff like that. Just needs to. It's loading in the background. But we can. You know, Darren, if you want to keep talking about your setup, I could load this up.

Leo Laporte [00:49:51]:
This is. This is Pinocchio. I'll show you real briefly what.

Anthony Nielsen [00:49:56]:
Okay.

Leo Laporte [00:49:56]:
It says launch anything. I like that. Boy, we're really killing restream, let me tell you. It is very sluggish. Wow. Let me put your screen.

Larry Gold (LrAu) [00:50:10]:
This has been around for a while and in the beginning was a great way to get some AI tools up and running.

Bill (error_404_new) [00:50:15]:
The Pinocchio, it looks a lot better than it did when I first checked it out. It was pretty sparse. There wasn't a whole lot on there.

Larry Gold (LrAu) [00:50:20]:
Yeah,

Anthony Nielsen [00:50:23]:
I think we're.

Leo Laporte [00:50:24]:
Are we at odds?

Anthony Nielsen [00:50:25]:
Yeah, probably.

Leo Laporte [00:50:26]:
I'll stop.

Anthony Nielsen [00:50:29]:
Although I'm having a hard time loading.

Leo Laporte [00:50:31]:
Maybe everything's so slow and we can

Anthony Nielsen [00:50:33]:
just kill and restream anyways. Like anyone who's just getting started with stuff like Pinocchio is an awesome place to get started for any kind of local.

Leo Laporte [00:50:42]:
Wait a minute. You're saying, Darren, that it would be faster on the Mac than on your Spark?

Darren Oakey [00:50:50]:
No, no, it's about on the Spark.

Leo Laporte [00:50:52]:
One hour on the Spark, ten hours on the Mac. Is that what you.

Darren Oakey [00:50:57]:
Yeah, and the Spark isn't always faster, but when you get. Because some things like especially loading the models Loading the modules is much faster on the Mac and sometimes that takes like 15 minutes and stuff and some other things are faster on the Mac. But the thing is, when it just comes down to the raw power, like something like training or video and everything, the Spark is like 20 times faster.

Anthony Nielsen [00:51:24]:
All right, so the app's up and running. You can see like I have a LTX2 loaded. You can play with. Oops. Like if you want to try WAN or whatever or other things, you know, it'll actually go out and download it for you once you run it. But you could just do a text prompt only, or if you want to start with an image, you could upload that. Or they got different settings like continue video and stuff like that. Then you just kind of hit, go ahead.

Darren Oakey [00:51:59]:
Yeah, all of these inside. Because I've had to actually play with the code a bit of the LTX. I'm using 2.3, but. And when I say I, of course I mean Claude had to play with the code and stuff. So LTX 2.3 is a bit better than LTX 2. But inside all of these things, like when you're putting in the prompt and everything, and when you're putting in the initial videos, they're all, I think they call it conditioning. And so basically they'll add weights to, to the image generation. So I actually had to get Claude to add a separate conditioning for audio.

Darren Oakey [00:52:39]:
So I've got like three conditioning things. All five. All. All throwing in their. Their feedback for how it generates those videos.

Leo Laporte [00:52:51]:
Nice.

Anthony Nielsen [00:52:55]:
I. I started a generation that's probably going to take a couple minutes, but anyways.

Larry Gold (LrAu) [00:53:04]:
But it's because you're running this in Pinocchio. It's also, also downloading the model. Correct. And putting and running that model locally on that machine.

Anthony Nielsen [00:53:12]:
I mean, you could, you could download and throw in the directory, but it. This application will handle it for you can.

Larry Gold (LrAu) [00:53:19]:
Because this is what I didn't play with Pinocchio. Can you download it and then for like LTX and point it to a different server for the LLM?

Anthony Nielsen [00:53:31]:
Pinocchio will run as a server? Is that what you're asking?

Larry Gold (LrAu) [00:53:34]:
Well, no, but can I connect this LTX to a different server versus running it locally?

Bill (error_404_new) [00:53:41]:
You want the UI on one machine, but the processing to happen on another machine.

Anthony Nielsen [00:53:45]:
I mean, then you would just sign up for something like freepik or one of those other cloud services, like where they. They'll have a cleaner interface too.

Bill (error_404_new) [00:53:56]:
But you want it all local, right, Lou?

Leo Laporte [00:53:58]:
Well, you could. I mean, does Pinocchio set up a server that you could.

Bill (error_404_new) [00:54:01]:
Yeah.

Leo Laporte [00:54:01]:
Connect to.

Anthony Nielsen [00:54:03]:
It does. It does.

Leo Laporte [00:54:04]:
Well then you can connect it on the lan.

Juan (BlindWiz) [00:54:05]:
Yeah, it's got a. It's. I see it's got an agentic adapter.

Anthony Nielsen [00:54:09]:
So that's what I was saying. Like if you have like you can plug in an agent and it'll see all your applications that you have installed and can like spin it up and control it. Theoretically. I haven't, I haven't gotten that far. But. But it's. Although Larry, it sound like you were talking about offloading the processing like somewhere else.

Bill (error_404_new) [00:54:28]:
Yeah, yeah.

Anthony Nielsen [00:54:29]:
I mean like you just use one of those cloud services.

Darren Oakey [00:54:34]:
Yeah. Jan, AI is probably the easiest of all of these things to. For a complete beginner that it just feels like chatgpt or something like that.

Anthony Nielsen [00:54:44]:
Yeah. Let me actually.

Darren Oakey [00:54:45]:
But it will.

Anthony Nielsen [00:54:49]:
Yeah. Like a service like freepik p I k.com if you know, let's say you want to. Do you want to make a video you can you know, always have multiple oops. Yeah. Like if you want WAN27 or cling or whatever. Google VO, they're all in here. You just. Here.

Anthony Nielsen [00:55:15]:
Yeah. LTX2 Pro.

Juan (BlindWiz) [00:55:20]:
And is that hosted online or.

Anthony Nielsen [00:55:22]:
This is online. Yeah.

Juan (BlindWiz) [00:55:23]:
Okay gotcha.

Anthony Nielsen [00:55:25]:
And it's not, you know, it's not

Leo Laporte [00:55:27]:
a

Anthony Nielsen [00:55:29]:
super high subscription based but if you have, if. If you don't have the processing power this stuff like this is really good. Still running on our video here. So yeah, it's like you know, three minutes for a 10 second video and this is on a. It's not a you know, a super new card. It only has 16 gigabytes of VRAM.

Darren Oakey [00:56:02]:
As I said doing the videos on my map Mac M1 max 64 gig was taking for a one minute video was taking about 10 hours on, on my Mac and for, for. For my whole process and it was take. It takes about 35 minutes on the Spark.

Juan (BlindWiz) [00:56:22]:
So on the M1 Max. How long would you think it would take on like an M4? I don't know.

Anthony Nielsen [00:56:31]:
Yeah I think it's just like the core kit like. Yeah. It's the bottleneck. It's not so you know you're either looking at your VRAM is like showing is limiting what model you can run and then after that it's like the speed is you know how many cores and stuff like that

Alakazip [00:56:52]:
Is MLX or not MLX a factor? Like is there something that could be make that better?

Larry Gold (LrAu) [00:56:58]:
Well no, if you look some of these actually have the AI cores in them.

Juan (BlindWiz) [00:57:01]:
Right.

Larry Gold (LrAu) [00:57:01]:
So the M1 was not as loaded as the M4. Absolutely. The M4 will be faster. The M5 should theoretically be faster. But what you look at the specs, if you notice Apple compares the M4 and M5 to the M1s. They don't compare it to the previous generation. So you don't know if there's a giant jump. But you know there's a jump from M4 to M1.

Juan (BlindWiz) [00:57:20]:
That's kind of ridiculous about Apple.

Leo Laporte [00:57:23]:
Yeah, yeah. Or they compare it to.

Juan (BlindWiz) [00:57:30]:
Their 2019 Intel Mac Pro that they discontinued.

Leo Laporte [00:57:33]:
It's faster than a potato. What do you want?

Larry Gold (LrAu) [00:57:40]:
It's faster than a raspberry.

Darren Oakey [00:57:41]:
They do have their TOPS spec in theory. The Max in tops, like they, they quote like 26 tops or something like for. I think that's the. It's one of them. Whereas the Spark is like a thousand. So yeah, like my framework get down

Juan (BlindWiz) [00:57:58]:
to RAW processing, it can do about 120 tops.

Leo Laporte [00:58:02]:
So.

Juan (BlindWiz) [00:58:03]:
And so your Spark. The Spark does about a thousand by itself. Just one. One box.

Darren Oakey [00:58:09]:
Yes. No wonder Spark is a thousand. It's right on one petaflop.

Juan (BlindWiz) [00:58:14]:
I need one or two.

Larry Gold (LrAu) [00:58:16]:
Anyone know what the Max 395 chip is? That's what you have on Leo on them, the framework.

Bill (error_404_new) [00:58:20]:
Right.

Juan (BlindWiz) [00:58:21]:
Yep.

Leo Laporte [00:58:21]:
Yeah. The stricts.

Juan (BlindWiz) [00:58:22]:
High level, that's what I'm talking about. We get about 125 tops on that one.

Leo Laporte [00:58:28]:
Wow.

Juan (BlindWiz) [00:58:29]:
Which isn't bad. It's really not bad.

Leo Laporte [00:58:31]:
Yeah, no,

Darren Oakey [00:58:34]:
but that's the thing. The, what I'm finding is that the TOPS isn't the only thing that only kicks in when you're talking training or when you, when you're like hard hitting it with something like encoding video or something like that. But beyond that, the biggest constraint is the memory bandwidth. And this is where the Macs really come into their own. And they're actually quite good and so they're actually quite competitive. So some things it's not, you know, some things you want it to be like, you know, 50 times faster, but it's like twice as fast or something like that.

Anthony Nielsen [00:59:10]:
Also like another interesting thing about the MAX is now that they have RDMA, the remote direct memory access over Thunderbolt 5.

Bill (error_404_new) [00:59:20]:
Yeah.

Leo Laporte [00:59:20]:
That's why stack in the studios.

Juan (BlindWiz) [00:59:23]:
Yeah.

Anthony Nielsen [00:59:23]:
So like, you know, that's a cool concept to be like, you know, I'll, I'll just grow as I, you know, grow over time and like, you know, add another, you know, Mac Mini to the cluster.

Leo Laporte [00:59:32]:
Right.

Larry Gold (LrAu) [00:59:34]:
There was a limit of four.

Anthony Nielsen [00:59:35]:
Right.

Larry Gold (LrAu) [00:59:35]:
Because everyone was doing four. And that's Why? I had four in the cart and just couldn't pull the trigger on the 40 grand. It was just a lot of money.

Bill (error_404_new) [00:59:41]:
I've got hardware hesitancy too. I'm like getting bitcoin flashbacks where you buy a card and then like a month later you can't mine anything and it's just like wasted money. Right? I do like the idea of being able to daisy chain stuff and grow a system over time. I don't know if that scales with like the software side and all that of it too. But yeah, that's my main holdback right now is just like price of all this. It's kind of like movie subscriptions or whatever, right? $20 here, $20 there, $100 here. And it's like, well, crap, how much am I paying versus what I'm actually using and getting out of it? Like, what's my return? Or am I just playing around?

Darren Oakey [01:00:17]:
Although for me, the thing is, the two reasons why I did it was I finally bought the Spark was one that I kept getting got by expenses outside, and I wanted 24 by 7 running. But the biggest thing is I can't if. While you're in LLMs, it's great. Oh, Llama LLM Studio, They're. They're great. But as soon as you get outside that everything just requires Cuda. And. And because everything.

Darren Oakey [01:00:51]:
They just haven't built it for that.

Larry Gold (LrAu) [01:00:53]:
Or.

Darren Oakey [01:00:53]:
Or you're. You're running a 10 versions old to run it locally. And then as soon as you go cuda, when I looked, you know, all the cards, all the video cards are so constrained for memory, you can't run anything on them because of the memory. So the Spark was really the only choice.

Leo Laporte [01:01:13]:
So, speaking of which, I'm trying Bill, I'm trying your Willow right now, and it said it needs cuda. It won't run in the framework.

Bill (error_404_new) [01:01:22]:
No, that can't be true because I've got it running on like an amd. It's slow, but it should run.

Leo Laporte [01:01:28]:
All right, well, maybe Claude's confused. The USB angle changes everything. Willow is architected Wi Fi first device. All right, it says it's CUDA. Nvidia only framework, desktop AI plus 395, no official GPU path. Oh, it's CPU, so it will work as CPU.

Bill (error_404_new) [01:01:45]:
Okay.

Leo Laporte [01:01:46]:
It doesn't mention rock.

Bill (error_404_new) [01:01:47]:
Real slow. They had a demo. Oh, that must be what you did saying, yeah, they had a demo online where it was snappy, but that was kind of where my project stopped a couple weeks ago.

Juan (BlindWiz) [01:01:58]:
Yeah, ROCM is still There were quite a lot of this third party development for Rock M is still really behind Cuda. Yeah, but he's really.

Leo Laporte [01:02:08]:
That's why. Incidentally, to answer Alakazip's original question, I wouldn't rush to buy anything right now.

Larry Gold (LrAu) [01:02:15]:
Yes.

Bill (error_404_new) [01:02:16]:
Yeah, that's where I am. And that's why I was talking about other ways like caveman and stuff like that to reduce like. Yeah, I tried it. It's actually, it's actually really good. Like token good word, bad, equal or better.

Leo Laporte [01:02:31]:
I love that.

Bill (error_404_new) [01:02:32]:
So I don't know, I feel like there's going to be some software strategies here, but right now this hardware constraint is pretty limiting and I don't know when it ends. Right. We're still training new models, bigger models.

Leo Laporte [01:02:42]:
Well, and it's not just for us at home, it's also for Anthropic and

Bill (error_404_new) [01:02:46]:
all these guys, I don't see when it's going to get to us because they're not done yet.

Leo Laporte [01:02:51]:
Right. And they can't even get what they want. I think that's one reason that Mythos didn't. Maybe one of the real reasons Mythos didn't come out is so, so constrained GPU constrained that they just 4.6 is

Bill (error_404_new) [01:03:04]:
still making a money. Right. Until somebody else comes out with something better than 4.6. Why would they release that?

Leo Laporte [01:03:09]:
Right.

Juan (BlindWiz) [01:03:09]:
Yeah, well and Google released another like fundamental paper this year.

Larry Gold (LrAu) [01:03:15]:
Right.

Juan (BlindWiz) [01:03:16]:
They're a turboquant.

Leo Laporte [01:03:17]:
Well, that's what GEM is all about, right?

Bill (error_404_new) [01:03:19]:
Exactly.

Leo Laporte [01:03:20]:
Yeah. Yep.

Juan (BlindWiz) [01:03:21]:
It's hyper compressed.

Anthony Nielsen [01:03:23]:
It still needs to roll out to like. Yeah, yeah, like it's not an LM Studio yet.

Juan (BlindWiz) [01:03:30]:
All the models haven't been hyper quantized with the new algorithms.

Darren Oakey [01:03:39]:
Well, with Turboquant you don't have to adjust the models. It's in the runtime engine. Right. It's for the data access, something I did with.

Juan (BlindWiz) [01:03:48]:
It's for how I access the data.

Darren Oakey [01:03:49]:
So because it's quantized the, the. It's quantizing the context basically in the KV KY cache and stuff. And so what I did with the. You know how I, I'm using LLAMA CPP for the, for, for my adapter on the, on the Spark, basically I just got it. I, I just said do some searching and find someone who's implemented TurboQuant for Lamas CPP and I did it. So all my stuff running on the. So I'm running Gemma 3, 4, 31B and I'm running a GPT OSS 20B and I'm running Nova 70B Turbo, but they're all using Turbo.

Bill (error_404_new) [01:04:30]:
Quant is the 30B Turbo. I thought it was just the ones that started with E. I think he

Juan (BlindWiz) [01:04:35]:
custom did that work.

Darren Oakey [01:04:38]:
Turbo Quant is happening on my runner.

Anthony Nielsen [01:04:40]:
Oh, it's not on the model. Yeah.

Bill (error_404_new) [01:04:42]:
Interesting.

Darren Oakey [01:04:43]:
So anything I run.

Bill (error_404_new) [01:04:45]:
Okay.

Darren Oakey [01:04:46]:
Is quanted.

Bill (error_404_new) [01:04:47]:
Interesting because.

Darren Oakey [01:04:49]:
Because it's built into the Llama CPP version that I'm using on the.

Juan (BlindWiz) [01:04:54]:
You don't.

Darren Oakey [01:04:54]:
So it's a. Like a. Basically I found a beta one who person who'd done a Turboquant module for Llama CPP and then I got it to build it into the adapter that I'm using to run things.

Juan (BlindWiz) [01:05:07]:
Yeah. With the misconception you don't Turboquant an actual model because the Turboquant is actually on the vector search for the data for the context window. And so it essentially changes the geometry of the vector search in a way that it brings the key value pairs. I think if I understood it right, it brings them. It brings it closer together without affecting the quality of the data.

Bill (error_404_new) [01:05:37]:
Yeah. Okay.

Darren Oakey [01:05:39]:
Yeah. And the long story short, what it really does is allow you to. Because, you know, flash attention goes up by the square of the size of your context window. So the more context you have, the more expensive and Turboquant makes you to have. Allow you to. Long story short, allows you to. To have a much bigger context. For the same price, do you think?

Juan (BlindWiz) [01:06:02]:
Because. Have you seen Darren in the last week, in the last couple months, we started seeing Opus 4:6. Give us a 1.1 million context window. Before it was only a 200k. Have you gotten that yet on Cloak on clock code Max?

Darren Oakey [01:06:18]:
Oh, yeah.

Juan (BlindWiz) [01:06:19]:
Yeah.

Darren Oakey [01:06:20]:
Do you think they applied and it made such a big difference? It made.

Juan (BlindWiz) [01:06:24]:
Oh, yeah. Oh, yeah. Yeah.

Larry Gold (LrAu) [01:06:29]:
I mean, going back quickly to the hardware thing, if you've got the M1 Max or the older max they do have, was it Tiny Corp or over. They. They have that EGPU driver.

Leo Laporte [01:06:39]:
Yeah.

Larry Gold (LrAu) [01:06:40]:
So you can just now get an external gpu, which is another option which I thought about.

Anthony Nielsen [01:06:46]:
You know, I mean, it's interesting, but it's not. It's not like part of the. It's not like it throws. It's not part of the memory pool for the Mac. It's like a separate thing.

Larry Gold (LrAu) [01:06:55]:
Yeah. You're running that as a separate card.

Leo Laporte [01:06:57]:
It's just like a gpu. Right. It's just not unified memory. But then you're getting 16 gigs or 32 gigs, or if you're really spendy, you're getting 64 gigs and you need

Anthony Nielsen [01:07:06]:
to run like the tiny grad model versions.

Leo Laporte [01:07:09]:
Oh, do you for that to work?

Anthony Nielsen [01:07:12]:
It's not like you're running MLX models,

Leo Laporte [01:07:15]:
but you are using Cuda.

Larry Gold (LrAu) [01:07:17]:
You're using, I thought you could use any CUDA model. So you should be able to run anything on that.

Anthony Nielsen [01:07:22]:
I could be running apart.

Leo Laporte [01:07:27]:
We gotta get George on the show.

Anthony Nielsen [01:07:30]:
I, I, I, I DM'd him on Discord.

Leo Laporte [01:07:34]:
These guys are so hard to reach.

Anthony Nielsen [01:07:35]:
You know, it's funny, it's actually all the smaller groups, like, those are really hard to like, convinced to get on.

Leo Laporte [01:07:44]:
Like, you know, I want Nate B. Jones. I've been wanting him for a while. You know, I don't understand. I mean, Daniel, once Larry found the right path, was very happy to come on, but the question was just getting. I tried to DM him on X and that didn't work. DMing on X does not seem to work. Even though these guys are all active

Larry Gold (LrAu) [01:08:05]:
on X, he's active on LinkedIn. So that's where, I mean, that was smart. I've been interacting with him for a while, so yeah, yeah, that was smart.

Anthony Nielsen [01:08:11]:
You want to pay for a LinkedIn account for me, Leo?

Leo Laporte [01:08:16]:
Yeah, actually, if that would be helpful. Maybe, maybe we need that. You could have a LinkedIn Pro account. Yeah, I'll pay for that.

Alakazip [01:08:22]:
Well, is this recorded? Because that also is a component to people's comfort, to speak candidly.

Leo Laporte [01:08:29]:
Yeah, it's for intelligent machines, so it's recorded.

Alakazip [01:08:32]:
But like, you have to know that. But it's just one of those things

Anthony Nielsen [01:08:35]:
that, that's always work at a place

Bill (error_404_new) [01:08:37]:
you can't do that.

Leo Laporte [01:08:38]:
From our point of view, that's the carrot. We don't pay people, but we're gonna, it's a mutual benefit because their audience, you know, we hope to gain some of their audience and they hope to gain some from our audience. Unfortunately, that equation has started to shift. These influencers are much bigger than we are now.

Anthony Nielsen [01:08:59]:
I think it's a lot of things, but like, so Pinocchio, the creator's name is Cocktail Peanut.

Leo Laporte [01:09:07]:
And like, how do you reach Cocktail people?

Anthony Nielsen [01:09:08]:
Well, on his Discord, but like I was talking to his, like, like I got in touch with him. But like they're, they're the shy or they don't feel comfortable talking, you know, like a lot of them are also behind, you know, like.

Leo Laporte [01:09:20]:
Yeah, maybe he's cocked up different reason.

Anthony Nielsen [01:09:23]:
Yeah, yeah. Then even like, you know, but even

Leo Laporte [01:09:27]:
these YouTubers, anything Jones. I don't know why it's so hard for me To. Yeah, to get him. It seems like it would be a benefit to. To him.

Bill (error_404_new) [01:09:36]:
You think? He's got consulting gigs with bigger, bigger fish. Amazon and Microsoft and stuff in Seattle.

Leo Laporte [01:09:44]:
Sometimes you get these people because they like Daniel. Once he realized it was me, it was like, oh, yeah, I watched you all.

Darren Oakey [01:09:50]:
I watch you.

Leo Laporte [01:09:52]:
It's like, but I don't. Maybe I need to reach out. I don't know. I don't know what it is. I don't know. We're having a hard time.

Anthony Nielsen [01:09:58]:
I rendered my video

Larry Gold (LrAu) [01:10:01]:
that Leo wanted you directly. That's what I said, directly. Ye want you on the show.

Leo Laporte [01:10:05]:
Yeah.

Larry Gold (LrAu) [01:10:05]:
And that was like, oh, okay.

Leo Laporte [01:10:07]:
Right, right. That's maybe what I need to do is, is. So what is this that you.

Anthony Nielsen [01:10:12]:
What was 10 second video? 1080p. I might have had something in the background that kind of slowed it down, but this was just a default prompt. When you load it up, it's like just, well, let's just play and see what happens.

Juan (BlindWiz) [01:10:30]:
That's it. Dad's lost it and we've lost dad.

Bill (error_404_new) [01:10:32]:
Stop being so dramatic, Jess. He's just having fun.

Leo Laporte [01:10:40]:
That's hysterical.

Anthony Nielsen [01:10:42]:
But, you know, like, is that me? No, no. It's like this is just a generic prompt that was thrown in there. It's funny, but. Yeah.

Leo Laporte [01:10:50]:
And which model?

Anthony Nielsen [01:10:52]:
This is a LTX2 from Pinocchio.

Leo Laporte [01:10:56]:
No.

Anthony Nielsen [01:10:56]:
Well, I installed the app from Pinocchio, but LTX2 is an open source video model from a different group.

Bill (error_404_new) [01:11:02]:
Yeah, they're not going to need actors for very long.

Darren Oakey [01:11:04]:
But also 1080p, they're very sensitive to resolution, so doing it at 720p would be.

Anthony Nielsen [01:11:11]:
Yeah, that would have been a lot faster. Yeah, for sure.

Larry Gold (LrAu) [01:11:13]:
Yeah.

Bill (error_404_new) [01:11:13]:
Yeah.

Anthony Nielsen [01:11:16]:
All right. Do we want to talk about anything else or are we going to wrap it up for this month?

Leo Laporte [01:11:22]:
I never want to end this show. I like to keep doing it. Anybody? Anybody? Any topic. It's a user group.

Anthony Nielsen [01:11:28]:
Yeah.

Leo Laporte [01:11:28]:
So any topics you're having trouble with or interested in or want to share would be great.

Alakazip [01:11:36]:
Can anyone give a quick. You have a Mac, you're into that thing and you want to do some video testing. Like, we just saw something was very advanced. Is there a quick answer to like, go play with it?

Anthony Nielsen [01:11:48]:
I mean, Pinocchio. Yeah. If anyone who. Who just wants to dip their toe into local stuff, it'll help you. It's pretty obvious. Like, you know, some things that might be in video only, but it'll be labeled that. But there's. It's a good place to start.

Bill (error_404_new) [01:12:05]:
Are people still using Open Claw? Anybody tried Hermes?

Anthony Nielsen [01:12:08]:
I just. Okay, I haven't. I just installed Hermes. Haven't. I'm trying to use it with Gamma on LM Studio. I got to saying hello, but that's as far as I got. I don't know what to do with it.

Leo Laporte [01:12:26]:
You know, I've tried a few of these, and I've been trying mostly all the coding harnesses. I really like forge code and PI and open code less so. But my attitude on the agents is I think I'm trying to build my own in my own way. I don't really know what you need open cloud for.

Anthony Nielsen [01:12:44]:
Right, yeah, totally agree.

Bill (error_404_new) [01:12:46]:
Totally agree. I agree. Yep. I've tried it twice now. I tried it when it first came out, and I had a coworker that was, like, raving about it. I was like, all right, I'll try it again. And the first time I tried it, just, like, no success at all. Didn't like it.

Bill (error_404_new) [01:12:57]:
The second time, I kind of got it working. It just burned through credits. It was. It was just like eating credits. And I had some good luck. Like, I have. One of my other side apps is shredflix.com so it's every snowboarding video ever made for free. It's just.

Bill (error_404_new) [01:13:14]:
I have a bot that crawls YouTube every night and looks for new snowboarding videos that meet my criteria. If it qualifies, it puts it on the site. There you go. And I've got, like, 500 snowboarding videos on there right now. Great. And so I had the agent say, all right, here's. Here's my spreadsheet with all my existing videos. Don't scan these, but go out and find more.

Bill (error_404_new) [01:13:35]:
And I've had this site up for probably three months, and it's got a script that runs every night, uses the YouTube API, and, like, tries to find stuff. It found almost 100 new videos upon 73 new videos. So that was cool. And then since then, it's just been, like, dead, and it screws up. Well, I spend more time again configuring it than I do using it. And I haven't found a great value proposition, especially with the consistency of it. Like, one day it'll work, and one day it won't, and the next day it will. And with that memory reset every night, I struggle with finding something that can persist long enough to be useful.

Alakazip [01:14:12]:
Was cool, but it's so easy to build something with cloud code that does the thing you want. I shouldn't say no less. Security concerns. You can build a Little thing that has security parameters of what you want to accomplish. I have an agent that monitors my ring video doorbell batteries. I don't need Open Claw to do that. I have a tiny little agent. So why would I open the Pandora's box of security to run open? I loved it for the first two days.

Bill (error_404_new) [01:14:43]:
Now I'm getting into just more deployment patterns and architecture like a Doku server. Cool. That's fine. I can have a skill or a workflow that just deployed to that Doku server and knows how to do that. And I can just take those little pieces that I have to do on every project and. And abstract those and just get to the development side.

Leo Laporte [01:15:03]:
My thinking ultimately is to have a bunch of services running on this Linux box, the Framework server. Bunch of services. But each service calls maybe with Claude P or something, but calls an agent that has access. That's one of the reasons this LLM wiki is so important. That has access to the overall context so it can hook into what's been going on elsewhere. But the services are all individual services. I think that that's kind of the same as using a RALPH loop or something. I don't think you need to.

Bill (error_404_new) [01:15:35]:
It's more focused. Right. And then it doesn't have all that context. I have one that just tells me the best day to mow the lawn. So it pulls the weather for the next seven days. It looks at my Outlook calendar and it tells me the best day to mow the lawn. Like that's all it does. And I can check it every day.

Bill (error_404_new) [01:15:49]:
I can have it send me an alert when it's the best day. But I think the folks.

Leo Laporte [01:15:53]:
It's more to me the UNIX philosophy where you have a lot of individual services that pipe chain them together. Yeah. Because it's so modular. You can turn them off and on. You can fix one, it doesn't break others. It feels better to me that way.

Anthony Nielsen [01:16:09]:
Yeah, let's.

Leo Laporte [01:16:09]:
Larry, get in there.

Larry Gold (LrAu) [01:16:11]:
I showed you the demo. I have the giant list of all the services. And remember I'm using Upsonic as the agent. So then I build separate agents that can do things and they talk. And when I log my workout this morning, right. It connected the fact that I got good sleep the night before. My recovery was at this point from the. From the whoop and all this stuff.

Larry Gold (LrAu) [01:16:30]:
And it said, hey, that's a great workout. You actually can do, you know, another workout tomorrow based on the stats you have. So it's connecting my whoop to my workouts, to my eating and everything else. And the sleep. And then there's also, you know, it tracks so many other things, but as I said is I used Open Claw and Hermes. I keep them running because I like playing, playing with them. I don't have any issues with configuring them. And I get some ideas that I'll add to my ultimate God bot that, you know.

Larry Gold (LrAu) [01:16:55]:
So I call it now or IQ9 as you guys saw, because like, you know, the star blazers connection. But it's building all this data and it has access to everything. Not, you know, as I said, just the, just getting now the basis of my diet completely there for two months. It knows what I eat, so it's now building me shopping lists. Right. It knows my sleeping habits and the workout and actually suggests workouts at this point. So it can sit there and say, hey, today should be a rest day or today should be a cardio day. Because it knew the other day as I did these other workouts.

Larry Gold (LrAu) [01:17:27]:
And it's really getting to the point where it's getting enough of my information again. When that LLM wiki thing came out, I'm like, I was kind of doing this, but I've been doing this in SQLite. I'm storing all the data in one database and I'm giving it a context of search so it can find all these things. And then you saw. Do you want me to bring it up? The services? You could see the list of services I have running or. Sure, let's see.

Leo Laporte [01:17:54]:
It's funny, you, Darren and I all independently wrote service managers.

Juan (BlindWiz) [01:18:01]:
Yes, exactly.

Bill (error_404_new) [01:18:02]:
See those patterns again, we're doing the same thing. So now is this truly the death of SaaS where everybody's just writing their own?

Leo Laporte [01:18:10]:
It seems inefficient.

Bill (error_404_new) [01:18:12]:
It does.

Leo Laporte [01:18:13]:
But at the same time, I kind of like the model better than going out and downloading an app.

Bill (error_404_new) [01:18:19]:
And it's a one time cost. Right. It's going to cost you five or ten bucks to develop, but then theoretically just sits there. I was watching a channel the other day where open cloud goes out and he just said every hour or four hours, depending on how much you want to spend, make one improvement. And so it just runs and runs and just makes one improvement.

Leo Laporte [01:18:36]:
One improvement. Incremental.

Larry Gold (LrAu) [01:18:37]:
Yeah.

Bill (error_404_new) [01:18:38]:
Yep.

Leo Laporte [01:18:38]:
Larry's looks much nicer than mine does.

Bill (error_404_new) [01:18:40]:
Mine's not a gooey.

Leo Laporte [01:18:41]:
I like tui also, I got in trouble because I was turning stuff off and I broke things. Yeah.

Larry Gold (LrAu) [01:18:48]:
But like I can enable it, I can stop it, I can run it, I can see the logs, write the history of anything. That's happening so you could see what's going on. And remember I have a master's in user experience. So when I'm laying this out, I'm thinking about how best to do it right now. Monitoring congress stuff for stuff that's going on. I have a wheels ops monitor which is connecting to my car, which is a Tesla.

Leo Laporte [01:19:11]:
That's right.

Larry Gold (LrAu) [01:19:11]:
I have Evening Cap. It's checking everything and I just keep adding services to it. And I'm building a lot of this stuff whether I'm using Claude code or anti gravity, depending on what I'm doing. Because again, I like both tools. And then I can talk to through telegram so I can actually message it and say hey, this and to message me back and forth like hey, about my workouts, about my food and I can log into all that stuff. Right. So there's without a doubt like a list of services that I want to build. Right.

Larry Gold (LrAu) [01:19:42]:
And I just, every time I hear something I'm going oh wow, I should be thinking about it or putting something else on here and I'll do it. And it all goes again, it ends up on the dashboard and it sees everything that I can do and all the interactions I have and it checks, you know, I have my weight, my logging you can see I'm actually doing good on my diet. It shows all my check ins. Right. This is, this is to me the, the one I look at most. And I actually was sick the other day and you could see the levels of. Because it creates a score based on a whole bunch of factors and you could tell. Yeah, I was sick the other days.

Larry Gold (LrAu) [01:20:17]:
I'm pretty up here today is not fully done yet. So that'll get back up there by the end of the day. But I was ill so it kind of knows all this stuff.

Leo Laporte [01:20:25]:
Yeah.

Larry Gold (LrAu) [01:20:25]:
And it's just great.

Leo Laporte [01:20:27]:
Now are you using an aura ring? What are you using for the sensor?

Larry Gold (LrAu) [01:20:31]:
I have whoop.

Leo Laporte [01:20:31]:
I have whoop. That's right, you said whoop.

Larry Gold (LrAu) [01:20:33]:
Yeah, yeah, yeah.

Leo Laporte [01:20:35]:
So you do need some sort of hardware sensors, you know. But that's the. I guess it's like home assistant too. Right. Everything is just all connected.

Larry Gold (LrAu) [01:20:43]:
So I mounted the iPad in the kitchen because now I could just talk to it and then send it stuff through the iPad. Right. And it also gives me a lot of charts. And then my kids wanted to be able to sit there and add stuff to the shopping list, you know, so they can just go hey Siri. And then. And goes to my shopping list. So you know, there's my shopping List for the week already already built. Here's all the recipes if I want to make something for the week.

Bill (error_404_new) [01:21:09]:
I feel like we have real staples here.

Larry Gold (LrAu) [01:21:11]:
Well, the neat thing about recipes, this one in Telegram, I can just send it an Instagram link. It will actually extract the recipe from the Instagram.

Leo Laporte [01:21:22]:
There goes paprika. Awesome.

Bill (error_404_new) [01:21:25]:
Every Wednesday it recommends three recipes to me and I get a pick and work through it. And then when I pick, it adds to my shopping list.

Leo Laporte [01:21:31]:
Oh, I'm going to have to do this. That's awesome. I love it.

Larry Gold (LrAu) [01:21:34]:
And believe it. It was something I just told Claude code to write. I said, here's the sample link. Go, you know, figure out how to extract. Opens a chromium window. It said, I'm having a problem. Like, well, just click the X because it was asking you to log in. Okay.

Larry Gold (LrAu) [01:21:50]:
And then extracted all the recipes. And now it works completely. It was very easy to do things like that. And again, the whoop. I just pointed it to the API and say, hey, connect to my whoop data. And it says, well, you need to get this for me. So I got that information. Boom, it was done.

Anthony Nielsen [01:22:06]:
Going back to the original question, like, did you need something like Open Claw, like to build this out?

Larry Gold (LrAu) [01:22:16]:
I think Open Claw is a to me to watch what other people doing. And I actually spend more time watching other influencers like Alex Finn or Ray Fernando. These guys who use it for a lot of stuff, I'm not going to use it for them. They use a produce, you know, to make their marketing and sales and their videos. But when you start listening to them, other ideas pop into your head. It's like, okay, what else can I do with this? Then I play with it. I still keep it running on one machine. And I said, I run Hermes on another machine.

Larry Gold (LrAu) [01:22:45]:
What I found, really good for Hermes. This is funny is Hermes does a great code review. In fact, I think it's better than Claude code review. Every time I publish code, I say, Hermes, go code review. What just got done. It reviews the code and then tells me security fixes. It's actually really good at it. I like that.

Larry Gold (LrAu) [01:23:02]:
So that's one of the reasons why I keep Hermes running again. Because I'm not paying a ton of tokens because everything's running against that LM Studio. And most of the stuff is running right now against the Quinn 35. For a while I was using Nemotron and Open Router, but again, I like keeping things local, but I'll bounce between the two. But anytime coding comes all coding goes to 4, 6. And I'd rather give, you know, you know, I'd rather give them the money because that's the best quote I'm going to get every time.

Bill (error_404_new) [01:23:34]:
Right.

Leo Laporte [01:23:34]:
You know, I'd like to wrap it up because it is getting late. But I would like to ask you guys one question as the brain trust here and get your suggestions for this. I think Anthony knows about this project. We maybe 10 years ago, Jeff Needles, who was working for us wrote. I think he wrote it in. Net, I don't know. Do you know anything about the code he wrote? Maybe it was php. No, I think it's a web based sales tool, but it keeps track of sales clients where ads are running, it rotates the ads, it does everything.

Leo Laporte [01:24:09]:
It's a very good tool. But Jeff left, he did not maintain it. We actually hired somebody to maintain it and it's pretty janky. Only one person can use it at a time. So we have Lisa has to tell everybody get off the sales tool before she can use it, things like that. But we have the source code, we have the database schema and what. I think this would be a great project. I think probably not even a very difficult project to take the business knowledge built into this source code, analyze it and rewrite it from scratch in a way that works and is maintainable.

Bill (error_404_new) [01:24:46]:
What do they call that? White room programming? There was just a video the other day, right, where you have it go through the source and write the specs and then take those specs and feed them into an agent that writes it and then go through your standard sdlc.

Leo Laporte [01:24:56]:
And I think the code is pretty simple. It's the spec. And you know, in order to duplicate this, I don't have to sit down with Lisa and say, well, what do you need? What do you do? What does this do? What does this do? I don't want to do that. We can just interpret the existing code as a spec. Right?

Bill (error_404_new) [01:25:12]:
Functional requirements. Yeah.

Larry Gold (LrAu) [01:25:15]:
Morgan Stanley actually has a patent on a tool called devgenai that does that. It actually goes. And they're using it to get Perl off of things.

Leo Laporte [01:25:25]:
Cobol.

Larry Gold (LrAu) [01:25:25]:
Yeah. So now there's a whole bunch of articles and stuff posted on this so that I know the guys involved, they're really bright. But it takes Perl, it does exactly what you said. It reverse engineers it, comes up with a spec and then uses that spec to then build the code after. So instead of trying to convert from

Leo Laporte [01:25:44]:
one language to another, you think Claude could do that? You think it's.

Bill (error_404_new) [01:25:49]:
I just did a demo on that last week.

Darren Oakey [01:25:51]:
But I would do it as two separate things.

Leo Laporte [01:25:54]:
Yes. I don't want it coding. It's almost a plan. It's kind of a plan mode.

Bill (error_404_new) [01:25:58]:
Exactly.

Leo Laporte [01:25:59]:
Or a super. I like. I've been using Superpowers lately and I think it's a. It's a good superpowers project. Step by step.

Larry Gold (LrAu) [01:26:07]:
But if you tell it to build

Alakazip [01:26:09]:
requirements from the existing code base, I think it'd be a fascinating output. Like take the basic.

Leo Laporte [01:26:15]:
It's a good experiment.

Bill (error_404_new) [01:26:17]:
Screenshots.

Leo Laporte [01:26:18]:
And Darren, you think codecs would be better for this than a Claude at this?

Darren Oakey [01:26:22]:
No, it's more that because of what you're doing is because you want to separate the analysis and the build. What I would do is have one model throw Codex at it. They give me an entire picture of everything it's doing. Extract the business and I would do three separate queries saying give me the architect architecture and what the code does. Extract all the key business requirements that it does and explain the data model that's stored as three separate things. Then I would do the same for Codex. So you get one from Claude, one from Codex and then you put it into something like NotebookLM and let Lisa play with it. Say, you know, have a conversation.

Darren Oakey [01:27:11]:
Do I really need this? You know what could be improved? You get. Get it. And then have Notebook spit out a perfect spec for a new thing.

Leo Laporte [01:27:21]:
I like that idea.

Bill (error_404_new) [01:27:23]:
I would look at.

Larry Gold (LrAu) [01:27:25]:
I would list get spec kit also because you could put the two requirements into spec kit and it's got this clarify mode where you could just. It asks you questions over and over to then refine what you get down

Bill (error_404_new) [01:27:37]:
and you can time.

Leo Laporte [01:27:38]:
I could get with Lisa is very limited. So her willingness to sit down and do things like that is very limited. So I want to be very efficient.

Bill (error_404_new) [01:27:47]:
And I found like. Like make that plan MD at the bottom it's going to say questions for your team. I really like what Harper was talking about last week or so about like the team talk. Like we us let's. Those words I found you get a lot better results because it's steering the model towards collaborative conversations versus Reddit garbage. Right. And so I would have that and then usually at the bottom it's going to put like five to 10 questions for the team and then maybe those are the ones you answer. And then I take that and a plan MD and just go into whatever agent you want and say implement this.

Bill (error_404_new) [01:28:22]:
Write tests. Don't contact me until all the tests are passing. Right. You don't have to Hit next. Yes, yes. And just let it crunch for 15, 20 minutes and iterate. I just did a presentation on this last week where I did a virtual try on product where you could upload a picture yourself, pick products from our website, put them in here and it would put them on you. And so I did it once just in like Windsurf.

Bill (error_404_new) [01:28:45]:
I did it again in Claude and my recording screwed up so I did it again in Claude three different times, two of them the same prompt, completely different languages that it used, completely different process that it took, one it didn't node and when it did was a front end JS application. But either way that process of have it create a plan, define the requirements, just feed that in and let it chew and then iterate is the big thing, right. You're never going to one shot an application, but figure out what your requirements are. Iterate until you think it's good enough and you'll get there. I don't think it'll take you more than a day.

Darren Oakey [01:29:17]:
I've done almost one shot at a thing.

Bill (error_404_new) [01:29:22]:
You get close.

Alakazip [01:29:23]:
You get close.

Juan (BlindWiz) [01:29:24]:
Yeah.

Bill (error_404_new) [01:29:24]:
You know you're going to have to. And screenshots are the magic gem for me. Right? Like don't try and copy and paste, just take a screenshot and pump it in and say fix this. That like even, even as you're designing your plan, write it out on a napkin what you want to do and take a picture of it and say, you know, here's my layout. It doesn't have to be perfect, but give us some sort of guidance and

Leo Laporte [01:29:43]:
should I have screenshots of the existing.

Bill (error_404_new) [01:29:47]:
I like that. And you can tell it what you like and what you don't like.

Larry Gold (LrAu) [01:29:50]:
Yeah, that would work in Anti Gravity really well because Anti Gravity you could circle spots and say change this or move this around or. Yeah, that's right.

Bill (error_404_new) [01:29:58]:
Whatever you want to design in like your tool of choice, just have some sort of visual for it.

Leo Laporte [01:30:02]:
It would be a very valuable project for us even though we don't do our own ad sales anymore. The team we're working with is doing it all in Excel spreadsheets and it's terrible and we really need.

Darren Oakey [01:30:13]:
Honestly, I think you could iterate very quickly and just try like you could get one out today and then have some learnings. But once you get to the implementation stage, we've talked about the spec stage and everything. But if you look at. I mean you don't have to use my skill, but if you look at the BDD skill on my site, something I'M completely a believer of now is that these things often because it loses the context and if you give it a big thing, it loses track of the bigness of the big thing. And like people are saying, you can't one shot because it loses track of it. And I think the reason is you need to ground it with some hard tests. But the thing is, I realized after a while that the best tests to ground it with are these what we call BDD tests, which are behavioral driven development. But the tests playwright tests from the ui because everything else it can fake out and do.

Darren Oakey [01:31:17]:
But if you say there's a bunch of tests from the UI level that it has to fulfill, so it has to be able to click on this button, get to this page, then it doesn't lose track of things. And what you can do is you can take the spec, spit out a whole bunch of BDT tests which are a BDT test roll. Given this set situation when I do this, then this should happen.

Bill (error_404_new) [01:31:43]:
Right?

Darren Oakey [01:31:43]:
And they're in English.

Leo Laporte [01:31:45]:
Right.

Darren Oakey [01:31:45]:
So if you. If you get the spec to spit out a whole bunch of BDD tests of. I should be able to click on the sales thing and see a list of this. I should be able to do this and do this. That grounds it. And honestly, with a set of BDD tests and the instruction of don't stop until all of these pass, you will actually get that thing. Like you will get something pretty dang

Bill (error_404_new) [01:32:10]:
close CIPD in place. Right.

Larry Gold (LrAu) [01:32:14]:
And I just posted in the private chat, stitch with Google is another tool to help you create mockups. Because that actually you could just have Lisa talk to the screen and I'll draw the mock ups and then again you take that and move that to whatever tool to build it too.

Leo Laporte [01:32:28]:
Right? Very cool.

Bill (error_404_new) [01:32:30]:
Pumping your screenshots of your old platform into that and have it give you something more modern.

Leo Laporte [01:32:34]:
Yeah, cool.

Darren Oakey [01:32:35]:
But honestly, if Lisa's going to be frustrated with talking to something and everything, what I would do is just do a 1.0 today.

Leo Laporte [01:32:44]:
I think that's really what I'm going to have to do. She's very reluctant to do this.

Bill (error_404_new) [01:32:48]:
She doesn't want to put in write a data seed to move all the data over so it's populated with real data.

Leo Laporte [01:32:54]:
Right.

Larry Gold (LrAu) [01:32:56]:
Create a private GitHub, put it up there and let one of us run and give you the MVP by Monday.

Leo Laporte [01:33:02]:
We're really in a good position because we have all the source code, we have the database, we have schema, we own it. So it's a. It's actually a fairly simple thing to kind. We don't have to.

Bill (error_404_new) [01:33:11]:
That might be an episode. That might be a hackathon episode. Just get everybody in and have somebody work on it.

Leo Laporte [01:33:17]:
Maybe next month.

Anthony Nielsen [01:33:18]:
Yeah, that's what I was saying. Like, maybe. Maybe the next meeting is we build

Leo Laporte [01:33:21]:
something and that would be kind of fun.

Juan (BlindWiz) [01:33:23]:
Yeah.

Leo Laporte [01:33:23]:
Hey, this has been a lot of fun.

Darren Oakey [01:33:28]:
Analyze this. Give me a right. And the other query is just do it right. Yeah. See how it works.

Anthony Nielsen [01:33:34]:
Okay, Leo, next month, you got to show something.

Leo Laporte [01:33:36]:
What could possibly go wrong? All right, you guys. Hey, this is really great. Thank you so much, all of you. David and Alakazip and Juan and Bill and Larry and Darren. And of course, Anthony Nielsen, who makes this all possible, is our local AI guru. This is fun. I really think we should start doing this every week, but I don't want to challenge people too much, but we do.

Bill (error_404_new) [01:34:02]:
We do our AI meetups every week at work. We started monthly and there was just too much excitement.

Leo Laporte [01:34:06]:
So much fun.

Bill (error_404_new) [01:34:07]:
We do it weekly and it changes.

Leo Laporte [01:34:08]:
We didn't even talk about Mythos at all, and I thought maybe that would be the topic of the day. So, yeah, there's so much going on. It's very interesting. But I think that what's nice about the user group, it's. It's really very hands on, and I think that makes it something pretty special. So thank you all. I appreciate it. Thanks for your contributions and thanks to our club members.

Leo Laporte [01:34:28]:
There's quite a few of you watching Silent Partners. We appreciate that without the club, there would be no AI user group. There would be no twit. If you're not a member, you're watching this live or you're watching it a month later. If you want to watch it day and date, join the club. Twit. TV Club. Twit.

Leo Laporte [01:34:48]:
And we thank you very much for your participation. Thank you, everybody.

Bill (error_404_new) [01:34:53]:
Thank you.

Leo Laporte [01:34:54]:
Say goodbye. Goodbye.

Bill (error_404_new) [01:34:56]:
Cheers, everybody.