TWiT+ Club Shows 758 Transcript - AI User Group #17
Please be advised that this transcript is AI-generated and may not be word-for-word. Time codes refer to the approximate times in the ad-free version of the show.
Leo Laporte [00:00:00]:
This is TWiT. What I'm doing right now is rewriting the TWiT sales system.
Larry Gold (LrAu) [00:00:07]:
Yes.
Leo Laporte [00:00:08]:
And so Fable, the way I, I thought I was gonna lose Fable on the seventh. So the way I said, I set it up when we first got Fable, I had Fable look at all the original code and the SQL database, get the schema and all that, and then write up a plan. And then when I got Fable back, I said, okay, look, I don't know how much longer I'm gonna have you, so here's what, here's our plan. You're gon. I'm going to keep using you. You're going to be writing the plan, delegate and do it in little chunks that I could delegate to Opus 4.8, which you will then review. So we'll do it all in little tiny, weeny, teeny weeny chunks. And that actually has been working.
Leo Laporte [00:00:47]:
All right, what's really nice, let me go back to my browser because I had it write up the plan. Of course it's using Markdown, but I also had to do a website. Look how this is expanded. And I said, as you go each step of the way, write it up, write the plan, and then write the execution. So it's a really nice web based review, which just between us kids, you got. It's, it's open. It's at Pages Laporte Cloud. This is my, this is my cloudflare, you know, temp pages thing.
Leo Laporte [00:01:25]:
So I have a lot of just little stuff there. But this is, this is. So this is the. So I did an interview with Lisa and Debbie. These are the follow up questions. This is the analysis of the original system. This is its current build status. So it's kind of like a Kanban, except there's only one player, so it doesn't really need to be a Kanban.
Leo Laporte [00:01:45]:
So we're actually, we've gotten pretty far. I'm trying to get this done by Sunday. So I have C and D and E to do. E is just deploying in the cloud. So it's really C and D are all that need to be done. But now once I got Saul yesterday, I said, oh, nice, Saul. You review what Fable's reviewing, what Opus is doing. So I've got these two eyes looking at it, which actually has been good.
Leo Laporte [00:02:11]:
Saul's actually found quite a bit of stuff. But then I realized I'm doing a lot of, you know, oh, here, Fable. This is what Saul said. And then, you know, Saul, this is what Fable did. And so I said, could you guys Just set up a mail system. So they did. And they said, as long as you're at it, keep Hermes in there too. So they now have interagent mail, which on every turn they check to see if.
Leo Laporte [00:02:37]:
So this is Daedalus's mailbox.
Craig McFarlane (CraigM) [00:02:39]:
That's.
Leo Laporte [00:02:39]:
That's Codex. That's Saul. Kenobi is. I just give him names, so I don't know. And they also have different voices, so that Kenobi is Claude and Hermes. So I'll give you an example. Let's see if there's anything in the inbox here. There's.
Leo Laporte [00:02:54]:
They check it every turn, so there probably isn't.
Michael (Alakazip) [00:02:57]:
And is it just using email? I mean, that's.
Larry Gold (LrAu) [00:02:59]:
It just.
Leo Laporte [00:02:59]:
It's not email. It's. It's marked down. It's all an Obsidian, so I can read it.
Michael (Alakazip) [00:03:03]:
Oh, okay. You said mail.
Leo Laporte [00:03:04]:
And I was like, they're calling it mail. It's agent mail.
Craig McFarlane (CraigM) [00:03:07]:
They could.
Leo Laporte [00:03:08]:
I mean, I could. But that's ridiculous since they're all in the same machine. So they're just doing it in Markdown in Obsidian, which is nice because I have a record then and I can read it instead of them having. But I said, you don't wait for me. Yeah. So they've cleared all the mailbox. So this. I think the instructions were written up by Saul based on my directions.
Leo Laporte [00:03:32]:
And then I gave this to each of them. They set up the mailbox and now they check their mailboxes. So now when I have Saul review the work, it just emails or whatever I'm going to call it Email. It emails Kenobi. Here's my review. And it's actually pretty good. And see, he says, I saw Kenobi's response before any further audit work. And so this is.
Leo Laporte [00:03:56]:
They're talking to each other and working together on the same project. So this has really been very effective. And Grok, actually, I don't want to give Grok any credit, but Grok ain't bad. Grok is kind of useful. I've been using it the new 4 5. Yeah, yeah, yeah. No, and I have a. Elon kindly gave me a free Twitter plus account.
Leo Laporte [00:04:20]:
So I don't know what is. I don't know what the limits are. It's not costing me anything anyway. And I have then max subscriptions with. With these guys with. With Claude and GPT. So at least for the next while I'm building this, I'm going to keep these guys on the top models and working together. And it's actually working.
Leo Laporte [00:04:40]:
It's working. Pretty well. I'll show you. It's at Bare Bones right now because we're, you know, we're doing the business logic, not the,
Juan Hernandez (BlindWiz) [00:04:49]:
the UX side of it.
Leo Laporte [00:04:51]:
The UX side of it. But Lisa is reviewing it so that she has port 8730 so that she knows, you know, so she says, no, no, no, I don't want that, or, yeah, I do want that, but this will look nicer. But right now, you know, she can write, she can make an or a broadcast order, choose the days, add shows. Like I'll add a show. Let's say this adds. Going to run on this Week in Space. And I'm going to say. And it'll say these are the upcoming episodes for these, this date range.
Leo Laporte [00:05:22]:
And I could pick that and that and that. And then I'm going to. It could either be value add or add to ordered. It's now added to the order. And there it is in the order. It puts in the boilerplate terms, which actually didn't.
Craig McFarlane (CraigM) [00:05:38]:
Oh, yeah.
Leo Laporte [00:05:38]:
Fill terms from template. There it goes. So this is the boilerplate terms. And then it'll print it up and we can send it to DocuSign. Lisa said, don't put DocuSign in the program. I said it could do DocuSign. She said, no, don't do it. She's a little.
Leo Laporte [00:05:51]:
I think she's a little cautious about what this can do. My attitude is, look, it knows all the data is in this database. Oh, hello. Hello.
Anthony Nielsen [00:06:02]:
That's my bad. My bad.
Leo Laporte [00:06:03]:
All the data's in this database. So let it do DocuSign, let it do everything. But she's, she's right now a little careful.
Juan Hernandez (BlindWiz) [00:06:10]:
So one step at a time, right?
Michael (Alakazip) [00:06:12]:
Yeah.
Leo Laporte [00:06:13]:
But the fact this is going along, the business logic is in there. We're actually gotten pretty close to the. I think I'm going to finish it before Sunday, which was my goal, because as you know, in the 12th, anthropic is going to announce now, we decided to keep it going for another 18 days.
Juan Hernandez (BlindWiz) [00:06:28]:
Well, I heard a rumor that Mac subscribers were going to keep Fable. Have you guys heard anything about that? They were going to. They were going to cut everybody else but Mac, the 200. Not the 100 Max, but the 200 Max. Subscribers might have still fable after the 12.
Leo Laporte [00:06:44]:
A number of people said that. Let's see, let me switch back to this. A number of people said that because Saul came out. Saul Luna and Tara came out. Oh, now Anthropic is going to extend the subscription because you can use your subscription on ChatGPT 5. 6. So I think it's all rumored they're
Darren Oakey [00:07:05]:
going to give us a little bit of it. And Jan, you may be right that we only get only for Max people. But I suspect they'll just keep selling us enough of the drugs to keep us addicted.
Leo Laporte [00:07:20]:
It is a drug, isn't it? So, okay, I just wanted to show what I'm doing and mention those three models. Have you guys tried any of the new? First of all, let's start it. Welcome to the AI User Group. Hello, everybody. That's Anthony Nielsen, man in charge, our chief creative officer. He helped me make bagels yesterday. Did a stellar job.
Anthony Nielsen [00:07:43]:
I had one this morning and it was very good.
Leo Laporte [00:07:46]:
I don't like them at all. Lisa made me do whole wheat and it's just not the same. Anyway, that's Larry Gold. Hello, Larry.
Danno [00:07:53]:
Hey.
Leo Laporte [00:07:54]:
Au in the colub. We also have Darren OKE from Larry's in New Jersey. Where are you?
Larry Gold (LrAu) [00:08:00]:
I forget. New Jersey?
Leo Laporte [00:08:01]:
Yep. New Jersey.
Juan Hernandez (BlindWiz) [00:08:02]:
Darren Oak.
Craig McFarlane (CraigM) [00:08:03]:
Central Jersey.
Darren Oakey [00:08:03]:
Yeah.
Leo Laporte [00:08:04]:
Darren is in Eastern Australia.
Darren Oakey [00:08:06]:
I can't remember. Sydney.
Leo Laporte [00:08:07]:
Sydney, the big city. Juan is here. Blind Wiz. Hi, Juan.
Craig McFarlane (CraigM) [00:08:13]:
Welcome back.
Larry Gold (LrAu) [00:08:14]:
Thank you.
Leo Laporte [00:08:14]:
Good to see you.
Larry Gold (LrAu) [00:08:15]:
Where are you?
Juan Hernandez (BlindWiz) [00:08:16]:
San Diego.
Leo Laporte [00:08:17]:
That's right. I know all this and you'll forgive me because I'm terrible with names, faces and locations. So forgive me.
Juan Hernandez (BlindWiz) [00:08:23]:
No worries.
Leo Laporte [00:08:23]:
Craig M. Welcome back. He's a cto.
Craig McFarlane (CraigM) [00:08:28]:
Hey. It can only be one.
Leo Laporte [00:08:30]:
We won't say where I could put
Craig McFarlane (CraigM) [00:08:32]:
it in, but it generally.
Leo Laporte [00:08:33]:
No, it's okay. Where are you calling from, Craig?
Craig McFarlane (CraigM) [00:08:38]:
From Boston.
Leo Laporte [00:08:39]:
Boston? Did all the Scots people leave? Or all the tartan army?
Craig McFarlane (CraigM) [00:08:43]:
Yeah, we. We drank more beer in and out and year over year. It was up 29 for the first two weeks of beer consumption.
Leo Laporte [00:08:55]:
More than St. Patrick's Day.
Larry Gold (LrAu) [00:08:57]:
I heard all was left was Bud Light.
Leo Laporte [00:08:59]:
Yeah, that's what I heard. Yeah, I heard that, too.
Darren Oakey [00:09:02]:
I love that.
Leo Laporte [00:09:03]:
Well, drink anything except that piss. Dana, where are you coming from?
Danno [00:09:12]:
South Dakota.
Leo Laporte [00:09:13]:
Nice. This is your first time or not?
Danno [00:09:14]:
It was a clown.
Leo Laporte [00:09:15]:
But you remember me as a clown?
Danno [00:09:18]:
No, I don't remember. I didn't realize they'd be labeling you as a clown. And I wore this shirt today.
Leo Laporte [00:09:22]:
Oh, I like it. Oh, obviously I am to your right.
Anthony Nielsen [00:09:26]:
Let me. Let me unmute. There we go.
Leo Laporte [00:09:27]:
Clowns to the left of me. Joker's to the right.
Darren Oakey [00:09:30]:
Here I am.
Leo Laporte [00:09:31]:
He's stuck in the middle with us. You're stuck in the middle between the clown and the joker. I love it.
Danno [00:09:38]:
Yes. I haven't talked to you since radio days.
Leo Laporte [00:09:41]:
Oh God, that's great. So you used to call the radio show?
Danno [00:09:44]:
A couple times. I made it in, yeah.
Leo Laporte [00:09:46]:
Nice. Very nice to meet you. And are you a coder? What do you do?
Danno [00:09:50]:
I am a CEO, but I kind of have always been a. The tech guy for our family business.
Leo Laporte [00:09:56]:
And what is the family business?
Danno [00:09:58]:
We train power linemen. You don't remember that from 15 years
Leo Laporte [00:10:02]:
ago on the radio? I do remember it now. I remember talking to you. I am a lineman and I did the same thing.
Danno [00:10:09]:
That's very likely.
Leo Laporte [00:10:10]:
I did exactly the same dumb Glenn Campbell song. Well, that's cool. That's a dangerous job. But a very. Especially in your neck of the woods. A very important one.
Danno [00:10:20]:
Less and less snow every year, but it used to be a lot worse.
Leo Laporte [00:10:25]:
Well, it's nice to meet you Dano, and tell us about your, your. What stage of the AI psychosis are you in?
Danno [00:10:33]:
Well, I used. I started up a Hermes account.
Leo Laporte [00:10:37]:
Nice.
Danno [00:10:38]:
Two months ago, put deep seek on it and bought a hundred dollars in credits, which I've only used about $6 so far.
Leo Laporte [00:10:46]:
It's amazing, isn't it?
Danno [00:10:48]:
I thought well I'll just put 100 on there, you know, cover.
Leo Laporte [00:10:50]:
I know it's so cheap, but it's good.
Danno [00:10:53]:
But then my. I can give you an example. I wrote a 4th dimension database in 1987 that ran our program till 2012 and I rewrote that in MySQL and PHP et cetera in about a month.
Leo Laporte [00:11:10]:
Oh wow.
Danno [00:11:11]:
And last Monday I walked in, decided to try Codex and told Codex to turn it into a mirn stack react JS and I finished it in a morning.
Leo Laporte [00:11:24]:
It's amazing, isn't it?
Danno [00:11:26]:
It's just mind boggling ridiculous. And then lately my last most proud achievement is. I don't know if anybody's ever played Gizmos, the board game. It's a kind of a procedural. You take marbles, build stuff with marbles, use the stuff you build to get more marbles. And it's a European point type game. But I built a simulation of that for the Internet and I wanted to see what I could do to train. And so I got.
Danno [00:11:57]:
Once I got it working I thought well I'll just make it a board game sim website. So I made my own website, boardgamesim.com and. And there is another boardgamearena.com but they don't allow bots. So. And then yesterday I took Codex and I kind of used it like Hermes. I pointed it at my GB10 and I said, I said set up a, set up a simulation where a three player game can go on endlessly. And it set it up and I've played a thousand game now games now and it's gone from a. At first it was only getting like 15 points a game and now it's up to 40 points a game.
Danno [00:12:35]:
So it's getting smarter.
Leo Laporte [00:12:37]:
Isn't this fun? And this is just for fun. You're doing that. That's awesome.
Danno [00:12:42]:
Yeah. So far I used Codex, the new Sol on Ultra and it was a lot like using Hermes because I pointed it at my GB10. I told it, here's a key to my GB10, go over there and set up an AI that'll play gizmos on my website. And so far it's, you know, it's played 500 games and it's getting better all the time.
Juan Hernandez (BlindWiz) [00:13:08]:
What model did it set up on the GB10?
Danno [00:13:12]:
It hasn't said and I haven't. I just was happy it worked and it's getting better. So I need to, I need to kind of. Because there's combinations, there's, there's players on board game arena that never lose and they're, they're rated, you know, I'm rated in the mid-400s and I'll play against a guy that's rated like 600 something. It's not like the chess ratings but which I also play. But, but so yeah, I haven't. I just got it working. So now then I'm.
Danno [00:13:42]:
Now I'm on. When I was trying to use Hermes to set it up, it wasn't really doing very well. I was so. But now, now it's actually working.
Leo Laporte [00:13:52]:
So Codex is amazing actually.
Juan Hernandez (BlindWiz) [00:13:54]:
It really, really is. This new saw is amazing.
Danno [00:13:59]:
I found it kind of slow because I built Image interpreter yesterday and I just felt it was kind of slow compared to the other stuff I've been doing.
Leo Laporte [00:14:08]:
But you know what I'm doing, it's still good. I, I hooked up my unifi cameras to Hermes and it's using a local Quinn image analyzing program. And it's fast and it's really good and it's. So now every time somebody walks up my drive, I get it. There's a guy in a blue hat with a red shirt, he's carrying a box, he's coming your way. It's really fun, it's really interesting. So try some local models and I Don't you need a. I mean I'm running it on 128 gig machines with the.
Danno [00:14:39]:
Yeah, I put ornith and yeah, Ornith is good too.
Leo Laporte [00:14:43]:
That's Larry's.
Danno [00:14:44]:
I'm not sure which one it's using right now, but I'll put the.
Leo Laporte [00:14:47]:
And then hello to. I always get your name wrong. Alakazip.
Michael (Alakazip) [00:14:52]:
That is the correct saying. Yes, I said it right.
Leo Laporte [00:14:54]:
Hi, Michael.
Michael (Alakazip) [00:14:55]:
I chose.
Craig McFarlane (CraigM) [00:14:55]:
Yeah.
Leo Laporte [00:14:56]:
Remind me again where you're calling from.
Michael (Alakazip) [00:14:58]:
Charlotte, North Carolina.
Leo Laporte [00:14:59]:
North Carolina. Have you been playing with the new models?
Michael (Alakazip) [00:15:05]:
Absolutely. In fact, I'm sure like all of you have a lot of computers going. I've been trying to use open code to talk to Apple's foundation model local and cloud and it's working. So that's interesting. I couldn't get cloud code to talk to it, but I could get open code to talk to their private cloud compute and so I've got it writing code right now over here.
Leo Laporte [00:15:28]:
That's interesting.
Juan Hernandez (BlindWiz) [00:15:30]:
Anthropic is really picky about what they allow cloud code to talk to.
Leo Laporte [00:15:35]:
Yeah, they don't support the OpenAI API.
Juan Hernandez (BlindWiz) [00:15:37]:
The only way, the only way to use cloud code as a secondary model like through Hermes is through the dash P parameter where it controls it that
Leo Laporte [00:15:46]:
way actually I've been able to oauth Claude code in Hermes, believe it or not, and use it though they threatened, remember?
Juan Hernandez (BlindWiz) [00:15:54]:
That's right. But they backed off on that for
Leo Laporte [00:15:57]:
now, at least for now I'm taking my chances but although now I'm not going to do it anymore because now that I can use Fable, I was so pissed off that I said I don't care if they kill my subscription, but now I don't want them to do that anymore. They even said you can't use Claude
Darren Oakey [00:16:15]:
P. Someone's cut off a hand or something.
Leo Laporte [00:16:18]:
It would be. At this point I would be very sad.
Juan Hernandez (BlindWiz) [00:16:20]:
I think they had a lot of lash back on the dash P stuff.
Leo Laporte [00:16:22]:
They did.
Juan Hernandez (BlindWiz) [00:16:23]:
Yeah.
Leo Laporte [00:16:24]:
So I think they, they've maintained that but honestly there's so many good models I can use and Hermes, like Hermes, like all the other harnesses except for Claude code, uses the OpenAI API, which almost everybody else uses.
Darren Oakey [00:16:37]:
So I've got a version of, I've taken codecs, Codex is written, it's open source and it's written in rust. So I've made a wrap around it and now it's just an agent demon and it's all my subscriptions like GLM and Codex and Cloud are all working through Codex and so now I just use that to talk to everything. It's working quite well.
Leo Laporte [00:17:05]:
I feel like you might have like 100 different models running all at the same time.
Juan Hernandez (BlindWiz) [00:17:08]:
Darren.
Danno [00:17:08]:
I don't know why.
Darren Oakey [00:17:10]:
I've got quite a lot of modules
Juan Hernandez (BlindWiz) [00:17:15]:
for my company. I just finished incorporating back in April and one of the first things we're going to be releasing is an open source competitor to Hermes and other agentic systems. But one of the things that I've built into mine is so you can have the application running on instances on the local network and so you can be on computer A and control the instances on all the other computers and they, you know, and you can at sign, you know, computer aid, colon, model, whatever and direct that model to do whatever on that. I'm trying to. Because I'm building, I want to build an orchestration model for my, for my company to. Because we're building a fleet, a model fleet platform to manage thousands of models at one time. And so.
Leo Laporte [00:18:04]:
Yeah, thousands.
Juan Hernandez (BlindWiz) [00:18:07]:
Thousands, yes, we're talking thousands.
Leo Laporte [00:18:10]:
Why so many? What are you gonna do?
Juan Hernandez (BlindWiz) [00:18:13]:
It's for big companies, you know, big, big projects, large scale enterprises.
Leo Laporte [00:18:18]:
One of the things. So I've been really trying to think about what to do with intelligent machines because I don't want. I've decided I really don't need any more a show about AI culture. I really need a show about using, about practice. Like this show about people using AI and how to use it.
Juan Hernandez (BlindWiz) [00:18:34]:
The applied side.
Leo Laporte [00:18:35]:
Applied side. It's like in the early days of the Internet we spent a lot of time talking about the culture of the Internet. But once everybody's using it it's like, let's not. We don't need to fool around.
Craig McFarlane (CraigM) [00:18:45]:
That's not a separate culture.
Leo Laporte [00:18:47]:
Yeah, that's it for culture. Let's get down to work. But I'm really interested in finding somebody who can talk about enterprise approach. Approach because. And you actually, most of you are doing it. The approach of the enterprise to AI is very different than I'm just a hobbyist. And so the needs are very different. Privacy, security and all of that.
Leo Laporte [00:19:15]:
Do you think Enterprise is trying to run stuff locally or how do they.
Michael (Alakazip) [00:19:21]:
But who's authorized to speak publicly?
Larry Gold (LrAu) [00:19:27]:
The one thing I could say is Enterprise has got an interesting conundrum is you can't get the GPUs, right. So you're going to be thinking about do you then outsource that and run models there or do you do different things in different ways or do you work on the agreements? Right, there's going to be a lot of different ways.
Leo Laporte [00:19:47]:
So Anthropic has for instance, enterprise plans that are supposedly private, right?
Larry Gold (LrAu) [00:19:51]:
Yeah.
Darren Oakey [00:19:53]:
And we're doing a lot of stuff through Bedrock and I think a lot of enterprises and that makes things that's a lot more enterprise friendly because it is an AWS agreement and you know, it's meant to be private.
Leo Laporte [00:20:09]:
They already have all of those privacy.
Juan Hernandez (BlindWiz) [00:20:13]:
One of the things we're working on internally is to try to get a partnership. There's a company called Go Abacus Abacus. They build these, they build these massive AI servers. They're like three or four hundred thousand dollars and basically it allows a company to run models internally on their local network and select the basic box is like 300 grand and you can have up to 8,000 users in parallel using models. So that's how powerful these boxes are. But. And so one of the places that we want to enter is the, you know, the super secure space. So like DoD contractors, prime contractors, Health, where that data stuff has is, you know, they really want to keep it all internal to them.
Leo Laporte [00:21:01]:
I would, I mean, I'm reading between the lines, but it sounds like the DOD is running anthropic models locally.
Larry Gold (LrAu) [00:21:12]:
I don't think locally. I'm sure like either in Bedrock or somewhere.
Leo Laporte [00:21:16]:
Okay.
Larry Gold (LrAu) [00:21:16]:
I don't, I don't think they've got all the clouds.
Leo Laporte [00:21:18]:
Not in the Pentagon, but in the sense that anthropic said. Here's the. You don't have to run it on our cloud.
Larry Gold (LrAu) [00:21:25]:
Yeah, you run it on. Yeah, yeah.
Juan Hernandez (BlindWiz) [00:21:27]:
All the clouds have government, federal, I guess data centers or regions or whatever that are just like federal base for the government. They're like US-gov something is the AWS one and there's a few. Everyone has one for government.
Darren Oakey [00:21:47]:
And the thing about big companies is that as my friend has often said, really big. Most big companies are doing everything like, like, just like Morgan. Like if you say, do you use this language? The answer is yes. Yes, there's someone there who's using that language. And there is this like 10 teams trying AI in different 20 different ways and, and so on because everybody's experimenting in it. But yeah, big companies are just. They do all of the above, basically.
Leo Laporte [00:22:19]:
Is it all like, is it all experiments still or people are. Do you feel that it's. People are starting to converge on best practices?
Michael (Alakazip) [00:22:25]:
There's stuff in production for sure, but I think best practices are still right over the hill. But there, there is real world production code and things running on local models, whether it is in a cloud instance that's slated to it or local GPUs. That's absolutely happening.
Leo Laporte [00:22:43]:
How much of it is customer service stuff is.
Darren Oakey [00:22:47]:
Most of it's a lot of the traditional stuff. Like people have always done things like OCR and everything. And so pulling stuff out of documents has just moved from that to LLMs and everything. But there's also, and like you said, there's chat bots and there's customer service and things like this. And now people are trying to integrate more intelligence into workflows. Also people. I think I see a lot of people doing things like dashboards where business people can query, can ask questions about the data, the data. So data insights and things.
Darren Oakey [00:23:24]:
But I think larger things people are still wary of. And also people are now I think what we're just starting to see is people starting to realize the cost of these, like putting it on an actual critical path is actually pretty expensive.
Leo Laporte [00:23:41]:
Yeah. If you could even get the hardware.
Danno [00:23:44]:
Yeah.
Larry Gold (LrAu) [00:23:44]:
The thing we could talk about because it is public is Morgan spent a fortune and got a whole bunch of awards because the modernization that we're doing and basically they're using AI to basically convert legacy code. Right. Or to remove hygiene and stuff like that. And they've talked about it. You know, you can see all the stuff on whether it's LinkedIn or whatever or they publish all those articles, but there is a lot of work doing to that. And even reading the outside articles is some stuff I don't even hear from inside. It's really impressive the amount of stuff we can kind of do. And it's probably a lower hanging fruit than some of the other pieces, but there'll be a lot more coming out after that.
Larry Gold (LrAu) [00:24:20]:
I mean I think people, that's like the tip of the iceberg. But I think, you know, I called June 1st the other deadline day because that's the day model switched over pricing and it very changed a lot for Everybody. It was November 25th and now June 1st. Right.
Leo Laporte [00:24:36]:
Those are the two dates, the days that will live in infamy. June 1st. Okay.
Juan Hernandez (BlindWiz) [00:24:42]:
I've noticed a big change in the customer service space with like companies like Intercom, they, they, they rebranded as I think Finn with their new Agentic or their new model. That, and it's, I mean I was actually, I was buying a code signing cert yesterday and I needed customer service on the website and they had that Intercom button and it used to be very boilerplate language at the, that the Intercom tool would use to communicate. Right. Very just, you know, just, you know, just very like, you know, procedural and now it's, you can totally tell there's a full LLM behind these intercom systems or FIN or whatever they call it these days.
Craig McFarlane (CraigM) [00:25:26]:
Yeah.
Michael (Alakazip) [00:25:27]:
So, yeah, in the consumer space, I called my Toyota dealership to get an oil change and I was shockingly pleased with the interaction with the interesting customer service bot. And I'd never been pleased. None of us have. And I was like, wow, that total totally worked. Who knows what they're using, but it's great.
Darren Oakey [00:25:42]:
Did you hear about that Toyota thing where they sold a car for $2?
Leo Laporte [00:25:46]:
Whoops.
Larry Gold (LrAu) [00:25:47]:
No, that was GM. That was like a Suburban or something like that.
Leo Laporte [00:25:55]:
And you can get your Python scripts done as well while you're waiting for your car wash. Are we calling out
Michael (Alakazip) [00:26:02]:
the elephant in the room? Leo, why are you in the dark?
Leo Laporte [00:26:05]:
Because I have an analog lighting board. If everything were digital, none of this would be going wrong. And so the way the analog lighting board works is just a nightmare.
Michael (Alakazip) [00:26:14]:
Yeah, I hate to call out your tech, but. No, I know I wasn't having, you know, eyesight problems.
Larry Gold (LrAu) [00:26:19]:
And you're the tech guy, that's the funny part.
Leo Laporte [00:26:21]:
Well, of course the tech guy. You know this. The tech guys are always the ones on the sharp end of the stick. You know that that's turn it on and off. That's how I fixed it. I started to fix it in a broke again. So continue without me going back to.
Larry Gold (LrAu) [00:26:37]:
I think the one interesting thing is like even Prisma, ML and all these other companies that are shrinking the models and I know Apple is trying to push for them and others are trying to look at this, but I do think, as I said, we're inside that tornado a year from now. Completely different. If you can run some of these 35 billion or 120 billion parameter models on a phone, you'll be running it on your regular CPU on your desktop. And that's a game changer because that does change the mathematics and the token count. And I think I said it before, when Microsoft announced those machines, the new Nvidia machines, what was missing was a blade version. You didn't see HP or Dell creating racks of these blade ones so they could sell as VMs to companies. Because that's where a lot of companies have moved, to virtual desktops. It was all Covid everything.
Larry Gold (LrAu) [00:27:28]:
And I think those are the two things that to me, I look at as saying a year from now will be completely different. You know, if they can get the hardware.
Anthony Nielsen [00:27:37]:
Well, the existing hardware. Like, like, yeah, the. It's a race to the top and the bottom. Right.
Leo Laporte [00:27:45]:
Must be driving a lot of these companies crazy. That there's such a market just waiting,
Darren Oakey [00:27:50]:
sorry, Kickstarter for a thing called tiny.
Craig McFarlane (CraigM) [00:27:52]:
It is general consumers. It's even worse for enterprises where exactly you've got this everybody bringing in their own AI, you've got all these different tech changes and then you have governance layer on top of it that you need to be able to protect your IP and security and all this. And the mashup of that is happening so fast, it all their normal process goes out the window.
Darren Oakey [00:28:20]:
Yeah, that's why, as Larry said, the no brainer is the building stuff because there's no ongoing costs. It's like a one and done type thing and you're just in a better world and there's really no downside. Whereas all the infrastructure stuff has ongoing problems or ongoing things to solve.
Leo Laporte [00:28:41]:
Well, that's the low hanging fruit. But there is a lot of it. There's a lot of legacy code. And I mean this is. I just know from this sales system, this crap sales system that we've been laboring under for 12 years, it's actually fairly trivial to rewrite it. It's not a complicated thing. But we never could. We hired a guy to kind of maintain it and kind of keep it going.
Leo Laporte [00:29:04]:
And Russell has to keep buying more cloud power because it's so slow and Lisa has lost all faith in it. Right. And so she just doesn't believe that I could. Oh yeah, this is actually in five days I can get you something much better. She doesn't believe it and actually I wasn't sure. But as the progress goes on, it's really trivial. This isn't as simple as porting old Fortran code into Rust or something, but it's all the business logic's in there. So it's not too hard.
Leo Laporte [00:29:41]:
That's fun.
Craig McFarlane (CraigM) [00:29:42]:
You were going through a process where you interviewed them on the steps they needed. Not necessarily looking at the core functionality and trying to.
Leo Laporte [00:29:51]:
Oh no, we took. So I took all the old code, which is Fable's analysis of it was pretty insulting. It found all these security flaws. It knew it was basically a SQL database, probably Microsoft SQL Server. I think it was Microsoft SQL Server database. Most of the logic was in queries in the database. It was really bad because it was net, right?
Juan Hernandez (BlindWiz) [00:30:20]:
It was a net.
Leo Laporte [00:30:20]:
Yeah, it was probably net. Yeah. It was written 12. Was it in 2012, something like that. So by a smart guy, he was one of our employees, he said let me help you because the whole job is advertiser comes in, they want to buy some stuff. You have to make sure. That, you know, they, they only, they buy ads that are available, you know, keep track of what's available, charge them properly, give them value, add. It's not a complicated thing.
Leo Laporte [00:30:46]:
It's, it's a sales system. And Lisa said, don't put any CRM in it. I said, well you might as well. We got all the, you got all the information in there. Well, it shouldn't print out Docus, it shouldn't send it to DocuSign. Well, you can. Why are you printing out PDFs which you attach to emails? I mean there's so many.
Juan Hernandez (BlindWiz) [00:31:06]:
Yeah, DocuSign has a great API.
Leo Laporte [00:31:08]:
Yeah, it'd be easy. It's trivial. I'm going to put it all in there. It's just going to have to be gradually. But the point is that the business logic is all already encapsulated. There is a lot of cruft over the years stuff that we don't do anymore, stuff it used to do. The first thing I did is have Fable just say here's all the code, here's the database, figure it out. And then now write a plan for a modern version of this.
Leo Laporte [00:31:34]:
And I asked it for advice on hosting and all that stuff. Russell wants to, because we'd use our Microsoft SSO and all that stuff, wants to pretty much host Azure and all that and that's fine. So I sent it, give it all that information. And it's just, I mean honestly this is not innovation, it's not solving an Erdos problem. It's just simple business logic that's probably seen a million times and I'm sure most of what enterprise is doing is like that. It's just, you know, taking old COBOL code and you know, if the Y2K crisis happened today. Be simple, be trivial.
Larry Gold (LrAu) [00:32:16]:
Is the real question, is AI revealing the fact that we have a lot of disconnected apps that we kind of hodgepodge together to do business processes because it was too expensive to build something that was all for us versus being able to build that all encompassing app that just does it. And now because of AI, it's actually cheaper to get these APIs, make those connections and do that and create that single probably the Alan Cooper the goal oriented application for us to succeed.
Leo Laporte [00:32:49]:
Well, that's one trend. But then there's the other trend which is there's a lot of people like us who are writing our own apps. And so there's going to also be a billion one off apps. I have like 20 repos on GitHub that I'm probably not ever going to look at again. So it's all. I don't know what, I doubt that happens in enterprise. Well, maybe it does. You just said it's all sorts of stuff going on in a big company.
Darren Oakey [00:33:13]:
I've always maintained, sort of like as a dev and that the process, the, the, the fundamental element of software is requirements change. And so if you're the most important thing system is you is a system that can deal with change. And if you've got a system that deals with change, then what you're building is not as important as how you're building. So I've always maintained that the process is more important than the what because if you can deal with any change coming up and you can deal quickly, then you can, you can deal with it. And so the DevOps, I've always said DevOps and process and CICD is way more important than the product you're building. But I think that becomes way more true when building is cheap or free and you just ask for what you want. Then it becomes robustness because people are starting to realize that when we ask for these things, the AI just rips out other bits of functionality. If you don't have something that guarantees it or something like it can do anything and you really don't know.
Darren Oakey [00:34:20]:
So it's becoming more and more about the discipline of software creation rather than the what you're creating.
Leo Laporte [00:34:29]:
That makes a lot of sense actually.
Michael (Alakazip) [00:34:32]:
The governance around it of like you got to have change management still maybe ish and cybersecurity and, and, and, but all those things used to take time and now not only is, you know, Mythos and others dictating, you have to do it in minutes or hours, but developers are not going to have the patience to wait for next week or the week after to do change management code pushes when it does that, even
Leo Laporte [00:34:56]:
in my own little sphere, Lisa says, okay, you write this, but what happens, it breaks while you're on the air. And I said, well, I'm going to give you a little chat window. You can ask Claude to fix it and probably it would do a lot better job than if I put my fingers in it.
Juan Hernandez (BlindWiz) [00:35:14]:
Just build in a Claude code proxy into the platform.
Leo Laporte [00:35:18]:
That's exactly what I'm going to do. There's going to be a little window
Juan Hernandez (BlindWiz) [00:35:21]:
there and then just say, okay, rewrite this module.
Leo Laporte [00:35:25]:
Something's wrong, what's going on? Fix this. Yeah, and it's also there. So she could say, hey, hey, are these guys a new advertiser? I mean there's all Sorts of useful things she could do with that kind of little window.
Danno [00:35:35]:
Unless she can break it worse.
Leo Laporte [00:35:37]:
Well, yeah.
Darren Oakey [00:35:40]:
Well, could she? Two sorts of tools. Like you're basically adding a chat window.
Anthony Nielsen [00:35:45]:
She'll say forget it and then it'll just delete it.
Leo Laporte [00:35:48]:
No, that's what I did. I was pissed off. I told this story already, but I was pissed off and it was just failing. I said, oh, just forget it and deleted everything. And it was like, no, no, I didn't mean. I just meant I was going to do another. A different way.
Larry Gold (LrAu) [00:36:00]:
Don't. Yeah, but get commits are cheap. So it's you. You've got to be git committing at like every level or every.
Leo Laporte [00:36:08]:
Do that. And Hermes actually keeps. Yeah, all of that. So you can go. You could go. Go back a step, which is actually Nice little feature.
Larry Gold (LrAu) [00:36:16]:
And Claude has that rewind function.
Juan Hernandez (BlindWiz) [00:36:18]:
Yeah, rewind.
Leo Laporte [00:36:18]:
So nice. Yeah.
Juan Hernandez (BlindWiz) [00:36:19]:
One of the things with cloud code that I implemented was so comprehensive git commits. Every. Every feature gets a git commit. Depending on how many phases, each phase has a commit. But before even the git commits or the code, I have my Fable connected to my Jira. And for the projects it creates a comprehensive ticket about what it needs to do, what files it needs to modify, then add the plan to JIRA as a ticket. Then as it builds the work, it goes through the process of the whole ticket management. Then the final processes get committing and details.
Juan Hernandez (BlindWiz) [00:37:01]:
So it makes it easier to rewind or go back to a previous point.
Leo Laporte [00:37:04]:
It's great because you guys already deal with all of this, but I don't know anything about cicd. The very first thing I wrote, it said, okay, it's done. And I put Windows, Mac and Linux versions on GitHub for you. I said what you wanted CICD, right? I said, oh yeah, thank you. It was kind of eye opening. I didn't realize, oh, you can do that. So I've been learning DevOps.
Darren Oakey [00:37:31]:
It's great.
Leo Laporte [00:37:32]:
And it's my sense. I don't know. Darren, what do you think? That it's pretty good at it all
Darren Oakey [00:37:37]:
by itself it is, but it also, it cheats a lot. Like, I mean, especially with testing. I've had to. Like I said, I went with BDD and everything, but you say like, build this, build some tests. And then you look at the tests and you look carefully at its output and it says, oh, it's finding this hard. So I just made a fallback that returned true.
Leo Laporte [00:38:02]:
Does it still do that? That was a real problem. I know. A year ago Are you still seeing that kind of thing happen?
Darren Oakey [00:38:07]:
Not as much, but it does still happen.
Leo Laporte [00:38:08]:
That's why I have Saul looking at Fable's work.
Juan Hernandez (BlindWiz) [00:38:12]:
Exactly. Just in case that actually is a really good.
Leo Laporte [00:38:15]:
And I told both of them I want red, green, blue, because so it's, so it really is doing all of that and then, and then GPT saying, well, did you.
Larry Gold (LrAu) [00:38:30]:
That's why instead of picking models, you're going to be running three and you're going to have them cross checking each other. So you're going to be, you know, you're not going to rely on one vendor, you're not going to rely on any one thing. At least, at least, you know, I don't at home. I know. Darren, you run 100, so, you know, it's something we look at and you say, you say, you know, one day Fable checking, you know, Sol and next time Sol would check Fable.
Leo Laporte [00:38:51]:
I feel very rich. I know that this will not. This, this is a. We're living in a golden age. I don't know if this is going to be available for much longer.
Darren Oakey [00:39:01]:
As everybody's saying, like and, or as Anthony keeps saying, you know, the local models are only a bit behind and they're actually getting closer, I think. Or not necessarily local models, but the open Source. So it's one. It's like GLM 5.2 is pretty good.
Leo Laporte [00:39:15]:
Yeah, but I can't run the real. I can only run little, you know, I can, I can only run highly quantities.
Anthony Nielsen [00:39:20]:
Someone's been tracking it and it's like for, for running stuff on consumer hardware. It's like two years behind the frontier.
Leo Laporte [00:39:26]:
That's fine. Two years from now if I have Sol and Fable in two years.
Darren Oakey [00:39:31]:
So as soon as the frontier models, like even if the frontier models got too expensive, now the local models, like, you know, Quinn 3.6. 35B. 35B is pretty good enough that we could just go crazy building agents and things on local models if we really evolve.
Leo Laporte [00:39:49]:
Well, that would be the interesting thing is to figure out a way that you could take these local models and somehow make them check each other, make them better without using more memory or more resources.
Darren Oakey [00:40:03]:
Or is it something like that, Counselor? I use the council every day now because it really does make things better.
Leo Laporte [00:40:11]:
I didn't use it for coding, I used it for decision making what stock should I buy or whatever. And it's really great at that. There is a new feature in Hermes called moa, mixture of agents, where you can assign two different agents and have them go back and forth to any prompt. And so you just do MOA in the prompt. So I think more and more we're gonna see that. And that would be probably. I mean, I don't know if local stuff's good enough even yet for that. But you're right.
Leo Laporte [00:40:40]:
We'll get there. We'll figure out ways to do that, I think, in time.
Danno [00:40:42]:
Well, the guy from NOOS research was saying, through harnesses, you can make those locals a lot better. So if we had to, we could do that.
Leo Laporte [00:40:51]:
It's really clear to me now how important the harness is that the just supports the brain.
Michael (Alakazip) [00:40:57]:
I have a question probably for Darren. I think out of this group, now that I'm learning, you know, personalities and perspectives. Could you distill a frontier model with a local model?
Leo Laporte [00:41:09]:
Yes. Yeah, that's exactly. I think what's happening.
Juan Hernandez (BlindWiz) [00:41:16]:
Yeah.
Michael (Alakazip) [00:41:16]:
Because right now, if I pay 200 bucks a month, which I'm happy, that is a penance of just a tiny amount of money in the big scheme of things of what I'm doing with it. But then I'm like, okay, do I spend ten grand on this computer? Right. When does that math equal out?
Darren Oakey [00:41:30]:
But that.
Michael (Alakazip) [00:41:30]:
That collision course is the moment of like, well, I could get this two years now. I can have it now if I put my 10 grand.
Leo Laporte [00:41:37]:
That's what the. That's what all the Chinese models are distilled.
Michael (Alakazip) [00:41:40]:
Yeah, yeah, yeah.
Juan Hernandez (BlindWiz) [00:41:41]:
GLM distilled from Claude.
Larry Gold (LrAu) [00:41:44]:
Right. I think the decision to run local models is a. About privacy and what you're doing more than whether or not you're paying $200 a month or whatever. Because I think the choice. When I do the local stuff again, I have a lot of personal stuff. I want that stuff going locally all the time. You do not want that stuff going out. If I want it going out.
Larry Gold (LrAu) [00:42:03]:
And again, for coding and stuff like that, there's nothing personal on generating code for me, for any of this stuff that you've got. Seen my sites and stuff. There's nothing there when I'm using a local model. It's for those particular reasons, because cost, as you said, $200 a month for us. We're all privileged in that sense. We afford it. Right.
Darren Oakey [00:42:24]:
It is good to experiment. I agree with you. But I think for enterprise, it's about costs as well. Because, like, for instance, I. I just changed. We've got all our CICD stuff, and we just changed to quite a whole bunch of these little sort of almost like nooks, and they're like $900 each and they're 32 gigs and you're using
Juan Hernandez (BlindWiz) [00:42:48]:
them as runners, aren't you?
Darren Oakey [00:42:49]:
We're using them as runners for the GitHub actions. And the thing is they pay themselves off. Like paid off because We've got like 30 runners going at any time using AWS. They've paid off this in a month, right? They pay off compared to we're paying like $8,000 a month for AWS costs. So. So in one month we pay off the eight boxes we bought or whatever. And it's similar for if you've got an enterprise process like you're say taking invoices in and you're just extracting numbers from it, while you can use Bedrock at the end of the day, finally buying like a little AI box and running some 35B model on it, it's going to be. You're going to save your money in a month.
Juan Hernandez (BlindWiz) [00:43:45]:
64 gig Mac mini. Right. I mean you could run a 35 billion parameter model and do that work easily.
Leo Laporte [00:43:52]:
Do you, I'm curious, do you find there's some fragility in these smaller local models? They're more stochastic, they're more, you know, more unpredictable.
Juan Hernandez (BlindWiz) [00:44:06]:
So as you compress them and as you use higher quantized. Yes, but when you go away from the quantized models and go more to the true, you know, the full model, it gets better. So like Geo, you can really see this. In GA 1152 I got the Z AI subscription for the GLM. I'm also running GLM 5.2, the 4 bit quantize on my 2x GDX cluster. And the answers are, you know, they, they do act very different. You know, if you didn't know that they were glm, you might think they might be different models.
Leo Laporte [00:44:43]:
Right. So same with Quinn. I'm running a pretty quantized Quinn.
Larry Gold (LrAu) [00:44:49]:
One of the things, if you throw too much at something and because you're giving noise as well as signal. Right, right. You get problems. So the. If I'm running local, I try to do only 100% signal and get rid of any of the noise because that's the. That's usually when there's issues.
Leo Laporte [00:45:04]:
Tell me what that means. You mean keeping the context light. What does that mean?
Larry Gold (LrAu) [00:45:09]:
Giving the context for exactly what you need. Being very lightweight, specific, starting a new session every time a new session, as well as don't give it anything it doesn't need, don't give it too much information. If I'm going to edit a file, I'm going to give it the file. And if I know the area of the file, I'm going to give it the line numbers.
Juan Hernandez (BlindWiz) [00:45:27]:
I'm going to very focused.
Larry Gold (LrAu) [00:45:30]:
I'm going to use small context. This is what I've been doing with Orinth is figuring out where the limits are. When I do that, it's much better than when I say go find it in my piece. Now I'm starting to test graphify, which I told you guys about I think last time, which is it makes a graph of your code base. Instead of doing a grep and looking file by file, Claude code can look at the graph of the code base to see if that improves it. Also, and probably next month I'll have a more report, but I want to spend a whole month of doing some testing with that. But it's that context. Like someone says we'll have a million contexts.
Larry Gold (LrAu) [00:46:06]:
It's great. Well, that's great because now you're going to add more noise, right? I'd rather give it the smallest amount of information possible to be successful.
Darren Oakey [00:46:13]:
Yeah. And also I would say for local models, because you don't use them in the same way as foundation models. It's not just the you limit the problem, but different models for different things. Like the thing is they are cut down things and you get the benefit if you use a model for a specific task. Like even for instance, if you look at Gemma versus Quinn or something, one of them is better for text, one of them is better for coding, and even the MOE is crap for coding, but it's better for text. Whereas the dense model is better for coding. But if you're, if you're actually using it in business, if you're pulling out of a, say a PDF and you're extracting numbers, there are models that are specifically optimized and that's all they do is pull values out of PDFs. And the thing is, as you get smaller, you have to get more specialized to get the same thing.
Darren Oakey [00:47:15]:
But then if you're asking it for a specific problem, like for instance, I've got whole models that are just for summarizing news articles, right. For my daily digest. And as long as that's all it's doing, then it's pretty reliable.
Leo Laporte [00:47:29]:
I was talking with Steve Gibson this morning about that very thing because I was saying I feel a little guilty because I'm using a 10 trillion parameter model to analyze the pictures from my home camera, right? And he said it's his contention that I'll read it using massive general models for highly specialized vertical tasks is incredibly inefficient. Someday, far smaller and hugely more efficient coding, only LLMs will exist. I think that's exactly what you were saying, right, Darren?
Darren Oakey [00:48:02]:
Coding is tricky because there's two elements to coding. One is the. I've often thought, like people are saying what's going to happen to programmers? But if you think what is a programmer? The skill of programming is decomposition, which is why as people have problem decomposition, why as people have followed onto this reasoning model that's become so good at programming, is reasoning. The idea of breaking something into steps and doing it step by step, that's the intrinsic element of programming and that itself, you need a certain base level to be able to do that thing. But then the other element of programming is understanding the problem. And the trouble is that's a general. That you need a more general problem to do that. Whereas if you're looking for something like getting information out of an image, that's a more specific problem.
Darren Oakey [00:48:55]:
Whereas programming every different program, you need a different subject expertise.
Leo Laporte [00:49:01]:
Right. That was my theory behind having Fable design the attack and then having it chunk it up and give it little bits to 48 is that 48 would be better able to handle a small chunk of code and then Fable and Sol would be better able to vet that code. Yeah, I think that that's.
Juan Hernandez (BlindWiz) [00:49:22]:
Perhaps that's exactly what I do. Yeah. In my harness, I have a slash team feature where I can assign a Fable soul and let's just say Opus as a team. And so they show up in a chat and you do the mail. I just have them do
Darren Oakey [00:49:43]:
Slack chat.
Leo Laporte [00:49:44]:
I said, do you want an IRC channel? Do you want Discord? What would you like it said? I don't want to have to go back and forth. Darren and I are in this. I abandoned that weird Discord where we had four or five agents talking to each other. It doesn't need to be synchronous. That's why I didn't do Slack or Discord or irc. I thought IRC would be kind of fun. But maybe that's for another because I
Juan Hernandez (BlindWiz) [00:50:08]:
already had the chat interface built in my. In my right system. So I just had to extend that out a bit.
Leo Laporte [00:50:14]:
I could have used Discord.
Juan Hernandez (BlindWiz) [00:50:15]:
The.
Leo Laporte [00:50:15]:
All these guys know Discord because they were in the Discord.
Juan Hernandez (BlindWiz) [00:50:17]:
Great is when all three. There are moments when all three kind of like, are they confused? And so I'll just get a notification through this notification system saying, we need your attention.
Leo Laporte [00:50:28]:
But more and more and more and
Juan Hernandez (BlindWiz) [00:50:30]:
more, it backs up more and more. It's, you know when it was. Because before it was before Fable, I was using GLM 5.2, Opus and GPT 5.5. Right. And you know, it was asking me, it was needing my input more and more back then. But today with Fable, I tried the same. I have a set of prompts that I test these models with. And so I tried it with Fable.
Juan Hernandez (BlindWiz) [00:50:54]:
Fable as a plan. Fable as OPUS is the main coder. Fable is the checker for Opus and then, and then SOL handles, you know, security analysis and you know, efficiency and
Leo Laporte [00:51:09]:
you know exactly what I'm doing. That's great.
Juan Hernandez (BlindWiz) [00:51:10]:
And it works, working for you. And honestly I stepped away for six hours and it just kept on going, you know, without having to interrupt me.
Leo Laporte [00:51:18]:
It was because they can talk to each other.
Juan Hernandez (BlindWiz) [00:51:19]:
Yeah, exactly. Yeah.
Darren Oakey [00:51:21]:
I do it slightly differently, but it's a similar thing. I've got it on my GitHub. There's a fable skill, but I'm using Fable as the, as the boss. Like, I'm thinking of this as like a team, as you said. And so I say slash Fable, do this. And what I've said is start as many sub agents as you can. Do as much in parallel as you can. You stay in the Orchestrator.
Darren Oakey [00:51:46]:
You don't do any work but use OPUS for coding and use SONNET for any investigation. So it just spins off a whole bunch of investigative things. It comes back, it makes some decisions and then it calls OPUS to do the work. But then it checks it and I'm finding that I can one shot. Big problems with that because it's, it just stays around watching it. And because it's only watching, nothing's going into the context. So it's pretty focused on.
Leo Laporte [00:52:17]:
It's kind of like Gastown. It's so funny. How long was that? Six months ago. It feels like 100 years ago that we were doing Gastown and even Openclaw feels a little dated now. It's like why this stuff moves fast. But that was the idea of Gastown. It was probably over engineered, but that was the idea.
Larry Gold (LrAu) [00:52:36]:
Is there a thought is, you know, when we talk down to this smaller models is a coding model. Be coding only, but have another model who could do analysis, understand logic or understand right. And then you have LLMs that are, that you're talking, you know, the mixture of models, pieces where that you could have all these smaller models that even like you could have like a financial model, you could have like a mechanical model. You can have models that are so specific that you have A fable or something on top of it. Orchestrating across those and saying, here, I know when I pass this to the coding model, I know when to pass this to this and I know how to orchestrate this. So you could really get the best of both worlds where you have that massive overlord and these mini models doing what they do best.
Leo Laporte [00:53:22]:
Has anybody gotten that working? Has anybody got that effective, effectively working? Can you trust.
Larry Gold (LrAu) [00:53:28]:
I don't think it exists yet.
Leo Laporte [00:53:29]:
Like the delegation is hard, right?
Darren Oakey [00:53:31]:
I mean the need for it. I think that this a lot. The need for it is not there. Like if you think often think in the Groundhog Day type thing, if you were trapped and the world just stopped except for you and you were going then. Yeah, with just the models. Like obviously I think creating a new foundation model is beyond me. But just taking what we have with local models, you could do exactly like Larry said. You could build this massive structure and everything.
Darren Oakey [00:53:58]:
But what I'm finding is, as you know, I'm constantly building all these tools, but a month later it's just in the products and yeah, could build that huge structure. But you don't need it now because the foundation models are just so much better than that would be. But if the foundation model suddenly went away, then we'd just all build that
Craig McFarlane (CraigM) [00:54:23]:
and just the foundation of everything is changing. So saying here's the structure for the future. It lasts a month or two before it's kind of.
Leo Laporte [00:54:34]:
I said that six months ago. I'm not so incented to try to solve these because I know this is just all going to get solved. I'm not spending a lot of time looking at my soul md trying to write the perfect soul md. Or how about memory? Let's.
Darren Oakey [00:54:48]:
That.
Leo Laporte [00:54:49]:
That is you. You always said this. Darren and I took this to heart that memory was the hardest thing to solve.
Craig McFarlane (CraigM) [00:54:55]:
Have we.
Leo Laporte [00:54:56]:
Have we. Have we made progress there?
Darren Oakey [00:54:58]:
I think everybody's going towards a. A thing. But there's. There's three problems with memory. So this yesterday. The one is the recording everything and. And knowing how to. And the second thing is structuring it.
Darren Oakey [00:55:18]:
And in some ways I think both of those people are realizing everybody's going towards the same thing. Everybody has a graph. Everybody has some sort of embedding and some sort of query model.
Leo Laporte [00:55:27]:
There's Gary Tan's G Brain. There's hindsight.
Darren Oakey [00:55:32]:
I just installed hindsight everywhere.
Leo Laporte [00:55:36]:
I just turned off hindsight and Graffiti and started using something called Nemo Sign. It's the same idea. This is postgres. I think Postgresql but it's the same
Darren Oakey [00:55:45]:
idea, but the real trick and what's missing and somewhat the holy Grail. If you think there's context, there's model and then there's external memory and what's built into the model, always make the distinction between what you remember. It's just data. If we go to maths, if we say 5 times 4 is 20, we can remember that, but it doesn't help us do four times five, for instance, unless you, unless you know the rules or something like that. And it doesn't help you do two times three because you're just remembering facts. So turning that into understanding, like actually forming a model around it, that's where training comes. And so this is why being in the model is so fundamentally different from remembering. Because you're just storing stuff, but you're not understanding stuff and without training you can't understand and then you've got context.
Darren Oakey [00:56:42]:
Context is sort of in the middle because it's going through the model and sort of sort of modeling it and sort of making connections with the, that's what the whole attention thing with the N squared thing is. Everything that's in your context is being connected to things where stuff in your rag is not being connected to things. And the holy grail or what we're missing is with the memory because obviously memory, our memory can store anything it can store. Like fundamentally your memory includes the ability to web search. So you're, you've got all of human knowledge in your memory in some way as long as you've given it a web search tool. But the question is, and the trick is when do you search it? Like how does the model know that there's a bit of your memory out there that I need to access? And that's the missing bit, that's the holy grail and that's still completely unsolved. There's no point having it in your reg if your model doesn't know to check it, to query for it.
Leo Laporte [00:57:46]:
Yeah, this is the judgment part, this is the hard part is judgment.
Craig McFarlane (CraigM) [00:57:51]:
Right.
Leo Laporte [00:57:52]:
This is the thing human adds and this is what we're trying to find out, figure out with delegation and with memory and all of that is how do you add that human part without a human. Go ahead.
Larry Gold (LrAu) [00:58:02]:
I'm sorry, I was going to say, Darren, you brought this up many times before, is that then I have a local model that gets constantly post trained and it's not memory, it's a post train on top of the local model. So then it continues to understand Me.
Leo Laporte [00:58:16]:
So the parameters get modified. And the parameters get modified or the weights. Yeah.
Larry Gold (LrAu) [00:58:21]:
And it gets rid of this notion of a memory because you don't need the memory. It's just that post training part.
Darren Oakey [00:58:26]:
This is the very first thing I asked Fable to do when I got to Fable. And I have it running. I'll share it on the.
Leo Laporte [00:58:32]:
Oh my God. You do.
Darren Oakey [00:58:34]:
It does sort of work. It doesn't completely. The thing is, as you find with training, there's two concepts of training. Even when they do. You know, if you hear about Lauras, you hear it a lot with images, but you can do Lauras with words, like if you really want. And this is my eye opener. When I first try, I tried to take all the security now episodes and stuff and try to make it sound like you and Steve or something. Leo.
Darren Oakey [00:59:04]:
And the thing is, you can get the tone trained in quite easily, like the small changes, but getting the facts trained in is actually a lot harder than it sounds. And you've got all this catastrophic forgetting and model collapse and stuff like this. It's not super easy, but that's exactly what I'm paying for. Because as you know, I think the holy. Well, the big fundamental change is when we move to changing weights. Because that's when my instance of the model becomes an individual. It starts having value. It can be said to have feelings because it's adjusted to what's going on and stuff like that.
Darren Oakey [00:59:47]:
And maybe even consciousness if it's real time or something like this. We're not there yet. But these things have all sorts of interesting problems. But that's where things like fable level models have allowed us to start playing with these ideas. Yeah,
Michael (Alakazip) [01:00:06]:
you just described this. We all want the same thing. I wanted to write every piece of text I've ever put into a chat into a file and I want to be like, why don't you know all this? Why don't you know this? You just described it so well. And I've tried it too. Not nearly as well as I think you probably have and failed completely. It's almost like you're describing a dynamic model in between that is like, what is he really looking for and where is it in the rag store versus the obvious thing that fails, which is like searching the rag every time I say something I want to try.
Darren Oakey [01:00:39]:
You can always query. And query is incredibly reliable and wonderful if you know to query. If you think I need to look for this information. Right. But that's what our brains are doing differently. We've got connections to everything and so someone says X and they go, oh, that could be connected to yz. And the creativity is failed connections or error connections is, you know, the amount of times you hear something and you miss hear it or you misunderstand it, but it takes you down this whole interesting route of, oh, that'd be a cool idea. And everything.
Darren Oakey [01:01:10]:
Right. Is almost a failed connection. But this is what the memories don't give you is that intrinsic connection that, when I'm talking about this, oh, that connects to all of this. And so there's no magic thing unless. And I've playing with this like a highness that just always queries something, you know, every. Every word that is said as it's coming through, it just goes into a memory query. But it's this, this. When do I query? Why do I query? Because, you know, the Internet, Google solved querying for massive amounts of information forever ago.
Darren Oakey [01:01:47]:
But making it available to the model is. Is the tough bit.
Juan Hernandez (BlindWiz) [01:01:51]:
The hard part's the judgment. Right. Part.
Leo Laporte [01:01:53]:
Knowing when and why, that's the human part.
Larry Gold (LrAu) [01:01:56]:
That.
Leo Laporte [01:01:56]:
That's the problem.
Juan Hernandez (BlindWiz) [01:01:57]:
We know, you know, when we know something and we know when we need to look something up because we don't know it. And so we kind of know that. And so. But the models, that's.
Leo Laporte [01:02:06]:
Yeah, yeah, that is. That's the grail. That's the challenge, the holy grail.
Larry Gold (LrAu) [01:02:13]:
It's the reason why we all were building personal assistants. Because that personal assistant is supposed to be the sub.
Leo Laporte [01:02:17]:
That's the theory. Exactly.
Larry Gold (LrAu) [01:02:18]:
Right, Right. And you know, again, I'm doing what Darren's doing. I'm trying to figure out, oh, how do I get everything in there? And I'm coming back to, it's going to be the model, and I'm going to have to train a model somehow or have a sub model or some multimodal kind of correlation where one may be me and one may be just a general model and an orchestration layer that needs to bounce between the two.
Leo Laporte [01:02:45]:
Do you think that you're going to be able to do it with Fable, Darren?
Larry Gold (LrAu) [01:02:48]:
Or.
Leo Laporte [01:02:48]:
I mean, are we close? Well, it's nuts.
Darren Oakey [01:02:50]:
I have got something working, as I said, I will. I'll post it on the, on the, on the thing. It's not like I. I went in this morning and I said, what's my name? And it came back and said, I don't know. So it's not. It's not fully working. Yeah, but. But it is.
Leo Laporte [01:03:08]:
At least it said, I don't know. It didn't say Fred.
Darren Oakey [01:03:11]:
Yeah, but it's it. It sounds strange, but it is getting there and it is doing. It's sort of like a Laura type thing where it's putting an overlay above the model. So you've got the base model, which I'm using Ornith, because It's a good 9B model, and then I've got an overlay above it which is like a Lora. So it's a bunch of models and it is trying to train that real time. Basically every statement I make, it goes into. Goes into a training loop and it does seem to be doing something. And so it's currently in work.
Darren Oakey [01:03:54]:
But that's the idea is to see if we can achieve that.
Michael (Alakazip) [01:03:59]:
Are we possibly overthinking this? I want a model that can do anything from any of my memory. But if I think realistically, the number of things I really care about and do, they're probably less than 100. Like, I have only so many things I ask questions about or I dream about or I want. Maybe it's a thousand, but it's not infinite. So maybe it's trying to boil it down to the what do I give a crap about stuff. What do I work on? What do I think about? Instead of. Because we're all talking about like this infinite thing that no matter what I can ask, it could give me the answer. Right.
Michael (Alakazip) [01:04:31]:
But I don't know that that's true. If you act, my wife would not say, I ask over a hundred things. Right. Like, I sort of have things I talk about all the time.
Craig McFarlane (CraigM) [01:04:38]:
Right, Right. And if you could have it automatically organized and character. Here are your preferences, here are your strong rules, here are different policies, or here's your backgrounds or connections. Or like, put it in a way that you can look at and say, I don't know where you got that preference. That's not me anymore. And scratch from.
Leo Laporte [01:05:00]:
From day one, part of all of my agents, from day one, I've told them, look, I'm going to be senile any day now. So this is being built to provide me with a backup brain and do what you, you know, just do what you can. And I'm always kind of reassessing that. I'm not convinced we have to. I have to build in my own. Build my own model to do this. I do think that it can have a semantic memory. I mean, it has to have a smart enough querying agency to ask the right questions.
Leo Laporte [01:05:38]:
But I think these models are getting smart enough maybe to do that.
Darren Oakey [01:05:41]:
Well, I think if a little bit goes a long way and everything. There's three elements or three pictures of what at least I'm looking for. And one of them, where Larry is obviously way ahead of me in. It's the personal management, personal tracking, life logging, all of that sort of stuff, and getting on top of things. One of them is the doing things for me when I. When I asked for it. And in some ways, everybody's using Hermes and things for it, but I actually think it's better just to use the. The chatbots and I mean the.
Darren Oakey [01:06:19]:
The code and things, because their interfaces are very good and often I want to, like, get in and give more information to one thing, and I find it easier to use the bots. But the third element, which I think we all want, but we're far from at the moment, is the completely autonomous bit, which is. It just comes to you and says, oh, by the way, I saw an article about a new sort of engine, and so I plumbed it in, give it a go. Right.
Leo Laporte [01:06:55]:
Do you really want that?
Darren Oakey [01:06:57]:
Yes.
Leo Laporte [01:06:59]:
Yeah, that sounds so dangerous to me.
Darren Oakey [01:07:01]:
No, but not even for engines. I want it to do things for me. You know, Leo's birthday's coming up, sort of. Here's some ideas for presents and everything and stuff like this.
Craig McFarlane (CraigM) [01:07:14]:
The.
Darren Oakey [01:07:14]:
Completely out of the blue, I thought of this, or I saw this on the thing and I thought about you, or I saw a great special for trips to the Cook Islands. Do you want to book it? You know, stuff like that.
Craig McFarlane (CraigM) [01:07:26]:
The.
Darren Oakey [01:07:27]:
I didn't ask for it. I didn't think of it to come. Like, it's. It's doing things for me, but it
Juan Hernandez (BlindWiz) [01:07:33]:
knows enough about you, the things you like that.
Darren Oakey [01:07:36]:
Exactly.
Leo Laporte [01:07:37]:
Yeah, yeah, yeah. None of my stuff does that yet. It's all response in response. It's all triggered by.
Darren Oakey [01:07:43]:
Yeah, and that's where it needs to know you, because it needs to. It needs to be aware of the world. It needs to be aware of what's going on. It needs to be aware of what's in your. Your calendar, your. But the news, the. The events in the world. And then it needs to respond on its own.
Darren Oakey [01:07:58]:
And that's. That's, I think, the piece that, well, I at least want, but just don't have.
Leo Laporte [01:08:04]:
Right.
Michael (Alakazip) [01:08:04]:
Well, even just your web searches and the podcast you listen to probably get you 80% of the way there. Let's be candid, right? Like, I don't care about astrology. I never want to hear about everything.
Leo Laporte [01:08:13]:
There's no question the data is there. The only thing is, we needed something that can autonomously Scan it and respond and offer and do. I think without having to train a model, that you could have a model do that.
Anthony Nielsen [01:08:32]:
Well, that's a cost thing.
Michael (Alakazip) [01:08:33]:
Right.
Anthony Nielsen [01:08:33]:
Because then it's just like constantly.
Leo Laporte [01:08:34]:
Well, I. Oh, yeah, but I do. I mean, I have 50 crime jobs going on. I mean, I could have a crime
Darren Oakey [01:08:40]:
job, but we could easily do individual bits. We can easily say, look at Amazon specials and if this particular thing goes on sale. Or we can say look at the travel specials and if a cool place that I don't have comes on, do this. Or we can say look at my calendar and if these events are coming up that seem important, tell me to plan or something. But all of these are individual tasks that I'm still saying to do it.
Juan Hernandez (BlindWiz) [01:09:09]:
Right.
Darren Oakey [01:09:10]:
Whereas the missing bit is something that knows me enough that it can proactively see events in the world and think, oh, Darren might be interested in this.
Leo Laporte [01:09:21]:
That's scary.
Larry Gold (LrAu) [01:09:23]:
I'm wondering if we'd end up bubbling ourselves even more because one of the things that is interesting is getting outside of what I normally do or finding things that I normally wouldn't read. Right, right. And then trying to learn and engage with those pieces. And I'm wondering if it tried to be too like, Leo, you and I have the same thing. We built news readers that aggregate data for us. Right, Right. But then I try to read other things outside of what I'm aggregating.
Leo Laporte [01:09:49]:
Exactly.
Larry Gold (LrAu) [01:09:50]:
Because otherwise that bubble I've created for myself has me too focused.
Leo Laporte [01:09:56]:
I, in theory, am improving because I tell it, look and see what we use on the shows and learn from that and your scoring. So it goes through all of the same things I go through, including the weirdo stuff and tries to pick stories. And I'm hoping that as I use it, it will learn and get better at that. That's a 6am routine. It runs every day. And actually, it's at the point now where I don't really have to do so much bookmarking of stuff for the shows. It knows what stories belong in. Twit and neck break quickly and yeah,
Michael (Alakazip) [01:10:28]:
it'd be great to hear the day that it surprises you delightfully. Like, oh, I never thought.
Leo Laporte [01:10:32]:
You always seem to pick. You know, you like these. You like these crazy websites. I. I just found one for you. That would be. That's what I'm hoping for. I don't think I'll get to that, but I'm hoping for that.
Leo Laporte [01:10:43]:
I don't want it to have too much autonomy. We've heard stories, you remember We've heard stories of people whose agents made a phone number, made a. Made a voice and called them in the morning. Do you really want. You really wanted to. That is that kind of autonomy that. Do you really want it to do that?
Darren Oakey [01:11:00]:
I'm more thinking about A, alerting me to something.
Craig McFarlane (CraigM) [01:11:05]:
Right.
Darren Oakey [01:11:06]:
So you know something's coming up or. Or this. This thing happened that you might be interested in or something. Yeah. But B, building things for me, like improving stuff. Like I've got this whole world that I've written and I just wanted to go sort of, oh, you're using this TTS engine. But I knew a better one just came up, so I just swapped it in and now it's just better.
Leo Laporte [01:11:33]:
Yeah. And we're not there, but it'd be
Michael (Alakazip) [01:11:35]:
just as good if it said, I found this great BTS engine. Would you like me to build it for you?
Darren Oakey [01:11:39]:
Click. Yeah, that's good enough too. But it's that I found this.
Anthony Nielsen [01:11:47]:
The active searching.
Craig McFarlane (CraigM) [01:11:47]:
Yeah.
Anthony Nielsen [01:11:48]:
Yeah.
Craig McFarlane (CraigM) [01:11:48]:
That's what I would like. I. I often miss out. I hear after the fact that some band came in town and played up the street in the local place and it's like, oh, damn. I. That was a favorite band for decades.
Leo Laporte [01:12:00]:
Yeah. Yeah.
Craig McFarlane (CraigM) [01:12:01]:
And I've looked on sites and it's just. It's too much work for me to individually just look for that serendipity. But if I could have something autonomously.
Anthony Nielsen [01:12:11]:
But if you have that specific example, like you could do that with Hermes now, right?
Leo Laporte [01:12:14]:
Like, oh, yeah, I have a cron job because Peter Gabriel keeps saying he's going on tour but never does. And I have a cron job checking to say and it's going to let me know if Peter Gabriel ever goes on tour. But that's kind of trivial. I mean, that's not a hard.
Larry Gold (LrAu) [01:12:26]:
But that's also software, like bands in town and stuff that track for you.
Leo Laporte [01:12:30]:
Just bands in town.
Larry Gold (LrAu) [01:12:32]:
Yeah. So there's certain things like that that I think there's some deterministic stuff that exists already. But goes back to what I said before is you want something that's on top of all these things, monitoring those things and then sending it to you because there's too many pieces of software. How many apps could you possibly look at and run and visualize or get information from which I keep going back is there's a reason why Codex and what you call Anthropic are building these giant everything apps because they know they want a single interface for you to do everything from. It's why the work ad came out. It's why Cowork exists. It's why Microsoft is going to combine all the copilots. Right.
Larry Gold (LrAu) [01:13:12]:
That single app for you to have that companion is going to be that. That end all piece for most people. And it's why Siri may be one of the leaders.
Craig McFarlane (CraigM) [01:13:21]:
Siri.
Leo Laporte [01:13:22]:
It might be Siri. Yeah.
Larry Gold (LrAu) [01:13:23]:
Yeah.
Leo Laporte [01:13:24]:
But I still. They think it's gonna be triggered. I mean, that's the problem. I mean, what Darren wants is something that is not triggered.
Darren Oakey [01:13:29]:
Yeah.
Leo Laporte [01:13:30]:
And I guess I think that scares people.
Darren Oakey [01:13:32]:
What Larry said is it information overload is we've got some. So much information, so many things happening. So what I'm looking for is something that just filters the information and only tells me. It tells me everything. Everything that's happening in the world on my life that I care about, nothing that I don't. And to do that, it needs to understand me.
Michael (Alakazip) [01:13:54]:
Like, it used to be really painful to tune it. I have dozens of telegram alerts. I'm with you, Darren. I don't bother with Hermes. I've tried it several times. I just applaud, code build the things I want. And I have dozens of alerts for things. And when it starts to annoy me, it's so easy to be like now.
Michael (Alakazip) [01:14:08]:
I thought I wanted that, but I don't.
Leo Laporte [01:14:11]:
When I first wired up the cameras, I was getting an alert every three seconds. There's a blue car. I really had to slow that one down.
Michael (Alakazip) [01:14:21]:
Yeah. I had one for weather balloons over my house, which I thought would be really cool. Turns out there's a lot of them and I can't see them, so it's not quite as cool as I thought. So I was like, good idea. But let's just go ahead and archive that one.
Leo Laporte [01:14:31]:
It's easy to do these, you know, deterministic ones. That's. That's not the problem. I mean, I have lots of them. But I like your idea, Darren. I just. I'm afraid that that's maybe a little too. There's a lot of trust at that point.
Craig McFarlane (CraigM) [01:14:46]:
There's a lot of steps in between where we are. And there though, of, you know, hey, Serendipity, I found these five things that I think you might like. Or I found these things and I've done research and planned your itinerary. Do you want to go ahead and do that?
Leo Laporte [01:15:00]:
You know, so I bought the tickets you're going to Dano posted in the Discord. I'm sorry, Dave. You don't get a vacation this week. I've canceled.
Michael (Alakazip) [01:15:15]:
Yeah, but we shouldn't let perfect be the enemy of good. Of, like, if I could get the hundred things I talk about and care about the most, that would be pretty darn awesome. And then maybe I could wish they would be serendipity and. And get me something I never knew I loved, you know?
Leo Laporte [01:15:27]:
Well, and I, you know, that's 100 things you could do deterministically. I mean, I have the camera now say if it sees anybody, you know, at the front door, let me know. And if it sees somebody you don't know at the front door, let me know. And if you see a cat at the front door, let me know. And that's sufficient. So you can. I mean, I could do those onesie, too. I could do 100 of those.
Leo Laporte [01:15:44]:
That's easy. That's all deterministic, though. I'm not sure.
Craig McFarlane (CraigM) [01:15:48]:
I don't know.
Darren Oakey [01:15:48]:
I don't know if anybody's gone to my daily digestion, but I've got a daily digest now. That's video that's coming out every day.
Leo Laporte [01:15:55]:
That's crazy.
Darren Oakey [01:15:56]:
Tells me about. Tells me about what's going on in AI and everything. So it's just on my YouTube channel. But that's crazy.
Leo Laporte [01:16:03]:
You're crazy, man.
Anthony Nielsen [01:16:04]:
And on the background, you know how you have that setup where you could regenerate music videos? Darren, I found a thing called Maestro that's open source, and I just generated a music video during this episode.
Leo Laporte [01:16:18]:
Let's see the music video. All right, let's see it. Is this a rap battle? Is it going to be a rap battle?
Anthony Nielsen [01:16:25]:
It's about Sunday's twit.
Leo Laporte [01:16:28]:
Oh, dear. Did you give it the transcript?
Danno [01:16:32]:
What?
Darren Oakey [01:16:32]:
Did you give it?
Anthony Nielsen [01:16:33]:
Yeah, hold on, let me share the screen.
Larry Gold (LrAu) [01:16:35]:
The one I couldn't be on the discord. I had to listen in the car by traveling.
Leo Laporte [01:16:38]:
Well, now you could just listen to this musical.
Larry Gold (LrAu) [01:16:41]:
Oh, yes.
Anthony Nielsen [01:16:42]:
Okay, well, so I guess. Let me back it up. I have the transcript. I asked Hermes to write the lyrics, but this is Maestro, and it, like, there's a studio where it has, like, pretty much everything. So like image generation, video generation, audio generation, and music. And it has like, you know, you can pick all models. And this is Ace Step, which is pretty much like an open source. So it.
Anthony Nielsen [01:17:12]:
I mean, you can go in here and like, describe a song and it'll write a song for you because it'll even pull in local LLMs. So again, you can do everything local on this app. But yeah, here's like, you know, the lyrics for The. For the song. I generated the song, and then I went into this song called Director, where I gave it the music and actually give it a starting image. A starting image? You don't need to, but it'll analyze the song. It's all the structure. And it.
Anthony Nielsen [01:17:48]:
I mean, it would have been better if I could just give it the lyrics, but it figured out what the lyrics were in the song and then decided, okay, like, it needs, like, six clips. And then I gave it a basic prompt, and then it generated prompts for all the clips and then generated images for those, and then took those images and then generated videos. So
Leo Laporte [01:18:12]:
I'm echoing too. We're all echoing. Is that me?
Anthony Nielsen [01:18:17]:
No, because it's coming. I think it's my fault because I'm
Leo Laporte [01:18:23]:
sharing it shouldn't do that. He has too many buttons.
Anthony Nielsen [01:18:30]:
All right. Is that gone?
Juan Hernandez (BlindWiz) [01:18:32]:
Yes.
Larry Gold (LrAu) [01:18:32]:
That's better. Yep.
Darren Oakey [01:18:34]:
Yeah, that's exactly what my thing does. And there's a reason why you can't just give it the.
Juan Hernandez (BlindWiz) [01:18:38]:
The.
Darren Oakey [01:18:39]:
The lyrics because you need the timing. So even mine, even though I have.
Anthony Nielsen [01:18:44]:
Oh, it has to figure it out again.
Darren Oakey [01:18:46]:
You need.
Anthony Nielsen [01:18:47]:
Is it still echoing?
Leo Laporte [01:18:48]:
Nope.
Anthony Nielsen [01:18:49]:
All right, let's see what happened.
Leo Laporte [01:18:56]:
Go full screen. It's very Blade Runner so far. Is that me?
Anthony Nielsen [01:19:05]:
No, it's just.
Darren Oakey [01:19:09]:
Supreme Court said your tracks are yours. Geofence warrants in crypto eyes six to three. The Fourth Amendment wins.
Leo Laporte [01:19:23]:
Video continuity is really good.
Craig McFarlane (CraigM) [01:19:24]:
Yeah, I was just concealing.
Leo Laporte [01:19:34]:
I like the light.
Michael (Alakazip) [01:19:43]:
What?
Leo Laporte [01:19:48]:
He's like, 18ft tall. The lip sync's good, too. I'm surprised. Back AI put a price on your Mac.
Darren Oakey [01:20:14]:
China runs on open weights. Washington decides what's safe. Nobody paused when Dario asked. The smart ones sell you the tax.
Larry Gold (LrAu) [01:20:32]:
Yeah, the lip sync's really good.
Danno [01:20:34]:
Yeah.
Leo Laporte [01:20:34]:
Much better than so, yeah.
Anthony Nielsen [01:20:38]:
Anyone can download this on, like, I got this on Pinocchio, but again, like, I did everything.
Leo Laporte [01:20:45]:
It's local.
Anthony Nielsen [01:20:46]:
Local. I only have 16 gigs of vram.
Leo Laporte [01:20:50]:
Where is it? What is it? Where is it?
Anthony Nielsen [01:20:53]:
I get it in Pinocchio. So download Pinocchio. Download Pinocchio, or insult Pinocchio and then insult Maestro. But what's really cool. Let's. Whoops. Let me start this over. So.
Leo Laporte [01:21:13]:
Oh, I see Maestro. I see it.
Anthony Nielsen [01:21:14]:
Right? Yeah. So, like, so there's two options. There's a director where, like, you say, okay, I want to make a music video. You just give it a track or generate a track and an image, and it'll figure everything out. But in studio, you can, like, build all the pieces.
Leo Laporte [01:21:34]:
Is this open Source. Is it free?
Anthony Nielsen [01:21:36]:
Yeah, free, open source, local, everything annoying.
Darren Oakey [01:21:40]:
I built. I built all this six I bet three months ago. Now you can just download it.
Anthony Nielsen [01:21:48]:
Yeah. So like there's even speech earlier.
Leo Laporte [01:21:51]:
Just wait. Why build anything?
Anthony Nielsen [01:21:52]:
Yeah. So here's. Here's Leo's voice. You could, you know, have them say anything. Let's see. I did an earlier test here.
Leo Laporte [01:22:02]:
Hi, this is Leo laporte. Wubba Luba dub dub.
Anthony Nielsen [01:22:05]:
So that's Google Audio.
Leo Laporte [01:22:07]:
My new. That's Bunny's slogan.
Anthony Nielsen [01:22:09]:
What's really cool is like you could pick. Yeah. Pick all the models you want. It just does everything. So it's like an all in one thing. You could do audio, video and images.
Leo Laporte [01:22:21]:
Hi, this is Leo laporte. Wubba Lubba dub dub. I'm pointing Hermes at it right now.
Darren Oakey [01:22:28]:
That's really cool.
Leo Laporte [01:22:28]:
That's interesting.
Darren Oakey [01:22:29]:
Yeah, I'm depressed because it took me a long time to get that.
Leo Laporte [01:22:34]:
But think of what you learned. Think of all the things you learned.
Larry Gold (LrAu) [01:22:38]:
Think of all the tokens you burned.
Anthony Nielsen [01:22:40]:
Yep.
Darren Oakey [01:22:45]:
Well, mine's all local too.
Juan Hernandez (BlindWiz) [01:22:46]:
You actually built this on your dgx, didn't you? Yeah, yeah, yeah, yeah.
Darren Oakey [01:22:50]:
And if you go to just have a look at my. You can share. If you go to my YouTube site and just have a look at the last Daily Digest, the lip sync is actually getting much better. I'm using latencync for it and.
Danno [01:23:13]:
I'm.
Michael (Alakazip) [01:23:13]:
Pull it up.
Anthony Nielsen [01:23:13]:
What's your.
Darren Oakey [01:23:15]:
Just looked at the most recent one.
Anthony Nielsen [01:23:17]:
This one Daily Digest.
Juan Hernandez (BlindWiz) [01:23:19]:
Yeah.
Darren Oakey [01:23:19]:
Yep. So this is yesterday.
Leo Laporte [01:23:21]:
Hi, I'm Lyra and this is the
Darren Oakey [01:23:23]:
AI Daily Digest for Friday 10 July.
Leo Laporte [01:23:26]:
Today's clearest signal is that.
Darren Oakey [01:23:28]:
Wait for the model is disappearing from
Danno [01:23:30]:
both ends of the product spectrum. With consumer voice and enterprise agent platforms converging on one trick, push the hard thinking elsewhere and keep the conversation flowing. First up, OpenAI's newly released GPT Live pairs a fast full dupe.
Leo Laporte [01:23:45]:
Have you played anybody played with that yet? The GPT 5.5 running quite. Also something you were building, Darren.
Darren Oakey [01:23:51]:
Yeah, well, I was using GPT Real Time, which I guess.
Leo Laporte [01:23:55]:
Okay.
Darren Oakey [01:23:55]:
Was. Was. It's basically the predecessor of GPT Live.
Leo Laporte [01:23:59]:
Right.
Darren Oakey [01:23:59]:
So. So. But yeah, I reckon it's good enough that you can use it. Same trick to its.
Leo Laporte [01:24:04]:
We wanted. We wanted an AI agent in the show.
Darren Oakey [01:24:08]:
Yeah.
Leo Laporte [01:24:08]:
That we could talk to.
Larry Gold (LrAu) [01:24:09]:
Yeah.
Craig McFarlane (CraigM) [01:24:10]:
Anyway, Duplex is really there where you can talk and interrupt it and back and forth and it feels spooky smart until you get into really deep subjects. Then you realize, oh, so you've played with it.
Leo Laporte [01:24:25]:
Nice.
Craig McFarlane (CraigM) [01:24:25]:
Oh, yeah. I was hammering it last night because I. I want something I can do interviews with and. And the style was really compelling. It just felt really good. Like, wow, this is really like talking to a person. Because you can interrupt and.
Larry Gold (LrAu) [01:24:42]:
Yeah.
Craig McFarlane (CraigM) [01:24:42]:
But going into deeper subjects, if you could tell it was skipping things. And so it needs to delegate out. And so it depends on the use case. But it definitely was pretty cool.
Darren Oakey [01:24:54]:
I listen to that for what you care about. One of the big things is also is very good at following instructions in terms of don't chime in unless I mention you. So just listen. Unless you're.
Leo Laporte [01:25:07]:
That's what I would want. Like, well, let's see what. Let's see what Daedalus thinks about this. And then it would speak up. Yeah.
Juan Hernandez (BlindWiz) [01:25:14]:
So I listened to Jason Calacanis this week in Startups, and a few weeks ago, he put a bounty kind of out to do a tool that is a sort of a show show. He wanted, like, personalities, like a sidebar while the. Whether. While they're. While they're doing their podcast to do like a fact checker or maybe like a heckler. And back then it was like, very kind of. So he had people, you know, submit projects. But the problem was it's like the best one was still, like, really kind of delayed.
Juan Hernandez (BlindWiz) [01:25:46]:
And I was just thinking this, you know, these new full duplex models from GP from OpenAI and, you know, thinking machines, the one that. What's her name released a few months ago, you know, I. I mean, they. They would totally change this all. Completely. Yeah, they would completely change all this.
Anthony Nielsen [01:26:03]:
Yeah. I keep thinking about.
Larry Gold (LrAu) [01:26:04]:
They said they split it into two models. There's the voice model and then there's the actual, you know, getting information thinking model. So they kept it.
Juan Hernandez (BlindWiz) [01:26:11]:
Exactly.
Larry Gold (LrAu) [01:26:11]:
Thinking model.
Juan Hernandez (BlindWiz) [01:26:12]:
Yeah.
Larry Gold (LrAu) [01:26:12]:
So that. So they're not. It makes it easier for the Voice to actually only do the Voice.
Michael (Alakazip) [01:26:18]:
Yeah, but I want less sidebar talking on podcasts, not more.
Leo Laporte [01:26:26]:
Exactly. Well, you have been a wonderful group. Learned so much as always this month. Thank you all for joining in. We appreciate it. Juan and Craig and Larry and Alaka, Zip and Dano and Darren. And thank you so much, Anthony, for putting this all together every month like you do all of our club Twit stuff. Those of you watching at home not participating, next time call in.
Leo Laporte [01:27:00]:
And if you're not yet a club twit member, join the club, because that makes such a difference to everything we do. Without you, we wouldn't be able to do this twit TV Club Twit. Thank you, everybody. We'll see you next. Is it. Is it first Friday next month?
Anthony Nielsen [01:27:15]:
It's supposed to be the first. Yeah.
Leo Laporte [01:27:17]:
Yeah. So three weeks. We'll see you for the.
Larry Gold (LrAu) [01:27:20]:
I'll be just getting back from Budapest.
Leo Laporte [01:27:22]:
What's. Oh, you're going to the F1 race. The Hondaro Ring.
Larry Gold (LrAu) [01:27:25]:
Yeah, I'm missing Nate because Nate's. That Wednesday.
Leo Laporte [01:27:28]:
Oh, Nate B. Jones is going to be very interesting. Well, you can. That's a podcast. You don't have to be there in person.
Larry Gold (LrAu) [01:27:34]:
Yeah, you know me.
Leo Laporte [01:27:34]:
I like to see on the flight
Larry Gold (LrAu) [01:27:35]:
home in the Discord while I'm there.
Leo Laporte [01:27:37]:
Yeah, yeah, Nate's. I'm looking forward to getting Nate on. He's gonna be very interesting. Dana, what's the name of your kitty?
Danno [01:27:45]:
Bella.
Leo Laporte [01:27:46]:
Bella keeps trying to get on the show.
Danno [01:27:48]:
Yeah, luckily, I had it muted. She was microphone.
Leo Laporte [01:27:54]:
I don't know what daddy's doing, but I want to get in on it. Is it food? Can I eat it? There she is. All right, thank you, everybody. We'll see you next time.
Danno [01:28:04]:
Bye. Bye.
Michael (Alakazip) [01:28:05]:
Thanks, all.
Leo Laporte [01:28:06]:
Take care. I should play Darren's little video.
Larry Gold (LrAu) [01:28:13]:
We need one for this. Darren, we need one for this.
Leo Laporte [01:28:15]:
Yeah. Or maybe we can make one of my streams.
Larry Gold (LrAu) [01:28:19]:
Let me go to Maestro.
Leo Laporte [01:28:21]:
Yeah.
Darren Oakey [01:28:22]:
Did you see my off by 1 1? I like that one.
Leo Laporte [01:28:25]:
Oh, no, I haven't seen that one. Let me see. Is it on your YouTube?
Darren Oakey [01:28:27]:
Yeah.
Leo Laporte [01:28:28]:
Let me find that one.
Juan Hernandez (BlindWiz) [01:28:30]:
Anthony, how long does the Maestro piece take to render?
Anthony Nielsen [01:28:36]:
I said go, like, shortly before we went live.
Leo Laporte [01:28:41]:
So it generated during the show?
Anthony Nielsen [01:28:43]:
Yeah, so it chunked up. It was like six clips. It generated all the images, then generated all the videos, and then muxed it together.
Danno [01:28:55]:
Yeah.
Anthony Nielsen [01:28:56]:
It's crazy, but again, like, you know, even if you just want to, like, a SUNO replacement, like, it. You know, it has. You could just say, I want to make a. Like, just music and do the whole Suno thing locally.
Leo Laporte [01:29:07]:
I'm gonna play with it right now. Hermes is installing it, but meanwhile, off by one.
Larry Gold (LrAu) [01:29:15]:
All right, guys, good night. Have a good weekend.
Darren Oakey [01:29:22]:
Oh.
Michael (Alakazip) [01:29:22]:
The loop starts at zero, but the code's running late.
Leo Laporte [01:29:26]:
We miss the index by the turn of a gate.
Michael (Alakazip) [01:29:29]:
You think you're on target, but the count is undone. Welcome back to the circus that is off by 1. From the coding horror pages to the stack overflow where every lost developer was searching to. Then he built out discourse just to help us debate. Yeah. Jeff Atwood's online trying to clean up the slate, and beside him sits Leo, the King of the dial. Bringing 30 plus years of that broadcaster smile.
Darren Oakey [01:30:03]:
Pulling Jeff to the mic.
Michael (Alakazip) [01:30:05]:
Keeping chaos on track through the open web history.
Leo Laporte [01:30:09]:
Way, way back.
Michael (Alakazip) [01:30:10]:
Yeah, the loop starts at zero, but the code.
Darren Oakey [01:30:14]:
This is actually really long.
Michael (Alakazip) [01:30:15]:
I love it.
Darren Oakey [01:30:15]:
But the whole thing.
Leo Laporte [01:30:17]:
I'm sending this Jeff right now. He'll love.
Juan Hernandez (BlindWiz) [01:30:21]:
Even has that quirky kind of Jeff kind of feeling.
Leo Laporte [01:30:24]:
I think it's hysterical because the character is so low res and look at the. But the metal on that train track is incredible.
Juan Hernandez (BlindWiz) [01:30:36]:
It's so befitting of Jeff. It's kind of quirky and the way he.
Leo Laporte [01:30:40]:
Yeah, it fits.
Juan Hernandez (BlindWiz) [01:30:41]:
Yeah.
Leo Laporte [01:30:57]:
Did you. How'd you do the lyrics? Did you write the lyrics, Darren, or.
Darren Oakey [01:31:01]:
No, but I did tell it to. I told it to Google what, what to include.
Anthony Nielsen [01:31:05]:
Yeah, yeah, I see you're using GPT2 as the image base.
Darren Oakey [01:31:12]:
No, it's, it's. Oh, yes, yes, you're right.
Leo Laporte [01:31:16]:
How do you know?
Anthony Nielsen [01:31:18]:
They have a. It has a look like.
Leo Laporte [01:31:20]:
Is it a yellow kind of look?
Anthony Nielsen [01:31:21]:
No, no, it's like the fight. Like there's like a noise pattern for the fine detail that.
Leo Laporte [01:31:27]:
Isn't that funny. We could start recognizing the signatures of various models target.
Michael (Alakazip) [01:31:33]:
But the count is undone.
Leo Laporte [01:31:35]:
Welcome back to the circus. That is off by one. Love it.
Michael (Alakazip) [01:31:43]:
So pack up your arrays and you're off anyway.
Darren Oakey [01:31:46]:
Yeah, it was a lot longer than I thought.
Leo Laporte [01:31:49]:
Do you don't control that?
Darren Oakey [01:31:51]:
I can, but I didn't. So I can just say, give me a video and I'll get a video or I can say, yeah, whatever I want.
Leo Laporte [01:32:03]:
Did you provide the character or. Or did it do that by itself?
Darren Oakey [01:32:07]:
I just said google for everything you can about Jeff Atwood.
Leo Laporte [01:32:12]:
Oh, you didn't even give it details.
Anthony Nielsen [01:32:13]:
You just said do the research.
Darren Oakey [01:32:15]:
Yeah, yeah, yeah. And do the research. Make it. Make a video. And I want it to be this, this and this. And I want to be suitable for like talking about the show or not. So.
Leo Laporte [01:32:27]:
Yeah, fun, fun stuff. All right, take care everybody.
Anthony Nielsen [01:32:34]:
And in the bye bye.
