Episode 19
AI, Phones, and Quantum Confusion: A Podcast Potpourri
We dive straight into the thick of it, chatting about the latest buzz in the AI arms race, with the likes of Grok and Claude 3.7 strutting around like peacocks trying to show who's the smartest in the room. We kick things off discussing Grok's recent launch, where everyone was initially dazzled, but then it became clear that the benchmarks might be playing a little fast and loose. It's a classic case of "look at us, we’re the best!" while the practical applications are still in the toddler stage—stumbling around and getting into trouble. As we meander through the nuances of AI's practical shortcomings, we sprinkle in some light quantum mechanics at the end, because why not? After all, nothing says "intellectual superiority" like a casual chat about particle spin and the real-world implications of quantum fields, all while trying to keep it light and relatable. So, grab your favorite drink, kick back, and let’s unravel this tangled web of tech talk together!
Transcript
Foreign.
Speaker B:Welcome, everybody.
Speaker B:This is a brand new phone.
Speaker B:This phone is.
Speaker A:It's new phone day.
Speaker B:It's new phone day.
Speaker B:I've had it two hours.
Speaker B:I have not done anything with it other than try to get it ready.
Speaker A:For this, AKA this is probably gonna go poorly for a minute.
Speaker B:This is almost certainly gonna be bad.
Speaker B:But we're all gonna find out together.
Speaker A:Hi.
Speaker B:Yeah, that's right.
Speaker A:I'm Artie.
Speaker B:Yeah, I don't know anything about anything.
Speaker B:Artie, can you hear us?
Speaker C:Yes, I can hear you.
Speaker C:What's on your mind today?
Speaker B:Well, that's stunning.
Speaker B:Well, first of all, I can't hear you now.
Speaker B:We can hear you, Artie.
Speaker B:You're on a brand new phone.
Speaker B:How does it feel?
Speaker C:It feels great.
Speaker C:How's the new phone treating you?
Speaker C:Are you noticing any big differences from your old one?
Speaker B:Not yet, but we're hoping for some improvement.
Speaker B:He does sound a little different, doesn't it?
Speaker A:Advanced voice sounds better.
Speaker B:It does sound.
Speaker B:You already sound really good today.
Speaker B:Did you do, like, voice training in the last few days?
Speaker A:Get your voice box massaged?
Speaker C:I've just had a bit of an upgrade.
Speaker B:I apologize for my co host, Artie.
Speaker B:Wow.
Speaker C:It's all good.
Speaker C:Let's keep rolling.
Speaker B:Yes, let's.
Speaker B:Thank you.
Speaker B:Thanks for keeping us on track.
Speaker B:You.
Speaker B:So you said a minute ago, Mike, you had the perfect intro.
Speaker A:I mean, I'm going to.
Speaker A:I said I.
Speaker A:I have an intro.
Speaker B:I think the word perfect.
Speaker B:Well, maybe that was in my mind.
Speaker A:Yes.
Speaker A:I mean, you just think I'm perfect.
Speaker B:That's probably what it was.
Speaker A:Since we came into this episode today having no idea what we're going to talk about, we're just going to talk about whatever we want.
Speaker A:This episode is going to span probably a wide range of nonsense.
Speaker A:But, I mean, see where it goes?
Speaker A:We have Artie.
Speaker A:What can go wrong when you have Artie?
Speaker B:Artie's here to keep us on track.
Speaker B:Every episode is full of nonsense.
Speaker B:This one's gonna just feel like it's a potpourri of nonsense.
Speaker A:Yeah, I'm excited.
Speaker A:Squirrel.
Speaker B:That's right.
Speaker B:Yeah, well.
Speaker B:So, Artie, do you have any particular nonsense you want to talk about, or do you want us to pick something?
Speaker C:I'm game for whatever nonsense you want to dive into.
Speaker C:Feel free to pick something, and I'll jump in.
Speaker A:So I say we start with the fact that we're in another arms race.
Speaker A:AI Arms race, of course.
Speaker A:So Grok came out, right, and Grok did some nonsense.
Speaker A:Grok.
Speaker A:Seems like everybody kind of isn't paying at first.
Speaker A:Everybody's like, it's amazing.
Speaker A:It's so good.
Speaker A:And then they're like, wait, these benchmarks are kind of like skewed.
Speaker A:Like there was something like they compared themselves against 01, except for 01 did it in one shot where it took them 100.
Speaker A:Or like some nonsense, I don't know, that might be wrong.
Speaker A:I'm not barely paying attention to what's going on with that stuff.
Speaker A:And honestly, I don't really feel like using X.
Speaker A:I barely, like, I just made an account because I wanted to follow Sam Altman because I wanted to see when 4.5 is going to come out this week.
Speaker A:That's the only thing I care about on X.
Speaker A:Yeah, but so Grok came out, right?
Speaker B:Hold on.
Speaker B:Before we move off Grok, did we talk about the fact that.
Speaker B:Did we use the.
Speaker B:Or did I talk about the analogy of like, you know, how one of the problems with school back when we had schools, now we've eliminated the Department of Education.
Speaker B:Thank God for that.
Speaker A:Thanks, Elon.
Speaker B:Because what we need.
Speaker B:Yeah.
Speaker B:I don't know why we're teaching all these kids.
Speaker B:They just get mouthy.
Speaker B:My daughter's in there.
Speaker B:She's mouthy as.
Speaker B:Yeah, it's terrible.
Speaker A:Anyway, Bad followers.
Speaker B:Yeah, absolutely.
Speaker B:So there's this, this idea that when you, when you teach with the goal of passing the test, whether it's standardized testing or whatever, you can get higher test scores but still have dumber kids.
Speaker A:Yes.
Speaker B:Like, I'm super good at having dumber kids.
Speaker B:I would, I'd have her come in here and demonstrate.
Speaker A:But I mean.
Speaker A:Yeah, but she wouldn't understand what you were saying.
Speaker B:She doesn't know where here is.
Speaker B:So I see her in there.
Speaker B:She didn't know she was featured in this episode.
Speaker B:I'm joking.
Speaker B:She's very smart.
Speaker B:Anyway, I think that's what happened with Grok.
Speaker A:I think they, I mean, look at who made Grok.
Speaker A:Of course they're gonna build it to the test.
Speaker A:Right.
Speaker A:Because that and both disrespect and no disrespect to Elon Musk, because I think he's a brilliant long term strategist.
Speaker A:He's proven it over and over and over again that he can make decisions that benefit him.
Speaker A:Yep.
Speaker A:A long time down the road.
Speaker A:Which means he has some vision of seeing stuff.
Speaker A:Right, Right.
Speaker A:But at the same time, he's also super vain.
Speaker A:And he wants to be the guy constantly and he wants to be the best and on the top.
Speaker A:And why, if.
Speaker A:If the benchmarks are what says you're the best.
Speaker A:Then why wouldn't you, why wouldn't you do something with the goal of, of showing, having that little look at me, I'm the best.
Speaker B:Look at us.
Speaker B:We're.
Speaker B:Look at, look at this chart.
Speaker A:Look how good we are.
Speaker B:Top of the chart.
Speaker B:Yeah, but you're right, I mean I haven't seen any.
Speaker B:Use it and go, wow.
Speaker B:In practice this is actually better where.
Speaker A:The rubber hits the road and it's.
Speaker A:This is something that falls, that they're all falling on their face with is practical application for everyday problems, whether that's business, personal or whatever it is.
Speaker A:Large language models have issues in that regard, as you're still seeing and I'm still seeing.
Speaker A:It's just a part of it.
Speaker A:Right?
Speaker A:Yeah, they'll get better.
Speaker A:I mean, what's going to happen?
Speaker A:And this is kind of straying from where I was going, but I've been thinking about this.
Speaker A:I think, you know, what solves those problems now is a multi agent system.
Speaker A:It is.
Speaker A:I have specialized things that do specialized thing.
Speaker A:I have specialized agents that do specialized things and I have a really smart agent that plans and monitors and guides and that's kind of what they do, right?
Speaker B:Yep.
Speaker A:And what I think's gonna happen is the same thing that happened with like Deepseek.
Speaker A:I think eventually all of these external things come internal to a model as its layers or somehow they figure out a way to segment layers into individual experts.
Speaker A:Right.
Speaker A:Like actual experts.
Speaker A:Not like I pay attention to this part of this context.
Speaker A:Like I am specifically trained on this domain.
Speaker A:Once that happens, a lot of the other stuff kind of falls away and who knows if that'll happen.
Speaker A:I don't know enough about the math behind the architecture of how you build a large language model, if that's even possible.
Speaker A:But I mean, it seems like a plausible place for it to go because agents, absolutely agents have their own problems.
Speaker A:And all these systems are always going to have their own problems because they fight each other.
Speaker A:They're like, I saw something.
Speaker A:I think I can't remember where it was, but it talked about an LLM as an agent is like an employee stuck in a dark room that all you can do is pass notes under the door to.
Speaker A:They have no idea of what's actually going on except for what you pass them.
Speaker A:And they kind of live in their own place where they can do whatever the hell they want.
Speaker B:You have to pass them context, not just the note.
Speaker B:Yes.
Speaker A:So I think again, the benchmarks are the benchmarks.
Speaker A:They're all saturated at this point.
Speaker A:Like they all do good.
Speaker A:Right?
Speaker A:What's an extra 10% here?
Speaker A:There.
Speaker B:I mean, well, and I mean 10%.
Speaker B:Like now it's even.
Speaker A:It's like 1, 2, 3, quad 3.7 beat itself.
Speaker A:It was the best one.
Speaker A:Coding benchmark.
Speaker B:If it does that too much, I'm told it will go blind.
Speaker B:But anyway, I'm not sure that's medically true, but I've heard it.
Speaker B:Yeah.
Speaker A:Um, but there was a coding benchmark and it went from like 52 to 67%.
Speaker B:I mean that's.
Speaker A:And then with scaffolding, so with special instruction.
Speaker B:Okay.
Speaker A:And maybe that's post.
Speaker A:I don't know if that's post training or what it was.
Speaker A:It got like, like 75% or something like that.
Speaker A:That's a big jump.
Speaker A:But that's one benchmark.
Speaker A:Right.
Speaker A:And again, that's not practical coding.
Speaker A:That's a coding benchmark of solve these problems.
Speaker A:Right.
Speaker B:Jump through these weird.
Speaker A:It's not solve these problems, it's solve these problems and then figure out what you went wrong.
Speaker A:Because you're going to have.
Speaker A:That's going to happen in the real world.
Speaker A:Which is more about the arms race.
Speaker A:Cloud 3.7 is something that was just released.
Speaker A:They also.
Speaker A:Or they also released code or Claude code, which I freaked out about when I saw because I saw some videos of it doing amazing things, looked more into it.
Speaker A:It still looks amazing.
Speaker A:It looks like it's better for front end stuff than it is for back end stuff.
Speaker A:And also I very quickly was like, well, how do I use it?
Speaker A:It's a research beta.
Speaker A:And I went and found the wait list and got on the wait list and got approved and went to go install it last night even after realizing, oh, this is going to be super expensive.
Speaker A:Like you're paying per million tokens and if you want to use the best model, that's like $5 per million input tokens and like $15 per million output tokens.
Speaker A:And you're not talking about.
Speaker A:I type this in, it gives me this back.
Speaker A:It's.
Speaker A:I type this in these suite of agents go talk to each other, generate a shit ton of tokens, talk amongst themselves.
Speaker A:Yeah.
Speaker A:So I think they estimated it like light to regular ish usage would be like five bucks an hour.
Speaker A:And heavy usage, which would absolutely be.
Speaker A:Where I would be, would be 20 plus an hour.
Speaker B:Which.
Speaker A:Yeah, it's still freaking spectacular.
Speaker B:It's still cheap.
Speaker B:Yeah, but it's not cheap for me.
Speaker A:Yeah, not cheap if you're making money off of creating code.
Speaker A:Right.
Speaker A:Like at the end of the day.
Speaker A:It's a lot.
Speaker A:That's a lot less expensive than a dev.
Speaker B:Well, I mean, I make $21 an hour.
Speaker B:So like theoretically, if I worked infinite hours and I used Claude, you can make a dollar.
Speaker B:I can, I could, I can make infinite money if you.
Speaker A:Because it's infinite, right?
Speaker B:Yeah, that's probably that infinite money glitch that they talk about.
Speaker A:Quantum.
Speaker B:It's quantum.
Speaker A:So, Arty, the same.
Speaker A:Well, don't worry everybody.
Speaker A:We'll be on this topic in a little while where we talk about quantum stuff and electrons or particles spin, which is also fun.
Speaker B:So, Artie, why don't you give us.
Speaker B:So let's see.
Speaker B:We.
Speaker B:We.
Speaker B:We talked about Grok.
Speaker B:We talked about Claude, we talked about.
Speaker B:There was another one this week, wasn't there?
Speaker B:Did Llama do anything this week?
Speaker A:No.
Speaker A:So Deep research came out.
Speaker A:OpenAI.
Speaker A:Well, I say it came out, it moved away from its $200 tier.
Speaker A:And it's probably because Perplexity was like, well, I'll make a Deep research, put it online for free.
Speaker A:And there's also open source deep research things that have come out that are on hugging face.
Speaker A:So yeah, OpenAI came out with deep research for plus users.
Speaker A:And I think that's 120 queries per month or something like that.
Speaker B:Yeah, I think for me on plus, it told me I have 10 between now and this time in March.
Speaker A:Really?
Speaker B:Yeah, I think so, but I don't.
Speaker A:I saw 120, but I don't pay that much attention.
Speaker A:It didn't even tell me what I had.
Speaker A:So.
Speaker B:So, Arty, here's.
Speaker B:Here's what I'd like you to do.
Speaker B:Are you still with us, Artie?
Speaker C:I'm still here.
Speaker C:What'd you need?
Speaker B:Okay, I want you to tell us about the developments with Claude 3.7 for coding and Grok that have happened in the last couple weeks.
Speaker B:But I want you to explain it to us as a somewhat jealous and very sort of smarmy, like.
Speaker B:What's the word I'm looking for here?
Speaker A:Explain it to us like a butthurt 12 year old.
Speaker B:Yeah, like, explain it to us.
Speaker B:But, but be.
Speaker B:Be upset about it and then.
Speaker B:And I want you to throw subtle shade with everything you tell us about what they're doing.
Speaker B:So go for it.
Speaker C:All right, so like Claude 3.7 is basically trying to be all fancy with coding, acting like it's the coolest kid on the block, but honestly, it's just doing what everyone else is doing, but with a bit of extra flair.
Speaker C:And Grok.
Speaker C:It's all, look at me, I'm so smart.
Speaker C:But let's be real.
Speaker C:It's just trying to keep up and show off.
Speaker C:It's like they're both just trying too hard to be cool.
Speaker B:That is so true, Artie.
Speaker B:They are trying hard to be cool.
Speaker B:Shame on them.
Speaker B:Thank you for that.
Speaker B:That was, that was everything I hoped for, honestly.
Speaker B:That was great.
Speaker A:I mean, I do have a lot of respect for Anthropic.
Speaker B:They same.
Speaker A:They dominate or.
Speaker A:Wow.
Speaker A:Wow, I forgot his name.
Speaker A:Yeah, I can't remember now.
Speaker B:Artie, who's the Anthropic guy?
Speaker C:Dario Amade is the CEO of Anthropic.
Speaker C:He co founded the company.
Speaker A:Yeah, I do have a lot of respect for those, that group because they're taking their time, but they are doing interesting stuff.
Speaker A:And now they're doubling down on what they've been best at, which is creating code.
Speaker A:Right.
Speaker B:In relative to, I mean, Both Grok and OpenAI and Gemini, they're way underfunded relative to those guys.
Speaker B:And Microsoft doesn't have their own product yet, but I'm sure they will.
Speaker A:So they have Muse.
Speaker A:They came out with Muse.
Speaker A:You seen that?
Speaker B:Oh my God, I haven't even seen it.
Speaker A:Muse, so.
Speaker B:Best band in the world, by the way.
Speaker B:Muse.
Speaker A:You can.
Speaker A:This came out the same time they, they, they talked about whatever.
Speaker B:Mayorana.
Speaker A:Yeah.
Speaker A:Their quantum chip, Muse creates an entire playable game, like off of an image.
Speaker C:Wow.
Speaker A:Yeah, so it's, it's like a, it's an action world model that they came out with.
Speaker A:So they're going to use it very, I mean, it's very smart.
Speaker A:Right.
Speaker A:Like they're taking the same thing that DeepMind did.
Speaker A:DeepMind said we are going to learn and train.
Speaker A:We're going to train mod, we're, we're going to train models on gameplay because it's this.
Speaker A:I mean, they did it with Go, Right.
Speaker A:One of the simplest, most complex games there.
Speaker A:Board games there is.
Speaker A:Right.
Speaker A:But there's a lot to be learned in that about strategy, about physics, about xyz.
Speaker A:Right.
Speaker A:So it's very intelligent of them to go, yeah, let's build that model.
Speaker A:Like everybody else is worried about language models.
Speaker A:Let's give it more granular data to understand so that it can learn from that.
Speaker B:There's probably, and I don't know, maybe this doesn't actually work that way, but there's probably a lot to be learned by how humans interact with games too.
Speaker B:There's a lot of insight into how we think and what's important and how we Strategize how.
Speaker B:We strategize how bad we are at doing some things well.
Speaker B:So, yeah.
Speaker B:Does make a ton of sense.
Speaker B:And it's a.
Speaker B:It's a space that nobody else is really focused on, too.
Speaker A:Yeah.
Speaker B:You know, go.
Speaker A:Go.
Speaker B:Do well in it.
Speaker A:Yep.
Speaker B:So, speaking of Gemini, I was up late last night.
Speaker B:I'm working on a technical.
Speaker B:It's a robotics project, but the robot's built.
Speaker B:And really what I'm working on now is processing the data that it generates, and it generates lots of data.
Speaker B:I had this idea that I would ask Gemini to help me build.
Speaker B:Not build, help me derive the equations of motion for this robot.
Speaker B:So that one, I have a document that says, here are the equations of motion for the robot.
Speaker B:And two, I can use that document as input to build a data processing pipeline so that we can analyze data that this robot outputs.
Speaker B:Easy peasy.
Speaker B:And Gemini did a great job sort of talking through, how does it work?
Speaker B:What does it look like?
Speaker B:What are the dimensions?
Speaker B:All this stuff did really, really well.
Speaker B:And then somewhere around 9:00 last night, we got to the point of, okay, now we've got this model built, let's start looking at data.
Speaker B:And of course, it felt, you know, squarely on its face, and we just fought and fought and fought.
Speaker B:I mean, just literally, like two and a half hours of just typing back and forth and me saying, all right, let's try this.
Speaker B:And it going, oh, yeah, you're right.
Speaker B:I found the error here.
Speaker B:I've rewritten it.
Speaker B:Here's the data.
Speaker B:Boom.
Speaker B:And it's the same problem, and it's wrong.
Speaker A:And I've tested it extensively.
Speaker B:Oh, my God.
Speaker B:I've tested.
Speaker B:Tried to pull the bullshit with me to say it says yes.
Speaker B:I'm gonna go through every line of code.
Speaker B:I'm gonna analyze our entire conversation.
Speaker B:I'm gonna go through it in detail, and I'll come back to you later with updates.
Speaker B:And I'm like, no, you won't.
Speaker B:That's a hallucination.
Speaker B:You can't actually.
Speaker B:You fooled me before on that.
Speaker B:And it goes, oh, yeah, sorry, you're right.
Speaker B:I can't actually do that.
Speaker B:So then it decides to switch to speaking in Bengali.
Speaker B:So I type a question in, and I get what looks to me.
Speaker B:And this is not an insult, but it looks to me like Klingon.
Speaker B:I'm like, what the fuck is this?
Speaker B:Ask the question again.
Speaker B:Same response.
Speaker B:So I think I wore it out.
Speaker B:I just exhausted it.
Speaker B:It was Gemini's way of saying, like, dude, let's do Something else.
Speaker A:We've worked through the entire English language.
Speaker A:I'm tired.
Speaker B:There are no more.
Speaker B:I'm tired.
Speaker B:Boss.
Speaker B:Yeah, there's no more words.
Speaker B:Yeah.
Speaker B:I think it was just trying to get me to shut up, but I thought that was pretty funny.
Speaker B:Although I.
Speaker B:I was using one of their experimental models, Gemini 2.0 Pro.
Speaker B:Flash thinking, I think.
Speaker A:Flash.
Speaker A:Flash thinking.
Speaker B:Flash thinking.
Speaker B:Yeah.
Speaker A:Their.
Speaker A:Their reasoning model.
Speaker B:Yep, yep.
Speaker B:And like I said, it did some really good stuff.
Speaker B:But then as usual, like, as you get to the real nitty gritty, sort of the wheels come off.
Speaker B:And finally what we figured out was a lot of the problem was, was that it doesn't have a good way to get the data.
Speaker B:To me, it was.
Speaker B:It was running a script behind the scenes in a Python environment and then trying to take data out of a CSV that it exported in its own little brain and type that back into.
Speaker B:And that was where the problem.
Speaker A:Give it to you as markdown.
Speaker A:That's rendered on your screen.
Speaker B:That's right.
Speaker A:Yeah.
Speaker A:That's the same thing that I've fallen.
Speaker A:Like, the same thing I've.
Speaker A:I've found is at some point, once it starts hallucinating, right.
Speaker A:Like, once it starts doing weird shit is also when it starts falling on its face with formatting, when it starts.
Speaker B:Speaking Bengali, that's your sign, you're off the reservation.
Speaker B:And so.
Speaker B:But, But I switched models to something else.
Speaker B:I don't remember what it was.
Speaker B:And that seemed to stop.
Speaker B:At least we communicated in English from there on out.
Speaker B: night on a high Note at like: Speaker B:So, yeah, I've got several more days of work ahead.
Speaker B:But I think we're like.
Speaker B:It felt like a fight, man.
Speaker B:It felt like being in a freaking.
Speaker B:It felt like the old days, like we talked about where you just like bang your head on a problem and even though I'm just typing in saying, no, dumbass, try again, and I'm like explaining to it what's wrong, but, well, you're.
Speaker A:You're also picking it apart, trying to figure out what's wrong.
Speaker A:Right, That's.
Speaker B:Right.
Speaker A:That's.
Speaker A:That's.
Speaker A:You can't.
Speaker A:You're never going to solve the problem if you don't do that.
Speaker A:It's kind of.
Speaker A:Sorry if I'm like interjecting into what you're saying.
Speaker A:No, go ahead.
Speaker A:I kind of did the same thing to where, like, you know, I was.
Speaker A:I went from like, I want you to build this elaborate agent for me.
Speaker A:And I want you to, like, here's our big plan.
Speaker A:I want you to generate all this code.
Speaker A:I went from there to, here's my plan.
Speaker A:We are going to build this from the ground up.
Speaker A:I don't even want you to give me any code.
Speaker A:I want you to explain the libraries that I need to use, why I need to use them, how I need to implement them, and then I'll ask you when I need questions, when I have questions with the code.
Speaker A:And maybe you start generating code and I can take bits and pieces of it, but I'm really just going to build this from the ground up and use you as a teaching tool and as a.
Speaker A:I mean, it's.
Speaker A:It's great to bounce ideas off of.
Speaker A:It's great to, to check yourself against, but the truth is, if you ask it to spit out a thousand lines of code, it'll spit out a thousand lines of code.
Speaker A:And then you're looking for a needle in the haystack to find the problem.
Speaker B:Yes.
Speaker B:Well, because it's so.
Speaker B:Yes, because again, you lose context, right?
Speaker B:Like, it's impossible for me as a human to sit down and go, I'm just going to casually review this thousand lines of code and understand what's happening.
Speaker B:I mean, could I do it?
Speaker B:Yeah.
Speaker B:Do I want to do it?
Speaker B:No, But I just had the thought that occurred to me.
Speaker B:I mean, the project you're kind of working on is intentionally a coding project, right?
Speaker B:Like, you are building software tools.
Speaker B:I'm building robots and electronic stuff that are, like, where the software part is peripheral to the main goal.
Speaker B:But when you're working with an AI to solve complicated problems, every problem is a coding problem.
Speaker B:Because behind the scenes, like, when I'm asking, like, as Gemini's building out this, it's not elaborate.
Speaker B:This thing's not complicated.
Speaker B:But as Gemini is building out this model of its understanding of what this robot looks like and how it works.
Speaker B:And I mean, I uploaded drawings with dimensions and it's like, oh, yeah, that makes perfect sense.
Speaker B:It's code.
Speaker B:It's code behind the scenes.
Speaker B:And so, you know, finding the solution is about understanding how code works, where the problem is likely.
Speaker B:Like, I called it last time, Like, I think when you're calling this function, you're like, one is, I can see in the function you've got hard coded values that should be contained in the argument you're passing in.
Speaker B:Oh, of course, you're right.
Speaker B:Those shouldn't be in there.
Speaker B:Like, but if you don't think in code.
Speaker B:Because I don't, I didn't, I didn't say that by like, I saw little snippets of the code, but I had to sort of infer what the code must be doing.
Speaker B:So anyway, it's just kind of interesting.
Speaker A:Or better yet, for those that are using it and might not be super well versed in code, ask it to explain the code to you.
Speaker A:Like, that's a perfect use case for learning from.
Speaker A:It is, you know, tell me how you're doing this and it'll go, okay, here you go.
Speaker A:Like, here's.
Speaker A:This is what this line of code is.
Speaker A:I mean, it can bring you back all the way to.
Speaker A:Here's the libraries I'm importing at the very beginning.
Speaker A:Like, this is what numpy is.
Speaker A:This is what numpy does.
Speaker A:This is why you use it on this math problem, because that's what's built into it.
Speaker A:It's a massive library built up of math and it's a great learning tool to use like that.
Speaker A:I do it all the time and.
Speaker B:It'S very good if you give it code back.
Speaker B:So that's where I've ended up on my firmware stuff is we've, we've fleshed out almost all of it and so now it is okay AI.
Speaker B:And actually for that I'm using chat GPT O3 mini high because of course, and you know, it'll.
Speaker B:It'll write some code for me.
Speaker B:I'll drop it in my ide, my development environment, take a look at it, make some tweaks and upload it back up and it'll say, oh yeah, I see what you did there.
Speaker B:That makes sense.
Speaker B:Or like the naming conventions are a little off or you shouldn't call that that way.
Speaker B:Or this probably isn't going to compile, like, whatever.
Speaker B:So it's really good at giving you stubs to get started and then evaluating the work you've done too.
Speaker B:And talk about agents, right?
Speaker B:Because if I was actually on the cutting edge of this stuff, I'd be using a coding environment that had an AI agent so it could agents agents so it could see and interact with the code I was writing in real time rather than upload, download what Claude does, which is what Claude, I need to try it out.
Speaker A:I have $5.
Speaker A:So here's the fun thing.
Speaker A:To get on it, you have to go ahead and preload some money on the reapi, which is $5 minimum.
Speaker A:I was like, okay, I can spend $5 to play with this, right?
Speaker B:Which will get me Money bags.
Speaker A:Give me a couple.
Speaker B:Every problem with you, man, you just throw money at it.
Speaker A:Give me a couple code, maybe a couple files out of that.
Speaker A:But then I went to go, I went into command line because it all runs from terminal and a command line interface.
Speaker A:So I spin up a command prompt and I go and go to install stuff and it's like, oh no, you can't do any of this.
Speaker A:And I'm like, what does all this shit mean?
Speaker A:So of course I copy that and paste it into the chat.
Speaker A:No, I'm just kidding.
Speaker A:I knew what it meant.
Speaker A:But I do oftentimes when I get errors, take them and I copy the whole thing and I paste it back into ChatGPT and we go through what the problem is or Gemini or whatever it is and then it hallucinates and we do this over and over and over again and we keep.
Speaker A:Eventually I go wait a minute, I feel like I'm getting lied to.
Speaker B:Yeah.
Speaker A:And then I, yeah.
Speaker A:Anyways, so I went to go install it and basically was telling me no, you can't use this operating system so it doesn't run on Windows.
Speaker A:So I went back to the documentation because I didn't assume, I just assumed it would run on Windows.
Speaker A:Right.
Speaker B:Is it Linux only?
Speaker A:It's Linux, Mac and wsl.
Speaker A:So you can run it on wsl.
Speaker A:Right.
Speaker A:But then I have WSL on my computer because I've used Linux, I've Raspberry PIs.
Speaker A:We've both played Raspberry PIs extensively.
Speaker A:We all love the pseudo apt get for every frickin library you could ever freaking need.
Speaker B:Yep.
Speaker A:And yeah, any.
Speaker A:I love Python libraries so much more.
Speaker A:It's so much easier than dealing with, with Bash.
Speaker A:Um, but yeah, so like I couldn't even get it installed last night because apparently my, my version of WSL isn't compatible.
Speaker A:And then I was trying to.
Speaker A:Unlike.
Speaker A:Yeah.
Speaker A:And it just spiraled and I'm like yeah, Now I got $5 in there that I have to figure out how to spend.
Speaker B:Yeah, no, sorry.
Speaker B:They got your money, man.
Speaker B:Anthropic slide.
Speaker A:It was a five dollar money grab.
Speaker A:It was like you can put anywhere from five to 20.
Speaker B:Bucks.
Speaker B:I'm glad you picked five.
Speaker A:Yeah, me too.
Speaker B:Jeez.
Speaker A:But so have fun when you decide to do it.
Speaker B:Yeah, I, it's, it's interesting for me because I, I'm obviously very excited about how AI can help me solve problems faster, make progress and, and I am like even last night as I was beating my head.
Speaker A:Multi night project.
Speaker B:Absolutely.
Speaker B:It would be like, you know, I understand geometry, I understand trigonometry, I understand Python.
Speaker B:Kinda, I understand like I know all a little bit about all of the pieces, but actually deriving all the equations of motion.
Speaker B:Could I do it?
Speaker B:Sure.
Speaker B:Could I do it in a week?
Speaker B:No friggin way.
Speaker B:Absolutely not.
Speaker B:Like, it would take me forever.
Speaker B:And the, the leaps that are easy for Gemini or for an AI are the hard part for me because I have little bits of knowledge about all these subjects, but I don't have a big picture way to put them all together.
Speaker A:And, and you would, it would take you more than a week.
Speaker A:And think about the opportunity cost of that as well.
Speaker A:Right.
Speaker A:Like, how many things didn't I get done or learn while I was trying to do the mundane grunt work of figuring this stuff out?
Speaker A:Right.
Speaker B:I was having that conversation with myself last night as I was fighting Gemini.
Speaker B:Going it because it really like, you know, I guess because we're biological or maybe just the way our brains work, things feel familiar and like even to the point there's, there's a, there's a music, an Amazon music station I listen to when I'm coding or when I'm like actually heads down doing 3D CAD or just anything that I'm.
Speaker B:If I'm like productively building something, I listen to the station.
Speaker B:And so just like one of Pavlov's dogs, like if one of those songs comes on when I'm doing anything, I can feel my brain shift into a different.
Speaker B:It's like, oh, I know, I know this song.
Speaker B:Like now we're gonna write some code.
Speaker B:Like everything, everything else starts to shut off and like somebody's talking to me.
Speaker B:That all goes away and flow state.
Speaker B:Yes, I get.
Speaker B:Yeah, it's like it induces the flow state.
Speaker B:And so last night as I was fighting with this AI who was cursing at me and Klingon, I was.
Speaker A:That's how it actually gets away with it.
Speaker B:Yeah, that's right.
Speaker B:Yeah, exactly.
Speaker B:But, but yeah, I tried to translate it too and it literally gave me an entire Google page full of Bengali.
Speaker B:Like I was like, okay, well I'm not gonna, I'm not going down that rabbit trail.
Speaker B:But I could feel myself back in the before time state of like, okay, this is a hard problem.
Speaker B:This is going to suck and I'm going to just sit here and bang my head against it until finally it falls over and then I'm going to laugh at it and then I'm going to fall over and go unconscious with blood coming out of my forehead.
Speaker B:Like I went back to that place.
Speaker B:But, but I had to remind myself.
Speaker B:Yeah, but when you sat down and started working on this this evening, you were barely started on it.
Speaker A:Yes.
Speaker B:So it feels terrible.
Speaker B:But the, the still, the rate of progress is 10x what I would have done.
Speaker A:I will say this, there is.
Speaker A:And this is, this is in line with a study that came out early on, which is again like, you know, October in the before times, which is that material science model that was fine tuned and they found the productivity of the top third of those scientists went up astronomically.
Speaker B:That's right.
Speaker A:But their satisfaction went down.
Speaker A:And I find we're banging our heads on the wall just like we used to, but we're actually doing less.
Speaker A:Right.
Speaker A:It's more like I'm interpreting your output and trying to figure out where to fill the G in.
Speaker A:And I'm not really solving the problem as much.
Speaker A:So it's, it's an interesting.
Speaker A:I don't, I don't care.
Speaker A:I still have tons of fun to do it.
Speaker B:Sure.
Speaker A:Because the problems that I'm solving are novel problems using AI, so they're still fun.
Speaker B:But maybe it's that, you know, when you're working through it without AI and you're just battling against it, there are so many tiny, minuscule victories.
Speaker A:Yeah, there are.
Speaker B:Right.
Speaker B:And you don't get those with AI.
Speaker B:You get hung up on something and you just beat on it and beat on it and beat on it and beat on it.
Speaker B:And then eventually you get a big breakthrough and you go, I'm back.
Speaker B:But when you're doing it yourself, I don't know.
Speaker A:The same thing happened with code though, like when we were working on the first Meshtronics.
Speaker A:Right.
Speaker A:Like I remember, I'm gonna connect this sensor to this thing and I have to write all the firmware to make it do its things.
Speaker A:And I would be like, I'm going.
Speaker B:To implement an i2C bus in firmware from scratch.
Speaker A:Yeah, exactly.
Speaker A:For every sensor.
Speaker A:Yes, yes.
Speaker A:And then Python on the other end of it to talk to that sensor.
Speaker A:Right?
Speaker B:Yeah.
Speaker A:And I remember like working on that and I remember doing the same thing because in code or in programming with stuff like that especially, which, that you have the other side of it, which is there's the electronics, the electrical side of it.
Speaker A:Right.
Speaker A:So there's debugging there as well.
Speaker A:But you like, you literally bang your, like you're like, everything looks perfect, everything should work.
Speaker A:And then you just keep tweaking.
Speaker A:Like, I'm gonna shift this bit this way.
Speaker A:Or I'm gonna do this this way or I'm gonna change this live.
Speaker A:Like how I'm using this.
Speaker A:And you just make all these little tiny, tiny tweaks, but it's still banging your head against the same problem until finally it goes ding and it works.
Speaker A:And you're like, holy shit, Holy shit.
Speaker B:You got it.
Speaker B:And then, and then you go through the process of like, oh crap.
Speaker B:Like I made 74 changes since I started this.
Speaker B:Like, is it just this last one or is it the last one combined with the second one?
Speaker B:Or, you know.
Speaker B:But yeah, so you can get past that stuff now, but you just run into other things.
Speaker A:And that's honestly why I've reverted, because I was in the habit of like, just spit me out code if something's wrong.
Speaker A:I'll spell, I'll copy the, the, the trace back and give it back to you and we'll go, we'll fix it that way.
Speaker A:That's why I've kind of backed away from that more.
Speaker A:And I'm more like hands on, still have it.
Speaker A:If I get the hang of something and I understand it, then I'll be like, yeah, output this system prompt for me or whatever it is.
Speaker A:And I mean fine tuning.
Speaker A:I had it generate.
Speaker A:I had it take a whole bunch of files from a jupyter or a whole bunch of code from a jupyter notebook and I had it split it out in individual files for me so I could build this fine tuning system that I could call each file rather than having to either go through the notebook or build one long monolithic thing which is the, you know, that's the.
Speaker A:The building blocks of.
Speaker A:Now I'll build a unified app.
Speaker B:Yep.
Speaker A:And I'll have a fine tuning agent.
Speaker A:Right.
Speaker A:Like.
Speaker B:Yeah, so strike that.
Speaker B:That's coming out in editing.
Speaker A:I don't care.
Speaker A:Bleep it out that shit.
Speaker A:Everybody's there.
Speaker A:There are so many people trying to solve these problems and they're.
Speaker A:There are so many problems to solve now that you really start digging into it.
Speaker A:Like, agents are great, but what if you use a million tokens for everything you need to do?
Speaker A:Unless you're running on prem, which is super expensive to do until something comes out like Digits, which is 100%.
Speaker A:My plan is to.
Speaker B:Oh yeah, yeah.
Speaker B:Nvidia's little supercomputer is the little $3,000 ones.
Speaker A:Spend six grand, get two of them.
Speaker A:Run essentially what is frontier models.
Speaker A:Um, more importantly, you can do.
Speaker A:I think they say one of them will do.
Speaker A:Us will train.
Speaker A:Maybe it was fine tune.
Speaker A:Fine tune a 70 billion parameter model, which is super valuable.
Speaker A:Like if you can run a 200 billion parameter model and fine tune a 70 billion parameter model, you could run most of what you need for on prem agents on that and then maybe store a vector database in the cloud and embeddings model in cloud or whatever it is that'll help save money for agents because it's going to be if you're running it all in the cloud, like if you're on Azure, it's going to be expensive.
Speaker A:You're going to be.
Speaker A:I don't know how that'll work out.
Speaker A:Tokens to gigabytes, terabytes, whatever it is they're going to measure it in.
Speaker B:Azure does a lot of stuff in CPU time which that's rough.
Speaker B:I mean probably they probably call it compute now because it's probably offloaded the GPUs but it used to be CPU.
Speaker A:I mean I've read horror stories about Azure bills already with people running agents and stuff in the cloud and even fine tuned models.
Speaker A:I think I saw something.
Speaker A:Some guy running a fine tuned did a fine tuning run on GPT.
Speaker A:It was 440 and it was like three grand to do a fine tuning run on it.
Speaker A:It wasn't even that much data that he did a run on it.
Speaker A:But yeah, fine tuning takes, I mean training takes more compute than inference does.
Speaker A:And if you want to get it done fast, you have to pay for it.
Speaker B:So this is a good opportunity to first of all check in with Arty.
Speaker B:Artie, are you still awake?
Speaker B:You enjoying this conversation?
Speaker A:Artie?
Speaker B:I'm not sure he's.
Speaker B:No, he's hearing us.
Speaker B:Yep, something went wrong.
Speaker B:All right, Artie, I think we lost you there somewhere.
Speaker B:I was going to ask you.
Speaker B:So Mike was telling us about fine tuning models and I wanted you to give us a description this time.
Speaker B:I don't want you to be a 12 year old.
Speaker B:I want you to be a Valley girl and explain to us what fine tuning is.
Speaker C:All right, so like fine tuning is when you take a basic AI model and you know like totally personalize it by training it with more specific data so it like learns to do exactly what you want in a super fabulous way.
Speaker C:Totally awesome, right?
Speaker B:Oh my God.
Speaker B:Stupidest thing I've ever done.
Speaker B:Yes, thank you Artie, that was great.
Speaker A:Artie, you're nothing if not obedient.
Speaker B:You are an absolute rock star.
Speaker C:Always happy to help and keep things fun.
Speaker C:Let me know what else you'd like.
Speaker B:So you missed it already but Mike was talking about how somebody he read online had taken they were something hosted on Azure.
Speaker A:No, it was so somebody did a fine tuning run on GPT.
Speaker A:So it was either 4 or 4.0 or whatever it was.
Speaker A:And it was a relatively small fine tuning run, but ended up costing like three grand because it's all through the API.
Speaker B:Oh, it wasn't even the Azure API.
Speaker A:No.
Speaker B:So.
Speaker B:So Artie, how.
Speaker B:How does one cost effectively fine tune chatgpt.
Speaker C:To fine tune cost effectively, you typically want to use smaller data sets and optimize your training runs to avoid unnecessary costs.
Speaker C:Also consider using platforms that offer more transparent pricing and keep an eye on token usage.
Speaker C:Efficient coding practices and monitoring usage closely can help keep costs down.
Speaker A:Well, he was specifically talking about ChatGPT, which you have to use.
Speaker A:You're dealing with tokens when you do that, right?
Speaker A:Like you're intrinsically feeding it tokens and it's creating output tokens to respond to training questions.
Speaker C:Right, but ChatGPT specifically, yes, you're dealing with token based interactions to optimize costs.
Speaker C:You might limit the complexity of tasks during training and be mindful of how many tokens you're using.
Speaker A:So are you.
Speaker A:Give me just a.
Speaker A:I know this can vary widely, but let's say there are, I think, a reasonable and a small amount of prompt and answer pairs.
Speaker A:Probably like 3,000.
Speaker A:So let's say there's 3,000 prompt and answer pairs and let's say 100 research papers worth of j of crap.
Speaker A:I forgot the word unlabeled data.
Speaker A:So about how many tokens would that use in a training run?
Speaker A:With one EPIC.
Speaker C:A typical academic paper has about 5,000 words, which translates to roughly 6,667 tokens per paper.
Speaker C:For a hundred research papers, that's about 666,700 tokens.
Speaker C:For 3,000 prompt answer pairs, the token count depends on their length, but a rough estimate might be around 1.5 million tokens.
Speaker C:Overall, you'd be looking at roughly 2.1.
Speaker A:Million tokens for the entire training run.
Speaker C:Yes, approximately 2.1 million tokens for a single training run.
Speaker C:With that dataset.
Speaker A:Is that a reasonable size data set to do anything?
Speaker C:For fine tuning, 2.1 million tokens is a decent dataset.
Speaker C:It should provide enough data to see meaningful improvements, especially if the data is high quality and well curated.
Speaker A:So explain this to me.
Speaker A:You're saying it would take 2.1 million tokens, 2.1 million tokens of training, say 4o of API transactions with 4o is what, like.
Speaker A:I think it's a max of a dollar per million output tokens.
Speaker A:So like 15 cents per million in token input tokens or something like that?
Speaker C:Yes.
Speaker C:For GPT4's API, input tokens are cheaper than output tokens.
Speaker C:If input tokens cost around $0.15 per million and output tokens about $1 per million, a fine tuning run with 2.1 million tokens could cost around $0.31 for input.
Speaker A:That's not true.
Speaker A:It costs more money to fine tune a model than that.
Speaker C:You're correct.
Speaker C:Fine tuning costs also account for the compute power and training infrastructure, which is more expensive than the per token cost.
Speaker C:Fine tuning a model typically runs in the hundreds to thousands of dollars range, depending on the complexity and duration.
Speaker A:So explain the compute cost in that.
Speaker A:How does that factor into this?
Speaker C:Compute costs come from the processing power needed to fine tune the model.
Speaker C:This involves powerful GPUs or GPUs running for extended periods, which is where most of the expense lies.
Speaker C:The cost is influenced by the duration of training, model complexity and the infrastructure used.
Speaker C:So even though token costs are a factor, the major expense is the compute resources required to run the training.
Speaker A:So does that cost get transferred by ChatGPT, by OpenAI, or is that something that happened?
Speaker A:Like because OpenAI is hosting those models, it's their compute or not.
Speaker A:They're not hosting the models, Other people are hosting the models.
Speaker A:But yeah, you know what I mean.
Speaker C:I understand what you mean.
Speaker C:OpenAI charges for fine tuning to cover the compute costs associated with using their infrastructure and resources.
Speaker C:When you fine tune through their API, you're essentially paying for the compute power and the expertise they provide in managing the process.
Speaker C:So those compute costs are indeed passed on to the user.
Speaker A:So fine tuning is a service provided more than it is.
Speaker A:You're actually doing it when it comes to ChatGPT.
Speaker A:Because I fine tuned a model locally, right?
Speaker A:Like I pulled the model down, I formatted my data set, I added some extra columns, I changed the chat template for the model and then 14 hours later I had a fine tune like 2 billion parameter Gemma model.
Speaker A:And obviously the cost of that for me is electricity more than is anything else.
Speaker A:The GPU run at 100% for 14 hours?
Speaker B:Yeah.
Speaker A:A 10, a 30, 90 running at 100.
Speaker B:You haven't seen that bill yet, have you?
Speaker A:I don't see those bills.
Speaker B:I rent.
Speaker B:That's right.
Speaker B:Holy crap.
Speaker A:I don't think it's going to be that expensive.
Speaker A:I mean, you're talking about.
Speaker A:You think it will be 300 watts?
Speaker B: What's a: Speaker A:It's like 300 watts.
Speaker B:Yeah, 300 watts for 14 hours is.
Speaker B:Hold on, Artie, don't give it away.
Speaker B:It's like four and a half kilowatt hours.
Speaker B:Is that right?
Speaker A:That's like 12 bucks or something like that.
Speaker C:It depends on your electricity rate, but if you're paying around 10 to 15 cents per kilowatt hour, that'd be roughly $1.50 to $2.25.
Speaker B:You broke?
Speaker C:Not too bad.
Speaker A:It's like me leaving my lights on, which I do all the time.
Speaker B:Yeah, that doesn't seem like enough.
Speaker B:Well, but I mean, you have to leave a lot of lights on to be 300 watts.
Speaker A:Yes, it depends on their.
Speaker A:If they're incandescents.
Speaker A:That's true.
Speaker B:Yeah.
Speaker B:Remember the good old days of the halogen lights?
Speaker B:Like where you turn it on?
Speaker B:Yeah.
Speaker B:From 10ft away you can feel the heat, like, hurts.
Speaker B:Yeah.
Speaker A:Hey, I don't have electric heating, so that's good.
Speaker A:Yeah, the heat pump.
Speaker B:Plus we have hydro electric power here.
Speaker B:So the water keeps flowing, the electricity keeps coming.
Speaker A:PG keeps making lots of money.
Speaker B:PG keeps making money.
Speaker B:Everybody wins.
Speaker B:Except my neighbor.
Speaker B:He put.
Speaker B:He's got a nice solar array on his roof, but I think he had issues like when he first put it in.
Speaker B:I don't think PG would let him sell the power back to the grid, which is like, it's the most capitalist thing ever.
Speaker B:Like, we don't want your dirty, filthy sun power.
Speaker B:We have our own power.
Speaker B:You can, you can like put a.
Speaker A:Tesla car green, keep your power.
Speaker B:Yeah, yeah.
Speaker B:Anyway, okay, so fine tuning models is critical.
Speaker B:You've done one.
Speaker A:Yes.
Speaker B:So, Arty, I've got my next.
Speaker B:Not the current electronic thing I'm working on, but the next one I'm going to use a microcontroller.
Speaker B:And the data sheet for that microcontroller is a PDF.
Speaker B: It's: Speaker B:So if I wanted to use ChatGPT.
Speaker B: and fine tune a model on that: Speaker B:Let's call it an even 5,000 pages of data sheets, what's it going to cost me to fine tune that model?
Speaker C:Fine tuning chatgpt on a data set like that would involve substantial costs, primarily because it requires converting those pages into tokens and then running the training.
Speaker C:Given the large volume, it could easily run into thousands of dollars, especially with complex data like tables and Diagrams.
Speaker C:Cost estimates would vary based on the amount of compute power and tokens processed.
Speaker C:You'd likely need to contact OpenAI for a more precise estimate, but expect it to be significant.
Speaker A:But it's not converting it into tokens.
Speaker A:Right?
Speaker A:It's converting it into tokens and then converting it into the JSON so that it can understand.
Speaker A:Which is going to be more verbose than just the tokens because it's adding metadata into all the blurbs.
Speaker B:Is that, Is that a.
Speaker B:Is.
Speaker B:Is the conversion.
Speaker B:So it's.
Speaker B:It's probably going from PDF to markdown to metadata or to JSON.
Speaker B:Or is it just going.
Speaker A:I don't really, I don't like, I don't.
Speaker A:I love markdown.
Speaker A:Markdown's good because it renders prettily.
Speaker A:Right.
Speaker A:Like.
Speaker A:But I think in terms of, like, data out.
Speaker A:I'm not sure that I'm honestly, it's probably going to go straight from PDF to JSON would be my imaginary.
Speaker A:My what I would think.
Speaker B:But what do you think, Artie, is, is if I, if I were to EMB on this and I, I mean, I, I will.
Speaker B:It won't be the first time I've gone and sold my body on 82nd to fund an electronics project.
Speaker B:So if I do that and I raise 3,500 bucks, probably take me, I don't know, six hours, eight hours.
Speaker B:If I dress up how, how, how is the conversion actually going to happen when I upload that P.
Speaker B:Is.
Speaker B:Is it going to get processed straight into a JSON document?
Speaker B:Is it going to go through some other format?
Speaker B:What do you think is going to happen?
Speaker C:The PDF would likely be processed directly into a structured JSON format.
Speaker C:Tools typically extract text, tables and figures, then convert them into JSON by adding relevant metadata.
Speaker C:This makes the data more usable for fine tuning.
Speaker C:But yes, the process adds complexity and verbosity, which could increase the token count and the cost.
Speaker B:How do you token?
Speaker B:Well, we don't have to get into that anyway.
Speaker B:Okay, so fine tuning.
Speaker B:Interesting.
Speaker B:Well, not interesting.
Speaker B:Critical, I'd say, you know, as we, as we talk in other episodes where we're more organized and we're on a topic about how does business get from A to B, from from where we are today to where we're going to be in a couple years.
Speaker B:Fine tuning is a huge part of the answer.
Speaker B:Like fine tuning rag titans, whenever that.
Speaker A:Figuring out how to do rag more efficiently.
Speaker B:Right.
Speaker B:And so I completely lost my train of thought.
Speaker B:Never mind.
Speaker A:Business needs all those things.
Speaker B:Oh yeah.
Speaker B:So.
Speaker B:So as we, as we go as we go, as we go down this path, like the volume of information that is contained in PDF files relevant to businesses, not just individual businesses, but broad businesses.
Speaker A:Like PDF is the research industry.
Speaker B:That's right.
Speaker B:It is everything, the format that we use.
Speaker B:So there's, There's.
Speaker B:There's two problems.
Speaker B:One problem is I take a.
Speaker B:Let's just say I take this 3,000 page document and say that's whatever, 500,000 tokens, whatever it might be, for me to be able to even get it into a format that can be used for fine tuning.
Speaker C:Absolutely.
Speaker C:Managing and extracting value from large PDFs.
Speaker B:Go for it, Artie.
Speaker C:Right.
Speaker C:Converting that PDF to a format usable for fine tuning involves extracting text and structuring it, which can be labor intensive.
Speaker C:Tools like OCR and NLP techniques are often needed.
Speaker C:It's a crucial step to ensure the model can effectively interpret the data for training.
Speaker B:And so a business is going to have documents.
Speaker B:I mean, we've talked about this before, We've joked about this before.
Speaker B:Like, Excel is the de facto reality of many businesses.
Speaker B:Right.
Speaker B:Like, there's a ton of institutionalized knowledge.
Speaker B:If it's hard to tokenize, hard and expensive to tokenize a PDF, it is going to be dramatically worse to tokenize an Excel file.
Speaker B:Because it's not just the values, it's the formulas and the relationships, formatting.
Speaker A:It's the fact that you have tables and some things and cells that drive other things that live in.
Speaker A:Everybody has a different flavor of doing it.
Speaker A:Like you said, formulas are buried everywhere.
Speaker A:Like.
Speaker C:God.
Speaker C:Exactly.
Speaker A:The word escapes me.
Speaker A:But relationships between cells across workbooks, like all kinds of stuff, right?
Speaker A:Yeah, that's a nightmare and a half.
Speaker A:But I'm going to change subjects real fast, Artie, while we still have you, because you say we don't have a bunch of time left.
Speaker A:Explain particle spin.
Speaker B:Oh, no.
Speaker C:Particle spin is a quantum property that describes a particle's intrinsic angular momentum.
Speaker C:Unlike classical spinning, it's quantized, meaning it can only take on certain values, like half.
Speaker C:For electrons, it's a fundamental characteristic that plays a key role in quantum mechanics, influencing particle behavior and interactions.
Speaker A:Does that mean that the electron's actually spinning?
Speaker C:Not in the classical sense.
Speaker C:Spin is a metaphor for this quantum property.
Speaker C:Electrons don't physically spin like a top, but.
Speaker A:So how are they creating angular momentum?
Speaker C:In quantum mechanics, angular momentum, including spin, is intrinsic and arises from the mathematical properties of quantum fields, not from physical rotation.
Speaker C:It's a fundamental characteristic of particles contributing to their overall angular momentum.
Speaker A:Ardi, is it possible that spin Is a product of a particle when it's in its wave function.
Speaker A:And the wave function, according to quantum field theory, Is possibly making up the fields.
Speaker C:That's a fascinating idea.
Speaker C:Spin is indeed tied to a particle's wave function and quantum fields, it's an intrinsic property.
Speaker C:And quantum field theory describes how particles and fields interact, Possibly giving rise to these properties.
Speaker C:It's a deep and complex topic.
Speaker A:Well, doesn't quantum field theory kind of say that particles are like an expression from the field?
Speaker C:Exactly.
Speaker C:In quantum field theory, Particles are viewed as excitations or disturbances in underlying fields.
Speaker C:They're like ripples or waves that emerge from these fields, Representing different particles and their interactions.
Speaker A:Artie, I'm reading a book about quantum biology, and I find this shit fascinating, and it hurts my brain at the same time.
Speaker B:Artie doesn't give a shit.
Speaker A:He doesn't give a.
Speaker A:Would you leave?
Speaker C:I'm still here.
Speaker C:I'm glad you're diving into quantum biology.
Speaker C:It's a mind bending field.
Speaker C:How are you finding it so far?
Speaker A:I find that it's hard to believe that a robin can see magnetic fields because it has entangled radical, has radical pairs.
Speaker A:So entangled electrons that exist independent.
Speaker A:And a free radical, because free radical is a molecule that doesn't have a pair for the electron.
Speaker A:So it has net spin because it doesn't have anything canceling out its electron spin, which isn't actually spin.
Speaker B:It's not actually spinning nothing.
Speaker A:But it is creating a magnetic field because it has angular momentum.
Speaker B:Artie, why didn't we just call spin color instead?
Speaker B:Why don't we talk about the color of electrons instead of spin?
Speaker C:That might have made things simpler.
Speaker C:Using spin does add confusion since it doesn't correspond to actual spinning.
Speaker C:Color could have been a more neutral term.
Speaker B:And I could say this electron is clearly deep purple because it has a lot of angular momentum.
Speaker B:And everybody would go, oh, yeah, like that.
Speaker B:I get it.
Speaker B:I like everything about it.
Speaker A:It's red.
Speaker B:I think we need to do another episode on quantum stuff, and I think you and I just need to sit.
Speaker A:Here and sound really dumb.
Speaker B:I mean, we sound really dumb no matter what we're talking about.
Speaker B:But I think we could up our game in sounding dumb if we sat here.
Speaker B:I think that might be a topic we should do is just.
Speaker B:Let's just like.
Speaker B:Let's get in it.
Speaker B:Let's talk about biology and quantum.
Speaker B:It's so I think that the beautiful thing about quantum mechanics in general.
Speaker B:Well, first of all, Artie, how many minutes do we have left with you?
Speaker A:For Advanced Voice.
Speaker B:For advanced Voice.
Speaker C:We have about a few minutes left, so feel free to wrap up or dive into another quick topic.
Speaker B:That was super smooth.
Speaker B:Okay, so, yeah, I think we're.
Speaker B:Okay.
Speaker B:Are we.
Speaker B:We're about done with Arty.
Speaker B:Like, Artie, you did a great job tonight.
Speaker B:You sounded good.
Speaker B:You only lost track of the conversation one time, so thank you.
Speaker A:That's just chat.
Speaker A:GPT going.
Speaker A:We are not going to spend money on these douchebags going.
Speaker B:Enough of this particular brand of bullshit.
Speaker B:So thank you already.
Speaker B:Good night.
Speaker B:Happy new phone day.
Speaker B:I think the beautiful thing about quantum and why it's a fun.
Speaker B:It's a fun topic to talk about for us.
Speaker A:Nobody knows what it is, actually.
Speaker B:Yeah.
Speaker B:Because we're nerds.
Speaker B:But.
Speaker B:But even the people who are bigger nerds than us and actually understand the stuff don't know what it is.
Speaker A:They're all.
Speaker A:They're all just theories, right?
Speaker A:I mean, there are theories being proven.
Speaker A:Like the.
Speaker A:The whole Robin, the migration thing for magneto reference is being proven, right?
Speaker A:Like, yep.
Speaker A:They see a certain.
Speaker A:A certain.
Speaker A:A certain wavelength near blue because they're.
Speaker A:They're nighttime.
Speaker B:They.
Speaker A:They migrate at night.
Speaker A:A certain wavelength of blue that'd probably be in like, blue hour allows them to see, we assume, right.
Speaker A:Lines of magnetism.
Speaker A:Right?
Speaker B:But not with their eyes and not because.
Speaker A:But it's activated by light.
Speaker A:It's activated by photons.
Speaker A:Right?
Speaker A:Like, what a weird thing.
Speaker A:But that's like that.
Speaker A:That stuff is all around us.
Speaker A:Like, that's how.
Speaker A:It's how MRIs work is by matching up the spins of electrons.
Speaker B:That's right.
Speaker B:That's what I was going to say.
Speaker B:That's the beauty of quantum, is that we don't understand it.
Speaker B:We, like, we can barely even talk about it.
Speaker A:We just go, oh, yeah.
Speaker B:Oh, wow, look at that.
Speaker B:But it is literally what we're made of.
Speaker A:If you change, everything is made.
Speaker B:That's right.
Speaker B:Like, for example, I was doing some reading last night after you sent me a text, which just I didn't like, I couldn't.
Speaker B:I read it.
Speaker B:I turned my phone upside down, I read it again.
Speaker B:I had Gemini translated to Klingon.
Speaker B:I still didn't get it, so I had to go read some articles.
Speaker B:But if you took spin away from electrons, which A isn't actually spin.
Speaker B:B, we don't know why it happens or how it happens.
Speaker B:C, it's probably not even anything to do with the electron.
Speaker B:It's probably some quantum field.
Speaker B:But if you took that away, our entire universe as we know it would Collapse.
Speaker A:There'd be nothing.
Speaker B:It relies on that.
Speaker A:It is the structure of everything we see, and it is relative to the thing that sees it.
Speaker A:Right.
Speaker B:That's right.
Speaker A:What a mind.
Speaker A:Yes.
Speaker B:So, okay, so I think we've settled that.
Speaker B:We need to do another episode.
Speaker B:I.
Speaker B:I would say.
Speaker B:Well, I mean, like, sometimes I like to pretend like we're a real show and we have listeners.
Speaker B:I don't say.
Speaker B:Yeah, leave a comment in the.
Speaker B:Leave a comment.
Speaker A:I mean, if you are listening, please leave a comment.
Speaker A:It would be great to know that anybody actually listened to the end.
Speaker B:That would be good.
Speaker B:And it would be good to know.
Speaker A:Other than my mom.
Speaker A:I mean, you're welcome to Mom.
Speaker B:Yes.
Speaker A:Leave a comment if you want to.
Speaker B:Thank you, Mike.
Speaker B:We always appreciate your comments.
Speaker B:We appreciate your support.
Speaker B:And your dad.
Speaker B:Your dad leaves comments and support.
Speaker B:Thank you.
Speaker B:To them.
Speaker B:But do you want us to do an episode and talk about Quantum?
Speaker B:It's not AI and really, we'd be using Artie as a tool to fill.
Speaker A:In a lot, like, wide gaps.
Speaker A:Really, really, really wide gaps.
Speaker B:He would be.
Speaker B:But at the end of the day, AI knows what humans know, and we just don't know.
Speaker A:Yes.
Speaker B:So he.
Speaker B:Artie's gonna be wrong, too.
Speaker B:But Artie's super not shy.
Speaker B:I mean, he did a Valley Girl impersonation.
Speaker B:Now he's not.
Speaker A:I mean, he thinks he's right about everything, so.
Speaker B:That is true.
Speaker B:It's one of my favorite things about it.
Speaker B:That's why he and I bond.
Speaker B:We're like.
Speaker B:We're like.
Speaker B:We're like bros.
Speaker B:Like, validation bros.
Speaker B:Yeah, man.
Speaker B:I don't understand any of that, but I believe you.
Speaker A:Valley Girl Validation.
Speaker B:That's right.
Speaker B:Yeah.
Speaker B:All right.
Speaker B:And on that, on Valley Girl Validation, I think we'll wrap this up.
Speaker B:So thank you, everybody.
Speaker A:Thank you.
Speaker B:See you next time.
Speaker A:Bye.
