Dwarkesh's blogging confuses me, because I am not sure if the message is free-associating, or, relaying information gathered.
ex. how this reads if it is free-associating: "shower thought: RL on LLMs is kinda just 'did it work or not?' and the answer is just 'yes or no', yes or no is a boolean, a boolean is 1 bit, then bring in information theory interpretation of that, therefore RL doesn't give nearly as much info as, like, a bunch of words in pretraining"
or
ex. how this reads if it is relaying information gathered: "A common problem across people at companies who speak honestly with me about the engineering side off the air is figuring out how to get more out of RL. The biggest wall currently is the cross product of RL training being slowww and lack of GPUs. More than one of them has shared with me that if you can crack the part where the model gets very little info out of one run, then the GPU problem goes away. You can't GPU your way out of how little info they get"
I am continuing to assume it is much more A than B, given your thorough sounding explanation and my prior that he's not shooting the shit about specific technical problems off-air with multiple grunts.
Dwarkesh has a CS degree, but zero academic training or real world experience in deep learning, so all of his blogging is just secondhand bullshitting to further siphon off a veneer of expertise from his podcast guests.
Better to be honest than say nothing, plenty of people say nothing. I asked a polite question thats near-impossible to answer without that level of honesty.
I thought your question was great. I read the Dwarkesh post as scratch space for working out his thinking - so, closer to a shower thought. But also, an attempt to do what he’s really great at, which is distill and summarize at a “random engineer” level of complexity.
You can kind of hear him pull in these extremely differing views on the future from very different sources, try and synthesize them, and also come out with some of his own perspective this year - I think it’s interesting. At the very least, his perspective is hyper-informed - he’s got fairly high-trust access to a lot of decision makers and senior researchers - and he’s smart and curious.
This year we’ve had him bring in the 2027 folks (AI explosion on schedule), Hinton (LLMs are literally divorced from reality, and a total dead-end), both Ilya (we probably need emotions for super intelligence, also I won’t tell you my plan), Karpathy and Dario (Dario maybe twice?), Gwen, all with very very different perspectives on what’s coming and why.
So, I think if you read him as one of the chroniclers of this era his own take is super interesting, and he’s in a position to be of great use precisely at synthesizing and (maybe) predicting; he should keep it up.
I teach and mentor lots of folks in my world. What I don’t do is feign expertise to rub shoulders with the people doing the actual work so I can soak money from rubes with ad rolls.
Cosign, 10 hours in and comments are exclusively people who seemingly know each other already riffing on top of something that's not clear to an audience outside the community, or replying to a coarser version of request with ~0 information. (some tells: referring to other people by first name; having a 1:1 discussion about the meaning of a fork by some other person)
I’m sad the mid to late 2010s alt reality era has come for HN the last 6 months, after 16 years here.
This was my last respite for intellectual argumentation.
Now, every day there’s multiple stories where you have to be caught up on this cinematic universe where AI is fake and doesn’t work and no one uses it and anyone who does is a grifter and or amateur and or embarrassing and any data centers they build will be a waste and they’re probably not even being built and OpenAI is JUST like pets.com so this is basically the web bubble from 1999…so therefore, $X! (In this case, X = RAM supply shortage is fake and actually just coordinated price gouging)
As my MD friend noted wisely a couple weeks ago: it’s noteworthy how this became a culture after LLMs became ubiquitous and user friendly. It was tons of fun and happy times when we were going to reduce # of radiologists, not software engineers.
(I agree on the DRAM price-fixing hypothesis being wrong.)
We (people writing code) are the ones using it more than anyone else to directly aid in / do our work, we probably more closely understand the limitations of LLM's than people working in other industries that just get fearmongering pseudo-technical babble whispered through streams about the power of the technology while using it as a substitute for Google search (which committed suicide a few years ago).
I also think part of what you're observing is the reddit/hive-mind effect, fear drives the crowd so its consensus will tend to emerge as "Nothing works, nothing will ever work, nothing is even being attempted!"
Not saying that you're wrong, but of course if there is a real AI bubble, then "not letting it pop" only escalates the damage when it does eventually pop. The best outcome all-around is for the market to form accurate expectations quite promptly about the real potential and capabilities of AI, regardless of the immediate consequences.
My own wild guess is that this spike in RAM and storage costs is more of a potential drag on the tech sector as a whole than AI companies specifically. Maybe we'll see some systems being reengineered to cut the waste and the technical debt throughout and be a bit leaner and meaner, since that will be making a real difference to the bottom line.
That’s not remotely true. There are plenty of health conditions that affect this. Alcohol consumption can affect it. Completely normal diets with specific foods can affect how stinky your poop is. Non standard but healthy diets also can do the same. There’s a lot more to it.
While yes to a certain extent cutting some processed food and adjusting your diet may help with it, eating a few boiled eggs would set you back massively. It’s not that straightforward.
Extractor, what I’d call a bathroom fan, fair that effectively stops airflow, I’ll go with “close enough to negative pressure for civvies that they fool themselves” (I.e. ain’t actual negative pressure like a cdc lab)
I mean, it literally does. Put something that smokes in a bathroom, open the window, close the door, and caulk the gaps. See how much smoke phases through the door.
I forgot HN isn’t exactly the demographic that tries this and finds out exactly how well it works. I guess for this demographic, I’d suggest telling your tenants it’s okay to smoke in the bathroom as long as the door is closed and the fan is on.
One of the least insightful comments I’ve seen in my 16 years here. “it’s because everyone here is dumb and knows it, and they are panicking and lying because they don’t want you to blow up their scam.”
I'm not calling anyone dumb. It is fine to not have experience with Hetzner or to have only with AWS. It's fine for someone to not know how to cook at home.
About people who work with it, I'm just alluding to the famous quote "It is difficult to get a man to understand something when his salary depends upon his not understanding it".
- "A lot of [Them] are making bank [and are] terrified of their skill set being made obsolete."
- "They have no idea what it means to use a dedicated server."
- "[They believe] bare-metal means “server room here in the office”"
FWIW, it definitely plays great: we all love to believe everyone who disagrees with us is ignorant, usually I'd upvote something like this and downvote a reply like mine, but this was so bad it hit an internal tripwire of "This is literally just a bunch of comments about They, where They is all the other comments."
You can easily play it off with "I didn't call other commenters DUMB, I just said they don't know a server is just a computer and they don't have to be in your office or in Amazon's data center to serve things!"
To riff on the famous quote you "just meant to allude to": "It is difficult to get an [interlocutor] to understand something [about their argumentation] when [they're playing to the crowd and being applauded]" I hope reading that gives you a sense of how strong of a contribution it is to discussion, as well as how well-founded it is, as well as what it implies.
At FAANG, open source is de rigeur for things you can’t make money off of, either because it’s an ecosystem play or someone asked their boss.
You’d be surprised how little drama there is around this.
I’d note that the department that made open LLMs hasn’t produced any work since they produced a Gemini 2.5 Flash equivalent with much better tool calling, because the God King threw a fit. Without reasoning. And they had a reasoning model on deck that was cancelled too.
Well, GPUs are getting more VRAM, although it's pricey. But we didn't used to have 96GB VRAM GPUs at all, now they do exist :) But for the ones who can afford it, it is at least possible today. Slowly it increases.
Hehe me too…went all out on a MBP in 2022, did it again in April. Only upgrade I didn’t bother with was topping out at 128 GB of RAM instead of 64. Then, GPT-OSS 120B comes out and quickly makes me very sad I can’t use it locally
Same. I repeatedly kick myself for not getting the 128GB version, although not for the GPT-OSS model because I really haven’t been too impressed with it (through cloud providers). But now it’s best to wait until the M5 Max is out due to the new GPU neural accelerators that should greatly speed up prompt processing.
"Inline verification of images following the prompt is awesome, and you can do some _amazing_ stuff with it." - could you elaborate on this? sounds fascinating but I couldn't grok it via the blog post (like, it this synthid?)
ex. how this reads if it is free-associating: "shower thought: RL on LLMs is kinda just 'did it work or not?' and the answer is just 'yes or no', yes or no is a boolean, a boolean is 1 bit, then bring in information theory interpretation of that, therefore RL doesn't give nearly as much info as, like, a bunch of words in pretraining"
or
ex. how this reads if it is relaying information gathered: "A common problem across people at companies who speak honestly with me about the engineering side off the air is figuring out how to get more out of RL. The biggest wall currently is the cross product of RL training being slowww and lack of GPUs. More than one of them has shared with me that if you can crack the part where the model gets very little info out of one run, then the GPU problem goes away. You can't GPU your way out of how little info they get"
I am continuing to assume it is much more A than B, given your thorough sounding explanation and my prior that he's not shooting the shit about specific technical problems off-air with multiple grunts.
reply