Discussion about this post

User's avatar
Taymon A. Beal's avatar

I think Jerusalem here slightly misunderstands Wiesenthal's position, or at least the interpretation of Wiesenthal's position that I'm personally most worried about, which is not primarily about writing quality as a proxy for take correctness.

If we had LLMs steelmanning everyone's arguments to find the hardest-to-assail version of them, then yes, that's an asymmetric advantage for whoever's actually right, reduces the value of various forms of sophistry, and ultimately serves to improve the quality of debate for truthseeking purposes.

And maybe in another six months we'll have LLMs that can do that. But today's LLMs are decidedly mediocre at analytic rigor; good enough for some purposes, but not good enough to much help an intelligent reader figure out the right answers to complicated questions about politics and policy. (See, e.g., the final section of https://www.slowboring.com/p/ai-progress-is-giving-me-writers; apologies to the Slow Boring non-subscribers, as it's paywalled.)

What they're very good at is smooth-sounding prose that superficially appears to hold together. My worry is that this makes it harder to spot flaws in an argument; not impossible, but you have to read a lot more carefully and develop fairly specific skills for detecting rhetorical cover-ups. I've been trying to get better at this but so far I'm not that good at it, and consequently find judging LLM-assisted arguments (in the sense of being able to go like "that's wrong, here's specifically why") to be harder than the fully human-generated ones that I used to typically run across.

Granted, there've always been specialist humans who developed the skill of hiding flaws in their arguments behind this kind of rhetorical slipperiness, but they used to mostly only show up in high-profile contexts, because doing it well was a rare skill. Now everybody has a machine in their pocket that can do it on demand, and *that* seems to me very plausibly bad for readers' epistemic health.

Aaron Bailey's avatar

It’s the excellent “chocolate or vanilla” scene from Thank You For Smoking!

https://youtu.be/xuaHRN7UhRo?si=Kir8DtCCKG_aNyBm

2 more comments...

No posts

Ready for more?