1. This kind of technology is inevitable and necessary. I'm glad Pangram exists.
2. Like just about every supportive piece on this topic, this one underestimates the technological challenges to what AI writing detection is doing. I think this is a vestige of the fact that wordcels are having these arguments, frankly. Pangrams one in 10,000 claimed false positive rate is impressive, but too many people are treating that as though it's indistinguishable from one in 1 million. A false positive rate of one in 10,000 means that every single day millions of false positives are being produced by humans beings. And I have no idea why discussing that basic technological reality is treated like a sign of bad faith.
3. Pangrams own founder has been much more cautious and limited in his claims then most of the boosters of his product. It's all very strange.
I don’t think this piece disagrees with you! That’s what I was saying with “I’m happy to concede that Pangram is imperfect and possible to over-rely on. If an author’s posts are mostly non-AI and one scans as partially AI, this could be a mistake. It’s also possible to defeat Pangram with intentional effort. If you think we need something more robust and more reliable, fine! But if you don’t like AI and don’t like Pangram, your attitude really ought to be that it’s a shame that our existing tools for AI detection are imperfect, and it’s important to invent better ones.”
Okay I’m at least somewhat persuaded by this reasoning, but that then nudged me to ask - Kelsey, what do you make of people who are overtly/publicly anti-AI but also using AI to do their writing?
Is it that their public position is performative? Cognitive dissonance? Viewing only their own use of AI as acceptable? (I’m skeptical of the last one because using AI to author whole essays goes way beyond using it for research — and I say this as someone who uses AI a ton).
I guess this just ties into a larger theme that I’ve been wondering about, which is how salient anti-AI / anti-data center views really are. It’s easy to oppose something on the internet, and slightly harder but not actually that much of a burden to show up to a city council meeting to scream nonsense at your local government… but when actual push comes to shove? Perhaps all these people using AI to write suggests something.
I think at least some people are lying to themselves as much as to their audience - AI is bad, the thing they're doing isn't bad, so the thing they're doing isn't AI. It might really feel like the AI just articulated exactly what they would have typed themselves, and if so who cares that they didn't type it?
I also think some people start out genuinely just using it for style edits and then escalate to fully AI-generated pieces. Some disclosures are probably true to how they started out using it.
I do also think you get a lot of cynical dishonest behavior, though. Substack's a business! People want to make money!
I use AI as a search engine and as a fact-checker (paste in a claim, ask whether it thinks it's true, wrong, distorted, etc). I don't use it to write. I don't think it's necessarily immoral if you disclose it, but I also think that people really really underestimate how much less clear and well developed their thoughts typically end up being if they use AI to write them down.
(Odd that it doesn't even flag the AI quotes but they may be individually too short).
Why? While I agree with your previous message in this thread, even the biggest proponents of pangram acknowledge that the false negative rate is far from zero.
Setting aside Pangram’s false positive / negative rates (which could be the entire explanation here), there is to me a significant contextual difference between AI authored prose appearing in quotes and not. It’s possible the underlying models in Pangram process that distinction and don’t regard the piece as AI-authored if its quotes.
It would be an interesting experiment to try variations of Kelsey’s article along these themes to see if there is a dividing line.
I think this is right about the motivation of most people who are anti-pangram, but there's a second group, which I expect the WIRED author is in. It's people who think AI is bad, defecting AI is bad, and the correct solution is to just all agree to eliminate all of it from our society.
It's similar to people who think that car culture is bad but speed cameras are also bad because we should just pedestrianize our cities and invest in mass transit.
If you want to eliminate AI from society (not my goal, but not a goal I have zero sympathy for), you will need the ability to tell whether people are using it!
My working theory of the kind of person who frothingly attacks Pangram is that their primary axis of insecurity is of knowing they themselves are a fraud. They seethingly resent ‘billionaires’ because their success calls attention to their own lack of talent. They loath Pangram because of the prospect their lack of ability to write will be revealed for the world to see.
Oh boy, more professors and media types being exposed as frauds and hypocrites, who attempt to deflect with wild, baseless claims of racism. I assume the next step is the accuse Pangram of editorial genocide or whatever. Meanwhile the credibility crisis of once-respected institutions like academia and the press continues unabated.
Is my understanding correct that AI-watermarking will mostly solve this issue? Sure, people can probably find jailbroken or non-EU-law-compliant models and use those instead but they will likely be much worse than the frontier and easier to detect. In addition, it will require a lot more effort to do this, potentially doing a way with AI-slop based low-effort, low-returns business cases.
AI can be a very good editor if you don’t let it go overboard. Write out something in more stream of consciousness or outline form use AI to clean it up into a coherent bit of writing, then edit yourself (possibly a couple cycles of this with different prompts to get AI to make the targeted changes you want).
I do this with work emails all the time, or if I need to email a contractor or something for my house, just for efficiency and to catch errors.
I am not sure how much of this I would do as an essayist, at least at present, and I am not sure what Pangram would say about text produced this way, but AI is too useful as a productivity tool for producing written content to go away. For good or ill, complaining about its use will soon seem as silly as complaining that accountants don’t do math by hand instead of using Excel.
Possible that some of the people complaining feel they are using it in this manner and text that they feel really is theirs but is created through AI use as well as their own efforts is being unfairly flagged.
I personally think it is really unlikely that someone who is a bad writer has a coherent enough voice such that it reliably shifted to being consistently flagged as AI after one or two years.
If this type of thing were to be plausible I’d suspect it would be a spectrum, and that there would be a number of people whose current long form writing gets *inconsistently* flagged by Pangram, and I’ve yet to see anyone claim to fall into that camp.
I largely agree, but I do wonder what “voice” someone who spends multiple hours a day researching/chatting/consulting with an LLM (and perhaps only an LLM, worsening the potential for mimicry) would have if not for the voice of the LLM itself.
If someone has 5 years of pre-AI writing and 2 years of post-AI writing, I would expect their original "voice" will still be pretty strong. But we're maybe starting to get to the point where a 25-year-old writer might have 3 years of pre-AI writing and 4 years of post-AI writing, if they were precocious at beginning to use AI.
True, but also bad writers often have a pretty weak and underconfident voice and style.
And someone overrelying on AI for editing might just tune their existing style more and more to suit the pseudo-preferences expressed by their LLM-editor.
Or they’re just lying and using AI to generate text, which still seems like the more likely option to me.
1. This kind of technology is inevitable and necessary. I'm glad Pangram exists.
2. Like just about every supportive piece on this topic, this one underestimates the technological challenges to what AI writing detection is doing. I think this is a vestige of the fact that wordcels are having these arguments, frankly. Pangrams one in 10,000 claimed false positive rate is impressive, but too many people are treating that as though it's indistinguishable from one in 1 million. A false positive rate of one in 10,000 means that every single day millions of false positives are being produced by humans beings. And I have no idea why discussing that basic technological reality is treated like a sign of bad faith.
3. Pangrams own founder has been much more cautious and limited in his claims then most of the boosters of his product. It's all very strange.
I don’t think this piece disagrees with you! That’s what I was saying with “I’m happy to concede that Pangram is imperfect and possible to over-rely on. If an author’s posts are mostly non-AI and one scans as partially AI, this could be a mistake. It’s also possible to defeat Pangram with intentional effort. If you think we need something more robust and more reliable, fine! But if you don’t like AI and don’t like Pangram, your attitude really ought to be that it’s a shame that our existing tools for AI detection are imperfect, and it’s important to invent better ones.”
Okay I’m at least somewhat persuaded by this reasoning, but that then nudged me to ask - Kelsey, what do you make of people who are overtly/publicly anti-AI but also using AI to do their writing?
Is it that their public position is performative? Cognitive dissonance? Viewing only their own use of AI as acceptable? (I’m skeptical of the last one because using AI to author whole essays goes way beyond using it for research — and I say this as someone who uses AI a ton).
I guess this just ties into a larger theme that I’ve been wondering about, which is how salient anti-AI / anti-data center views really are. It’s easy to oppose something on the internet, and slightly harder but not actually that much of a burden to show up to a city council meeting to scream nonsense at your local government… but when actual push comes to shove? Perhaps all these people using AI to write suggests something.
I think at least some people are lying to themselves as much as to their audience - AI is bad, the thing they're doing isn't bad, so the thing they're doing isn't AI. It might really feel like the AI just articulated exactly what they would have typed themselves, and if so who cares that they didn't type it?
I also think some people start out genuinely just using it for style edits and then escalate to fully AI-generated pieces. Some disclosures are probably true to how they started out using it.
I do also think you get a lot of cynical dishonest behavior, though. Substack's a business! People want to make money!
For anyone wondering:
Pangram: "100% of this text is Human Written."
(Even the AI quotes!!)
I use AI as a search engine and as a fact-checker (paste in a claim, ask whether it thinks it's true, wrong, distorted, etc). I don't use it to write. I don't think it's necessarily immoral if you disclose it, but I also think that people really really underestimate how much less clear and well developed their thoughts typically end up being if they use AI to write them down.
(Odd that it doesn't even flag the AI quotes but they may be individually too short).
I would call it more than odd....
Why? While I agree with your previous message in this thread, even the biggest proponents of pangram acknowledge that the false negative rate is far from zero.
Setting aside Pangram’s false positive / negative rates (which could be the entire explanation here), there is to me a significant contextual difference between AI authored prose appearing in quotes and not. It’s possible the underlying models in Pangram process that distinction and don’t regard the piece as AI-authored if its quotes.
It would be an interesting experiment to try variations of Kelsey’s article along these themes to see if there is a dividing line.
To your last sentence - this is entirely or almost entirely a function of how people react to the witch hunt accusations.
It's kind of impressive how Wired manages to seemingly be wrong about everything.
I think this is right about the motivation of most people who are anti-pangram, but there's a second group, which I expect the WIRED author is in. It's people who think AI is bad, defecting AI is bad, and the correct solution is to just all agree to eliminate all of it from our society.
It's similar to people who think that car culture is bad but speed cameras are also bad because we should just pedestrianize our cities and invest in mass transit.
If you want to eliminate AI from society (not my goal, but not a goal I have zero sympathy for), you will need the ability to tell whether people are using it!
I think if free consumer chat bots went away, then basically there would not be enough AI use in general prose to be a problem.
Are there people who are anti-cars and anti-speed cameras?
Do they drive?
My working theory of the kind of person who frothingly attacks Pangram is that their primary axis of insecurity is of knowing they themselves are a fraud. They seethingly resent ‘billionaires’ because their success calls attention to their own lack of talent. They loath Pangram because of the prospect their lack of ability to write will be revealed for the world to see.
Oh boy, more professors and media types being exposed as frauds and hypocrites, who attempt to deflect with wild, baseless claims of racism. I assume the next step is the accuse Pangram of editorial genocide or whatever. Meanwhile the credibility crisis of once-respected institutions like academia and the press continues unabated.
Doesn’t surprise me that a genderslop slinger like Beccia would be caught out like this.
So glad to see this.
Is my understanding correct that AI-watermarking will mostly solve this issue? Sure, people can probably find jailbroken or non-EU-law-compliant models and use those instead but they will likely be much worse than the frontier and easier to detect. In addition, it will require a lot more effort to do this, potentially doing a way with AI-slop based low-effort, low-returns business cases.
AI can be a very good editor if you don’t let it go overboard. Write out something in more stream of consciousness or outline form use AI to clean it up into a coherent bit of writing, then edit yourself (possibly a couple cycles of this with different prompts to get AI to make the targeted changes you want).
I do this with work emails all the time, or if I need to email a contractor or something for my house, just for efficiency and to catch errors.
I am not sure how much of this I would do as an essayist, at least at present, and I am not sure what Pangram would say about text produced this way, but AI is too useful as a productivity tool for producing written content to go away. For good or ill, complaining about its use will soon seem as silly as complaining that accountants don’t do math by hand instead of using Excel.
Possible that some of the people complaining feel they are using it in this manner and text that they feel really is theirs but is created through AI use as well as their own efforts is being unfairly flagged.
I personally think it is really unlikely that someone who is a bad writer has a coherent enough voice such that it reliably shifted to being consistently flagged as AI after one or two years.
If this type of thing were to be plausible I’d suspect it would be a spectrum, and that there would be a number of people whose current long form writing gets *inconsistently* flagged by Pangram, and I’ve yet to see anyone claim to fall into that camp.
I largely agree, but I do wonder what “voice” someone who spends multiple hours a day researching/chatting/consulting with an LLM (and perhaps only an LLM, worsening the potential for mimicry) would have if not for the voice of the LLM itself.
If someone has 5 years of pre-AI writing and 2 years of post-AI writing, I would expect their original "voice" will still be pretty strong. But we're maybe starting to get to the point where a 25-year-old writer might have 3 years of pre-AI writing and 4 years of post-AI writing, if they were precocious at beginning to use AI.
True, but also bad writers often have a pretty weak and underconfident voice and style.
And someone overrelying on AI for editing might just tune their existing style more and more to suit the pseudo-preferences expressed by their LLM-editor.
Or they’re just lying and using AI to generate text, which still seems like the more likely option to me.