Excuse Me, Your AI is Showing

Some clear examples of why generative AI requires human oversight

As a freelance content writer in the B2B SaaS space, I've been navigating our new AI-dominated reality for a while. I’ve lost work to generative AI and seen entire marketing departments laid off because of it. While no one said they were replacing me with ChatGPT or Claude, it was pretty obvious that’s what was happening. Now we’re far past the honeymoon phase of incorporating AI tools into content development. Most companies I work with are using these tools and they expect me to use them as well. And use them I do! I even have an effective (and ever-evolving) AI-assisted content writing process.

I am not a purist, after all. I don’t shun AI writing tools. I am, first and foremost, a writer. But I’m also cognizant that skills must be upped and minds must remain open if one wants to remain hirable. I’ve been using generative AI chatbots for four years at this point, including ChatGPT, Jasper, Claude, and (my current tool of choice) Abacus.ai. I think it’s important to understand how these tools work and read what they generate, if only so I can explain why their output can be problematic.

These tools are useful as writing assistants. Generative AI is the world’s best note-taker and meeting summarizer. It’s great at pulling statistics out of huge reports, creating lists of common items (benefits, challenges, best practices) from a bunch of source articles, and writing meta titles and descriptions. It can generate a passably good outline, though I’ve never had one that came out untouchably perfect. It's also good for drafting content, with the (huge) caveat that most of what it drafts will (or should) be heavily edited, revised, or completely discarded.

AI is great at processing huge amounts of data and generative AI is great at creating skimmable snippets to save you the time and effort of reading the source material — that’s what Google and Bing does when you search for something and you get a summarized answer to your question at the top of the search results.

Of course, sometimes this goes horribly wrong. That’s because generative AI tools are really good at making stuff up and presenting these lies with confidence. If you’re using these tools without any oversight, then it’s entirely possible that you’re telling people that the best way to keep cheese from sliding off of pizza is by mixing a bit of glue into the sauce.

Generative AI doesn’t need writing skills to summarize a sixty-minute webinar on omnichannel personalization, but it does need a sense of how words play well together on a page so that other humans will want to read a 1200-word article about that same webinar. Even the latest models of Claude and ChatGPT don’t yet possess this needed sense of craft.

I often wonder if writers were consulted when the mad AI scientists cobbled together ChatGPT’s large language model or if it was just engineers and data scientists piecing together language as though it were a puzzle for the LLM to decode.

I use generative AI for assembly, for extracting stats, summarizing large reams of information, and helping me find that information in the first place. I collect information from interviews or webinars or my own research, organize it with the help of AI, then I use my writing brain to make it readable, unique, interesting, and reflective of my client’s brand.

Is this writing? I don’t actually know anymore. But I do know that AI speeds up the process of writing about 20%, sometimes more if it’s a large piece involving lots of research. I don’t know what other people do or how much work they’re putting into revising and fact-checking AI-generated content.

I write and use these tools every single day. I’ve learned a lot about what they’re capable of and even more about their vast and dismaying limitations. This applies only to business writing, by the way. I will never use an AI tool to write an essay or a poem or a novel.

But I will grudgingly admit that AI, with some rather large caveats, can be quite helpful for business writing. That’s not a glowing endorsement. I have some thoughts, based on hours upon hours of using these tools, for every company who has (or plans to) replace their human writers with pleasantly cheerful, empty-eyed robots.

Go ahead and use generative AI if you want to create a blog post or something more complex like an eBook or report. But I implore you to hire a writer to oversee and refine whatever mediocre text comes out of your AI-powered tool of choice.

If you skip this step, 100% of your blog posts will start with a version of the sentence, “In today’s rapidly evolving digital landscape" or, "Today, digital transformation isn't just a buzzword, it's table stakes if you want to drive growth" or whatever.

And if you don’t have an experienced writer working to turn that AI-generated, cliché-riddled, poorly-paced content into a decent article, then you may as well not bother posting it at all. No one’s going to read it and Google’s getting better and better at detecting it.

You should also know and (I hope) care that if you’re using AI to pump out content as quickly as the robots can cook it up, then you’re regurgitating lots of mediocre, homogenized, un-brandworthy word salad. Why? because that’s largely what these tools were trained on.

You’re starting all your posts with, “In a world” and filling them with the same buzzwords and phrases everyone else is filling them with — seamless, scalable, cutting edge, level up, immersive…I could go on, but I might put myself to sleep.

Sometimes you’re selling a platform or a technology that enables things like individualized customer experiences (glances over shoulder at clients) and, in that case, it’s fine to use these terms ever-so sparingly. But they should be couched in actual examples from your actual client portfolio using actual statistics from actual case studies.

Have you ever found yourself reading long blocks of vague-sounding business-speakery with no concrete examples? This is the product of AI. Do you want your entire website to sound like that? I dearly hope not.

Remember how I said that AI likes to make stuff up? When this happens, it’s affectionately called a “hallucination”. It happens often and in ways that are so subversive, you may not notice until you’re giving a presentation in front of a couple thousand people and your AI chatbot says something so awful, you shut the damn thing down the next day. Sure, that's an old reference, but consider that, just this summer, Anthropic had to shut down their AI-authored blog a few weeks after launch.

The thing about generative AI and chatbots is that it's programmed to answer user prompts and, truth be damned, it will answer your question in a way that sounds so plausible, you may have no idea they’re making it up.

It’s becoming increasingly clear that chatbot hallucinations are an inevitable feature of generative AI. They’re not some random flaw. Per this article in Scientific American:

"Many machine-learning experts don’t view hallucination as fixable because it stems from LLMs doing exactly what they were developed and trained to do: respond, however they can, to user prompts.”

Hallucinations happen for various reasons that I won’t bore you with (though I did write about it for Coveo). Examples, based on my firsthand experience of using these tools, include false citations, made-up statistics, and references to clients, case studies, and events that don’t exist. AI can produce any number of true-sounding falsehoods that it cooks up in response to a user query.

You need humans to check this stuff, to look at the link to make sure it’s real, find that stat the AI is quoting to verify it’s on the source page, and question anything that doesn’t make sense. Are you doing that? If not, then it’s highly likely that some of the content on your carefully maintained corporate blog is flat-out wrong. I have never known an AI chatbot to say, “Wait, that doesn’t sound right. Let me go back and check my sources to make sure I don’t get you fired or embarrass you in front of a Reuter’s journalist.”

AI will cheerfully lie to you with absolute confidence. It literally does not care. If you question it, “Hey, Claude, you just made up a link and I can’t find this stat you mentioned in the source documents I provided. What’s up with that?” It will respond, instantly and without any sense of remorse, “My apologies. You are correct and I should not have done that.” It will then immediately do it again.

Generative AI tools, if left to their own devices, will calmly destroy all your credibility by pumping out fake facts as fast as their neural network can process them. But frequent breaks with reality aren’t their only flaw. They’re also bad at writing for several reasons that I will now smugly point out.

First, LLMs like ChatGPT were trained by scraping the web. This includes websites like Twitter, Wikipedia, and publicly accessible web pages (blogs, news sources, websites, forums, etc.) They also fed entire books into ChatGPT to help train it, but it’s impossible to know exactly what sources were used because this is top secret information.

These tools learn by consuming massive amounts of text data then predicting the most likely output based on patterns in that data. They turn words into numbers (e.g., "embeddings"), identify statistical relationships in those numbers, then pump out the response based on probability distributions.

If you ask it what the quick brown fox does, its likely response will be, “jumps over the lazy dog”, not because it knows anyting, but because that sentence may have come up over and over again in its training data. You know what else comes up over and over again in human writing? Clichés, mediocrity, bias, buzzwords like “synergy” and “low-hanging fruit.”

We have created the world’s most mediocre writer and we are outsourcing all our content to this thing.

The good news for anyone who wants to use generative AI to create business content (and that’s everyone) is it has a few obvious tells.

Long, overly complex sentences? Check. Nonspecific examples about even less specific business benefits? Check. Repetitive phrasing? An overreliance on bulleted lists? Colons in titles and subtitles? Any sentence that ends with “on the road to success” or “wealth of opportunities”? Check, check, check, and check.

You need to hire a writer if you want to turn the reams of bland AI text into something that represents your business. I realize, as a freelance content writer, it's self-serving of me to say this, but here we are.

Generative AI isn’t the quick content fix that many people thought it would be at the end of 2022. I’m not the only one who’s noticing this. My work began picking up again at the start of this year, although it remains inconsistent. Writers, and the companies that work with us, are in uncharted territory so this makes sense. My plan is to continue using these tools, reading about them, and figuring out the best way to incorporate them into my workflow so that I can continue to produce high-quality, human-centered content.

I’m not an AI hater. I use these tools in my day-to-day work and they’re helpful. They save me a ton of time in research and I'm continuing to find new ways to use them. Abacus.ai, in particular, is an excellent meta tool that let's me test out all the latest LLM models for a very affordable monthly fee.

They're getting better, but they remain incredibly limited and reliant on human input to produce what I consider high quality content. That means they’re not a replacement for human writers. At least, not yet. Now that you know some of the tells, you might want to give your company blog a thorough review. I guarantee that if you’ve been using ChatGPT or a similar tool to write most of your content, then your AI is definitely showing.

//

A version of this article was published on Medium on June 19, 2024.