A practical guide to writing with AI
I tried every writing app so you don’t have to
2023 is the year artificial intelligence. AI can now analyze your data, take actions in the real world, pass the bar exam, and offer medical advice. Initial studies suggest AI will automate huge chunks of knowledge work, including writing. Students are already using ChatGPT to cheat on their homework.
Generative AI opens the door to a new form of 'cheating'—using AI to do our jobs. AI has encoded the world’s knowledge and can be prompted to perform any task. But it is a black box which generates outputs that are believable, but not trustworthy. It is 93% accurate, but we don’t know why the other 7% is wrong (c.f. Benedict Evans).
Some answers can be clearly wrong, like a calculator giving a wrong result. Others can seem right, but aren't. An example being whether COVID originated from the Wuhan market (oops). Large language models (LLMs) function like a complex game of mad libs, responding to each prompt with one word after the other.
For this piece, I explore whether we can “cheat” by asking AIs to create great writing and I document my journey using more than 20 writing tools.
Great writing is short writing
In the internet era, AI-fueled content farms are flooding the internet with low quality articles to drive traffic. Yet, their prevalence doesn’t mean AI cannot write well. AI prompters choose to publish whatever it generates.
AI makes it easier to create content, exacerbating the problem of information overload. Our time and attention will forever remain limited. There will be tons of noise. Artifact, a startup created by the Instagram founders, uses a TikTok-like approach to choose articles based on user behavior. Google has said it will filter out AI generated content.
But it shows why great writing matters. Great writing is like a lighthouse at sea. It leaves a lasting impression. It sparks curiosity. It evokes imagery, senses, and emotions. And many authors offer the same advice — write with your eraser. George Orwell states “if it is possible to cut a word out, always cut it out.” Ruth Bader Ginsburg says “you’re not finished when you can’t think of anything more to add to your document; you’re finished when you can’t think of anything more that you can remove from it.”
Can writing be universally impactful? Often its influence often hinges on factors beyond the writing itself. Consider the following factors:
ethos — who is the messenger, what is their character, do they have credibility?
pathos — how does this piece of writing make me feel?
logos — what is the logic behind what they are saying
timing — is it the right time for me to receive this message, remember it, and act on it?
connection — how does it connect to my personal experiences?
Each person answers these questions differently, making universal appeal difficult to achieve. The writer doesn’t know who the reader is, their background, where they grew up, and why they are reading. Writers can make an educated guess. They can change the title of the piece. They can write in different publications to appeal to specific audiences. But it is a guess.
Sometimes stories do achieve mass appeal. To quote investor and writer Morgan Housel:
“The person who tells the most compelling story wins. Not the best idea…Tell people what they want to hear and you can be wrong indefinitely without penalty.”
Despite our frequent focus on what makes us different, we look to movies, books, and music to feel understood. We all belong to the same human experience. We all have dreams, anxieties, fears, and doubts. We are not alone.
AI helps you write, but at what cost?
To churn out content fast, writers ask chatGPT to pretend to be a certain persona and write about X.
For example, you can ask chatGPT the following:
“You are a university rhetoric professor. You write with clarity, simplicity, and structure. Please write a persuasive 5-paragraph essay that analyzes dating app trends and argues why meeting people in person is more effective.”
Publishing this verbatim is giving our jobs away to AI. But great writing is not produced from beginning to end as a stream of consciousness. These prompts are akin to an assignment written by your high school english teacher. Great writing comes from constant re-writing and editing. And writing involves two key dimensions: writing vs. refining, and ideas vs. details.
Let’s try AI tools for each use case.
Writing ideas
Writers need to generate compelling stories and titles to get noticed. But without good initial content, the suggestions fall flat. Inflection.ai was a better source for inspiring ideas. The dialogue feels like a chat with a friend. While the suggestions are often wrong, wrong ideas bring clarity, paving the way for good ideas. As Charlie Munger suggests, “invert, always invert.”
Refining ideas
AIs are incredible at structuring unorganized thoughts. Oddly, generative AI seem better at organizing content than generating it. Sometimes, AI misunderstands the original intent. When organizing existing thoughts, the AI uses the same language as the prompter, which feels easier to comprehend.
I used the following two messages with ChatGPT to critique this essay.
“You are a university rhetoric professor. Your help authors produce clear, simple, concise writing. Please critique the following essay outline.”
“Can you make a suggestion on how I should restructure it?”
As I added detail, I found myself changing section titles to communicate the salient point. And much like title generation, compelling section titles come from clarity. AI can only offer editorial suggestion. I tried garnering inspiration from viral tweet templates from Tweetlify. These feel especially cringe-worthy.
Writing details
Writers often need inspiration when they get stuck. Many tools offer auto-completion to expand salient points into paragraphs of text. But the auto-completion doesn’t create text people want to read. Writers also need to find evidence, which is more of a search problem. I used bard.google.com for generalized search and elicit.org to find research papers.
Refining details
Every word in a manuscript matters. It is always easier to write something new than to edit, shorten, simplify, or condense. This is where AI and software tools shine.
AIs can also give great feedback. It is not always correct, but large pieces of writing could take hours to get feedback from peers. AI gives it in seconds. For this, I used chatGPT and Lex.page, and boo.ai. Boo.ai’s interface is particularly charming! It is so simple, yet so powerful! Boo.ai had some hiccups though. It did not save my writing on browser refresh. Using it as my primary interface for real work would be a non-starter.
AI is a writer’s ally when used correctly
After reviewing all of the functions and their outputs, I give them the following scores:
AI is great at brainstorming, outlining, and rewriting — tasks which the end reader never sees, but helps the author clarify thinking. AI is not good at writing compelling titles, takeaways, or sentences yet, where every word matters a lot in contributing to an author’s unique voice and message.
This is why great writers are here to stay. They provide a unique voice. They exercise taste and preference. And with AI, their voice can become even more powerful with a writer’s new secret weapon.
To quote the maker of IA writer:
If you do not understand the foundation of what you write, it’s not likely that you will end up with a text that has been felt and understood in every aspect. Writing is rewriting and rewriting until the thought becomes clear.
Paul Graham, a venture capitalist, also had a similar sentiment on AI generated text.
Noticed something fascinating(ly digusting) about AI-generated summaries of essays: they don't just make them shorter, but also make the ideas more conventional. Which makes sense given the way the AIs are trained.
Writing AI will become even more powerful in the future
Language models will become better at mimicking a writer’s style.
Personalized AI will capture this over time without the need for prompt engineering as AI captures more data about our preferences and behaviors.
The user experience of AI writing will move away from chat bot paradigms.
See Maggie Appleton’s Language Model Sketchbook
Chatbots require a lot of words and a lot of reading. And the feedback given by AI is not localized. New apps and tools will be developed that utilize existing UX patterns to create better workflows that use AI “in-the moment.” Highlighting text, right click menus, and localized action will give writers more control and more power.
Writing will no longer require word by word editing.
Eventually, writers will be able to create personalized AIs, trained from their body of work. Writers will give AIs the salient points they want to communicate along with the evidence they want to use to create new pieces of work. They would then localize their feedback to specific pieces of text they want to change in tone, style, clarity, or brevity.
Appendix: How I write with AI
For this article I followed the following steps:
Write all my thoughts in bear.app, unstructured
Get inspirations, outlines, and feedback from chatGPT by dumping the raw text
Re-formulate the text into the suggested rough outline, and fold content underneath each section
Write the details underneath each section
Rewrite the section titles into a cohesive story
Copy to Hemingway to get clarity suggestions
Use chatGPT to generate a compelling title ideas
Get feedback from peers
Publish on substack
Appendix: All the writing software I tried
In order of preferred use
For writing
For editing and auto-completion
For idea generation, research, and article summarization