If you’re a content marketer worth your weight in SEO keywords, you’ve undoubtedly tried GPT-4 for writing long-form articles.
And like the rest of us, you’ve been disappointed by it. Yes, it can write content, and some of it sounds pretty good, but all of it sounds like ChatGPT. Then, when you try to steer it with a prompt, it goes way too far in one direction and is very hard to dial into your specific brand’s style and tone.
“Sigh, if I could just show it my blog and our library of past posts, then it would ‘get’ me.”
Of course, those beautiful past posts are too long to copy and paste in the prompt, because of context windows or whatever… Few-shot learning is a no-go.
But fine-tuning? That’s another story.
In fact, thanks to the recent release of GPT-3.5 Turbo for fine-tuning, we cracked the code on it.
That’s right, we can generate 3,000 word blog articles that match our style and tone using a custom-trained AI model.
It works in seconds and costs pennies.
Holy smokes — are you as excited as I am?!
Here’s how you do it.
First, we need our blog articles in a digestible format.
For this, we’ve chosen markdown formatting.
Markdown is a very common way to format text. It’s used on many Content Management Systems already and it’s a convenient way to show our AI model what we’ve created in the past and also to generate new posts. We don’t want the AI to get hung up on HTML syntax or anything more complicated than just the basics like headings and lists, which makes markdown a perfect choice.
To get my blog articles from Wordpress, I exported my posts and then used a wordpress-export-to-markdown script to convert the files in bulk to markdown.
To export your Wordpress articles, first go to Tools and then Export.
Next, select Posts. If you have a huge library of posts, you could start with articles from just one category, from a favorite author, or more recent ones that reflect the latest writing style.
Then, press Download Export File.
That gives us an XML file, which is not exactly what we want. But we can get there.
Note: PC users, please forgive me — I used to be one of you, but I’m going to proceed with Mac instructions here because that’s what I have now.
Rename your posts export file to "wordpress.xml" and move it onto your desktop. Leave the ".xml" off if you’re on a Mac that has file extensions hidden.
Now go to your Launchpad and type "Terminal." Open the terminal app.
Type "cd desktop" and press enter. This navigates to your desktop.
Now type or paste in this command:
It will ask if it can install the package (If not, you may need to install npx first). Type "y" for yes and press enter.
If the package installs successfully, it should say "Starting wizard…" and ask the path to the Wordpress export file. Since we named our file, moved it to the desktop, and navigated there already, you should just be able to type "wordpress.xml" and press enter.
Press enter when it asks the output folder and it will create a default. There are a couple more options that are up to you. When it asks whether to save images attached to posts, say no because it will take a long time and you don’t need ‘em.
After the last question, it will start saving posts:
Now you should have a .md (md = markdown) file for each of your blog posts in a folder called "output" on your desktop.
Let’s take a moment to hand-pick some blog posts.
We need at least 10 for our examples, but 25-30 is great if you can muster it. Remember, this is worth the effort, because even if this takes the time that one post would take for you to write manually, soon you’ll have the power to generate unlimited articles at your fingertips.
The key here is that we want to feed our model the best posts, the ones most representative of the content we want to create in the future. So you might choose some of your more recent blogs, or ones you know are better written, and start collecting those while discarding others.
Also, keep in mind that extremely long examples won’t work. They should be under approximately 3,000 words to work with GPT-3.5 Turbo, the model we will use for fine-tuning.
Entry Point is our software tool that lets you fine-tune AI models with ease. It makes the whole process much more accessible and you won’t have to type a single line of code. You can close that command line window!
And since the free tier includes 50 examples, it won’t cost a thing.
After you sign up, go to create a project, choose the Blog Writer blueprint, and press Create.
Now open ChatGPT in another window.
We’re going to have ChatGPT write purposefully bad summaries of our articles.
The idea is that our AI blog generator is going to write blogs for us, but it needs a prompt to steer it and provide it with any specific facts we want included, because otherwise it might pick some random topic or go in a wild direction we don’t care about.
This prompt doesn’t have to be a work of art. In fact, the worse it is, alongside the most excellent output, the easier our lives will be when we want to go and use our AI to generate a post.
For this article's purposes, our prompt will consist of a topic and some very rough notes that we could scribble down in 30 seconds if we had an idea for an article but didn’t have time to write it ourselves. That’s what we’re going to give our AI and get back a masterpiece. At least, that’s the idea.
This is how fine-tuning works: we give the model examples of some input and what the output should look like from it, and it will be able to do that again in the future. In this case, setting a really low bar for the quality of the input is good for us, because we’ll be able to just wing it on the notes, stream our partially coherent thoughts and push a button.
But the problem is we don’t actually have rough notes for our blog articles, do we? We just have finished, polished blog articles.
No problem. Let’s use GPT-4 to reverse-engineer some crummy, hastily written notes that could have been the basis for our blog articles.
Here is a prompt that I found to work fairly well for this task:
You are a flexible writer that reproduces the style of a very busy executive who is not a great writer but knows a lot and has good ideas. You will summarize the following article into rough notes.
These notes should have elements of:
-Stream-of-consciousness but full sentences. 20% of them can be fragments. -Just complete enough that a professional blog writer with domain knowledge could come up with the following article. -Include any specific numbers or facts that need to be cited, but not common industry knowledge. -Try to give the jist and leave some details out so a writer could take creative liberties -Keep it to one paragraph, no more than 7 short sentences. -No frills or personality. Keep it plain. -Limit to approximately 9th grade language. -Be linear and coherent. -No semi-colons or complex sentence structure. Article: [paste a full article here]
This prompt can surely be improved upon — it was the first iteration that worked pretty well for me, so I went with it. You may want to tweak it to your preferred style of note-writing.
The longer the notes are, the more closely they will dictate the full blog post, and the more our AI model will expect you to hold its hand through the content. The shorter the notes are, the more is left to interpretation and creativity, the more our model will expect that it is supposed to go off and create the ideas for content itself. These are two characteristics we need to try to balance.
Training AI models is half art and half science.
Anyway, we need to copy and paste each article in. This part is a bit of a grind, but it’s going to be worth it.
When you get the first set of notes back from ChatGPT, go to the Examples tab in Entry Point and press the plus button.
Paste both the notes and the polished article into the corresponding Entry Point fields for “notes” and “Blog Post.” There is just one empty field now: “topic.” This is included in the Blog Writer blueprint as another way to steer your content. Write a very quick key phrase for the “topic”— just a couple of words for what the article is about.
Then save it as your first example.
You have to do this process for each of the articles.
It should only take about a minute for each article, so you’ll be done in no time.
Now that you have a dozen or so examples saved, check the token lengths of your articles. Entry Point shows this on the examples tab.
If they are longer than about 4,000 tokens (approximately 3,000 words), they will be excluded from the training data. Chances are you would have already run into this limit when pasting the articles into GPT-4 for summaries, but it’s good to check.
If you want to train models that can produce even longer content, you’ll have to wait for the 8k and 16k context length models to come out later in 2023.
Open your Templates page from the sidebar.
In the prompt, it should have placeholders in double brackets for your “topic” and “notes” fields. In the completion, it should have a placeholder for “blog_post”. Change the display format for your blog post to Markdown.
You can remove the “Blog post:” label in front of the placeholder if you would prefer a cleaner output since it's not necessary.
Remember to press Save.
Now let’s train this AI model!
First, set up your OpenAI integration. Open Integrations from the top nav bar, click OpenAI, and paste your OpenAI key in.
Next, back in your Blog Writer project, go to Fine-tunes in the sidebar and click the plus button:
We’re going to use GPT3.5-Turbo. Look at how cheap that is!
When your model finishes, go to the Playground.
Now you can type a real topic and notes in the prompt area…and generate whole blog posts.
Make sure to set max tokens to 4,000 so your output doesn’t get cut off. I recommend doing one completion at a time, and experimenting with temperature in the range of 0 to 0.8, although you can go up all the way up to 2, just to get crazy. The closer you get to 2, however, the more likely you are to get weird off-the-rails kinds of output.
When you have some notes and settings to try, just press “Get completions”:
What I’ve found to be a solid and intuitive workflow is to write some notes and generate a post, then get ideas from the post and add to and revise my notes. Doing this 3-4 times, I can produce a pretty solid article in about 5 minutes.
Each of these 5-min "solid" articles are ones that would have easily cost me $300-500 if I paid a content writer for them. 🤯
You may have noticed that your blog writer makes up links and images that don’t exist. We can go back through our examples and remove these, or replace them with more generic placeholders like "[image here]." Or, if you like living on the edge, just make sure to replace all the bad references before publishing your article.
Also keep in mind that you should not trust any facts the model states, sources it cites, or quotes it attributes. These are likely made up (hallucinations).
With that in mind, the outputs from these models should be considered rough drafts. They need to be researched, reviewed, and revised before publishing.
Your AI blog writer won’t be good at factual articles like "10 real-life examples of brand activations," but it will be great at general knowledge tasks like "10 benefits of brand activations.”
Also, there are about a million ways you could configure your fields to steer your content. Instead of "topic" and "notes," you could add a dropdown for "category," have an "SEO keywords" list, or provide a complete blog outline in an "outline" field for more steerability and control.
Beyond the fields you can customize, consider if you trained two separate models:
One to generate a blog outline from your rough notes and ideas
Another to turn the outline into a post in your style
Having the separation between these steps would allow you to make modifications before the final article and get even more predictable outputs.
The possibilities are unlimited!
If you can get awesome results with this method using GPT-3.5 Turbo, imagine what it will be like when GPT-4 comes out for fine-tuning this fall. You’ll get an instant upgrade and it will be even more amazing.
At the same time, this is uncharted territory. Think about it: by following this guide, you are one of the first people in the world to have this kind of content-creation power.
There are a lot of unknowns. What does content creation look like in a year from now? Is the internet going to be flooded with these types of articles (is it already)?
At any rate, it's a good time to be an early adopter.
Entry Point provides the tooling that allows you to iterate rapidly with AI content creation and find what works best for your organization.
Thanks for reading. If I literally just gave you a superpower, please subscribe to a paid Entry Point plan to help us create even more guides like this! (yes, this article was written entirely by hand)