Automated text generation is generating huge buzz already in 2023. Oliwia Lewandowska, Director of Data Science, Looping Group cuts through the hype and explains the implications of new AI content tools for marketers.
The applications of artificial intelligence and machine learning for the marketing industry have been the stuff of numerous conference presentations and comment pieces over the past few years. But there is an area AI that has been draining attention in the tech press in recent months that opens up a whole new area of opportunity for brands.
Language processing, or automated text generation, has been around for a while, but recent technological advances have taken this developing area of AI to mainstream availability, offering users impressive quality, reliability, ease and speed. And this is getting a lot of tech companies very excited, with large budgets shifting into this area. A survey by natural language processing company John Snow Labs found that 60% of tech leaders have increased their budgets for AI language technologies by 10% since 2020. And Google recently announced a new project to develop a single AI language model that supports the 1000 most supported languages.
The opportunity for brands is to leverage AI to drive sustainable efficiencies in a traditionally costly and time-consuming area of marketing: content creation. News pieces, blogs posts and marketing copy can all now be produced using AI in a way that, until recently, simply wasn’t possible.
But as content has such a pivotal impact on brand reputation, most marketers will want to understand the technology in order to make an informed decision. So let’s look at the recent tech advances that have enabled language processing to become part of the marketing toolkit.
The technology that is behind all of this is neural network-based language models, or more simply, large language models. Trained on enormous amounts of text data, large language models are tens of gigabytes in size and. There are several large language models that have taken this field forward in recent months, most significantly Gopher and GPT-3.
Gopher has been developed by Google-owned DeepMind. The model has been trained on an eye-watering 280 billion parameters and has been used to beat grandmasters at chess. However, it is arguably GPT-3 that is really turning the dial in large language models.
GPT-3 (Generative Pre-trained Transformer 3) was created by OpenAI, which was founded by Elon Musk. It has 175 billions parameters and is a deep learning prediction model, which means it can take input and generate a sequence of text based on its prediction of what would be more useful of relevant. GPT-3 can perform a wide variety of natural language tasks, including text generation, classification (i.e. sentiment analysis), question answering, text summarisation, and language translation. GPT-3 is very sensitive to the input text or ‘prompts’ – so the better the input, the better the output.
And the power and potential of GPT-3 was highlighted at the end of 2022 with the launch of the chatbot ChatGPT. While it’s not more technically advanced than GPT-3, its immediate popularity showed that conversing in natural language is now a mainstream phenomenon. Within just five days of its launch, ChatGPT broke the one million user mark, with thousands of people sharing their ChatGPT prompts on Linkedin and Twitter.
The other important development in AI generated text is fine-tuned models. These are generally smaller than their large language counterparts and can improve a model’s ability to perform a task. This is important for marketers as it means that text can be fine-tuned to the language, tone and voice of a brand, and can avoid a near endless list of words and phrases that are deemed off-brand. This is a game-changing advance for the marketing industry. And because fine-tuned models are derived from existing language models, they don’t take nearly as much time to train or run.
Large language model technology is already producing exciting results in the arts and creative fields. After being fed all seven Potter tales, a large language model came up with the first chapter a new Harry Potter story. And a US script writer has been garnering interest with an AI language program that he is using to generate film scripts. One screenplay was created by inputting all the characters ever played by David Hasselhoff.
While it’s early days for marketing, the potential is huge. With marketing teams more stretched than ever and budgets under pressure, large language model-based solutions will simply make life easier for brands. Nobody is expecting AI to completely replace human content creation. But there are many situations where it is easier, faster and more efficient to use auto-generated text. And as we move forward, human and machine will work in greater collaboration, so it’s not an either/or but a harmonious relationship. In this way the technology becomes part of the marketing team, not just another platform.
Advances in large language modelling – and the development of GPT-3 – have seen a flurry of start-ups offering easy-to-use online AI solutions for content generation. These learn marketing data very quickly, incorporate the language and style of the brand and produce accurate content with rapid turnaround times. Some of these solutions are able to cut editorial costs by up to 80%.
Clearly all AI solutions require a level of human input and oversight. Text needs to be checked, inputs need to be tested and evolved, and fine-tuning requires intelligent decision making. And human involvement is essential to monitor for bias. Language models are optimised to mirror language systems, so it stands to reason that they can perpetuate stereotypes and biases hardwired into natural language. Meta was recently called out when it was discovered that its open-source large language model to help scientists, Galactica, could be prompted to provide ‘scientific research’ on the benefits of homophobia, antisemitism, suicide etc.
While the success of large language technology is clearly dependent on the inputs, what you can achieve with a very small amount of manual work at the outset is truly phenomenal. And you can continue to produce high-quality, brand-appropriate content time and again, with what is a very easy-to-use solution.
