When ChatGPT launched in late 2022, it awakened the world to the transformative potential of artificial intelligence (AI). The technology underpinning the powerful new chatbot is one of the biggest step changes in the history of AI—rather than simply analyzing or classifying existing data, generative AI is able to create something entirely new, including text, images, audio, synthetic data and more. Across business, science and society itself, it will enable groundbreaking human creativity and productivity.
From ChatGPT to DALL-E, the latest class of generative AI applications has emerged from foundation models, which are complex machine learning systems trained on vast quantities of data (text, images, audio or a mix of data types) on a massive scale. With recent advances, companies can now build specialized image- and language-generating models on top of these foundation models. Most of today’s foundation models are large language models (LLMs) trained on natural language.
The power of these systems lies not only in their size, but also in the fact that they can be adapted quickly for a wide range of downstream tasks without needing task-specific training. In zero-shot learning, the model uses a general understanding of the relationship between different concepts to make predictions and does not use any specific examples. In-context learning builds on this capability, whereby a model can be prompted to generate novel responses on topics that it has not seen during training using examples within the prompt itself. In-context learning techniques include one-shot learning, which is a technique where the model is primed to make predictions with a single example. In few-shot learning, the model is primed with a small number of examples and is then able to generate responses in the unseen domain.
The introduction of pre-trained foundation models with unprecedented adaptability to new tasks will have far-reaching consequences. According to Accenture’s 2023 Technology Vision report, 97% of global executives agree that foundation models will enable connections across data types, revolutionizing where and how AI is used. To operate in tomorrow’s market, businesses will need to lean on the full capabilities that generative AI provides.