A Large Language Model (LLM) is a deep-learning algorithm that uses massive amounts of parameters and training data to understand and predict text. This model can perform various natural language processing tasks beyond simple, probabilistic text generation, including content revision and translation.
Note that in machine learning, deep learning is a technique that uses multilayered neural networks to perform complex tasks such as classification, regression, and representation learning. Hence, LLMs are neural network-based language models.
The word “large” refers to:
- The parameters, or variables and weights, the model uses to influence the prediction outcome. There is no definition of the number of parameters. However, LLM training datasets typically range in size from 110 million parameters (Google’s BERTbase model) to 340 billion parameters (Google’s PaLM 2 model).
- The amount of data used to train an LLM. It can be multiple petabytes in size and contain trillions of tokens.
A predictive language model predicts a single word, such as the predictive text feature in text-messaging applications. However, an LLM can predict more complex content, such as the most likely multi-paragraph response or translation.
LLM Underlying Mechanism
An LLM is initially trained with textual content. The training process may involve:
- Unsupervised learning — the initial process of establishing connections between unstructured and unlabelled data.
- Supervised learning — the process of tuning the model to allow for more precise analysis.
After the training, LLMs undergo deep learning using neural network models known as transformers. This process converts one type of input into a different kind of output.
Transformers leverage self-attention, which allows LLMs to analyse relationships between tokens in an input and assign them weights to determine relative importance. When you input a prompt, the model uses those weights to predict the most likely textual output.
A few popular types of LLM applications
LLM-Powered Autonomous Agents
In computing, an agent is software (or occasionally a hardware component) that acts autonomously on behalf of a user, another program, or a larger system.
So, in an LLM-powered agent system, LLM functions as the agent’s brain, complemented by several key components: planning, memory, and tool use.
Retrieval Augmented Generation (RAG)
A technique that retrieves relevant documents based on user input and passes them to a language model for processing.
RAG enables AI applications to generate more informed and context-aware responses by leveraging external knowledge.
For a detailed explanation, with a flow diagram of how RAW works, see the section below Retrieval Augmented Generation (RAG).
Other applications
- Summarisation — A specific type of free-form writing.
- Classification and Tagging — A method that applies a tag, such as sentiment analysis or toxicity level, to a given input.