How an LLM is built: the 3 steps explained by Guillaume Lample

Calling all engineers!

Do you know what it takes to build, tune and operate an LLM?

Learn the basics of LLM / GenAI with Mistral AI co-founder: Guillaume Lample

Guillaume shares the basic principles, the hurdles they encountered and the techniques they used to overcome them.

He breaks down the LLM into 3 parts:

1. Pretraining

Build a base of language understanding from a super large text dataset (the web++).

• Task: The model learns to predict what word comes next by analyzing huge amounts of text data.
• How: The model learns by reading through huge amounts of text, using powerful computers to process all this information.
• Dataset: Raw text, containing trillions of tokens - huge!
• Training time: Weeks to months.

2. Instruction tuning

Teach the model to follow and understand tasks and instructions.

• Task: Train the model to predict next words based on the specific prompts and responses.
• How: The model learns by practicing with examples of prompts and their correct responses.
• Dataset: Paired (prompt, response) data with 10k-100k instructions.
• Training time: Few hours to days.

3. Learning from human feedback

Align the model's behavior with human preferences.

• Task: Optimize model responses using techniques like RLHF (Reinforcement Learning with Human Feedback) or DPO (Direct Preference Optimization).
• How: Use human preference data and iterative feedback to adjust the model's outputs.
• Dataset: Human-labeled preference data, typically 10k-100k feedback samples.
• Training time: Few hours to days.

Now you know how an LLM works, ready to build your own?