Build a Large Language Model (From Scratch)
Original price was: $59.99.$56.99Current price is: $56.99.
Learn how to create, train, and tweak large language models (LLMs) by building one from the ground up!
In Build a Large Language Model (from Scratch) bestselling author Sebastian Raschka guides you step by step through creating your own LLM. Each stage is explained with clear text, diagrams, and examples. You’ll go from the initial design and creation, to pretraining on a general corpus, and on to fine-tuning for specific tasks.
Build a Large Language Model (from Scratch) teaches you how to:
• Plan and code all the parts of an LLM
• Prepare a dataset suitable for LLM training
• Fine-tune LLMs for text classification and with your own data
• Use human feedback to ensure your LLM follows instructions
• Load pretrained weights into an LLM
Build a Large Language Model (from Scratch) takes you inside the AI black box to tinker with the internal systems that power generative AI. As you work through each key stage of LLM creation, you’ll develop an in-depth understanding of how LLMs work, their limitations, and their customization methods. Your LLM can be developed on an ordinary laptop, and used as your own personal assistant.
Purchase of the print book includes a free eBook in PDF and ePub formats from Manning Publications.
About the technology
Physicist Richard P. Feynman reportedly said, “I don’t understand anything I can’t build.” Based on this same powerful principle, bestselling author Sebastian Raschka guides you step by step as you build a GPT-style LLM that you can run on your laptop. This is an engaging book that covers each stage of the process, from planning and coding to training and fine-tuning.
About the book
Build a Large Language Model (From Scratch) is a practical and eminently-satisfying hands-on journey into the foundations of generative AI. Without relying on any existing LLM libraries, you’ll code a base model, evolve it into a text classifier, and ultimately create a chatbot that can follow your conversational instructions. And you’ll really understand it because you built it yourself!
What’s inside
• Plan and code an LLM comparable to GPT-2
• Load pretrained weights
• Construct a complete training pipeline
• Fine-tune your LLM for text classification
• Develop LLMs that follow human instructions
About the reader
Readers need intermediate Python skills and some knowledge of machine learning. The LLM you create will run on any modern laptop and can optionally utilize GPUs.
About the author
Sebastian Raschka is a Staff Research Engineer at Lightning AI, where he works on LLM research and develops open-source software.
The technical editor on this book was David Caswell.
Table of Contents
1 Understanding large language models
2 Working with text data
3 Coding attention mechanisms
4 Implementing a GPT model from scratch to generate text
5 Pretraining on unlabeled data
6 Fine-tuning for classification
7 Fine-tuning to follow instructions
A Introduction to PyTorch
B References and further reading
C Exercise solutions
D Adding bells and whistles to the training loop
E Parameter-efficient fine-tuning with LoRA
From the Publisher
“The most understandable and comprehensive
explanation of language models yet! Its unique
and practical teaching style achieves a level of
understanding you can’t get any other way.”
Cameron Wolfe, Senior Scientist, Netflix
“Sebastian combines deep knowledge with
practical engineering skills and a knack for
making complex ideas simple. This is the guide
you need!”
Chip Huyen, author of Designing Machine
Learning Systems and AI Engineering
“Definitive, up-to-date coverage. Highly
recommended!”
Dr. Vahid Mirjalili, Senior Data Scientist, FM
Global
about the book
Build a Large Language Model (From Scratch) is a practical and eminently-satisfying hands-on
journey into the foundations of generative AI. Without relying on any existing LLM libraries, you’ll
code a base model, evolve it into a text classifier, and ultimately create a chatbot that can follow
your conversational instructions. And you’ll really understand it because you built it yourself!
about the author
Sebastian Raschka is a Staff Research Engineer at Lightning AI, where he works on LLM research and
develops open-source software.
The technical editor on this book was David Caswell.
Publisher : Manning (October 29, 2024)
Language : English
Paperback : 368 pages
ISBN-10 : 1633437167
ISBN-13 : 978-1633437166
Item Weight : 1.35 pounds
Dimensions : 7.38 x 0.7 x 9.25 inches
luc beeusaert –
Zeer goed om zelf je eerste LLM te leren maken.
[...]Amazon Customer –
Excellent book
[...]Alessio Chiovelli –
Still reading it, but i’ve seen really good feedbacks from linkedin, and they were not wrong! Clearly expressed, and the concepts are easily explained. Looking forward to read all the chapters!
[...]Amazon Customer –
You get all the details you need to build a very good understanding of this technology. Bravo.
[...]Fabio –
Fui direto a parte em que estava precisando compreender e fiquei surpreso com a didática do autor ao abordar o tema.Continuo no estudo e gostando muito da leitura.
[...]Howard W –
A Comprehensive Guide to Mastering LLMs from Scratch
[...]I thoroughly enjoyed this book. It’s an excellent resource for learning about LLMs from the ground up. The textbook breaks down complex concepts into fundamental principles, making it ideal for both beginners and those looking to deepen their understanding of the latest techniques in the field. Whether you’re starting from scratch or aiming to refine your knowledge, this book offers a comprehensive, step-by-step approach to mastering LLMs
Sridhar Srinivasan –
Best LLM book out there!
[...]“𝐁𝐮𝐢𝐥𝐝 𝐚 𝐋𝐚𝐫𝐠𝐞 𝐋𝐚𝐧𝐠𝐮𝐚𝐠𝐞 𝐌𝐨𝐝𝐞𝐥 (𝐅𝐫𝐨𝐦 𝐒𝐜𝐫𝐚𝐭𝐜𝐡)” – An ABSOLUTE must-have for those who have a “how-things-work?” mindset. The flow of the book from data preparation through fine-tuning is awesome. Better to have a quick refresher on the PyTorch before delving into the chapters.
BRIAN –
Great Tutorial
[...]What an amazing book detailing how each component of the language models components fit together and work synchronously. It is not too difficult to read / follow along if you have previous coding experience with Neural Networks and PyTorch on Machine learning projects. It definitely was a great purchase to understand what it takes to build a local LLM. I had to remove 1 star because the book already tore a bit on the front cover on day 3 of reading.
TanelP –
Excellent book with practical focus
[...]This book was perfect for me. I’m a computer performance specialist, but haven’t yet gotten serious about ML and language models. I’ve read occasional overview articles, so have an idea what things like “vectors” and “matrix multiplication” are, but I didn’t see the full picture. I had bought some other machine learning books before that tried to cover everything about everything and never got even half-way through reading them. This book covers not only the practical examples (and source code) with all the steps for training your own toy language models (Python/pytorch code), but also it explained how all the training layers work together in unison. On the training architecture topic, this book did a better job in a handful of pages than all the deep papers I had read in the past, so I should probably have started from this book, not the other way around.Also, the book does a good job incrementally building the knowledge by adding a new layer after another as you progress through the book. Highly recommended!
Allan Tan –
A Comprehensive and In Depth LLM Book
[...]This is the most in-depth book about LLMs. If you want to understand how transformers work, including every layer in their architecture, this book explains everything in detail.For example, concepts like vanishing gradients and ReLU activation are thoroughly explained, providing the mathematical intuition behind the algorithms. The appendix, spanning 80 pages, delves deeper into PyTorch and neural network concepts for additional support.The book also includes example code covering byte pair encoding, attention mechanisms, and even direct preference optimization.Im happy to have bought and read this book as this gives me better intuition how the transformer model works and how to improve llm performance with fine tuning.
Mr Mike –
Very thorough with excellent explanations
[...]I worked with AI including a patent for Bayesian based diagnostics. This is very thorough and provides a good evaluation of what it will take to implement a complete LLM package.
Rathi –
Beginner friendly LLM starter
[...]Fantastic book for a beginner. Attention mechanism and other complex constructs explained in an easy to understand illustrated manner without dumbing down too much. Special mention to the prod ready code that comes with the book. Best book till date on LLMs.
Omid B. –
Awesome book to learn about LLMs
[...]The book dives deep into the fascinating world of language models, walking you through every step—from designing and coding components to training and fine-tuning models for specialized tasks. What stood out most to me is its practical, hands-on approach, breaking down complex ideas into something manageable—even for those experimenting on a personal laptop. 💻I highly recommend this read.