Transformer models are the de-facto standard in modern NLP. They have proven themselves as the most expressive, powerful models for language by a large margin, beating all major language-based benchmarks time and time again.
In this course, we learn all you need to know to get started with building cutting-edge performance NLP applications using transformer models like Google AI’s BERT, or Facebook AI’s DPR.
We cover several key NLP frameworks including:
And learn how to apply transformers to some of the most popular NLP use-cases:
Language classification/sentiment analysis
Named entity recognition (NER)
Question and Answering
Throughout each of these use-cases we work through a variety of examples to ensure that what, how, and why transformers are so important. Alongside these sections we also work through two full-size NLP projects, one for sentiment analysis of financial Reddit data, and another covering a fully-fledged open domain question-answering application.
All of this is supported by several other sections that encourage us to learn how to better design, implement, and measure the performance of our models, such as:
History of NLP and where transformers come from
Common preprocessing techniques for NLP
The theory behind transformers
How to fine-tune transformers
We cover all this and more, I look forward to seeing you in the course!
A brief introduction to the course, and how to get the most out of it.
An overview of everything we'll be covering in this course.
How to setup a local Python environment that aligns to the environment used throughout the course.
How to setup CUDA for CUDA enabled GPUs.
Here we'll start with our first NLP preprocessing technique, how we can use stopwords.
In the first part of exploration of tokens in NLP, we'll look at word, character, punctuation, part-of-word tokens and more.
In the second part of tokens in NLP, we'll look at model-specific special tokens.
We take a look and the Porter and Lancaster stemmers.
Here we take a look at reducing words to their lemma roots.
Here we will introduce Unicode Normalization and the two forms of equivalence, canonical and compatibility.
Here we take a look at the two different directions in Unicode Normalization, composition, and decomposition.
We'll move onto applying Unicode Normalization in Python with both NFD and NFC forms.
In the final Unicode Normalization session, we'll learn about and implement NFKD and NFKC forms.
In this session we'll work through an example of using windows to calculate sentiment for longer pieces of text.
An introduction to the two modes of Q&A, open domain (OD) and reading comprehension (RC).
An introduction to the three key model types we will be using in Q&A, retrievers, readers, and generators.
We introduce the SQuAD Q&A dataset.
How we process the SQuAD data to be more friendly for our use-case.
We take a look at refactoring our SQuAD processing data using pattern matching syntax introduced in Python 3.10.
We put together our first Q&A model.
Learn how to implement ROUGE scores using Python.
We take a look at applying ROUGE performance metrics to our first Q&A model.
We work through the intuition and mathematics behind ROUGE-N.
We work through the intuition and mathematics behind ROUGE-L.
We introduce the ROUGE metrics.
A quick introduction to the retriever-reader architecture and Haystack.
An quick overview of how Elasticsearch works, and why we use it.
We introduce Elasticsearch via Haystack.