8.1 C
Innichen
Sunday, June 6, 2021

Natural Language Processing with Transformers in Python

Learn next-generation NLP with transformers using PyTorch, TensorFlow, and HuggingFace!
Instructor:
James Briggs
4,753 students enrolled
English [Auto]
How to use transformer models for NLP
Modern natural language processing technologies
An overview of recent development in NLP
Python
Machine Learning
Natural Language Processing
Tensorflow
PyTorch
Transformers
Sentiment Analysis
Question and Answering
Named Entity Recognition

Transformer models are the de-facto standard in modern NLP. They have proven themselves as the most expressive, powerful models for language by a large margin, beating all major language-based benchmarks time and time again.

In this course, we learn all you need to know to get started with building cutting-edge performance NLP applications using transformer models like Google AI’s BERT, or Facebook AI’s DPR.

We cover several key NLP frameworks including:

  • HuggingFace’s Transformers

  • TensorFlow 2

  • PyTorch

  • spaCy

  • NLTK

  • Flair

And learn how to apply transformers to some of the most popular NLP use-cases:

  • Language classification/sentiment analysis

  • Named entity recognition (NER)

  • Question and Answering

  • Similarity/comparative learning

Throughout each of these use-cases we work through a variety of examples to ensure that what, how, and why transformers are so important. Alongside these sections we also work through two full-size NLP projects, one for sentiment analysis of financial Reddit data, and another covering a fully-fledged open domain question-answering application.

All of this is supported by several other sections that encourage us to learn how to better design, implement, and measure the performance of our models, such as:

  • History of NLP and where transformers come from

  • Common preprocessing techniques for NLP

  • The theory behind transformers

  • How to fine-tune transformers

We cover all this and more, I look forward to seeing you in the course!

Introduction

1
Introduction

A brief introduction to the course, and how to get the most out of it.

2
Course Overview

An overview of everything we'll be covering in this course.

3
Environment Setup

How to setup a local Python environment that aligns to the environment used throughout the course.

4
CUDA Setup

How to setup CUDA for CUDA enabled GPUs.

NLP and Transformers

1
The Three Eras of AI
2
Pros and Cons of Neural AI
3
Word Vectors
4
Recurrent Neural Networks
5
Long Short-Term Memory
6
Encoder-Decoder Attention
7
Self-Attention
8
Multi-head Attention
9
Positional Encoding
10
Transformer Heads

Preprocessing for NLP

1
Stopwords

Here we'll start with our first NLP preprocessing technique, how we can use stopwords.

2
Tokens Introduction

In the first part of exploration of tokens in NLP, we'll look at word, character, punctuation, part-of-word tokens and more.

3
Model-Specific Special Tokens

In the second part of tokens in NLP, we'll look at model-specific special tokens.

4
Stemming

We take a look and the Porter and Lancaster stemmers.

5
Lemmatization

Here we take a look at reducing words to their lemma roots.

6
Unicode Normalization - Canonical and Compatibility Equivalence

Here we will introduce Unicode Normalization and the two forms of equivalence, canonical and compatibility.

7
Unicode Normalization - Composition and Decomposition

Here we take a look at the two different directions in Unicode Normalization, composition, and decomposition.

8
Unicode Normalization - NFD and NFC

We'll move onto applying Unicode Normalization in Python with both NFD and NFC forms.

9
Unicode Normalization - NFKD and NFKC

In the final Unicode Normalization session, we'll learn about and implement NFKD and NFKC forms.

Attention

1
Attention Introduction
2
Alignment With Dot-Product
3
Dot-Product Attention
4
Self Attention
5
Bidirectional Attention
6
Multi-head and Scaled Dot-Product Attention

Language Classification

1
Introduction to Sentiment Analysis
2
Prebuilt Flair Models
3
Introduction to Sentiment Models With Transformers
4
Tokenization And Special Tokens For BERT
5
Making Predictions

[Project] Sentiment Model With TensorFlow and Transformers

1
Project Overview
2
Getting the Data (Kaggle API)
3
Preprocessing
4
Building a Dataset
5
Dataset Shuffle, Batch, Split, and Save
6
Build and Save
7
Loading and Prediction

Long Text Classification With BERT

1
Classification of Long Text Using Windows

In this session we'll work through an example of using windows to calculate sentiment for longer pieces of text.

2
Window Method in PyTorch

Named Entity Recognition (NER)

1
Introduction to spaCy
2
Extracting Entities
3
NER Walkthrough
4
Authenticating With The Reddit API
5
Pulling Data With The Reddit API
6
Extracting ORGs From Reddit Data
7
Getting Entity Frequency
8
Entity Blacklist
9
NER With Sentiment
10
NER With roBERTa

Question and Answering

1
Open Domain and Reading Comprehension

An introduction to the two modes of Q&A, open domain (OD) and reading comprehension (RC).

2
Retrievers, Readers, and Generators

An introduction to the three key model types we will be using in Q&A, retrievers, readers, and generators.

3
Intro to SQuAD 2.0

We introduce the SQuAD Q&A dataset.

4
Processing SQuAD Training Data

How we process the SQuAD data to be more friendly for our use-case.

5
(Optional) Processing SQuAD Training Data with Match-Case

We take a look at refactoring our SQuAD processing data using pattern matching syntax introduced in Python 3.10.

6
Processing SQuAD Dev Data
7
Our First Q&A Model

We put together our first Q&A model.

Metrics For Language

1
Q&A Performance With Exact Match (EM)
2
ROUGE in Python

Learn how to implement ROUGE scores using Python.

3
Applying ROUGE to Q&A

We take a look at applying ROUGE performance metrics to our first Q&A model.

4
Recall, Precision and F1

We work through the intuition and mathematics behind ROUGE-N.

5
Longest Common Subsequence (LCS)

We work through the intuition and mathematics behind ROUGE-L.

6
Q&A Performance With ROUGE

We introduce the ROUGE metrics.

Reader-Retriever QA With Haystack

1
Intro to Retriever-Reader and Haystack

A quick introduction to the retriever-reader architecture and Haystack.

2
What is Elasticsearch?

An quick overview of how Elasticsearch works, and why we use it.

3
Elasticsearch Setup (Windows)
4
Elasticsearch Setup (Linux)
5
Elasticsearch in Haystack

We introduce Elasticsearch via Haystack.

6
Sparse Retrievers
7
Cleaning the Index
8
Implementing a BM25 Retriever
9
What is FAISS?
10
FAISS in Haystack
11
What is DPR?
12
The DPR Architecture
13
Retriever-Reader Stack

[Project] Open-Domain QA

1
ODQA Stack Structure
2
Creating the Database
3
Building the Haystack Pipeline

Similarity

1
Introduction to Similarity
2
Extracting The Last Hidden State Tensor
3
Sentence Vectors With Mean Pooling
4
Using Cosine Similarity
5
Similarity With Sentence-Transformers
You can view and review the lecture materials indefinitely, like an on-demand channel.
Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don't have an internet connection, some instructors also let their students download course lectures. That's up to the instructor though, so make sure you get on their good side!

Be the first to add a review.

Please, login to leave a review
30-Day Money-Back Guarantee

Includes

11 hours on-demand video
Full lifetime access
Access on mobile and TV
Certificate of Completion