注意

点击此处下载完整的示例代码

使用 torchtext 库进行文本分类¶

在本教程中，我们将展示如何使用 torchtext 库构建用于文本分类分析的数据集。用户可以灵活地：

访问原始数据作为迭代器

构建数据处理管道，将原始文本字符串转换为可用于训练模型的 torch.Tensor

使用 torch.utils.data.DataLoader 打乱和迭代数据

先决条件¶

在运行本教程之前，需要安装最新版本的 2.x portalocker 包。例如，在 Colab 环境中，可以通过在脚本顶部添加以下行来完成此操作：

!pip install -U portalocker>=2.0.0`

访问原始数据集迭代器¶

torchtext 库提供了一些原始数据集迭代器，这些迭代器会生成原始文本字符串。例如，AG_NEWS 数据集迭代器会将原始数据作为标签和文本的元组生成。

要访问 torchtext 数据集，请按照 https://github.com/pytorch/data 上的说明安装 torchdata。

import torch
from torchtext.datasets import AG_NEWS

train_iter = iter(AG_NEWS(split="train"))

next(train_iter)
>>> (3, "Fears for T N pension after talks Unions representing workers at Turner
Newall say they are 'disappointed' after talks with stricken parent firm Federal
Mogul.")

next(train_iter)
>>> (4, "The Race is On: Second Private Team Sets Launch Date for Human
Spaceflight (SPACE.com) SPACE.com - TORONTO, Canada -- A second\\team of
rocketeers competing for the  #36;10 million Ansari X Prize, a contest
for\\privately funded suborbital space flight, has officially announced
the first\\launch date for its manned rocket.")

next(train_iter)
>>> (4, 'Ky. Company Wins Grant to Study Peptides (AP) AP - A company founded
by a chemistry researcher at the University of Louisville won a grant to develop
a method of producing better peptides, which are short chains of amino acids, the
building blocks of proteins.')

准备数据处理管道¶

我们回顾了 torchtext 库中最基本的一些组件，包括词汇表、词向量、分词器。这些是用于处理原始文本字符串的基本数据处理构建块。

这是一个使用分词器和词汇表进行典型 NLP 数据处理的示例。第一步是使用原始训练数据集构建词汇表。这里我们使用内置的工厂函数 build_vocab_from_iterator，它接受迭代器，该迭代器会生成标记列表或标记迭代器。用户还可以传递任何要添加到词汇表中的特殊符号。

from torchtext.data.utils import get_tokenizer
from torchtext.vocab import build_vocab_from_iterator

tokenizer = get_tokenizer("basic_english")
train_iter = AG_NEWS(split="train")


def yield_tokens(data_iter):
    for _, text in data_iter:
        yield tokenizer(text)


vocab = build_vocab_from_iterator(yield_tokens(train_iter), specials=["<unk>"])
vocab.set_default_index(vocab["<unk>"])

词汇表块将标记列表转换为整数。

vocab(['here', 'is', 'an', 'example'])
>>> [475, 21, 30, 5297]

使用分词器和词汇表准备文本处理管道。文本和标签管道将用于处理来自数据集迭代器的原始数据字符串。

text_pipeline = lambda x: vocab(tokenizer(x))
label_pipeline = lambda x: int(x) - 1

文本管道根据词汇表中定义的查找表将文本字符串转换为整数列表。标签管道将标签转换为整数。例如，

text_pipeline('here is the an example')
>>> [475, 21, 2, 30, 5297]
label_pipeline('10')
>>> 9

生成数据批次和迭代器¶

对于 PyTorch 用户，建议使用 torch.utils.data.DataLoader（教程在此处）。它适用于实现 getitem() 和 len() 协议的映射风格数据集，并表示从索引/键到数据样本的映射。它也适用于可迭代数据集，其中 DataLoader 的 shuffle 参数为 False。

在发送到模型之前，collate_fn 函数会处理 DataLoader 生成的样本批次。 collate_fn 的输入是具有 DataLoader 中批次大小的数据批次，并且 collate_fn 会根据之前声明的数据处理管道对其进行处理。请注意此处，并确保 collate_fn 声明为顶级 def。这确保了该函数在每个工作程序中都可用。

在此示例中，原始数据批次输入中的文本条目被打包到一个列表中，并连接成单个张量，作为 nn.EmbeddingBag 的输入。偏移量是一个分隔符张量，用于表示文本张量中各个序列的起始索引。标签是一个张量，保存各个文本条目的标签。

from torch.utils.data import DataLoader

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")


def collate_batch(batch):
    label_list, text_list, offsets = [], [], [0]
    for _label, _text in batch:
        label_list.append(label_pipeline(_label))
        processed_text = torch.tensor(text_pipeline(_text), dtype=torch.int64)
        text_list.append(processed_text)
        offsets.append(processed_text.size(0))
    label_list = torch.tensor(label_list, dtype=torch.int64)
    offsets = torch.tensor(offsets[:-1]).cumsum(dim=0)
    text_list = torch.cat(text_list)
    return label_list.to(device), text_list.to(device), offsets.to(device)


train_iter = AG_NEWS(split="train")
dataloader = DataLoader(
    train_iter, batch_size=8, shuffle=False, collate_fn=collate_batch
)

定义模型¶

该模型由 nn.EmbeddingBag 层和用于分类目的的线性层组成。nn.EmbeddingBag 使用默认模式“mean”计算嵌入“包”的平均值。尽管此处的文本条目长度不同，但 nn.EmbeddingBag 模块在此处不需要填充，因为文本长度保存在偏移量中。

此外，由于 nn.EmbeddingBag 会动态累积嵌入的平均值，因此 nn.EmbeddingBag 可以提高处理张量序列的性能和内存效率。

../_images/text_sentiment_ngrams_model.png

from torch import nn


class TextClassificationModel(nn.Module):
    def __init__(self, vocab_size, embed_dim, num_class):
        super(TextClassificationModel, self).__init__()
        self.embedding = nn.EmbeddingBag(vocab_size, embed_dim, sparse=False)
        self.fc = nn.Linear(embed_dim, num_class)
        self.init_weights()

    def init_weights(self):
        initrange = 0.5
        self.embedding.weight.data.uniform_(-initrange, initrange)
        self.fc.weight.data.uniform_(-initrange, initrange)
        self.fc.bias.data.zero_()

    def forward(self, text, offsets):
        embedded = self.embedding(text, offsets)
        return self.fc(embedded)

初始化实例¶

AG_NEWS 数据集有四个标签，因此类数为四个。

: World
: Sports
: Business
: Sci/Tec

我们构建一个嵌入维度为 64 的模型。词汇表大小等于词汇表实例的长度。类数等于标签数，

train_iter = AG_NEWS(split="train")
num_class = len(set([label for (label, text) in train_iter]))
vocab_size = len(vocab)
emsize = 64
model = TextClassificationModel(vocab_size, emsize, num_class).to(device)

定义训练模型和评估结果的函数。¶

import time


def train(dataloader):
    model.train()
    total_acc, total_count = 0, 0
    log_interval = 500
    start_time = time.time()

    for idx, (label, text, offsets) in enumerate(dataloader):
        optimizer.zero_grad()
        predicted_label = model(text, offsets)
        loss = criterion(predicted_label, label)
        loss.backward()
        torch.nn.utils.clip_grad_norm_(model.parameters(), 0.1)
        optimizer.step()
        total_acc += (predicted_label.argmax(1) == label).sum().item()
        total_count += label.size(0)
        if idx % log_interval == 0 and idx > 0:
            elapsed = time.time() - start_time
            print(
                "| epoch {:3d} | {:5d}/{:5d} batches "
                "| accuracy {:8.3f}".format(
                    epoch, idx, len(dataloader), total_acc / total_count
                )
            )
            total_acc, total_count = 0, 0
            start_time = time.time()


def evaluate(dataloader):
    model.eval()
    total_acc, total_count = 0, 0

    with torch.no_grad():
        for idx, (label, text, offsets) in enumerate(dataloader):
            predicted_label = model(text, offsets)
            loss = criterion(predicted_label, label)
            total_acc += (predicted_label.argmax(1) == label).sum().item()
            total_count += label.size(0)
    return total_acc / total_count

分割数据集并运行模型¶

由于原始 AG_NEWS 没有有效数据集，因此我们将训练数据集以 0.95（训练）和 0.05（验证）的分割比例分割为训练/验证集。这里我们使用 PyTorch 核心库中的 torch.utils.data.dataset.random_split 函数。

CrossEntropyLoss 标准将 nn.LogSoftmax() 和 nn.NLLLoss() 组合到一个类中。当训练具有 C 个类的分类问题时，它非常有用。SGD 将随机梯度下降法实现为优化器。初始学习率设置为 5.0。StepLR 用于在此处通过 epoch 调整学习率。

from torch.utils.data.dataset import random_split
from torchtext.data.functional import to_map_style_dataset

# Hyperparameters
EPOCHS = 10  # epoch
LR = 5  # learning rate
BATCH_SIZE = 64  # batch size for training

criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(model.parameters(), lr=LR)
scheduler = torch.optim.lr_scheduler.StepLR(optimizer, 1.0, gamma=0.1)
total_accu = None
train_iter, test_iter = AG_NEWS()
train_dataset = to_map_style_dataset(train_iter)
test_dataset = to_map_style_dataset(test_iter)
num_train = int(len(train_dataset) * 0.95)
split_train_, split_valid_ = random_split(
    train_dataset, [num_train, len(train_dataset) - num_train]
)

train_dataloader = DataLoader(
    split_train_, batch_size=BATCH_SIZE, shuffle=True, collate_fn=collate_batch
)
valid_dataloader = DataLoader(
    split_valid_, batch_size=BATCH_SIZE, shuffle=True, collate_fn=collate_batch
)
test_dataloader = DataLoader(
    test_dataset, batch_size=BATCH_SIZE, shuffle=True, collate_fn=collate_batch
)

for epoch in range(1, EPOCHS + 1):
    epoch_start_time = time.time()
    train(train_dataloader)
    accu_val = evaluate(valid_dataloader)
    if total_accu is not None and total_accu > accu_val:
        scheduler.step()
    else:
        total_accu = accu_val
    print("-" * 59)
    print(
        "| end of epoch {:3d} | time: {:5.2f}s | "
        "valid accuracy {:8.3f} ".format(
            epoch, time.time() - epoch_start_time, accu_val
        )
    )
    print("-" * 59)

使用测试数据集评估模型¶

检查测试数据集的结果…

print("Checking the results of test dataset.")
accu_test = evaluate(test_dataloader)
print("test accuracy {:8.3f}".format(accu_test))

测试随机新闻¶

使用迄今为止最好的模型并测试高尔夫新闻。

ag_news_label = {1: "World", 2: "Sports", 3: "Business", 4: "Sci/Tec"}


def predict(text, text_pipeline):
    with torch.no_grad():
        text = torch.tensor(text_pipeline(text))
        output = model(text, torch.tensor([0]))
        return output.argmax(1).item() + 1


ex_text_str = "MEMPHIS, Tenn. – Four days ago, Jon Rahm was \
    enduring the season’s worst weather conditions on Sunday at The \
    Open on his way to a closing 75 at Royal Portrush, which \
    considering the wind and the rain was a respectable showing. \
    Thursday’s first round at the WGC-FedEx St. Jude Invitational \
    was another story. With temperatures in the mid-80s and hardly any \
    wind, the Spaniard was 13 strokes better in a flawless round. \
    Thanks to his best putting performance on the PGA Tour, Rahm \
    finished with an 8-under 62 for a three-stroke lead, which \
    was even more impressive considering he’d never played the \
    front nine at TPC Southwind."

model = model.to("cpu")

print("This is a %s news" % ag_news_label[predict(ex_text_str, text_pipeline)])

脚本的总运行时间：（0 分钟 0.000 秒）

下载 Python 源代码： text_sentiment_ngrams_tutorial.py

下载 Jupyter 笔记本： text_sentiment_ngrams_tutorial.ipynb

由 Sphinx-Gallery 生成的图库