TrainLoop中resume_checkpoint一般怎么设置举个例子

在 TrainLoop 中，`resume_checkpoint` 用于从已保存的模型检查点中恢复训练。下面是一个使用 `resume_checkpoint` 的示例：

import torch
from transformers import GPT2LMHeadModel, GPT2Tokenizer
from transformers.optimization import AdamW
from transformers.trainer import Trainer, TrainingArguments
from torch.utils.data import DataLoader, Dataset

class MyDataset(Dataset):
    def __init__(self, tokenizer, data_path):
        self.tokenizer = tokenizer
        self.examples = []

        with open(data_path, "r", encoding="utf-8") as f:
            for line in f:
                line = line.strip()
                if len(line) > 0:
                    self.examples.append(line)

    def __len__(self):
        return len(self.examples)

    def __getitem__(self, idx):
        return self.tokenizer.encode(self.examples[idx], add_special_tokens=True)

tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
train_dataset = MyDataset(tokenizer, "train.txt")
train_loader = DataLoader(train_dataset, batch_size=4, shuffle=True)

model = GPT2LMHeadModel.from_pretrained("gpt2")
optimizer = AdamW(model.parameters(), lr=1e-5)

training_args = TrainingArguments(
    output_dir="./checkpoints",
    num_train_epochs=3,
    per_device_train_batch_size=4,
    save_steps=1000,
    save_total_limit=2,
    fp16=True,
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    data_collator=lambda data: {"input_ids": torch.stack(data)},
    optimizer=optimizer,
)

trainer.train()

以上代码将使用 GPT-2 模型来训练一个文本生成器。训练开始后，训练过程中每隔 1000 步就会保存一次模型检查点。假设在训练过程中需要停止，然后再次从检查点恢复训练，可以使用以下代码：

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    data_collator=lambda data: {"input_ids": torch.stack(data)},
    optimizer=optimizer,
    resume_from_checkpoint="./checkpoints/checkpoint-1000",
)

trainer.train()

以上代码中，`resume_from_checkpoint` 设置为 `./checkpoints/checkpoint-1000`，表示从保存的第 1000 步检查点处恢复训练。注意，`resume_from_checkpoint` 的值应该是一个存在的检查点路径。