Fine-tuning a BERT model for text extraction with the SQuAD dataset
We are going to fine-tune BERT (from HuggingFace Transformers) for the text-extraction task with a dataset of questions and answers. The questions are about a given paragraph (context) that contains the answers. The model will be trained to locate the answer in the context by giving the possitions where the answer starts and finishes.
This example is based on BERT (from HuggingFace Transformers) for Text Extraction.
Before running the python scripts, it's necessary do the following exports:
export NCCL_DEBUG=INFO
export NCCL_IB_HCA=ipogif0
export NCCL_IB_CUDA_SUPPORT=1