Skip to content

Releases: huggingface/transformers

Added two pre-trained models and one new fine-tuning class

30 Nov 22:15
66d50ca
Compare
Choose a tag to compare

This release comprise the following improvements and updates:

  • added two new pre-trained models from Google: bert-large-cased and bert-base-multilingual-cased,
  • added a model that can be fine-tuned for token-level classification: BertForTokenClassification,
  • added tests for every model class, with and without labels,
  • fixed tokenizer loading function BertTokenizer.from_pretrained() when loading from a directory containing a pretrained model,
  • fixed typos in model docstrings and completed the docstrings,
  • improved examples (added do_lower_caseargument).

Small improvements and a few bug fixes.

26 Nov 09:57
Compare
Choose a tag to compare

Improvement:

  • Added a cache_dir option to from_pretrained() function to select a specific path to download and cache the pre-trained model weights. Useful for distributed training (see readme) (fix issue #44).

Bug fixes in model training and tokenizer loading:

  • Fixed error in CrossEntropyLoss reshaping (issue #55).
  • Fixed unicode error in vocabulary loading (issue #52).

Bug fixes in examples:

  • Fix weight decay in examples (previously bias and layer norm weights were also decayed due to an erroneous check in training loop).
  • Fix fp16 grad norm is None error in examples (issue #43).

Updated readme and docstrings

First release

17 Nov 11:23
Compare
Choose a tag to compare

This is the first release of pytorch_pretrained_bert.