Releases: huggingface/transformers
Releases · huggingface/transformers
Added two pre-trained models and one new fine-tuning class
This release comprise the following improvements and updates:
- added two new pre-trained models from Google:
bert-large-cased
andbert-base-multilingual-cased
, - added a model that can be fine-tuned for token-level classification:
BertForTokenClassification
, - added tests for every model class, with and without labels,
- fixed tokenizer loading function
BertTokenizer.from_pretrained()
when loading from a directory containing a pretrained model, - fixed typos in model docstrings and completed the docstrings,
- improved examples (added
do_lower_case
argument).
Small improvements and a few bug fixes.
Improvement:
- Added a
cache_dir
option tofrom_pretrained()
function to select a specific path to download and cache the pre-trained model weights. Useful for distributed training (see readme) (fix issue #44).
Bug fixes in model training and tokenizer loading:
- Fixed error in CrossEntropyLoss reshaping (issue #55).
- Fixed unicode error in vocabulary loading (issue #52).
Bug fixes in examples:
- Fix weight decay in examples (previously bias and layer norm weights were also decayed due to an erroneous check in training loop).
- Fix fp16 grad norm is None error in examples (issue #43).
Updated readme and docstrings
First release
This is the first release of pytorch_pretrained_bert
.