PEFT

DeepSpeed support (Experimental)

PyTorch Fully Sharded Data Parallel (FSDP) support (Experimental)

https://huggingface.co/docs/transformers/tokenizer_summary#subword-tokenization

https://huggingface.co/learn/nlp-course/en/chapter6/5