Skip to main content

NLLB

Conversion​

1. Sentencepiece with OpenNMT Tokenizer​

eole convert HF --model_dir facebook/nllb-200-1.3B --output ./nllb-1.3b --token $HF_TOKEN --tokenizer onmt

2. HuggingFace Tokenizer​

eole convert HF --model_dir facebook/nllb-200-1.3B --output ./nllb-1.3b --token $HF_TOKEN

Inference​

echo "What is the weather like in Tahiti?" > test.en

1. Sentencepiece with OpenNMT Tokenizer​

eole predict -c inference-pyonmttok.yaml

2. HuggingFace Tokenizer​

eole predict -c inference-hf.yaml