What it is
Open, fully reproducible 1.72B Brazilian-Portuguese base LLM with end-to-end pretraining and evaluation pipeline. It is strongest below 7B on LAMBADA-PT. Reproduce a Brazilian-Portuguese base LLM end to end, from open-data corpus and tokenizer training to Megatron pretraining, HF conversion, and paired benchmark evaluation.
Key results
- Best sub-7B LAMBADA-PT score here: 45.31
- Validated end to end on 2×24GB Ampere GPUs, no NVLink
- Stable from scratch: 0 skipped iterations and 0 NaN
Shows that a competitive, fully reproducible 1.72B PT-BR LLM can be trained on modest 2×24GB hardware.