Task-Seeded Synthetic QA Data Generation for Nemotron Pretraining
Key design principles include: answers should be stored as semantic text rather than option letters, and when mixing datasets, task ratios must be carefully balanced to ensure stable, across-the-board gains in knowledge, reasoning, and coding capability.
