全职 Machine Learning Engineer – Foundation Model Training 工作, 薪水 up to SGD 10,000, UMICHAT (SINGAPORE) PTE. LTD. 公司招聘中 - Ricebowl

About the Role

We are looking for a Machine Learning Engineer to develop and scale large language model pre-training. You will work across model architecture, training data preparation, distributed training and evaluation to improve model quality and training efficiency.

Key Responsibilities

• Develop and implement LLM pre-training strategies, including model architecture selection, training configurations and experiment planning.
• Build scalable data processing pipelines covering cleaning, deduplication, quality filtering, tokenization and dataset mixing.
• Configure and run large-scale distributed training, addressing performance bottlenecks, training stability and GPU utilization.
• Implement long-context training and context-extension techniques to improve performance on longer sequences.
• Monitor training runs, investigate loss anomalies and optimize convergence, throughput and resource efficiency.
• Evaluate model checkpoints, analyze results and translate findings into improvements to training data, architecture and hyperparameters.

Requirements

• Bachelor’s degree or above in Computer Science, Artificial Intelligence, Machine Learning, Natural Language Processing, Distributed Systems or a related technical discipline.
• Hands-on experience contributing to end-to-end LLM pre-training projects, including models with 7 billion or more parameters.
• Practical expertise with distributed training frameworks such as Megatron-LM, DeepSpeed or PyTorch FSDP.
• Experience running or supporting training workloads across 64 or more GPUs.
• Strong knowledge of pre-training data preparation, including cleaning, deduplication, filtering, tokenization, data mixing and quality improvement.
• Experience developing or optimizing data pipelines for large-scale language model training.
• Familiarity with long-context training methods such as RoPE scaling, NTK-aware interpolation and YaRN.
• Ability to troubleshoot training issues and systematically improve model performance and training efficiency.

Preferred Qualifications

• Experience pre-training models with 70 billion or more parameters.
• Experience with Mixture-of-Experts (MoE) architectures and pre-training.
• Experience improving GPU cluster performance, distributed training systems or AI training infrastructure.
• Relevant publications at NeurIPS, ICML, ICLR, ACL or EMNLP, particularly in foundation model training, model architecture or training optimization.

Pay: $5,000.00 - $10,000.00 per month

Benefits:

• Additional leave
• Food provided
• Health insurance
• Professional development

Work Location: In person