- Singapore
全职 Agent Algorithm Evaluation Engineer 工作, 薪水, Shopee 公司招聘中 - Ricebowl
工作地点
职位描述
岗位职责
Department Engineering and Technology
LevelExperienced (Individual Contributor)
LocationSingapore
The Engineering and Technology team is at the core of the Shopee platform development. The team is made up of a group of passionate engineers from all over the world, striving to build the best systems with the most suitable technologies. Our engineers do not merely solve problems at hand; We build foundations for a long-lasting future. We don't limit ourselves on what we can or can't do; we take matters into our own hands even if it means drilling down to the bottom layer of the computing platform. Shopee's hyper-growing business scale has transformed most "innocent" problems into huge technical challenges, and there is no better place to experience it first-hand if you love technologies as much as we do.
About the Team:
About Us
Sea Group is establishing a brand-new, strategic AI department. This department is dedicated to exploring the transformative potential of generative AI in revolutionizing human connection, self-expression and communication diversity, and social interaction. We are building the next generation of AI-native applications and a comprehensive Model-as-a-Service (MaaS) product support system. Based on massive multi-country data, we are building a leading multilingual AI ecosystem from the ground up. We look forward to more outstanding talents joining us to build leading Southeast Asian multilingual models and explore innovative AI-native applications.
The AI application team focuses on the intersection of social connectivity and artificial intelligence. Our mission is to leverage LLMs to create digital personas that can act as personal assistants and social bridges. This team operates with a startup's agility backed by our Group's robust resources, aiming to define how humans interact in the AI era.
Job Description:
- Evaluation Framework Design: Design and build multi-dimensional evaluation metric systems tailored to diverse business scenarios.
- Red-Team Testing & Safety Control: Simulate complex, extreme real-world user scenarios to conduct stress testing and red-teaming of AI models; identify and drive remediation of issues related to bias, hallucinations, policy-violating content, and values alignment.
- In-Depth Bad Case Analysis: Conduct root-cause analysis of erroneous model outputs and collaborate with algorithm engineers (Model/SFT/RLHF) to drive prompt optimization and fine-tuning improvements.
Automated Evaluation Tooling: Leverage LLM-as-a-Judge methodologies to build automated evaluation pipelines that enhance product iteration efficiency.
- User Experience Insights: Conduct in-depth research into user psychology within social and companion-style products, translating subjective, qualitative experience into quantifiable metrics to continuously enhance product experience.
Requirements:
- Proven experience enhancing automated evaluation workflows, including but not limited to conventional evaluation methods, LLM-as-a-Judge, and automated weakness mining.
- Proven experience enhancing automated data production pipelines, including but not limited to Self-Instruct, Evol-Instruct, and Synthetic Data RL.
- Strong software engineering fundamentals, with proficiency in Python and Shell scripting and hands-on experience in Linux environments.
- Master's degree or above in Computer Science or a related field, with at least 3 years' working experience.
Nice-to-Have
- Prior background in AI evaluation, QA automation, or NLP algorithm development at a major internet company.
- Extensive experience as a power user or developer of mainstream AI social products.
重要安全守则
申请工作时，切勿提供您的银行或信用卡详细资料。不要转账或完成无关的在线调查问卷。如果您发现可疑内容，请举报此招聘广告。
了解更多