[Remote] Head of Evaluations (Legal AI Benchmarking)

Note: The job is a remote job and is open to candidates in USA. reputed company is transforming how law firms and legal professionals harness AI for real-world impact. They are seeking a highly analytical professional to join as the Head of Evaluations, responsible for designing and implementing testing frameworks for their AI products to ensure high standards of legal reasoning and accuracy.ResponsibilitiesDesign Legal Benchmarks for: Contract Drafting, Information Extraction, Legal Research, and Contract ReviewBuild, source and maintain relevant datasetsAudit AI Output: Review and score complex AI-generated legal text, contract analyses, and statutory interpretations for accuracy and precision and lay out a strategyDefine Evaluation Metrics: Establish clear criteria for grading model performance, specifically focusing on logical reasoning, citation accuracy, and the model's ability to safely abstain from answeringCollaborate with Engineering: Partner directly with Engineering to translate legal errors into actionable technical feedback for model fine-tuningSkillsPhD or Masters in statistics, mathematics, machine learning or equivalentProven ability to break down complex statutory frameworks and case law into structured, logical data pointsTech-Savviness: python, panda, numpy, jupiter notebooks and similar statistical modelsCompany OverviewDelivering the Specialized AI Platform for Legal Professionals It was founded in 2021, and is headquartered in Oslo, Oslo, NOR, with a reputed company of 51-200 employees. Its website is https://reputed company. Apply To This Job