Note: The job is a remote job and is open to candidates in USA. Prolific is building the biggest pool of quality human data in the world, utilized by over 35,000 AI developers and researchers. They are seeking Senior AI and Machine Learning Engineers to help train and evaluate the next generation of LLMs, leveraging deep technical expertise to ensure model accuracy and efficiency.ResponsibilitiesEvaluate LLM Architecture Logic: review AI-generated explanations of model architectures, loss functions, and backpropagation for technical accuracyAudit Code & Notebooks: validate ML-specific code (e.g., training loops, data preprocessing scripts, or model evaluations) for efficiency and correctnessRefine RLHF Frameworks: provide the high-quality human feedback necessary to align models with human intent, safety, and helpfulnessAnalyze Model Reasoning: critically assess how an AI model navigates complex chain-of-thought (CoT) prompts and identify where the reasoning breaks downBenchmark Performance: conduct comparative testing between different model outputs based on specific technical taxonomies and performance metricsSkillsEducation: a BS, MS, or PhD in Computer Science, Artificial Intelligence, Robotics, or a related quantitative field with a focus on Machine LearningProfessional Experience: experience building, deploying, or fine-tuning ML models in a production environmentDeep Learning Mastery: professional-level understanding of neural network architectures (Transformers, CNNs, RNNs) and optimization techniquesLLM Specialization: hands-on experience with Prompt Engineering, RLHF (Reinforcement Learning from Human Feedback), or RAG (Retrieval-Augmented Generation) workflowsTechnical Rigor: the ability to audit complex model logic, identify training data contamination, and evaluate mathematical proofs behind ML algorithmsAnalytical Critique: high attention to detail in spotting 'hallucinations,' biased outputs, or logical failures in AI-generated technical contentFrameworks: expert proficiency in PyTorch or TensorFlow/KerasLanguage & Data: advanced Python (NumPy, Pandas, Scikit-learn) and experience with Hugging Face TransformersCloud & MLOps: experience with AWS (SageMaker), Google Cloud (Vertex AI), or specialized tools like Weights & Biases and LangChainVector Databases: familiarity with Pinecone, Milvus, or Weaviate for RAG evaluationBenefitsCompetitive pay ratesFlexible hoursAbility to work from homeCompany OverviewBuilding the most advanced global infrastructure for People Science. It was founded in 2014, and is headquartered in London, England, GBR, with a workforce of 51-200 employees. Its website is https://www.prolific.com/.Company H1B SponsorshipProlific has a track record of offering H1B sponsorships, with 12 in 2025, 7 in 2024, 16 in 2023, 39 in 2022, 16 in 2021, 8 in 2020. Please note that this does not guarantee sponsorship for this specific role.
Apply To this Job