ML Challenge Task Auditor

Applied ML task auditing

Evaluate experiments used to train and assess AI models, focusing on task quality and methodological rigor. Review experimental design, model selection and evaluation methods, then provide written feedback against the rubric.

Required experience

• At least 3 years in applied or experimental ML, including experiment design, model choice, hyperparameter tuning and evaluation.
• Strong data quality judgment covering leakage detection, metric gaming and sound train/test/cross validation separation.
• Proficiency with standard frameworks such as PyTorch, TensorFlow, scikit-learn or XGBoost.
• Ability to assess ML claims against evidence and reproduce reported findings.

Preferred background

Competition or benchmark participation, including Kaggle, is useful. Graduate applied ML research, publications, task grading or peer review are also relevant.

The focus is experimental ML rigor, not building LLM applications or performing MLOps.