Collaborate with our client to create, implement, and maintain operational, QA, and performance metrics; aligned with business-specific needs related to AI.
Define and interpret A/B tests that help answer critical questions about the quality of Expert Raters and trends toward best practices.
Establish a data-driven product development culture by driving the definition, tracking, and operationalizing of queries-, projects-, and program-level metrics.
Develop and socialize dashboards, reports, and other ways of enabling the team and program to answer product data questions in a self-serve way.
Help drive innovations through effective identification and application of analytics, causal inference, experimentation, and machine learning.
Drive and direct analysis of current workflows and systems to identify opportunities for process improvement, automation, and efficiency enhancement.
This is a high-impact role in which you will have a direct influence on how product decisions are made.
Lead the design, analysis, and interpretation of experiments proactively perform data exploration on engagement behaviors to discover future opportunities.
Support re-pricing exercises and/or new business through business intelligence data readiness.
Able to create, train, and teach AI-operational metrics: Kripendorf’s alpha, chance corrected agreement, raw agreement, confidence intervals, stratified sampling, pseudo sampling, structured sampling, and others. – Mandatory.
Requirements & Skills:
Advanced knowledge of statistical models and metrics; knows how to balance analytical rigor with pragmatism and actionability.
Ability to communicate technical and statistical concepts clearly and concisely among audiences at many different levels, and be solutions-oriented.
Ability to lead effectively and collaborate with cross-functional teams, bridging the gap between business and technology.
Willingness to embrace uncertainty and contribute to defining industry standards in an evolving landscape.
Synthesize quantitative data from multiple sources (surveys, behavioral, 3rd party), including visualization of data, into clear and compelling narratives in decks and memos, tailored to the needs and communication preferences of the audience.
An analytical mindset; you love digging into the data (including survey data, public “big data” repositories, and internally logged data) for hidden connections and patterns and proactively uncover insights based on critical business questions.
Demonstrated knowledge in NLP, large language models, or generative AI – Desirable.
English B2 – Required.
Oral and written comprehension.
Appropriate use of language.
3+ years experience in a quantitative role navigating highly ambiguous environments, ideally as a founding data scientist or product analyst at a hyper-growth product company or research org – Required.
Advanced degree in Statistics, Mathematics, Physics, Economics, or related quantitative field – Desirable.
Demonstrated experience working, digesting, and/or processing Big Data models – Desirable.
