LLM Evaluation Specialist

DESCRIPTION

We’re a small team building an AI-native data storytelling product that helps business professionals turn data into clear, honest narratives. We need a sharp human evaluator to review our AI output and give us reliable, actionable feedback.

RESPONSIBILITIES

Run recurring evaluation passes on AI-generated charts, headlines, captions, and chat responses against a defined rubric Judge accuracy (is the claim supported by the data?), chart-type fit, clarity, and tone Flag misleading framing, specious correlations, and awkward phrasing that automated evals miss Log findings in eval tooling and deliver a concise report to the team after each cycle Suggest rubric improvements based on patterns observed

Skills

Strong data analysis and visualization judgment Sharp editorial instincts and attention to detail Comfort with spreadsheets and eval tools (no coding required) Clear, concise written reporting Prior experience evaluating LLM output is a plus

Education

Prior experience evaluating LLM output

Bachelor’s degree in a related field such as data analytics, statistics, journalism, information science, or business, or equivalent practical experience

EEO

“Mindlance is an Equal Opportunity Employer and does not discriminate in employment on the basis of – Minority/Gender/Disability/Religion/LGBTQI/Age/Veterans.”