Data Scientist (Generative AI) - Hybrid Onsite

Marathon TS is looking to hire an onsite Data Scientist (Generative AI) to support a Federal Agency in Washington DC. We are looking for a hands-on data scientist to build and lead the quality of generative AI workflows that review large document sets and draft structured findings for expert reviewers in a regulated environment. You will write and tune the prompts, retrieval and output logic, and own how the program proves the outputs are accurate and well-evidenced enough for experts to rely on.

What you'll do

• Design and build generative AI workflows that review large document sets against defined criteria, identify potential issues, and draft structured findings and documents.
• Build the evaluation framework: gold sets from historical reviews, and metrics for accuracy, evidence linkage, unsupported statements and consistency, run as a regression check before every prompt or model change.
• Set prompt engineering and retrieval standards: chunking, retrieval over source documents and reference standards, and structured outputs that match application contracts.
• Build document chunking and retrieval over large, mixed-format document sets.
• Define provenance and confidence: prompt and model versioning in MLflow, a confidence score reviewers can act on, and links to the evidence behind each statement.
• Measure improvement from reviewer edits by comparing AI drafts with the final human versions, and feed the results back into prompts.
• Build monitoring of output quality and usage, and document each version and its results.
• Set guardrails for confidential data, and explain results to business leaders in plain language.
• At the Principal level, set standards across workflows and mentor other data scientists.

What you bring

• Senior: 8+ years in data science or machine learning. Principal: 12+ years. Both include 2+ years building production applications on large language models.
• Python and PySpark; Databricks notebooks and MLflow.
• Proven evaluation design for generated text: gold sets, automated scoring and human review loops.
• Databricks Mosaic AI: Model Serving, Vector Search and Agent Framework.
• Document understanding: extracting content from PDF, Excel and scanned files.
• Ability to explain model behavior and limits to non-technical domain experts.
• Experience with Azure OpenAI models.

Nice to have

• Audit, accounting or regulatory domain knowledge.
• Model-graded evaluation and human-in-the-loop review systems.

Working with us

• You will work inside a secure, private cloud environment with highly confidential data.
• You must be authorized to work in the United States and able to pass a background check.
• We are looking for people who can start within a few weeks.

#cjjobs

Marathon TS is committed to the development of a creative, diverse and inclusive work environment. In order to provide equal employment and advancement opportunities to all individuals, employment decisions at Marathon TS will be based on merit, qualifications, and abilities. Marathon TS does not discriminate against any person because of race, color, creed, religion, sex, national origin, disability, age or any other characteristic protected by law (referred to as "protected status ").