JobHabor
Menu

Automation Engineer - AI (26-09847) -

ClifyX · Location not stated

Strong hiring record · median 17 days open

100100

Hiring signal

Actively hiring

Open 1 day · Never reposted · How we score

First seen
19 November 2026
Reposted
Not since September 2026
All reasons
  • No location stated

In its favour

  • Posted today
  • Still listed on the employer's own board

Did you apply?

Automation Engineer - AI (26-09847) - at ClifyX. We’ll keep it in your applied jobs, so you can see it later and be warned before applying to the same employer twice.

Key Responsibilities

Design & Build AI Eval Pipelines

Architect automated evaluation frameworks measuring faithfulness, answer completeness, hallucination rates, and task completion across complex conversational and agentic workflows.

LLM-as-a-Judge & Heuristic Scoring

Develop, calibrate, and maintain rubric-based LLM-as-a-Judge evaluators.

Adversarial & Safety Testing (Guardrail Validations)

Design systematic stress tests for prompt injection, jailbreaks, PII leakage, guardrail boundaries, and adversarial edge cases.

Partner with ML/Platform Engineers

Inspect distributed traces, logs, and telemetry to pinpoint whether failures stem from prompts, models, RAG retrieval, or downstream connectors.

Synthetic Test Data Generation

Build programmatic pipelines to generate diverse persona simulations, multi-turn edge cases, and high-quality golden evaluation datasets.

CI/CD Integration

Integrate automated regression suites into build/release pipelines to gate prompt changes, model bumps, and backend API updates.

More jobs at ClifyX

All 204

Similar roles