
SDET AI Lead/Senior/Junior
Apptad · New York, NY, Alpharetta, GA, United States · Remote
Weak hiring record · median 66 days open
- First seen
- 17 November 2026
- Reposted
- Not since September 2026
- Last checked
- Today · still on the employer's board
All reasons
- 55% of this employer's openings are over 60 days old
In its favour
- Posted yesterday
- Still listed on the employer's own board
- Someone has already applied through it
Job Role: SDET AI Lead/Senior/Junior
Job Location
NY/Alpharetta, GA- Day 1 onsite (3 days work from Office, 2 days' work from home)
Job Type: Contract
Lead role- New York
Senior Role- Alpharetta, GA
Junior Role- Alpharetta, GA
About the Role
We are seeking a Lead Software Development Engineer in Test (SDET) to lead our quality engineering strategy and build scalable, reliable, and intelligent test automation solutions. This is a hands-on leadership role combining software engineering, test architecture, mentoring, CI/CD, and artificial intelligence.
The ideal candidate has strong expertise in Python, Playwright, pytest, API testing, automation frameworks, Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and AI evaluation. You will collaborate with software engineers, product managers, DevOps teams, data scientists, and AI engineers to improve quality throughout the product lifecycle.
Key Responsibilities
- Lead the design, development, and maintenance of automation frameworks using Python, Playwright, and pytest.
- Define test strategies covering UI, API, integration, end-to-end, regression, smoke, contract, exploratory, and non-functional testing.
- Create scalable, maintainable, and reusable test utilities, fixtures, mocks, test data, and service virtualization.
- Integrate automated testing into CI/CD pipelines using GitHub Actions, GitLab CI, Jenkins, or Azure DevOps.
- Test web applications, REST and GraphQL APIs, microservices, distributed systems, databases, and event-driven architectures.
- Review requirements, user stories, prompts, retrieval workflows, and technical designs to identify risks and define test coverage.
- Investigate test failures, reduce flaky tests, identify root causes, and collaborate with engineering teams to resolve defects.
- Establish coding standards, quality gates, test reporting, dashboards, metrics, and automation best practices.
- Mentor SDETs, QA engineers, and developers in software quality, automation, and AI testing.
- Evaluate and introduce AI-assisted testing, including test generation, intelligent test selection, failure analysis, defect prediction, and test maintenance.
- Design testing strategies for LLM applications, including prompt validation, response quality, hallucination detection, toxicity, bias, safety, latency, cost, consistency, and reliability.
- Test RAG systems, including document ingestion, OCR, preprocessing, chunking, metadata, embeddings, vector databases, retrieval, context assembly, grounded responses, citations, permissions, stale data, and retrieval failures.
- Build automated evaluation frameworks using benchmark datasets, expected responses, deterministic assertions, human feedback, and model-based evaluation.
- Promote the secure, responsible, explainable, and maintainable use of AI in testing.
Required Qualifications
- Lead-level experience in software testing, quality engineering, or SDET roles, with the ability to own strategy and influence engineering practices.
- Strong hands-on automation development skills using Python, Playwright, and pytest.
- Experience designing, implementing, and maintaining automation frameworks at scale.
- Strong knowledge of UI, API, integration, end-to-end, regression, contract, functional, and non-functional testing.
- Experience testing web applications, APIs, microservices, and distributed systems.
- Practical experience testing LLM-powered applications and RAG architectures.
- Understanding of embeddings, semantic search, vector databases, context retrieval, prompt engineering, and response generation.
- Experience creating AI test datasets, evaluation suites, prompts, expected outputs, edge cases, and quality criteria.
- Knowledge of LLM and RAG metrics, including correctness, relevance, faithfulness, groundedness, context precision, context recall, completeness, safety, and toxicity.
- Experience with Git, CI/CD, logging, debugging, test reporting, and Agile development.
- Strong leadership, mentoring, communication, analytical, and problem-solving skills.
Desired Qualifications
- Experience with Playwright tracing, screenshots, video capture, parallel execution, fixtures, and test sharding.
- Experience with requests, Postman, Allure, JUnit, and comparable testing tools.
- Familiarity with Ragas, DeepEval, promptfoo, LangSmith, or similar LLM evaluation frameworks.
- Experience with OpenAI, Azure OpenAI, Anthropic, Google Vertex AI, or Amazon Bedrock.
- Experience with Pinecone, Weaviate, Milvus, Qdrant, Chroma, pgvector, or other vector databases.
- Knowledge of Docker, Kubernetes, AWS, Azure, GCP, SQL, NoSQL, observability, performance, accessibility, and security testing.
Nice-to-Have
- Experience leading a quality engineering or SDET team.
- Experience testing conversational agents, AI copilots, autonomous agents, multi-turn workflows, and tool-calling systems.
- Familiarity with prompt injection, jailbreaks, data leakage, unauthorized retrieval, insecure output handling, and adversarial testing.
- Experience testing machine learning models, production RAG platforms, or regulated applications.
- Knowledge of responsible AI, privacy, security, governance, and compliance.
- Contributions to open-source testing or AI evaluation projects.
Skills
- Python
- Playwright
- pytest
- API testing
- CI/CD
- GitHub Actions
- GitLab CI
- Jenkins
- Azure DevOps
- REST
- GraphQL
- microservices
- distributed systems
- embeddings
- vector databases
- prompt engineering
- LLMs
- RAG
- OCR
- model evaluation
- Docker
- Kubernetes
- AWS
- Azure
- GCP
- SQL
- NoSQL
More jobs at Apptad
All 275- Salesforce Financial Services Cloud (FSC) Technical ArchitectApptad · Tysons, VA, Irving, TX, San Antonio, TX, United States · USD 90–95/hrtoday
- Sr. .Net DeveloperApptad · Texas · United Statesyesterday
- Java FullstackApptad · Remote · USD 55–60/hryesterday
- Java Full Stack DeveloperApptad · Pleasanton, CA, United States · USD 55–60/hryesterday
- Senior Product Manager – PaymentsApptad · South Carolina · United Statesyesterday
Similar roles
- Network Test EngineerCompQsoft Inc. - ACTIVE · FORT GEORGE G MEADE, MD · USD 135,000–210,000today
- Construction Inspector III- Water/WastewaterAECOM · Fort Worth, TX, United Statestoday
- QA Automation Engineer@Tampa, FLDiverse Lynx · Tampa, FL · USD 8–20/hrtoday
- Quality Assurance Engineer INorth Atlantic Industries · Bohemia, NY · USD 65,000–80,000/yrtoday
- Senior Manager– AI Governance & Controls EngineeringAbbVie · North Chicago, IL, United Statestoday
- Electronics Test EngineerPEAK Technical Services · Camarillo, CA · USD 95,000–120,000/yrtoday