AI Automation Engineer

Position: AI Automation Engineer
Location: Bellevue, WA (Onsite)
Openings: 1
Employment Type: Full-Time
Salary: Up to $150K

Experience Required

• Minimum 15 years of overall IT experience.
• 3 years of hands-on AI/ML experience.

Mandatory Skills

• Strong experience with ETL Pipelines.
• Hands-on experience in Machine Learning / Artificial Intelligence.
• Experience with AI Operations, including MLOps, AgentOps, and LLMOps.
• Strong programming skills in Python and SQL.
• Experience in System Design.
• Hands-on experience with MCP (Model Context Protocol) Client-Server Design.

Preferred Skills

• AIOps / MLOps
• Observability for AI
• Bash Scripting

Key Responsibilities

• Design and build end-to-end AI testing frameworks covering unit, functional, integration, and regression testing, including automated test case generation and model output validation against defined quality benchmarks.
• Develop load and performance testing frameworks to simulate high-concurrency scenarios, validating scalability, latency SLAs, and platform stability under peak workloads.
• Build LLM evaluation pipelines measuring relevance, groundedness, and hallucination metrics, integrating them into CI/CD workflows to enforce automated quality gates before every model or agent deployment.
• Own the AI platform regression testing strategy by maintaining automated test suites that validate agent behavior, RAG pipeline accuracy, and API contracts after every release to prevent quality degradation.

Have a Great Day!
Warm Regards,
Sania Wadhwa
Technical Recruiter
Contact Details: 609-755-5388
Email: sania@applabsystems.com
4365 Route 1 South, Suite 105
Princeton, NJ 08540
www.applabsystems.com