Senior AI Quality Engineer

At Mitratech, we are a team of technocrats focused on building world-class products that simplify operations in the Legal, Risk, Compliance, and HR functions. We are a close-knit, globally dispersed team that thrives in an ecosystem that supports individual excellence and takes pride in its diverse and inclusive work culture centered around great people practices, learning opportunities, and having fun! Our culture is the ideal blend of entrepreneurial spirit and enterprise investment, enabling the chance to move at a rapid pace with some of the most complex, leading-edge technologies available.

For over 35 years, the experts at Mitratech have been focused on solving the complex needs. Today, we serve 20,000 client companies of all sizes globally, representing 30% of the Fortune 500 and over 500,000 users in over 160 countries.

As we continue to grow, we’re always looking for resourceful, enthusiastic, and fresh perspectives. Join our global team and see what makes Mitratech a truly exceptional place to work!

Given our continued growth, we always have room for more intellect, energy, and enthusiasm - join our global team and see why it's so special to be a part of Mitratech!

Job Overview 

We are seeking a highly skilled Senior QA Engineer specializing in AI and Machine Learning to join our growing team. The ideal candidate will play a pivotal role in ensuring the quality, reliability, and performance of our AI-driven products and platforms. This position requires a strong foundation in software testing, automation, and QA strategy, combined with an understanding of machine learning workflows such as RAG pipelines, inference APIs, and generative AI applications. The successful candidate will collaborate closely with ML engineers, data scientists, and developers to design and execute comprehensive test plans that validate model behavior, system integrations, and end-to-end user experiences.

Responsibilities

• Design and execute manual and automated test cases for backend APIs, frontend interfaces, and end-to-end AI workflows.
• Conduct exploratory testing of new generative AI features and user interactions.
• Develop and maintain automated test frameworks for UI and API.
• Build automation scripts for AI/ML pipelines using pytest, unittest, or custom frameworks.
• Create comprehensive test plans, manage defects, and report results using Jira, or similar tools.
• Validate RAG (Retrieval-Augmented Generation) pipelines, including embedding accuracy, retrieval quality, and model response evaluation.
• Test AI agents and orchestration flows, ensuring correctness in multi-step reasoning and tool invocations.
• Perform performance and scalability testing for ML inference endpoints (SageMaker, Bedrock, custom APIs), monitoring latency and throughput.
• Conduct integration testing across data ingestion, preprocessing, inference, and output pipelines.
• Validate UI/UX and Gen AI interaction flows, ensuring a consistent and intuitive user experience across devices and browsers.
• Automate end-to-end test scenarios, including document upload, retrieval, summarization, and inference workflows.
• Integrate and maintain automated test suites within CI/CD pipelines (GitHub Actions, GitLab CI, Jenkins).
• Collaborate closely with ML engineers, data scientists, and developers to reproduce issues, validate fixes, and ensure model reliability.
• Ensure security, compliance, and data privacy standards are upheld for sensitive datasets during testing.

Skills & Experience

• Strong experience in manual testing of backend APIs, frontend apps, and full-stack systems.
• Hands-on with test automation tools: Selenium, Playwright, Cypress, Postman, REST-assured, or Python-based frameworks.
• Proficient in test case design, bug reporting, and defect tracking (Jira, Trello, or equivalents).
• Familiarity with AI/ML workflows — especially LLM-based applications, RAG pipelines, embeddings, and model inference.
• Experience with performance testing tools or frameworks; understanding of latency, throughput, and scaling metrics.
• Knowledge of CI/CD automation and integrating tests into build pipelines.
• Understanding of API testing principles: CRUD operations, authentication, edge cases, and error handling.
• Familiar with UI/UX validation and cross-browser compatibility testing.
• Strong analytical and communication skills — able to describe AI-related issues clearly and reproducibly.
• Awareness of security and compliance practices for testing sensitive or proprietary data.

Education:                         

• A Master’s degree in Machine Learning, Computer Science with a preference for specialization in the NLP domain.

We are an equal-opportunity employer that values diversity at all levels. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, national origin, age, sexual orientation, gender identity, disability, or veteran status.