Senior Test Automation Engineer – AI Solutions (Python)

EPAM Systems

  • Беларусь
  • Постоянная работа
  • Полная занятость
  • 1 мес. назад
We are looking for a talented and experienced Senior Test Automation Engineer to join our team and help productionize AI solutions leveraging cutting-edge technologies. This is an exciting opportunity to work on AI-driven systems, including Retrieval Augmented Generation (RAG), Large Language Models (LLMs), and AWS Bedrock capabilities. You will play a critical role in ensuring the functionality, performance, scalability, security, and reliability of these innovative solutions.At EPAM Belarus, employees have the flexibility to choose the environment that suits them best. You can work from any location in Belarus, whether it's your home or our offices in Minsk, Grodno, Brest, Gomel, Mogilev or Vitebsk.ResponsibilitiesDesign and implement comprehensive testing strategies tailored to AI workflows, APIs, and cloud-native solutionsDevelop detailed test plans, test cases, and scripts for AI systems, including data pipelines, knowledge bases, vector databases, and LLM-powered componentsValidate the performance and accuracy of LLM outputs, ensuring compliance with prompt engineering best practices and client-specific requirementsTest RAG-based architectures, including retrieval processes, embedding validation, and semantic search accuracyEnsure seamless integration of data ingestion, cleansing, transformation, and validation pipelines with vector databases and AI modelsBuild and maintain test automation frameworks for AI components and data pipelines using industry-standard tools and custom scriptsConduct performance, load, and scalability testing for AI/LLM endpoints, RAG workflows, and deployed pipelinesTest cloud infrastructure deployment (e.g., IaC via Terraform/CloudFormation) and ensure production-readiness for AWS Bedrock, S3, EKS, and vector database configurationsIdentify, log, and manage defects in AI workflows, APIs, data pipelines, and backend systems, collaborating with developers and DevOps teams to resolve issuesMonitor system reliability, availability, and fault tolerance in cloud environments to ensure production-level readinessRequirementsProficiency in test automation using Python (3+ years of experience)Proven experience in QA engineering, with a focus on testing AI systems and cloud-native solutionsStrong knowledge of testing frameworks, tools, and methodologies for AI workflows, APIs, and data pipelinesHands-on experience with LLMs, RAG architectures, and vector databasesProficiency in test automation using Python or similar scripting languagesExpertise in performance and scalability testing for AI systemsFamiliarity with cloud environments, particularly AWS services like Bedrock, S3, and EKSSolid understanding of Infrastructure as Code (IaC) tools such as Terraform or CloudFormationExcellent problem-solving skills and ability to collaborate effectively with cross-functional teamsFluency in English (both written and spoken) at a minimum B2 levelNice to haveExperience with AWS Bedrock capabilities and generative AI solutionsKnowledge of semantic search and embedding validation techniquesFamiliarity with continuous integration and delivery pipelines for AI systemsBackground in data engineering or data scienceWe offer/BenefitsWe connect like-minded people:
  • Delivering innovative solutions to industry leaders, making a global impact
  • Enjoyable working environment, whether it's the vibrant office or the comfort of your own home
  • Opportunity to work abroad for up to two months per year
We invest in your growth:
  • Leadership development, career advising, soft skills and well-being programs
  • Certifications, including GCP, Azure and AWS
  • Unlimited access to LinkedIn Learning and Get Abstract
  • Free English classes with certified teachers
We cover it all:
  • Monetary bonuses for taking part in the referral program
  • Comprehensive medical & family care package
  • Seven trust days per year (sick leave without a medical certificate)
  • Discounts from 800+ partners (sports activities, restaurants, stores and services)
EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our clients, our employees, and our communities. We embrace a dynamic and inclusive culture. Here, you will collaborate with multi-national teams, contribute to numerous innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to learn and grow continuously. No matter where you are located, you will join a dedicated, creative, diverse community to help you discover your fullest potential.

EPAM Systems