Oleg R.
AI / ML QA Engineer
Information
Available
Employment
Full-time
Part-time
Expertise
AI QA
ML QA
LLM Testing
RAG Testing
AI Quality Assurance
Model Evaluation
AI Monitoring
Location
Kyiv, Ukraine
Work Experience
4+ years
Skills
Selenium
Cypress
Playwright
Postman
Python
Langfuse
LangSmith
Promptfoo
DeepEval
Helicone
MLflow
Weights & Biases
SQL
LLM evaluation
Prompt testing
RAG testing
AI monitoring
Model validation
Performance testing
Languages
CV
Download PDFProfessional Summary
AI/ML QA Engineer with experience testing AI systems, LLM applications, and ML models. Specializes in assessing model quality, identifying errors, testing AI behavior in real-world scenarios, and monitoring system stability in production.
Focuses on verifying not only the correctness of responses, but also the stability, predictability, and business logic of AI solutions.
Key skills
AI / LLM QA
- testing LLM applications
- prompt robustness check
- detection of hallucinations
- assessment of the relevance of responses
- testing edge-case scenarios
ML QA
- checking accuracy, precision, recall
- model error analysis
- testing on different data segments
- assessment of classification quality
RAG QA
- retrieval quality check
- assessment of the relevance of sources
- testing the retrieval + generation bundle
Performance QA
- load testing
- latency check
- stability testing under load
Monitoring
- Setting up AI system monitoring
- log analysis
- identification of quality degradation
Experience
AVADA MEDIA
AI/ML QA Engineer
- testing AI agents and AI services;
- quality control of LLM answers;
- analysis of errors and unstable behavior of models;
- testing RAG systems;
- development of evaluation datasets;
- Setting up AI monitoring in production;
- conducting load testing;
- interaction with AI and ML teams.
Key projects
- testing the accuracy of conversation analysis
- checking the correctness of transcriptions and conclusions
Result: improved call analysis quality
AI agent for generating responses in CRM
- checking the relevance of answers
- testing dialogue scripts
Result: stable communication with clients
- testing the AI consultant's responses
- checking edge-case queries
Result: improved quality of consultations
- testing voice scripts
- latency and stability check
Result: system stability under load
Education
Bachelor of Computer Science
Strengths
- Deep understanding of AI systems
- Focus on quality and stability
- Working with LLM and ML models
- Ability to find critical errors
- Experience testing production AI solutions
- A systematic approach to QA
