Senior Analyst - AI QA (R-18975)

At Dun & Bradstreet, we believe data has the power to create a better tomorrow. As a global leader in business decisioning data and analytics, we help companies worldwide grow, manage risk, and innovate. For over 180 years, businesses have trusted us to turn uncertainty into opportunity. We’re a diverse, global team that values creativity, collaboration, and bold ideas. Are you ready to make an impact and help shape what’s next? Join us! Explore opportunities at dnb.com/careers.

We are looking for a highly skilled AI Tool/Agent Testing Engineer to evaluate, validate, and ensure the reliability of AI agents, AI automation tools, and agentic workflows used across our analytics platform. This role blends test engineering, GenAI understanding, Python/PySpark proficiency, and agent development lifecycle knowledge. You will work closely with Data Science, AI Engineering, and Platform teams to ensure that AI agents behave predictably, safely, and in alignment with business and compliance requirements

Build agentic workflows using Lang Chain/Lang Graph and similar frameworks

Develop autonomous agents for data validation, reporting, document processing, and domain workflows

Deploy scalable, resilient agent pipelines with monitoring and evaluation

Develop GenAI applications using models like GPT, Gemini, and LLaMA

Implement RAG, vector search, prompt orchestration, and model evaluation

Partner with data scientists to productionize POCs

Build distributed data pipelines (Python, PySpark)

Develop APIs, SDKs, and integration layers for AI-powered applications

Optimize systems for performance and scalability across cloud/hybrid environments

Contribute to CI/CD workflows for AI models—deployment, testing, monitoring

Implement governance, guardrails, and reusable GenAI frameworks

Work with analytics, product, and engineering teams to define and deliver AI solutions

Participate in architecture reviews and iterative development cycles

Support knowledge sharing and internal GenAI capability building

6-8 years of experience in AI/ML engineering, data science, or software engineering, with at least 2 years focused on GenAI

Strong programming expertise in Python, distributed computing using PySpark, and API development

Hands on experience with LLM frameworks (Lang Chain, Lang Graph, Transformers, OpenAI/Vertex/Bedrock SDKs)

Experience developing AI agents, retrieval pipelines, tool calling structures, or autonomous task orchestration

Solid understanding of GenAI concepts: prompting, embeddings, RAG, evaluation metrics, hallucination identification, model selection, fine tuning, context engineering

Experience with cloud platforms (Azure/AWS/GCP), containerization (Docker), and CI/CD pipelines for ML/AI

Strong problem solving, system design thinking, and ability to translate business needs into scalable AI solutions

Excellent verbal, written communication and presentation skills

All Dun & Bradstreet job postings can be found at https://jobs.lever.co/dnb. Official communication from Dun & Bradstreet will come from an email address ending in @dnb.com.

Notice to Applicants: Please be advised that this job posting page is hosted and powered by Lever, a subsidiary of Employ Inc. Your use of this page is subject to Employ's Privacy Notice and Cookie Policy, which governs the processing of visitor data on this platform.