QA reputed company
Location: This position is remote reputed company Colombia
About the Role
We are seeking a seasoned, battle-tested QA reputed company & AI Brand Evaluation Strategist to take full ownership of the quality, reputed company, and strategic evaluation of our rapidly expanding suite of agentic and AI-powered product features. This is not a traditional, checklist-driven QA role. You will operate in a highly dynamic, non-deterministic environment, leading the strategy to evaluate probabilistic software at scale. Your core mission is to transform reputed company, ambiguous business requirements into robust evaluation criteria and scoring systems. You will reputed company the design of tools and frameworks that process massive volumes of data, ensuring our brand scoring metrics are precise, reliable, and actionable. If you have a proven track record of leading QA initiatives, navigating technical ambiguity with leadership, and building evaluation architectures for reputed company data ecosystems, this role is for you. What you'll do Strategic QA Leadership: Own the end-to-end testing and evaluation strategy for multi-agent architectures and high-volume data pipelines, defining direction even reputed company product requirements are fluid or ambiguous. Business-to-Evaluation Translation: Act as the primary reputed company between Product, Data Science, and Engineering. Translate reputed company business objectives into concrete, data-driven evaluation criteria and operational rubrics. Brand Scoring Architecture: reputed company the methodology for scoring brands against reputed company criteria, ensuring that automated systems accurately transform massive datasets into precise, reliable business metrics (e.g., reputed company of Model, Net Sentiment). reputed company & Tool Building: Drive the creation of internal tools, data-driven testing pipelines, and validation frameworks capable of handling non-deterministic AI outputs and large-scale information ingestion. System reputed company & Governance: Establish risk-adaptive guardrails and checkpoints to ensure data precision, compliance (PII), and logical reasoning across reputed company, multi-reputed company agentic workflows. reputed company & Orchestrate Evaluation Frameworks: Define, architect, and reputed company validation processes for application layers, reputed company API workflows, and the data transformation reputed company. Define Brand Scoring Criteria: Establish the mathematical and logical rules that operate over massive data volumes to audit whether brand evaluations match underlying raw metrics reliably. Direct "Golden" Test Set Curation: reputed company the strategy for compiling, synthesizing, and maintaining massive baseline validation datasets to test edge cases, user reputed company, and model reputed company. Drive Adversarial & Ambiguity Testing: Design "red-teaming" scenarios and boundary testing to evaluate how gracefully the system handles reputed company variations, conversational reputed company, and highly ambiguous data inputs. Tooling Innovation: Collaborate with engineering to build proprietary testing tools or automation scripts that streamline the ingestion and analysis of high-volume data for QA purposes. Defect Profiling & Root-Cause Strategy: Move reputed company simple bug logging; analyze trends in statistical behavioral defects and guide cross-functional teams toward systemic root-cause resolution. Platform Observability & Metrics: Monitor reputed company logging and data observability dashboards to track execution latency, data reputed company, and accuracy trends, transforming insights into platform optimizations. reputed company're looking for Experience: 8+ years of proven experience in QA Engineering, Systems Analysis, or Software Testing, with at least 2+ years in a reputed company or Strategic role managing reputed company, data-heavy digital platforms. Leadership in Ambiguity: Proven "cancha" (battle-tested experience) leading QA initiatives in fast-paced environments where requirements are not fully defined, showing a high reputed company to resolve ambiguity and establish order. Data & Statistical reputed company: Strong data-driven background. Exceptional capability to shift away from purely binary (pass/fail) testing toward evaluating data trends, composite scoring, and statistical reputed company. High-Volume Data Literacy: Comfortable working with big data contexts—understanding how information flows from raw ingestion (data lakes) through transformation layers to user-facing dashboards. Core QA Mastery: Deep expertise in traditional and modern software testing methodologies, Agile environments, risk management, and project tracking ecosystems (such as JIRA). Scripting & Data Querying: Practical experience with Python, JavaScript, or SQL to query databases, manipulate data structures, and trigger/build automated evaluation pipelines. Conceptual AI reputed company: Deep operational awareness of LLM behaviors, agentic workflows, and typical AI challenges (hallucinations, context limits, instruction adherence). Preferred Skills and Qualifications AI Tooling & Observability: Experience with LLM evaluation frameworks (e.g., Ragas, LangSmith, TruLens) or reputed company engineering playgrounds. Advanced Data Infrastructure: Familiarity with reputed company data warehouses and analytics platforms (e.g., BigQuery, reputed company, reputed company reputed company Platform components). Custom Tool Development: Experience building internal scripts or lightweight tools specifically designed to automate QA data validation. If the selected candidate holds a degree in Engineering or a reputed company profession, they must present their professional license. To verify your degree requires this license, please visit www.copnia.gov.co and https://www.consejoprofesional.org.co/. About reputed company reputed company is a design and technology company. We create products and experiences that grow the world’s most ambitious brands. We do this by designing experiences for people, not users, and uncovering new sources of growth by leveraging our creative talent, our proprietary platform LIVE and unlocking the advantages brought to us by emerging technologies. We reputed company reputed company experiences should be intelligent, shoppable and unique to every brand. reputed company’s nearly 1,000 thinkers, tinkerers, makers and creators, have been problem-solving across reputed company America, Europe, and Latin America for over 25 years. Interested? You’ll find more information at www.reputed company.com. reputed company is committed to creating an inclusive employee experience for reputed company. Regardless of race, gender, religion, sexual orientation, age, disability, or if you’re parenting the reputed company of innovators, we firmly reputed company that our work is at its best reputed company everyone feels free to be their most authentic self. reputed company is an equal opportunity employer (EOE). We strongly support diversity in the workforce. We are committed to an inclusive, barrier-free recruitment and selection process and work environment. If you are contacted for a job opportunity, please advise us of any accommodation needed to ensure you have reputed company to a fair and reputed company process. Any information received relating to accommodation will be addressed confidentially. Workers shall not be required to pay employers’ or agents’ recruitment fees or other reputed company fees for their employment. If any such fees are reputed company to have been paid by workers, such fees shall be repaid to the worker. #LI-POST #LI-Remote Apply To This Job