M2 - Sr Data Platform Architecture Lead

Objective of the Role

Design and define the Data & AI Platform Architecture that must support the process of data collection, transformation and visualization, ensuring the technological evolution of such architecture and data infrastructure over the Data Lakehouse for supports business.

Main Responsibilities

• Design, deploy, evolve and ensure the integrity of the organization’s official data platform layer, defining a roadmap for the use of its capabilities.
• Develop and document the roadmap for the evolution of the Data Architecture, coordinating with the technical areas involved to gather improvement requirements.
• Disseminate standards and guidelines for data architecture in projects.
• Define the framework, guidelines, standars, and principles of data architecture platform, including modeling, metadata, security, backup, and reference data.
• Continuously update data guidelines, and standards, publishing and promoting them to relevant teams to ensure a unified and high-quality data design approach across the organization.
• Identity improvement opportunities in data ingestion, evaluating and proposing new technologies based on market best practices likes real and near real time.
• Identify roles and responsibilities within the Data Architecture and define the interaction model between roles to ensure a proper data lifecycle.
• Work together with the Chapters of Data Engineering Chapter Lead, DataOps Lead, Data Modeling Lead, Data Governance and the IT area to implement and maintain data pipelines supporting DataOps practices.
• Design comprehensive solutions such as the implementation of Data Marketplace, Data Sharing and similar initiatives.
• Autonomous Work: Thrive in an autonomous work culture, taking initiative and ownership of projects.

• Culture Ambassador: Promote and embody the company’s values, fostering a positive and collaborative work environment.
• Continuous Improvement: Identify opportunities for process improvements and implement solutions to enhance efficiency.
• Building trust-based relationships with user areas.

Required Knowledge and Experience

• Significant experience in similar positions.

Advanced knowledge

• Education: Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
• 10+ years of experience designing Datalake on cloud platforms or Big Data ecosystems in reteail companies preferably.
• 10+ years of experience in AWS/GCP/Azure cloud services, including real time events.
• 6+ years of experience with Databricks platform with focus in:
  • Unity Catalog
  • MLOps
  • DataOps
  • AI Architecture
  • Advanced Analytics
• Knowledge of data integration tools and technologies: for ingesting the Data Lakehouse with information from various sources, e.g., ETL tools, DTL, API connectors, and data ingestion frameworks.
• Knowledge of event hubs tools such as Apache Kafka and cloud-native event tools.
• Knowledge and understanding of best practices in data security and governance.

Familiarity with

• Agile methodologies, e.g., scrum and kanban.
• AWS infrastructure.
• GCP infrastructure.
• Azure infrastructure.
• Networking infrastructure.
• Databricks´s APIs.
• Unity Catalog.
• Data Catalog Tools.
• API Gateway

Additional experience required

• Experience with CI/CD: GitHub Actions.
• Experience with Python for data ingestion.
• Experience with APIs for data ingestion and availability.
• Experience with SQL and NoSQL databases.
• Experience in Data Modeling (Data Warehouse / Data Lake).
• Experience using visualization tools (Looker -QuickSigth - Power BI).
• Experience working under Agile methodologies.

Nice to have

• Experience in Machine Learning and Artificial Intelligence projects.

Spin está comprometida con un lugar de trabajo diverso e inclusivo. Somos un empleador que ofrece igualdad de oportunidades y no discrimina por motivos de raza, origen nacional, género, identidad de género, orientación sexual, discapacidad, edad u otra condición legalmente protegida. Si desea solicitar una adaptación, notifique a su Reclutador.