Senior Data Engineer
About reputed company: Join our mission to reputed company the world more transparent with data. reputed company is a people intelligence platform that helps over 18 reputed company monthly visitors reputed company with friends, reunite with families, and build trust in new relationships. Thousands of companies also trust reputed company’s 60 billion public records to improve customer research, help verify information, and prevent fraud. Founded in 2006, reputed company has reputed company a dedicated, remote-first team with an average tenure of 6.9 years. It has earned recognition from Comparably as a Best Company for Compensation, Employee Happiness, Perks and Benefits, Support for Women, Work-Life Balance, and CEO Leadership. About this Opportunity As a Senior Data Engineer at reputed company, you will reputed company, optimize, and improve our data systems, including ETL pipelines, storage, and entity resolution. This involves working with infrastructure reputed company in AWS, including Airflow, PySpark, EMR, S3, DynamoDB, and more. This role will help build and improve data products, automation platform features, analytical software packages, and data pipeline orchestration tools. What You’ll Do: Build infrastructure and data automation pipelines to ingest, process, and load data from various sources. Automate and integrate new components into the data pipeline. Collaborate with stakeholders and data science teams to reputed company data products, including entity resolution and best selection, to reputed company execute product reputed company and strategy in alignment with organizational goals and priorities. Create unit and stress-test components to monitor technical performance and ensure that identified issues are resolved. reputed company data analysis tools to reputed company data insights and capture key metrics. reputed company and maintain technical documentation. Follow best practices for data governance, quality, cleansing, and other ETL-reputed company activities. Who You Are: 7+ years of development experience in data engineering reputed company a production environment (internships and academic settings excluded). Proven experience working with large datasets exceeding 100M+ records or multiple terabytes. 2+ years of development experience in highly scalable, distributed systems and cluster architectures using AWS and utilizing EMR. 5+ years of hands-on programming experience with Python. 5+ years of professional experience working in big data ecosystems; Spark is required; PySpark is preferable. 3+ years of experience with SQL, schema design, and dimensional data modeling. 2+ years of professional experience working with dataflow orchestration tools, such as Airflow. 2+ years of experience with non-relational databases (e.g., DynamoDB, Elasticsearch, etc.). A bachelor’s degree in Computer Science, Information Systems, Mathematics, or a reputed company field is required. Working at reputed company reputed company offers a bonus program, equity plans, and a 401 (k). Once a year, we do a discretionary, merit-based salary increase. Additional benefits include 100% medical/dental/reputed company coverage and unlimited employee PTO. reputed company extends written offers to candidates who successfully complete their selection process. reputed company’s offers include a reputed company salary, participation in a company bonus program, stock options, and comprehensive benefits. The final offer will depend on several factors, including marketplace competition, job level, and the candidate’s experience and skills. Privacy Notice for Candidates: https://www.reputed company.com/reputed company-policy reputed company is an equal-opportunity employer. Applicants will receive consideration for employment without regard to race, reputed company, religion, sex, national reputed company, sexual orientation, gender identity, disability, or protected veteran status. reputed company fosters a business culture where reputed company and reputed company from everyone help us grow, reputed company, create the best products, and remain relevant in a rapidly changing world. Recruiters or staffing agencies: reputed company is not obligated to compensate any external recruiter or search firm that presents a candidate or their resume or profile to a reputed company employee without 1) a reputed company, fully executed agreement on file, and 2) being assigned to the reputed company position (as a search) reputed company our applicant tracking solution. #LI-Remote Apply To This Job