Design and maintain scalable ETL pipelines for data integration from platforms like YouTube, Google Ads, and Pinterest, ensuring seamless data ingestion and high-quality results.
Optimize data syncing algorithms to handle large datasets efficiently, improving scalability and performance.
Collaborate with AI researchers to transform machine learning models into production pipelines, delivering actionable insights in real time.
Implement automated testing, monitoring, and validation processes to ensure data reliability and accuracy.
Manage and optimize cloud infrastructure (e.g., AWS, GCP), focusing on cost efficiency and resource scalability.
Build fault-tolerant systems to support high data volumes and ensure platform stability under heavy usage.
Research and adopt emerging technologies to continuously improve data workflows and ML deployment.
Troubleshoot and resolve technical challenges quickly and effectively.
Work closely with product and engineering teams to align on technical goals and ensure seamless integration.
Document best practices and mentor junior engineers, fostering knowledge sharing and team development.
Requirements & Skills:
4+ years of experience in data engineering roles with expertise in building and maintaining complex ETL pipelines.
Strong programming skills in Python, with a deep understanding of system engineering and data infrastructure design.
Experience deploying machine learning models in production environments and integrating them into scalable data pipelines.
Proficiency with AI technologies such as PyTorch, TensorFlow, or Jax is a strong advantage.
Solid knowledge of distributed systems, data modeling, and storage solutions for high-volume, real-time data.
Familiarity with orchestration tools (Airflow, Temporal) and containerization (Docker, Kubernetes) for managing workflows.
Proficiency with cloud platforms like AWS, GCP, Snowflake, or Databricks, including cost-effective resource management.
Knowledge of ad-tech/mar-tech platforms and data integration from external APIs and large datasets.
Strong problem-solving skills, with the ability to troubleshoot complex data issues across pipelines and integrations.
Excellent collaboration and communication skills, with comfort working in cross-functional teams.
