Data Engineer to Build data solutions for the Advertising networks / App Store Platforms / Internal systems.
Collaborate with business teams to deliver data-driven automation solutions.
Proactively monitor crawler performance, troubleshoot issues, and ensure data stability.
Build and maintain efficient and reliable ETL pipelines for data processing.
Manage a large-scale crawler infrastructure (Python/Java) deployed on Argo Workflows.
Familiarize yourself with Business CRM systems like HubSpot, create data automation, and synchronize data between current existing systems and HubSpot CRM.
Understand game industry data tracking and event logging, manage the internal tracking system Thinking Data, and its accompanying data querying tool Theano.
Utilize BI tools like Quicksight and Superset, capable of building dashboards and data insights required by the company’s business departments.
Write, debug, and optimize complex SQL queries for data analysis and reporting.
Requirements & Skills:
5+ years in Python backend development with a strong focus on data crawling techniques and best practices.
3+ years of experience with cloud infrastructure (AWS, Alibaba Cloud, or Tencent Cloud).
Solid understanding of Kubernetes concepts and experience managing Argo services.
Ability to comprehend and utilize API documentation in both English and Chinese.
Proficiency in troubleshooting and resolving issues within Java-based crawler systems.
Strong SQL skills, including experience with complex queries, query optimization, and debugging.
Experience with Postgres, MySQL, Redshift, MongoDB, and Elasticsearch databases.
Experience with BI tools and building dashboards and data visualizations with QuickSight, Superset, and Metabase.