Help design, build, and manage DRW’s Unified Data Platform and support its users.
Work closely with Traders and Researchers to determine appropriate data sources and implement processes to onboard and manage new data sources for analysis to unlock future trading opportunities.
Design and develop data solutions to help discover, purchase, organize, track usage, manage rights, and control quality of data sets to address the needs of various DRW trading teams and strategies.
Continually monitor data ingestion pipelines and data quality to ensure stability, reliability, and quality of the data. Contribute to the monitoring and quality control software and processes.
Requirements & Skills:
7+ years of experience working with modern data technologies and/or building data-intensive distributed systems
Expert-level skills in Java/Scala or Python with a proven ability to output high-quality, maintainable, code
Strong familiarity with SQL and Bash
Experience leveraging and building cloud-native technologies for scalable data processing.
Prior experience with both batch and streaming systems and an understanding of the limitations those paradigms impose
Experience with an array of data processing technologies (e.g. Flink, Spark, Polars, Dask, etc.)
Experience with an array of data storage technologies (e.g. S3, RDBMS, NoSQL, Delta/Iceberg, Cassandra, Clickhouse, Kafka, etc.)
Experience with an array of data formats and serialization systems (e.g. Arrow, Parquet, Protobuf/gRPC, Avro, Thrift, JSON, etc.)
Prior experience dealing with schema governance and schema evolution
Prior experience developing data quality control processes to detect data gaps or inaccuracies
A desire to mentor less experienced team-members and champion both engineering best-practices and high code-quality standards.
Strong technical problem-solving skills
Proven ability to work in an agile, fast-paced environment, prioritize multiple tasks and projects, and efficiently handle the demands of a trading environment
