Designing, developing, and maintaining solutions that manage the lifecycle of data (data acquisition, integration, storage, processing, and supporting analysis) from a range of sources, whilst taking into account the security, compliance, scalability, efficiency, reliability, fidelity, flexibility, and portability.
Developing tools that help to provide data-driven solutions and automated tasks.
Continuous testing and monitoring of systems to isolate errors quickly and improve overall performance
Proactive learning and researching new technologies and software versions
Demonstrating stakeholder engagement by communicating with the wider team to understand the functional and non-functional requirements of the data and the tool.
Working with cloud technologies and different operating systems
Supporting the development and deployment of Machine Learning models
Working closely alongside DevOps engineers, Data Scientists and Data Analysts
Working with big data technologies
Requirements & Skills:
UK DV Clearance or the ability to obtain it
Demonstrable experience in data engineering
Experience with Python
Critical thinking and ability to problem-solve
Experience writing SQL (or similar) queries
Experience with data warehousing and database systems (eg. SQL and NoSQL)
Knowledge of Database Architecture
Exposure to cloud technologies (eg. AWS and Azure)
Experience with CI/CD
Experience with developing data pipelines and data orchestration tools
Implementation of ETL (Extraction, Transformation, and Load) flows
