Staff Data Engineer

Job summary

Responsible for delivering business analysis and consulting activities for the defined specialism using advanced technical capabilities, building and maintaining effective working relationships with a range of stakeholders, ensuring relevant standards are defined and maintained, and managing process and system improvements to deliver business value. Specialisms: Business Analysis; Data Management and Data Science; Digital Innovation.

For this role specifically, you will be part of bp’s Data & Analytics Platform organisation, the group responsible for the platforms and services that operate bp’s big data supply chain. The portfolio covers technologies that support the life cycle of critical data products in bp, bringing together data producers and consumers through enablement and industrial scale operations of data ingestion, processing, storage and publishing, including data visualization, advanced analytics, data science and data discovery platforms. You will be part of the Data Governance & Compliance Management team, providing the governance technology services to the data clearing house for all of bp’s big data and analytics requirements.

• Design, implement and maintain reliable and scalable data infrastructure, including design and development of industrial scale data pipelines on Azure and AWS data platforms and services, building data ingestion and publishing pipelines, and development and provisioning of data sets for wide scale access for data professionals.

• Design and develop software for distributed systems, data warehouses, execute on GDPR and other privacy requirements from digital security and need to have business context and knowledge about the data domains they are working in.

• Own the end-to-end technical data lifecycle and corresponding data technology stack for their data domain and to have a deep understanding of the bp technology stack.

• Write, deploy and maintain software to build, integrate, manage, maintain, and quality-assure data, and responsible for deploying secure and well-tested software that meets privacy and compliance requirements; develops, maintains and improves CI / CD pipeline.

• Adhere to and advocate for software engineering best practices (e.g. technical design, technical design review, unit testing, monitoring & alerting, checking in code, code review, documentation).

• Responsible for service reliability and following site-reliability engineering best practices: on-call rotations for services they maintain, responsible for defining and maintaining SLAs. Design, build, deploy and maintain infrastructure as code. Containerizes server deployments.

• Actively contribute to improve developer velocity.

• Participate in industry working group for standard evolution.

• Actively mentor others, contributes to or leads data engineering learning and development paths