Data Engineer will be part of a cross-disciplinary team, working closely with other data engineers, software engineers, data scientists, data managers, product owners and Portfolio managers
- Writes, deploys and maintains software to build, integrate, manage, maintain, and quality-assure data at bp.
- Responsible for deploying secure and well-tested software that meets privacy and compliance requirements; develops, maintains and improves CI / CD pipeline
- Responsible for service reliability and following site-reliability engineering best practices: on-call rotations for services they maintain, responsible for defining and maintaining SLAs. Design, build, deploy and maintain infrastructure as code. Containerizes server deployments.
- Actively contributes to improve developer velocity.
- Creating and code management of data pipelines and automation templates
- Creating Data Mappings and Pipelining technical documentations
- Testing new / changes to functionalities , creating test cases
Years of experience: 4 to 5 years with minimum of 2 to 3 years relevant experience
- Experience developing, maintaining, and documenting reliable and scalable data infrastructure and data products in complex environments
- Development experience in one or more object-oriented programming languages (e.g., Py-spark, Python, Java, C++)
- Experience of working in a cloud environment – AWS / Azure
- Excellent data modeling and dimensional modeling knowledge
- Excellent SQL knowledge
- Knowledge and preferably hands-on experience in technologies across all data lifecycle stages
- Experience of working in relational and non-relational data bases
- Experience using Azure DevOps, GitHub
Experience of Agile Development/Delivery methodologies