1. Home
  2. Careers
  3. Jobs at bp
  4. Senior Data Engineer

Senior Data Engineer

Senior Data Engineer

  • Location India - Maharashtra - Pune
  • Travel required Yes - up to 10%
  • Job category IT&S Group
  • Relocation available No
  • Job type Professionals
  • Job code 130855BR
  • Experience level Intermediate
Apply Search all jobs at bp

Job summary

Grade HResponsible for supporting the delivery of business analysis and consulting processes and procedures for the defined specialism using sound technical capabilities, building and maintaining effective working relationships, ensuring relevant standards are defined and maintained, and supporting delivery of process and system improvements. Specialisms: Business Analysis; Data Management and Data Science; Digital Innovation.

Years of Experience: 6-8

Essential domain expertise / skillset:

Experience in Big Data Technologies – AWS, Redshift, Glue, Py-spark
•Experience of MPP (Massive Parallel Processing) databases helpful – e.g. Teradata, Netezza
•Challenges involved in Big Data – large table sizes (e.g. depth/width), even distribution of data
•Experience of programming- SQL, Python
•Data Modelling experience/awareness – Third Normal Form, Dimensional Modelling
•Data Pipelining skills – Data blending, etc
•Visualisation experience – Tableau, PBI, etc
•Data Management experience – e.g. Data Quality, Security, etc
•Experience of working in a cloud environment - AWS
•Development/Delivery methodologies – Agile, SDLC.
•Experience working in a geographically disparate team

Reference of role definition as per the rate card - (to be filled by procurement)

Data Engineer:

Role Context:
- Data Engineers develop and maintain data infrastructure and write, deploy and maintain software to build, integrate, manage, maintain, and quality-assure data at bp. Data Engineers collaborate with their business stakeholders, Data Managers, Data Scientists, Software Engineers and Architects.
- Data Engineers will have broader responsibilities than the more narrow data pipeline building capabilities that were previously emphasized. For example, Data Engineers will be also now responsible for distributed systems architecture design, data warehousing, executing on GDPR and other privacy requirements from digital security and need to have business context and knowledge about the data domains they are working with (more overlap with Data Managers).
- Data Engineers will own the end-to-end technical data lifecycle and corresponding data technology stack for their data domain and to have a deep understanding of the bp technology stack.

Key skills and experiences:
- Data Manipulation: develop / actively contribute to, debug and maintain software across the entire data lifecycle (from ingestion to deprecation), augment the data tech stack in your immediate scope and integrate these with existing data systems. Understand the end-to-end lifecycle of the data and the data tech stack deployed across your business entity / enabler area. Enhance understanding of the business entity's / enabler area's key data technologies / and research alternatives to bp platform offerings. Influence your team’s technical decisions.
- Software Engineering: Proficient in at least one object-oriented programming language. Advanced SQL skills. Understand how to use the standard bp (e.g. cloud) tools to efficiently produce software at scale. Write clean, re-usable code and commit code in manageable chunks. Write efficient and optimized algorithms. Code written has extensive test, automated monitoring and alerting coverage. Proactively find and address bugs, technical debt and inefficient practices / tools. Demonstrate the ability to debug and solve complex issues by translating business problems into technical solutions. Understand the immediate codebase, as well as its surrounding systems. The software solutions deliverd perform as described, minimizing the number of unexpected failures or changes required. In-depth knowledge of and apply software engineering best practices and processes, e.g. Continuous Integration and Continuous Deployment practices, Kubernetes, AzureDevOps or similar.

  • The Senior Data Engineer will work as part of an Agile software delivery team; typically delivering within an Agile Scrum framework. Duties will include attending daily scrums, sprint reviews, retrospectives, backlog prioritisation and enhancements.
  • Will coach, mentor and support the data engineering squad on the full range of data engineering and solutions development activities covering requirements gathering and analysis, solutions design, coding and development, testing, implementation and operational support.
  • Will work closely with the Product Owner to understand requirements / user stories and have the ability to plan and estimate the time taken to deliver the user stories.
  • Proactively collaborate with the Product Owner, Data Architects, Data Scientists, Business Analysts, and Visualisation developers to meet the acceptance criteria
  • Will be very highly skilled and experienced in use of tools and techniques such as AWS Data Lake technologies, Redshift, Glue, Pyspark SQL, Athena

Apply Search all jobs at bp