Grade HResponsible for supporting the delivery of business analysis and consulting processes and procedures for the defined specialism using sound technical capabilities, building and maintaining effective working relationships, ensuring relevant standards are defined and maintained, and supporting delivery of process and system improvements. Specialisms: Business Analysis; Data Management and Data Science; Digital Innovation.
Years of Experience: 6-8
Essential domain expertise / skillset:
Experience in Big Data Technologies – AWS, Redshift, Glue, Py-spark
•Experience of MPP (Massive Parallel Processing) databases helpful – e.g. Teradata, Netezza
•Challenges involved in Big Data – large table sizes (e.g. depth/width), even distribution of data
•Experience of programming- SQL, Python
•Data Modelling experience/awareness – Third Normal Form, Dimensional Modelling
•Data Pipelining skills – Data blending, etc
•Visualisation experience – Tableau, PBI, etc
•Data Management experience – e.g. Data Quality, Security, etc
•Experience of working in a cloud environment - AWS
•Development/Delivery methodologies – Agile, SDLC.
•Experience working in a geographically disparate team
|Reference of role definition as per the rate card - (to be filled by procurement)|
- Data Engineers develop and maintain data infrastructure and write, deploy and maintain software to build, integrate, manage, maintain, and quality-assure data at bp. Data Engineers collaborate with their business stakeholders, Data Managers, Data Scientists, Software Engineers and Architects.
- Data Engineers will have broader responsibilities than the more narrow data pipeline building capabilities that were previously emphasized. For example, Data Engineers will be also now responsible for distributed systems architecture design, data warehousing, executing on GDPR and other privacy requirements from digital security and need to have business context and knowledge about the data domains they are working with (more overlap with Data Managers).
- Data Engineers will own the end-to-end technical data lifecycle and corresponding data technology stack for their data domain and to have a deep understanding of the bp technology stack.
Key skills and experiences:
- Data Manipulation: develop / actively contribute to, debug and maintain software across the entire data lifecycle (from ingestion to deprecation), augment the data tech stack in your immediate scope and integrate these with existing data systems. Understand the end-to-end lifecycle of the data and the data tech stack deployed across your business entity / enabler area. Enhance understanding of the business entity's / enabler area's key data technologies / and research alternatives to bp platform offerings. Influence your team’s technical decisions.
- Software Engineering: Proficient in at least one object-oriented programming language. Advanced SQL skills. Understand how to use the standard bp (e.g. cloud) tools to efficiently produce software at scale. Write clean, re-usable code and commit code in manageable chunks. Write efficient and optimized algorithms. Code written has extensive test, automated monitoring and alerting coverage. Proactively find and address bugs, technical debt and inefficient practices / tools. Demonstrate the ability to debug and solve complex issues by translating business problems into technical solutions. Understand the immediate codebase, as well as its surrounding systems. The software solutions deliverd perform as described, minimizing the number of unexpected failures or changes required. In-depth knowledge of and apply software engineering best practices and processes, e.g. Continuous Integration and Continuous Deployment practices, Kubernetes, AzureDevOps or similar.-