1. Home
  2. Careers
  3. Jobs at bp
  4. Staff Data Engineer

Staff Data Engineer

Staff Data Engineer

  • Location United States - Illinois - Chicago
  • Travel required Negligible travel
  • Job category IT&S Group
  • Relocation available No
  • Job type Professionals
  • Job code 126467BR
  • Experience level Senior
Apply Search all jobs at bp

Job summary

Job Profile Summary

Are you passionate about bringing together the world of data and business to motivate change? Data and Analytics are a cross-collaborative team of experts who build, integrate, manage, and maintain intelligent data products at rapid pace.
We’re looking for curious minds who are excited by the opportunity to create and deliver an exciting suite of digital products and services to advance the global energy transition. You will need to solve complex business challenges whilst collaborating and co-innovating globally, backed by a culture that encourages creativity and curiosity, and a team that brings out the best in each other.

Considering Joining bp?
At bp, we support our people to learn and grow in a diverse and exciting environment. We believe that our team is strengthened by diversity. bp is committed to encouraging an inclusive environment in which everyone is respected and treated fairly. There are many aspects of our employees’ lives that are meaningful, so we offer benefits to enable your work to fit with your life. These benefits can include flexible working options, a generous paid parental leave policy, excellent retirement benefits, and more!

Innovation & engineering

Join us in creating, growing, and delivering innovation at pace, enabling us to thrive while transitioning to a net zero ‎world. All without compromising our operational risk management.

Working with us, you can do this by:

  • deploying our integrated capability and standards in service of our net zero and ‎safety ambitions
  • driving our digital transformation and pioneering new business models
  • collaborating to deliver competitive customer-focused energy solutions
  • originating, scaling and commercialising innovative ideas, and creating ground-breaking new ‎businesses from them
  • protecting us by assuring management of our greatest physical and digital risks

Because together we are:

  • Originators, builders, guardians and disruptors
  • Engineers, technologists, scientists and entrepreneurs‎
  • Empathetic, curious, creative and inclusive

Legal disclaimer

If you are selected for a position in the United States, your employment will be contingent upon submission to and successful completion of a post-offer/pre-placement drug test(and alcohol screening/medical examination if required by the role) as well as pre-placement verification of the information and qualifications provided during the selection process. The drug screen requires a hair test for which BP must be able to obtain a sufficient hair sample for analysis (~4 cm/1 ½” scalp, or > 2 cm/¾” body – arms & armpits/legs/chest)
As part of our dedication to the diversity of our workforce, BP is committed to Equal Employment Opportunity. Applicants will receive consideration for employment without regard for race, color, gender, religion, national origin, disability, veteran status, military status, age, marital status, sexual orientation, gender identity, genetic information, neurodiversity/neurocognitive functioning or any other protected group status. We are also committed to providing reasonable accommodations for qualified individuals with disabilities and disabled veterans in our job application procedures. If you need assistance or an accommodation due to a disability, you may contact us or have one of your representatives contact us at PeopleCareHouston@bp.com; or by telephone at 833 478 7480 8am-5pm CST/CDT Monday-Friday.
Read the Equal Employment Opportunity is the Law poster and the poster supplement - for more information about Equal Employment Opportunities. (Spanish version)

Key Responsibilities:

  • Architects, designs, implements, and maintains reliable and scalable data solutions
  • Establishes metadata management, data catalogs, data lineage, data standards, data stewardship, and best practices across the organization
  • Leads the team to establish Data Governance model including people, process, and technology to sustain data quality
  • Researches and promotes new tools and techniques to shape the future of the data engineering environment
  • Works closely with data scientists, micro-service developers, and security authorities to build out data platform incrementally and securely
  • Develops data modelling best practices and acts as a subject matter expert for data models across all areas of the business
  • Leads the team to write, deploy, and maintain software to build, integrate, manage, maintain, and quality-assure data
  • Architects, designs, develops, and delivers large-scale data ingestion, data processing, and data transformation projects on the Azure cloud
  • Mentors and shares knowledge with the team to provide design reviews, discussions, and prototypes
  • Leads customer discussions from a technical standpoint to deploy, run, and audit best practices for cloud products
  • Leads the team to follow software & data engineering best practices (e.g., technical design and review, unit testing, monitoring, alerting, source control, code review & documentation)
  • Leads the team to deploy secure and well-tested software that meets privacy and compliance requirements; develops, maintains, and improves CI / CD pipeline
  • Leads the team in following site-reliability engineering best practices: on-call rotations for services they maintain, responsible for defining and maintaining SLAs. Design, build, deploy and maintain infrastructure as code. Containerizes server deployments.
  • Actively contributes to improve developer velocity
  • Part of a cross-disciplinary team working closely with other data engineers, software engineers, data scientists, data managers and business partners in a Scrum/Agile setup

Job Requirements:

Bachelor or higher degree in computer science, Engineering, Information Systems, or other quantitative fields


  1. Years of experience: 10 to 15 years with minimum of 10 years relevant experience
  2. Deep and hands-on experience (typically 5+ years) designing, planning, productionizing, maintaining, and documenting reliable and scalable data infrastructure and data products in complex environments
  3. Hands on experience with:
a) Databricks and using Spark for data processing (batch and/or real-time)

b) Configuring Delta Lake on Azure Databricks

c) Languages: Python, Scala, SQL
d) Cloud platforms: Azure (ideally) or AWS

e) Azure Data Factory

f) Azure Data Lake, Azure SQL DB, Synapse, and Cosmos DB
g) Data Management Gateway, Azure Storage Options, Stream Analytics and Event Hubs
h) Designing data solutions in Azure incl. data distributions and partitions, scalability, disaster recovery and high availability

i) Data modeling with relational or data-warehouse systems

j) Advanced hand-on experience with different query languages
k) Azure DevOps (or similar tools) for source control & building CI/CD pipelines

4) Understanding Data Structures & Algorithms & their performance

5) Experience designing and implementing large-scale distributed systems
6) Deep knowledge and hands-on experience in technologies across all data lifecycle stages
7) Stakeholder management and ability to lead large organizations through influence

Desirable Criteria:

  • Strong stakeholder management
  • Continuous learning and improvement mindset
  • Boy Scout mindset to leave the system better than you found it

Key Behaviors:

  • Empathetic: Cares about our people, our community, and our planet
  • Curious: Seeks to explore and excel
  • Creative: Imagines the extraordinary
Inclusive: Brings out the best in each other

Apply Search all jobs at bp