1. Home
  2. Careers
  3. Jobs at bp
  4. Cyber Security - Big Data Lead Engineer

Cyber Security - Big Data Lead Engineer

Cyber Security - Big Data Lead Engineer

  • Location United States - Texas - Houston
  • Travel required No
  • Job category IT&S Group
  • Relocation available No
  • Job type Professionals
  • Job code 126446BR
  • Experience level Senior
Apply Search all jobs at bp

Job summary

Role Synopsis
The goal of the Security Data Service is to empower its customers to understand and improve the security posture of BP by providing a consolidated, easy to use data platform for reporting and analytics.
We are looking for a Big Data Information Security Engineering Lead with strong technical expertise, who is interested in joining our team to create and manage our data infrastructure and tools, including collecting, storing, processing and analyzing a range of security data and data systems.
As a key member of this team, you will be working as part of the small empowered, customer-centric agile squad to deliver a consolidated and easy to use data platform for reporting and analytics.

Key Accountabilities

  • Collect and process raw data at scale for a variety of projects and initiatives.
  • Read, extract, transform, stage and load data to selected tools and frameworks as required and requested.
  • Build ingestion pipelines and ETL jobs to consume data feeds from security source systems (e.g. Azure feeds, O365, AD, Syslog, APIs).
  • Designing and implementing Continuous Integration / Continuous Delivery (CI/CD) solution using BP standards for the squad.
  • Creation and managing of data pipelines, transformation, and efficient storage.
  • Build monitoring and alerting mechanisms for data pipelines and transformations.
  • Managing infrastructure for Big Data and data pipelines.
  • Build flexible data solutions that support the ingestion, or the consumption needs of our data lake and its customers.
  • Lead conversations around how we can derive more value out of our data and enable operational excellence.
  • You will be a pivotal member of the squad and can shape and support the success of this service.

Essential Education

  • Focus on experience rather than education.
  • Advanced knowledge of Azure data engineering concepts.
  • AWS Foundational knowledge.
Essential Experience & Job Requirements
  • Extensive experience of developing using the Azure analytics components including Data Lake, Power BI, Data Factory, Azure Data Explorer, Azure Synapse, Data Warehouse, and Databricks.
  • Fully conversant with Agile and DevOps development methodology and concepts as applied to data driven analytics projects. Including CI/CD Coding, security testing best practice and standards.
  • Experience in DevOps in Azure cloud environments.
  • Experience with designing, building, and operating analytics solutions using Azure cloud and serverless technologies.
  • Data Management experience e.g. data profiling, large volume data handling.
  • Experience in automated data driven testing.
  • Advanced knowledge of scripting Languages like Python, Scala, or PowerShell (at least one of them).
  • Advanced knowledge of database programming including relational (SQL, Maria) and nonrelational (NoSQL, MongoDB, Cassandra) databases.
Desirable Criteria & Qualifications
  • Experience with security related data sets.
  • Experience with Machine Learning Models.
  • Object Oriented Programming knowledge is a plus.

Role Synopsis

The goal of the Security Data Service is to empower its customers to understand and improve the security posture of BP by providing a consolidated, easy to use data platform for reporting and analytics.
We are looking for a Big Data Information Security Engineering Lead with strong technical expertise, who is interested in joining our team to create and manage our data infrastructure and tools, including collecting, storing, processing and analyzing a range of security data and data systems.
As a key member of this team, you will be working as part of the small empowered, customer-centric agile squad to deliver a consolidated and easy to use data platform for reporting and analytics.

Key Accountabilities

  • Collect and process raw data at scale for a variety of projects and initiatives.
  • Read, extract, transform, stage and load data to selected tools and frameworks as required and requested.
  • Build ingestion pipelines and ETL jobs to consume data feeds from security source systems (e.g. Azure feeds, O365, AD, Syslog, APIs).
  • Designing and implementing Continuous Integration / Continuous Delivery (CI/CD) solution using BP standards for the squad.
  • Creation and managing of data pipelines, transformation, and efficient storage.
  • Build monitoring and alerting mechanisms for data pipelines and transformations.
  • Managing infrastructure for Big Data and data pipelines.
  • Build flexible data solutions that support the ingestion, or the consumption needs of our data lake and its customers.
  • Lead conversations around how we can derive more value out of our data and enable operational excellence.
  • You will be a pivotal member of the squad and can shape and support the success of this service.

Apply Search all jobs at bp