1. Home
  2. Careers
  3. Jobs at bp
  4. Cyber Security - Big Data Engineer

Cyber Security - Big Data Engineer

Cyber Security - Big Data Engineer

  • Location United States - Texas - Houston
  • Travel required No
  • Job category IT&S Group
  • Relocation available No
  • Job type Professionals
  • Job code 127051BR
  • Experience level Intermediate
Apply Search all jobs at bp

Job summary

Role Synopsis
The goal of the Security Data Service is to empower its customers to understand and improve the security posture of BP by providing a consolidated, easy to use data platform for reporting and analytics

We are looking for an Information Security Engineering Lead with strong technical expertise in Big Data, who is interested in joining our team to create and manage our data infrastructure and tools, including collecting, storing, processing and analyzing a range of security data and data systems.

As a key member of this team, you will be working as part of the small empowered, customer-centric agile squad to deliver a consolidated and easy to use data platform for reporting and analytics.

Key Accountabilities
  • Collect and process raw data at scale for a variety of projects and initiatives.
  • Read, extract, transform, stage and load data to selected tools and frameworks as required and requested.
  • Build ingestion pipelines and ETL jobs to consume data feeds from security source systems (e.g. Azure feeds, O365, AD, Syslog, APIs).
  • Designing and implementing Continuous Integration / Continuous Delivery (CI/CD) solution using BP standards for the squad.
  • Creation and managing of data pipelines, transformation, and efficient storage.
  • Build monitoring and alerting mechanisms for data pipelines and transformations.
  • Managing infrastructure for Big Data and data pipelines.
  • Build flexible data solutions that support the ingestion, or the consumption needs of our data lake and its customers.
  • You will be a pivotal member of the squad and can shape and support the success of this service.

Essential Education

  • Focus on experience rather than education.
  • General understanding of Azure data engineering concepts.
  • AWS Foundational knowledge.
Essential Experience & job Requirements:
  • Working knowledge of developing using the Azure analytics components including Data Lake, Power BI, Data Factory, Azure Data Explorer, Azure Synapse, Data Warehouse, and Data Bricks.
  • General understanding of Agile and DevOps development methodology and concepts as applied to data driven analytics projects. Including CI/CD Coding, security testing best practice and standards.
  • Experience in DevOps in Azure cloud environments.
  • Experience with designing, building, and operating analytics solutions using Azure cloud and serverless technologies.
  • Data Management experience e.g. data profiling, large volume data handling.
  • Experience in automated data driven testing.
  • General knowledge of scripting Languages such as Python, Scala, or PowerShell (at least one of them).
  • General knowledge of database programming including relational (SQL, Maria) and nonrelational (NoSQL, MongoDB, Cassandra) databases.
Desirable Criteria & Qualifications
  • Experience with security related data sets.
  • Object Oriented Programming knowledge is a plus.
  • General understanding of Machine Learning Models.

Role Synopsis
The goal of the Security Data Service is to empower its customers to understand and improve the security posture of BP by providing a consolidated, easy to use data platform for reporting and analytics

We are looking for an Information Security Engineering Lead with strong technical expertise in Big Data, who is interested in joining our team to create and manage our data infrastructure and tools, including collecting, storing, processing and analyzing a range of security data and data systems.

As a key member of this team, you will be working as part of the small empowered, customer-centric agile squad to deliver a consolidated and easy to use data platform for reporting and analytics.

Key Accountabilities
  • Collect and process raw data at scale for a variety of projects and initiatives.
  • Read, extract, transform, stage and load data to selected tools and frameworks as required and requested.
  • Build ingestion pipelines and ETL jobs to consume data feeds from security source systems (e.g. Azure feeds, O365, AD, Syslog, APIs).
  • Designing and implementing Continuous Integration / Continuous Delivery (CI/CD) solution using BP standards for the squad.
  • Creation and managing of data pipelines, transformation, and efficient storage.
  • Build monitoring and alerting mechanisms for data pipelines and transformations.
  • Managing infrastructure for Big Data and data pipelines.
  • Build flexible data solutions that support the ingestion, or the consumption needs of our data lake and its customers.
  • You will be a pivotal member of the squad and can shape and support the success of this service.

Apply Search all jobs at bp