As part of the Data Platforms team, the Data Engineer position contributes to the overall BPX Data Platform Strategy.
- Define data workflow, pipelines, security, and ELT/ETL guidelines, policies, and procedures
- Provide oversight to ensure that project teams are following the approved data workflow, pipelines, security, and ETL/ELT processing guidelines
- Evaluate, compare, and recommend new Data Platform vendor products and/or tools
- Partner with project teams to provide technical direction and new integration patterns, where needed, for complex and/or large data sets
- Present modifications to Data Platforms to the Architecture Review Board for approval
- Responsible for the development, guidance, and oversight for data ingestions and integrations for BPX cloud data platforms.
- Work with project teams to provide direction on approved patterns to meet the project data requirements for our Data Platforms.
- Strong SQL development knowledge required to provide support to project developers/users for database design, data flow and analysis activities.
- Development and deployment of innovative big data platforms for advanced analytics and data processing.
- Define and build the data rules and pipelines that will enable faster, better, data-informed decision-making within the business.
- Ensure stable solutions, stable infrastructure, and participate in the design and build out of data solutions.
- The database and data support roles are outsourced to a managed services partner that resides within the Data Platform team, and this role will require the ability to work and partner with those resources.
- Create data storage solutions optimized for performance and maintain data architecture standards across BPX Data Platforms.
- Partner with the Data Quality Lead to incorporate testing and validation scripts to support the accuracy of data transformations.
- Implement Master Data Management solutions, processes, and governance to support the Data Management team.
- Partner with the DataOps support team to troubleshoot data issues and provide guidance for solving these issues.
- Drive excellent, consistent customer service.
Essential Experience and Job Requirements
- BS in Computer Science, MIS, Mathematics, or equivalent degree preferred but not required.
- 3-5+ years of Data Engineering experience
- 3-5+ years of relevant work experience in IT/Data and/or Analytics space
- Experience in any cloud data warehouse, Snowflake preferred
- Experience with replication tools, Fivetran preferred
- Experience with transformation tools, dbt preferred
- Experience with programming tools, python preferred
- Experience with REST APIs for data ingestion
- Strong understanding of ETL/ELT processing with large data stores
- Experience designing and delivering large scale, 24-7, mission-critical data pipelines and features using modern big data architectures
- Stream processing services such as Kafka, AWS Kinesis, Apache Storm, Spark Streaming, Azure Event Grid etc
- Demonstrated experience working in large-scale data environments which includes real-time and batch processing
- Strong data modeling skills (relational, dimensional, and flattened)
- Strong analytical and SQL skills, with attention to detail
- Ability to aid in tuning and performance recommendations for poor performing SQL queries and/or python scripts
- Knowledge of Database Administration tasks – Indexing, SQL Tuning/Performance, Backup/Recovery, DR
- Ability to work with multiple external teams and accomplish shared goals by building consensus
- Strong communication (written/verbal) and collaboration skills
- Consulting, negotiation, and relationship skills
- Problem solving skills
We offer a reward and wellbeing package to enable your work to fit with your life. These can include, but not limited to, access to health, vision and dental insurance, flexible working schedule, paid time off policy, discretionary annual bonus program, long-term incentive program, and a generous 401K matching program. How much do we pay (Base)? $126,014-$194,530
*Note that the pay range listed for this position is a good faith and reasonable estimate of the range of possible base compensation at the time of posting.