Senior Data Engineer - Barcelona, España - Hewlett Packard

Hewlett Packard

Empresa verificada

Barcelona, España

hace 2 semanas

Publicado por:

Isabel García

beBee Recruiter

Descripción

The HP Ecommerce CRM enablement team is seeking for its CDP program a proven and hands-on Databricks process architect and engineering leader to drive our Databricks transformation, pipelines architecture, governance and engineering patterns.

Our CDP delivers a unified Customer data model across our global CRM operations.

In this role, you will be responsible for driving our Databricks migration, designing and executing a comprehensive pipeline architecture strategy that aligns with CDP team's skillsets, business objectives and drives compliant, efficient and effective data management.

Responsibilities:

Drive our AWS Databricks transformation from current relational DB setup.
Collaborate with our diverse teams, SQL developers/data engineers/CRM analysts, to develop, implement and maintain sustainable, highperformance, growthready dataprocessing and data integration systems
Provide best practice on CI/CD techniques with Git and Databricks
Provide best practice on Databricks cluster management.
Reviews and evaluates designs and project activities for compliance with privacy, architecture, security and quality guidelines and standards; provides tangible feedback to improve product quality and mitigate failure risk.
Design and maintain an appropriate metadata repository to enable the organization to understand its data assets.
Builds and manages relationships throughout the organization.
Drives innovation and integration of new technologies into projects and activities in the big data space.
Collaborates and communicates with project team regarding project progress and issue resolution.
Provides guidance and mentoring to less experienced staff members.

Requirements:

Bachelor's or Master's degree in Computer Science, Information Systems, Engineering, or equivalent.
10+ years of experience in data architecture, database design, and data management.
Expert in AWS Databricks
Expert in SQL & relational database systems (PostgreSQL preferred)
Proficient knowledge in working with distributed data processing frameworks like Apache Spark.
Handson experience in Python and Pandas
Experience in overall architecture of big data systems within AWS
Experience in Global Companies a great advantage.

Knowledge & Skills

Extensive experience with designing data engineering solutions to manage data streams in complex, distributed and parallel systems.
Excellent analytical and problemsolving skills. Ability to collaborate with and analyze source systems to drive data profiling/cataloging. Can present clear insights to colleagues to support the end use of the data.
Extensive experience with data engineering pipelines, frameworks to mine, cleanse and explore data.
Strong problem management. Initiating and monitoring actions to investigate patterns and trends to resolve problems. Determining preventative measures.
Evaluating forms and processes for database architecture testing and methodology, including writing and execution of test plans, debugging, and testing scripts and tools.
Excellent written and verbal communication skills; mastery in English and local language.

Complexity

Frequently contributes to the development of new ideas and methods.
Works on complex problems where analysis of situations or data requires an indepth evaluation of multiple factors.
Acts as an expert providing direction and guidance to process improvements and establishing policies.
Exercises significant independent judgment within broadly defined policies and practices to determine best method for accomplishing work and achieving objectives.
Provide mentoring and guidance to lower level employees.