Data Engineer - Madrid, España - Daniel J. Edelman Holdings

Daniel J. Edelman Holdings

Empresa verificada

Madrid, España

hace 1 semana

Publicado por:

Isabel García

beBee Recruiter

Responsibilities:

Design, build, and maintain scalable and robust data pipelines to support analytics and machine learning models, ensuring high data quality and reliability for both batch & realtime use cases.
Design, maintain, optimize data models and data structures in tooling such as Snowflake and Databricks.
Leverage Databricks for big data processing, ensuring efficient management of Spark jobs and seamless integration with other data services.
Utilize PySpark and/or Ray to build and scale distributed computing tasks, enhancing the performance of machine learning model training and inference processes.
Monitor, troubleshoot, and resolve issues within data pipelines and infrastructure, implementing best practices for data engineering and continuous improvement.
Diagrammatically document data engineering workflows.
Collaborate with other Data Engineers, Product Owners, Software Developers and Machine Learning Engineers to implement new product features by understanding their needs and delivery timeously.

Qualifications:

Minimum of 3 years experience deploying enterprise level scalable data engineering solutions.
Strong examples of independently developed data pipelines endtoend, from problem formulation, raw data, to implementation, optimization, and result.
Proven track record of building and managing scalable cloudbased infrastructure on AWS (incl. S3, Dynamo DB, EMR).
Proven track record of implementing and managing of AI model lifecycle in a production environment.
Experience using Apache Airflow (or equivalent), Snowflake, Lucenebased search engines.
Experience with Databricks (Delta format, Unity Catalog).
Advanced SQL and Python knowledge with associated coding experience.
Strong Experience with DevOps practices for continuous integration and continuous delivery (CI/CD).
Experience wrangling structured & unstructured file formats (Parquet, CSV, JSON).
Understanding and implementation of best practices within ETL end ELT processes.
Data Quality best practice implementation using Great Expectations.
Realtime data processing experience using Apache Kafka Experience (or equivalent) will be advantageous.
Work independently with mínimal supervision.
Takes initiative and is actionfocused.
Mentor and share knowledge with junior team members.
Collaborative with a strong ability to work in crossfunctional teams.
Excellent communication skills with the ability to communicate with stakeholders across varying interest groups.
Fluency in spoken and written English.

LI-RT9