Data Engineer - Madrid, España - Talan
Descripción
Company Description
Talan is an international advisory group on innovation and transformation through technology, with 5000 employees, and a turnover of 600M€.
- CONSULTING in management and innovation : supporting business, managerial, cultural, and technological transformations.
- DATA & TECHNOLOGY to implement major transformation projects.
- CLOUD & APPLICATION SERVICES to build or integrate software solutions.
- SERVICE CENTERS of EXCELLENCE to support the latter through technology, innovation, agility, sustainability of skills and cost optimization.
By understanding their challenges, with our support, innovation, technology and data, we enable them to be more efficient and resilient.
We believe that only a human oriented-practice of technology will make the new digital age an era of progress for all.
Job Description:
The data engineer is a key role in any of the different data squads in charge of development and maintenance of new big data data platforms and data products.
His/her main mission is to develop the different data pipelines which ingest, transform and prepare data from different sources into the different layers of our datahub platform and/or data products.
ROLES / RESPONSIBILITIES- Develop data pipelines with Spark and Scala to ingest and transform data from different sources (API, files, Kafka topics) into HDFS, IBM Cloud Storage, Parquet or SQL/NOSQL databases
- Manage big data storage solutions in the platform (HDFS, IBM Cloud Storage, Parquet)
- Implementation of schemas and queries in SQL/NOSQL databases like postgres, MongoDB or Oracle
- Develop and configure scheduling of data pipelines with a combination of shell scripting and autosys, oozie or AirFlow
- Configure Dremio Data Virtualization to interface with Parquet or as a way to expose the data in the different data products
- Configure Dataiku pipelines as a data preparation tool for loading and transformation of data from our big data platform into specific datamarts for data or business products
- Provide N3 level support to end users
Qualifications:
- You have a higher degree in the field of business intelligence or data science or computer science with at least 5 years of experience in a similar position.
- Your knowledge of the modern BI technology stack is your strength, ideally in the Talend / Snowflake / Tableau environment.
- You have a confirmed experience in leading workshop with the business for the definition of KPI, as well as in the development of data pipeline with Talend, from the sources to the data warehouse, and then the development of dashboards in Tableau.
- You are comfortable communicating with the professions and synthesizing the issues. You are structured in your approach while remaining result oriented.
- Your collaborative spirit makes you a key partner and you are familiar with agile methodologies.
- Experience in marketing data (Google Analytics, social media and custom solutions) is a plus.
- You are fluent in English and ideally also in French.
IT TOOLS
- Spark and Scala as legacy data pipeline development language
- SparkaaS and Scala as data pipeline development platform
- Management of legacy big data storage solutions (HDFS and Parquet)
- Management of big data storage solutions (IBM Cloud Object Storage and Parquet)
- Implementation of SQL/NO SQL database schemas and queries (MongoDB, Oracle, Postgres)
- Ksh, Autosys and Oozie as legacy data pipeline scheduling solution
- Ksh and Airflow as data pipelie scheduling solution
- Dremio as data virtualization too.
- Dataiku as data preparation toolMigration from SAS to Spark as a bonus.
BUSINESS SKILLS
Knowledge of banking industry and processes
Business / IT Relationship
BEHAVIOURAL SKILLS
Ability to understand, explain and support change
Ability to Deliver / Results Driven
Ability to collaborate / Teamwork
Ability to manage a project as a bonus
TRANSVERSAL SKILLS
Agile
LANGUAGES
English C1
French as a bonu
Más ofertas de trabajo de Talan
-
Junior Software Integration IAM
Málaga, España - hace 3 días
-
Azure Platform Engineer
Malaga, España - hace 2 semanas
-
QA Engineer
Madrid, España - hace 2 semanas
-
Data Scientist for transactions monitoring
Málaga, España - hace 3 semanas
-
Telecom & Network Product Manager
Madrid, España - hace 5 días
-
Senior Scrum Master
España - hace 1 semana