idealo internet GmbH Logo

(Senior) Data Engineer - Data Science Team at idealo internet GmbH (Berlin, Deutschland)

About the Employer

Job Description

The position provides the opportunity to work on a wide range of interesting topics from operationalizing deep learning models to training recommender systems on petabytes of data. As part of the data science team, you will be also given a lot of responsibilities to shape the direction of the team. If you would like to become part of this success story, please send your application. About your new role You will be part of the data science team and work closely with our data scientists to operationalize machine learning pipelines You will develop and implement effective data processing architectures You will also collaborate a lot with the data warehouse and data platform team You will participate at meetups, conferences and the research community and apply what you’ve learned back at your daily work Skills & Requirements A deep understanding of distributed computing frameworks such as Spark (particularly SparkML, SparkSQL, tune/optimize and debug Spark jobs), Hadoop and/or Flink Experience with big data at AWS, in particular using EMR and S3 Experience with Docker and container orchestration like Kubernetes, Swarm or similar Experience with pipeline management tools like Airflow, Luigi or NiFi Experience with programming languages such as Python, Go and/or Scala Good knowledge of SQL/RDBMS Experience with the command line, shell scripting and version control (Git) Excellent communication skills in English, both oral and written; German is nice to have Preferably experience with automatic configuration management like Terraform and Puppet Preferably experience with modern agile software development practices like microservices, test-driven development, pair programming, CI/CD etc. At idealo you can expect: Training opportunities for your individual management or specialist career The provision of all necessary equipment that you need to deliver top performances Employees are self-reliant and free to schedule their own work day Onboarding program including Welcome Day, Trainings and individual incorporation The chance to work together on our goals, while also working to create something new 700 motivated colleagues and a cool office loft in the trendy district of Kreuzberg