Big Data Engineer (m/f)
Responsibilities
- Participate in design and development of Big Data analytical applications from an early product vision to the implementation on a scalable cluster.
- Closely work together with data scientists and business owners to find best options for machine learning solutions in a mixed scenario of batch, real-time, structured and unstructured data. Evaluate implementation options and identify the most appropriate solution for a given analytical use case.
- Design and implement scalable big data structures on Hadoop with Spark, HDFS, Hive, HBase, as well as on classical RDBMS.
- Implement data ingestion and –cleaning workflows, analyze and optimize data structures and pre-process raw data, in particular real-time streaming data using.
- Integration of huge structured and unstructured data from different data sources, platforms, historical data or real-time streams.
Qualifications
- Master degree in Computer Science, Mathematics, or related technical discipline.
- First experience in data engineering on Hadoop, implementing solutions on Hive, HBase, HDFS using Spark and Scala.
- Experience in data engineering with RDBMS, SQL.
- Experience with public cloud services, preferably Azure
- First experience in programming real-time stream processing with Spark Streaming, Kafka and Storm.
- Experience in integrating Big Data machine learning solutions with MLlib, Python, R and Scala.
- Coding batch jobs and querying data in Java, Scala, SQL, bash, Python, pig.
- Experience with various ETL techniques and frameworks such as Flume or Talend.
- Fluent in English, German considered as a plus.
What we offer
We offer…
- the opportunity to shape the “Energiewende”,
- the chance to shape E.ON’s future,
- exciting business challenges to be solved,
- a space to develop ideas and create personal impact,
- an agile and hands-on environment.
Therefore, if you…
- want to work on complex and innovative projects,
- aspire to take the next step and inspire others with your ideas,
- grow on major challenges and just get started when others give up,
…you will be our perfect match!
This includes:
- flexible working hours,
- regular trainings,
- competitive salary,
- pension scheme,
- good development possibility,
- discretionary bonus scheme