Data Engineer - #1090776

Bachelor’s degree in a relevant field such as Big Data Platforms, Computer Science, or related
discipline
You have 5+ years of experience in building or designing large-scale, fault-tolerant, distributed
systems, (for example: data lakes, delta lakes, data meshes, data lake houses, data platforms, data
streaming solutions…)
In-depth knowledge and experience in one or more large scale distributed technologies including
but not limited to: Hadoop ecosystem, Kafka, Kubernetes, Spark
Migration experience of storage technologies (e.g. HDFS to S3 Object Storage)
Integration of streaming and file based data ingestion / consumption (Kafka, Control M, AWA)
Experience in DevOps, data pipeline development, and automation using Jenkins and Octopus
(optional: Ansible, Chef, XL Release, and XL Deploy)
Expert in Python and Java or another static language like Scala/R, Linux/Unix scripting, Jinja
templates, puppet scripts, firewall config rules setup
VM setup and scaling (pods), K8S scaling, managing Docker with Harbor, pushing Images through CI/CD 
Experience using data formats such as Apache Parquet, ORC or Avro Experience in machine learning algorithms is a plus.
Hands-on experience in integrating Data Science Workbench platforms (e.g. Dataiku)
Cloud migration experience might come handy
Experience of agile project management and methods (e.g., Scrum, SAFe)
Knowledge of financial sector and its products is beneficial
 

If you are keen to explore the above role, please send across your updated resume to [email protected] and we can discuss to proceed further