We are seeking an exceptional Senior/Lead Data Engineer to join our FD Data team. The ideal candidate will be a seasoned data engineer/lead with extensive experience in designing, developing, and optimising data pipelines and infrastructure.
Essential Requirements:
- Proven track record of excellent coding and software development within high-performing data engineering teams, and successfully shipping data products.
- Strong software development skills in Scala or another common programming language such as Java or Scala, adhering to best practices with a focus on low-level design documentation, readable, and well-documented code that is appropriately tested.
- Solid experience big data technologies such us with SQL and Spark.
- Experience Integrating Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data.
- Design and deploy data engineering solutions using containerization and orchestration techniques.
- Good understanding about CI/CD techniques to collaborate with DevOps teams to deploy the data solutions.
- Strong expertise in designing and developing big data ecosystems using Hadoop, Spark, and related tools (HDFS, Hive, Ranger, etc)
- Experience in ensuring data quality.
Preferred Qualifications (Nice to Have):
- Experience in the financial industry.
- Knowledge of additional data processing libraries and tools.
- Expertise in one of the major real-time data processing frameworks, such as Apache Flink or Kafka Streams.
- Experience in building event-driven and/or streaming data services.
- Experience with deploying and managing Cloud Database Services
Soft Skills:
- Excellent problem-solving skills and attention to detail.
- Strong communication skills and the ability to work collaboratively in a team environment.
