About the Client
Our client is a Bank and Financial Services Corporation.
Responsibilites:
- Design, develop, and implement Spark Scala applications and data processing pipelines to process large volumes of structured and unstructured data.
- Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data.
- Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in Elasticsearch.
- Collaborate with data engineers, data scientists, and other stakeholders to understand requirements and translate them into technical specifications and solutions.
- Implement data transformations, aggregations, and computations using Spark RDDs, DataFrames, and Datasets, and integrate them with Elasticsearch.
- Troubleshoot and resolve issues related to data processing, performance, and data quality in the Spark-Elasticsearch integration.
Requirement:
- Diploma or Degree in IT / Computer Science / Equivalent Qualification.
- Strong experience in developing Spark applications. Experience with Spark Streaming is a plus.
- Proficiency in Scala programming language and familiarity with functional programming concepts
- In-depth understanding of Apache Spark architecture, RDDs, DataFrames, and Spark SQL
- Experience integrating and working with Elasticsearch for data indexing and search applications.
- Solid understanding of Elasticsearch data modeling, indexing strategies, and query optimization
- Experience with distributed computing, parallel processing, and working with large datasets
- Proficient in performance tuning and optimization techniques for Spark applications and Elasticsearch queries.
Interested candidates, who wish to apply for the above position, please send in your resume to [email protected]
We regret that only shortlisted candidates will be contacted.
EA License No. 01C4394
EA Personnel Reg.No. R1104824 (Samsudeen Mohamed Basith)
“By sending us your personal data and curriculum vitae (CV), you are deemed to consent to PERSOLKELLY Singapore Pte Ltd and its local and overseas subsidiaries and affiliates collecting, using and disclosing your personal data to prospective employers/companies based in any country for purposes of evaluating suitability for employment, conducting reference checks, administering employment related services and such other purposes stated in our privacy policy. Our full privacy policy is available at www.persolkelly.com.sg. If you wish to withdraw your consent, please drop us an emailto let us know. Please feel free to contact us if you have any queries.“
