Key Responsibilities
- Architect and implement robust data pipelines using Apache Kafka for real-time data streaming and processing.
- Develop ETL processes to extract, transform, and load data from various sources into data warehouses like Hive, Teradata, and Clickhouse.
- Provide technical vision and create roadmaps to align with the long-term technology strategy
- Proficient in building data ingestion pipelines to ingest data from heterogeneous sources like RDBMS, Hadoop, Flat Files, REST APIs, AWS S3
- Key player in Hadoop Data Ingestion team which enables data science community to develop analytical/predictive models and implementing ingestion pipelines using Hadoop Eco-System Tools Flume, Sqoop, Hive, HDFS, Pyspark , Trino and Presto sql.
- Work on governance aspects for the Data Analytics applications such as documentation, design reviews, metadata etc
- Extensively use DataStage, Teradata\Oracle utility scripts and Data stage jobs to perform data transformation/Loading across multiple FSLDM layers
Requirements
- Bachelor’s degree in Computer Science, Engineering, or a related field.
- 6 - 10 years of experience with Data Engineering experience in the banking domain including implementation of Data Lake, Data Warehouse, Data Marts, Lake Houses etc
- Experience in data modeling for large scale data warehouses, business marts on Hadoop based databases, Teradata, Oracle etc for a bank
- Expertise in Big Data Ecosystem such as Cloudera (Hive, Impala, Hbase, Ozone, Iceberg), Spark, Presto, Kafka
- Experience in a Metadata tool such as IDMC, Axon, Watson Knowledge Catalog, Collibra etc
- Experience with distributed computing frameworks such as Apache Spark or Hadoop.
- Proficiency in programming languages like Python, Java, or Scala.
- Experience with cloud platforms like AWS, GCP, or Azure is a plus.
- Hands-on experience with data warehousing solutions and real-time data processing.
Disclaimer: The company is committed to ensuring the privacy and security of your information. By submitting this form, you consent to the collection, processing, and retention of the information you provide. The data collected (which may include your contact details, educational background, work experience and skills) will be used solely for the purpose of evaluating your qualifications for the position you're applying for. Your data will be stored securely and retained for the duration necessary to fulfill our hiring process. If you are not selected for the position, your data will be kept on file for a limited period in case future opportunities arise. You have the right to access, correct, or delete your data at any time by contacting us at Quess Singapore | A Leading Staffing Services Provider in Singapore (quesscorp.sg)
