x
Get our mobile app
Fast & easy access to Jobstore
Use App
Congratulations!
You just received a job recommendation!
check it out now
Browse Jobs
Companies
Campus Hiring
Download App
Jobs in Singapore   »   Jobs in Singapore   »   Information Technology Job   »   Data Engineer
 banner picture 1  banner picture 2  banner picture 3

Data Engineer

One North Consulting Pte. Ltd.

One North Consulting Pte. Ltd. company logo

One North Consulting is currently hiring Data Engineers with about 7+ years of experience as per details given below.

Interested Singapore Citizens / Singapore Permanent Residents can connect on email id - [email protected]

Skills & Responsibilities: -

As Data Engineer, you will focus on managing the Hadoop cluster, implementing data ingestion framework designing data models.

Your primary role will be to implement data lake & transform the data for business use.

  • Experience with at least one Cloud Infra provider (Azure/AWS)
  • Experience in building data pipelines using batch processing with Apache Spark (Spark SQL, DataSet / Dataframe API) or Hive query language (HQL)
  • Knowledge of Big data ETL processing tools
  • Write SQL scripts to test ETL results
  • Write scripts to regression test the ETL iterations already completed
  • Experience with Hive and Hadoop file formats (Avro / Parquet / ORC)
  • Basic knowledge of scripting (shell / bash)
  • Experience with CI CD tools such as Jenkins, JIRA, Bitbucket, Artifactory, Bamboo and Azure Dev-ops.
  • Understanding of DevOps practices using Git version control
  • Debug, fine tune and optimize large scale data processing jobs
  • Deliver big data solutions based on premise Hadoop or cloud-based systems like AWS.
  • Manage Hadoop cluster, participate in scale out planning & implementation.
  • Design ingestion layer for structured & unstructured data (text, voice, xml etc)
  • Implement specific data model for business & analytics use.
  • Deliver ELT solution including data extraction, transformation, cleansing, data integration and data management.
  • Augment with new sources of data including internal/external untapped data.
  • Contribute to the establishment and maintenance of cloud computing platform and big data services.
  • Provide support for analytics tools & environment like RServer etc & debug performance issues.

Competencies :-

  • Databases: RDBMS, SQL programming
  • ETL and Data Integration Tools: Azure Data Factory (ADF), Microsoft SQL Server Integration Service(SSIS), SAS Data Integration
  • Big Data: Hadoop (Hortonworks), Hive, Spark, Sqoop, etc
  • Programming and Scripting: Linux/Unix Shell Scripting, Java, Scala, Hive QL
  • BI/Dashboarding: SAS Visual Analytics, Qliksense

Working Experience:

  • 7+ years (no upper limit) in data engineering and modelling.
  • Hands-on in MS SQL. Should be an expert in MS SQL queries and scripting.
  • Hands-on experience in testing ETL systems is mandatory
  • Hands-on in managing data mapping, data quality and integrity, performance in data processing.
  • Develop and implement techniques and analytics applications to transform raw data into meaningful information
  • Experience in Agile software development.
  • Candidate should be able to create executable scripts for regression testing purposes
  • Experience in testing SSIS ETL mappings is good to have
  • Visualise, interpret, and report data findings and may create dynamic data reports as well

If interested, please email us your CV to [email protected]

✱   This job post has expired   ✱

Sharing is Caring

Know others who would be interested in this job?