Astek is proposing an opportunity for Data Engineer (Big Data) to support our project based in Singapore.
Responsibilities
- Use big data technologies like Snowflake and work closely with the data science to help build complex algorithms that provide unique insights into the data using.
- Use agile software development processes to iteratively make improvements to the back-end systems
- Model front end and back-end data sources to help draw a more comprehensive picture of user flows throughout our system and enable powerful data analysis
- Build data pipelines that clean, transform, and aggregate data from disparate sources
- Develop models that can be used to make predictions and answer questions for the overall business
Requirements
- At least 5 years of work experience as a data engineer or backend developer in a big data field.
- A good level of understanding for all Big Data components & Administration Fundamentals.
- Solid working knowledge of implementing the optimal data structures and algorithms to create efficient and scalable applications in Java or Python.
- Experience in application integration with Snowflake, Oracle and MS-SQL.
- Familiarity with Linux or Unix and shell scripting.
- Able to drive Devops best practices like CI/CD, containerization, blue-green deployments, 12-factor apps, secrets management etc in the Data ecosystem
- Skilled in big data and related technologies like Snowflakes, Spark, Presto, Airflow, Yarn etc.
