As a Data engineer, you will be a part of a highly qualified team, leading the delivery of modern data platforms using Azure Services and Databricks.
Responsibilities:
- Designing and delivering software using an agile and iterative approach based on Scrum or Kanban
- Following and contributing to improvement of client's software engineering practices e.g., source code management, test-driven development, automated testing, CI/CD
- Supporting the Lead Data Engineer in technical architecture and design
- Building an understanding of how data is used within client's commercial activity, and using this knowledge when working with Business Analysts or users to identify system requirements
- Analyzing and estimating IT changes giving input on technical opportunities, constraints, and trade-offs
- Creating documentation and presenting to technical and non-technical audiences
- Providing handover to Support teams and providing third line support for short periods after releases
- Owning their own learning so that they remain a technical subject matter expert.
- Any ad-hoc duties as assigned by supervisors.
Technical Skills:
- Having solid experience in data modeling, data warehousing, SQL Server database design and development (SQL, relational and dimensional modelling) Analytical querying using technologies such as SSAS cubes or columnar data stores, Python.
- ETL tools such as SSIS, Alteryx, Azure Data Factory
- Building data pipelines using code e.g. Pyspark
- Modern cloud-based data architectures and technologies such as Data Lake, Delta Lake, Data Lakehouse e.g. Azure Data Lake Gen 2, Azure Delta Lake Lakehouse architecture using Databricks, Hudi or Iceberg
- NoSQL using CosmosDB, MongoDB, DynamoDB etc.
- Python scripting, including data engineering and machine learning libraries
- Scripting languages such as Powershell, bash, DAX, M-language etc. (Note - We primarily use SQL, Pyspark and Python in our day-to-day work along with some coding around DAX, Bash, Powershell and other scripting languages. )
- Distributed computing mainly around Spark ยง Big data files formats e.g. Parquet, Avro, ORC etc.
- Big data technologies such as Hadoop, Hive, Map-Reduce, Partition design, performance tuning in Spark etc.
- Reporting and visualisation tools such as SSRS, Tableau, Power BI or Excel
Interested applicants, please email your resume to Karin Chan Wei Kien
Email: [email protected]
CEI Reg No: R1104584
EA Licence No: 99C4599
Recruit Express Pte Ltd
