Job Description
Avensys is a reputed global IT professional services company headquartered in Singapore. Our service spectrum includes enterprise solution consulting, business intelligence, business process automation and managed services. Given our decade of success, we have evolved to become one of the top trusted providers in Singapore and service a client base across banking and financial services, insurance, information technology, healthcare, retail and supply chain.
We are currently looking for Databricks Engineer who has proven track record in IT Industry. This is an exciting opportunity to expand your skill set, achieve job satisfaction and work-life balance. More details as below.
Databricks Engineer
A Databricks Engineer is responsible for migrating from as-is on-perm data warehouse to Databricks and sound
knowledge of Databricks. Requires good understanding of data architecture, cloud solutions, building and maintaining
a robust, integrated, and governed data infrastructure. The role involves extracting valuable insights from data while
ensuring data security, compliance, and high-quality data management.
Roles And Responsibilities:
• Lead end to end data migration project from on-premises environments to Databricks with minimal downtime.
• Work with architects and lead solution design to meet functional and non-functional requirements.
• Hands on experience in Databricks to design and implement the solution on AWS.
• Hands on experience in configuring Databricks clusters, writing Pyspark codes, build CI/CD pipelines for the
deployments.
• Highly experienced in optimization techniques (Zordering, Auto Compaction, vacuuming)
• Process near real time data through Auto Loader, DLT pipelines
• Must have strong background in python and able to identify, communicate and mitigate risks and issues.
• Identify and resolve data-related issues and provide support to ensure data availability and integrity.
• Optimize AWS, Databricks resource usage to control costs while meeting performance and scalability requirements.
• Stay up to date with AWS, Databricks services, and data engineering best practices to recommend and
implement new technologies and techniques.
• Proactively implement engineering methodologies, standards, and leading practices.
Requirements / Qualifications
• Bachelor’s or master’s degree in computer science, data engineering, or a related field.
• Minimum 5 years of experience in data engineering, with expertise in AWS or Azure services, Databricks, and/or
Informatica IDMC.
• Proficiency in programming languages such as Python, Java, or Scala for building data pipelines.
• Evaluate potential technical solutions and make recommendations to resolve data issues especially on
performance assessment for complex data transformations and long running data processes.
• Strong knowledge of SQL and NoSQL databases.
• Familiarity with data modelling and schema design.
• Excellent problem-solving and analytical skills.
• Strong communication and collaboration skills.
• Databricks certifications, and Informatica certifications are a plus.
Preferred Skills:
• Experience with big data technologies like Apache Spark and Hadoop on Databricks.
• Experience in AWS Services focusing on data and architecting.
• Knowledge of containerization and orchestration tools like Docker and Kubernetes.
• Familiarity with data visualization tools like Tableau or Power BI.
• Understanding of DevOps principles for managing and deploying data pipelines.
• Experience with version control systems (e.g., Git) and CI/CD pipelines.
• Knowledge of data governance and data cataloguing tools, especially Informatica IDMC.
WHAT’S ON OFFER
You will be remunerated with an excellent base salary and entitled to attractive company benefits. Additionally, you will get the opportunity to enjoy a fun and collaborative work environment, alongside a strong career progression.
To submit your application, please apply online or email your UPDATED CV in Microsoft Word format to [email protected] Your interest will be treated with strict confidentiality.
CONSULTANT DETAILS:
Consultant Name : Seema Verma
Avensys Consulting Pte Ltd
EA Licence 12C5759
Privacy Statement:
We take your personal data protection seriously and adhere to both EU and local data protction regulations.
Upon submission of your CV, you grant Avensys Consulting permission to retain your personal information in our electronic database, unless you specify otherwise. This data will be used to evaluate your suitability for current and potential job openings within our organization. Should you wish to have your personal data removed at any point, a simple notification to us will suffice.
Rest assured, we will not disclose your personal information to any third parties, and we remain steadfast in our commitment to providing equal opportunities to all applicants.
