Roles and Responsibilities
Cluster Setup and Maintenance:
- Install, configure, and maintain big data platforms like Hadoop Clusters, Elastic Search, Spark, Kafka, Hive etc.
- Set up and manage distributed file systems (HDFS) and NoSQL databases (HBase, Cassandra).
- Ensuring high availability and reliability of Hadoop, Kafka and Elasticsearch Clusters.
Performance Optimization:
- Monitor system performance, fine-tune jobs, and optimize cluster utilization.
- Manage and resolve performance bottlenecks in big data environments.
Security Management:
- Implement data security policies, user authentication (Kerberos), and access control (Ranger, Knox).
- Regularly review and plan patch systems to ensure data integrity.
- Apply updates, patches, and version upgrades to all managed systems.
- Ensuring the backward compatibility and minimal downtime during upgrades.
Backup and Recovery:
- Plan and implement backup and recovery strategies for big data environments.
- Develop disaster recovery solutions and test failover scenarios.
Workflow Automation:
- Design and implement workflows using tools like Apache Oozie or Apache Airflow.
- Automate cluster management tasks like upgrades and scaling.
- Developing shell scripts and Ansible Playbooks to automate routine tasks, including monitoring, backups, and deployments.
Monitoring and Support:
- Monitor logs, metrics, and system health using tools like Ambari, Cloudera Manager, or Prometheus.
- Provide 24/7 on-call support for production systems and resolve incidents.
Collaboration:
- Work closely with data engineers and developers to support ETL pipelines and data processing jobs.
- Collaborate with stakeholders to ensure big data systems meet organizational needs.
- Working transversally with other teams to guarantee high data quality and availability.
Cloud migration
- Migration planning from On-premises to GCP & AWS cloud
Required Skills and Qualifications
Technical Skills:
- Strong knowledge of big data ecosystems (Hadoop, HDFS, YARN, Hive, HBase, Spark, Kafka and Elastic Search).
- Proficiency in Linux/Unix system administration.
- Experience with monitoring tools like Nagios, Grafana, or Splunk, HPOMI.
- Scripting skills in Python, Ansible, Shell, or Bash.
- Familiarity with cloud platforms (AWS, Azure, GCP) and their big data services.
Soft Skills:
- Strong problem-solving and analytical abilities.
- Effective communication and collaboration skills.
- Ability to handle high-pressure production environments.
Preferred Experience
- 8-9 years of experience in big data administration or similar roles.
- Hands-on experience in scaling and managing large, big data clusters in production environments.
Interested applicants please send your resume in MS Word format and attention it to Tan Joanna (R1104661)
Shortlisted candidates will be notified
Data provided is for recruitment purposes only
www.ambition.com.sg
EA Registration Number: R1104661
Business Registration Number: 200611680D. Licence Number: 10C5117
