Summary:
We are seeking a highly skilled and experienced Data Engineer to design, develop, and deploy big data solutions. The ideal candidate will have strong expertise in designing and architecting large-scale data platforms using advanced technologies such as Cloudera, Azure Cloud, and Kubernetes, among others.
Key Responsibilities:
- Responsible for the design, architecture, and implementation of end-to-end Big Data solutions.
- Apply data mining and data modeling to build data analytic solution based on project requirements. Collaborate with data science and engineering teams to optimize the deployment of data models and machine learning pipelines.
- Design and architect scalable solutions using technologies like Cloudera Machine Learning, Elasticsearch, and AML applications. Actively involve in the integration of Cloudera CDH/CDP platforms for data management and analytics.
- Ensure high availability and disaster recovery of big data platforms using technologies like Kubernetes, VMWare, and Azure Cloud.
- Automate infrastructure and deployment processes using Chef and Shell scripting.
- Implement monitoring and observability solutions using Prometheus, Grafana, and other monitoring tools.
- Manage security frameworks for big data environments using CyberArk and Trend Micro.
- Perform infrastructure troubleshooting and manage incident resolution using Remedy.
- Optimize database performance and management using PostgreSQL.
- Collaborate with cross-functional teams to ensure secure, reliable, and scalable infrastructure on HPE servers.
Required Skills & Qualifications:
Education:
- Bachelor’s or Master’s degree in Computer Science, Information Technology, or related fields.
Experience:
- At least 5 years of experience in Big Data architecture and infrastructure.
Technical Expertise:
a) Cloudera CDH/CDP platforms and Cloudera Machine Learning.
b) Azure Cloud architecture and deployment.
c) Experience with Kubernetes, VMWare, RHEL, and container orchestration.
d) Monitoring tools: Prometheus, Grafana.
e) Security tools: Trend Micro, CyberArk.
f) Scripting: Shell scripting and automation with Chef.
g) Strong experience with Tableau and SAS for data visualization and analytics.
h) AML application expertise and experience with Elasticsearch.
i) Database expertise in PostgreSQL.
j) Strong experience in infrastructure management with HPE servers.
k) Proven experience in working with cloud and on-premise environments.
Preferred Skills:
a) Experience with disaster recovery, high availability, and security compliance.
b) Experience with DevOps practices and continuous integration (CI) pipelines.
c) Experience managing large-scale environments and multi-tier architectures.
Soft Skills:
a) Strong problem-solving and critical-thinking skills.
b) Excellent communication and leadership abilities.
c) Ability to work collaboratively in a fast-paced, dynamic environment.
d) Strong organizational and multitasking skills.
