We are seeking a highly skilled and experienced Big Data Lead Architect to oversee the design, development, and deployment of big data solutions. The ideal candidate will have strong expertise in managing and architecting large-scale data platforms using advanced technologies such as Cloudera, Azure Cloud, and Kubernetes, among others. As a Big Data Lead Architect, you will collaborate with cross-functional teams to ensure the scalability, performance, and security of big data environments.
Key Responsibilities:
- Lead the design, architecture, and implementation of end-to-end Big Data solutions.
- Oversee the integration of Cloudera CDH/CDP platforms for data management and analytics.
- Architect scalable solutions using technologies like Cloudera Machine Learning, Elasticsearch, and AML applications.
- Collaborate with data science and engineering teams to optimize the deployment of data models and machine learning pipelines.
- Ensure high availability and disaster recovery of big data platforms using technologies like Kubernetes, VMWare, and Azure Cloud.
- Automate infrastructure and deployment processes using Chef and Shell scripting.
- Implement monitoring and observability solutions using Prometheus, Grafana, and other monitoring tools.
- Manage security frameworks for big data environments using CyberArk and Trend Micro.
- Perform infrastructure troubleshooting and manage incident resolution using Remedy.
- Optimize database performance and management using PostgreSQL.
- Collaborate with cross-functional teams to ensure secure, reliable, and scalable infrastructure on HPE servers.
Required Skills & Qualifications:
- Education: Bachelor’s or Master’s degree in Computer Science, Information Technology, or related fields.
- Experience: 7+ years of experience in Big Data architecture and infrastructure, with a minimum of 3 years in a lead role.
- Technical Expertise:Cloudera CDH/CDP platforms and Cloudera Machine Learning.
Azure Cloud architecture and deployment.
Experience with Kubernetes, VMWare, RHEL, and container orchestration.
Monitoring tools: Prometheus, Grafana.
Security tools: Trend Micro, CyberArk.
Scripting: Shell scripting and automation with Chef.
Strong experience with Tableau and SAS for data visualization and analytics.
AML application expertise and experience with Elasticsearch.
Database expertise in PostgreSQL.
- Strong experience in infrastructure management with HPE servers.
- Proven experience in working with cloud and on-premise environments.
Preferred Skills:
- Experience with disaster recovery, high availability, and security compliance.
- Experience with DevOps practices and continuous integration (CI) pipelines.
- Experience managing large-scale environments and multi-tier architectures.
Soft Skills:
- Strong problem-solving and critical-thinking skills.
- Excellent communication and leadership abilities.
- Ability to work collaboratively in a fast-paced, dynamic environment.
- Strong organizational and multitasking skills.
