- Work with multiple customers to understand their requirements and design solutions around Cloudera data platform (CDP) migration, data lake and data warehouses.
- Cloudera data platform (CDP) and Cloudera Machine learning (CML) clusters design and setup for PROD and DR.
- Review and validate the existing cluster design for CDP and CML and provide the review report.
- Design strategies for data migration from old cluster to new cluster.
- Discuss and design the strategies and solutions for ELT jobs migration across clusters.
- Design migration strategies for data transfer from on-prem clusters to the cloud.
- Review the existing security and governance practices on the clusters and submit the report.
- Architect and design the data lake solution for the cloud including various layers and data ingestion and transformation.
- Propose optimized and robust security and governance solutions for data lake on the cloud.
- Design robust and reusable frameworks/solutions across various areas like cluster setup process, data reconciliation, data migration etc.
- Propose short term and long-term roadmaps to the customer based on their requirements and long-term vision.
- Manage, guide and work with large data architecture and cloud teams.
Job Requirements:
- At least 10+ years working experience in creating data lakes and Data Warehouses both on-prem and Cloud.
- Having experience on at least 2 migration projects from Big Data appliance (Oracle BDA, IBM BDA, Teradata BDA) to Cloudera data platform (CDP 7.1.7/7.1.9).
- Experience in setting up Cloudera Clusters (CDP & CML) for at least 2 projects.
- Must be able to propose solutions using the Cloudera data platform (CDP) and Cloudera machine learning (CML) tools.
- Having very good experience in automating the routine tasks using technologies like Python, Shell scripting etc.
- Expert knowledge in devising migration strategies for data migration from BDA cluster to the new CDP cluster.
- Deep technical knowledge and expertise in migrating ELT jobs from BDA cluster to the new CDP cluster.
- Expert in performance tuning for the ELT jobs across tools like ODI, Informatica etc.
- Should be well versed in various aspects like sizing, infra, networking, compatibility etc. with respect to the Cloudera clusters.
- Hands on knowledge on implementing data lake security and governance best practices.
- Must have worked on projects on Cloud platforms like AWS, Azure etc.
- Working knowledge of projects in banking and financial domain is a must.
- TOGAF Certification is mandatory.
- Must have architect level certification in at least 2 cloud platforms amongst AWS, Azure and Google cloud.
- Must have good communication skills so as to manage the customer in terms of requirements gathering and validation.
- Must have experience in leading and managing large teams.