Cloud Administrator
Cluster Management: Deploy, configure, and manage Databricks clusters on AWS, ensuring optimal performance and resource utilization.
User Access & Security: Manage user roles, permissions, and secure access to Databricks workspaces and data.
Monitoring & Optimization: Continuously monitor cluster performance, troubleshoot issues, and optimize jobs and workflows.
Automation: Develop and maintain automation scripts for cluster provisioning, scaling, and maintenance.
Backup & Recovery: Implement backup and recovery processes to ensure data integrity and availability.
Support: Provide technical support to data engineers and analysts using Databricks for ETL, data processing, and analytics.
Requirements:
Experience: Total Experience 5+ yrs with 2+ years in Databricks administration in an AWS environment. (NOTE: overall experience can be more, but experience in Databricks Admin is key here)
Technical Skills: Strong knowledge of Databricks, Spark, AWS services (EC2, S3, IAM), and Python/Scala scripting.
Certifications: Databricks or AWS certifications.
Soft Skills: Strong troubleshooting, problem-solving, and communication skills.
This role requires expertise in managing Databricks on AWS and a proactive approach to optimizing cloud-based data processing environments.