We are looking for a highly experienced Senior Engineer with a minimum of 7-9 years in data engineering, specializing in Talend, Informatica, and Azure Databricks. The ideal candidate will have a proven track record in designing and implementing scalable data solutions, managing ETL processes, and ensuring data quality in cloud environments. This role requires strong technical skills along with the ability to collaborate effectively with cross-functional teams and stakeholders.
Key Responsibilities:
- Data Integration: Design and implement ETL processes using Talend and Informatica to integrate data from various sources into Azure Databricks and other cloud-based platforms.
- Azure Databricks Management: Develop and manage data pipelines and workflows in Azure Databricks, optimizing performance and ensuring data reliability.
- Data Architecture: Collaborate with architects to design and implement data models and data warehouses that meet business requirements.
- Data Quality Assurance: Establish and monitor data quality standards and processes, ensuring the accuracy and reliability of data across platforms.
- Performance Tuning: Optimize ETL processes for performance and scalability, ensuring efficient data processing in large datasets.
- Stakeholder Collaboration: Work closely with business analysts, data scientists, and other stakeholders to understand data needs and deliver actionable insights.
- Experience: 7-9 years of experience in data engineering, with a strong focus on Talend, Informatica, and Azure Databricks.
- Technical Skills:Proficiency in Talend and Informatica for ETL processes.
Strong experience with Azure Databricks, including Spark SQL and data engineering practices.
Knowledge of data modeling, data warehousing, and data governance best practices.
Familiarity with programming languages such as Python or Scala is preferred.
- Analytical Skills: Strong analytical and problem-solving skills with attention to detail.
- Communication Skills: Excellent verbal and written communication skills, with the ability to convey complex data concepts to non-technical stakeholders.
