Job Description:
- A ETL Eng will oversee the migration specializing in Informatica PowerCenter to IDMC.
- This role involves leading the migration efforts, ensuring seamless transition, and leveraging the advanced features of IDMC.
- The ideal person will have extensive experience in data integration, migration projects and hands-on expertise with both CDI-PC and IDMC.
Roles And Responsibilities:
• Manage the end-to-end migration process from informatica PowerCenter (CDI PC) to Informatica
IDMC, ensuring minimal disruption to business operations.
• Collaborate with architects and stakeholders to design robust migration strategies and solutions that meet both
functional and non-functional requirements.
• Hands on experience in Informatica IDMC for creating mappings, workflows, setting up Secure Agents etc.,
• Completed or knowledge in utilizing Informatica CDI-PC for seamless migration from PowerCenter to IDMC
• Integrate data from various sources, both internal and external, into AWS and Databricks environments,
ensuring data consistency and quality, while leveraging Informatica IDMC for data integration, transformation, and
governance.
• Develop ETL (Extract, Transform, Load) processes to cleanse, transform, and enrich data, making it suitable
for analytical purposes using Databricks' Spark capabilities and Informatica IDMC for data transformation and quality.
• Monitor and optimize data processing and query performance in both AWS and Databricks environments,
making necessary adjustments to meet performance and scalability requirements. Utilize Informatica IDMC for
optimizing data workflows.
• Implement security best practices and data encryption methods to protect sensitive data in both AWS and
Databricks, while ensuring compliance with data privacy regulations. Employ Informatica IDMC for data governance and
compliance.
• Maintain clear and comprehensive documentation of data infrastructure, pipelines, and configurations in both
AWS and Databricks environments, with metadata management facilitated by Informatica IDMC.
• Collaborate with cross-functional teams, including data scientists, analysts, and software engineers, to
understand data requirements and deliver appropriate solutions across AWS, Databricks, and Informatica IDMC.
• Identify and resolve data-related issues and provide support to ensure data availability and integrity in both
AWS, Databricks, and Informatica IDMC environments.
• Optimize AWS, Databricks, and Informatica resource usage to control costs while meeting performance and
scalability requirements.
• Stay up to date with AWS, Databricks, Informatica IDMC services, and data engineering best practices to
recommend and implement new technologies and techniques.
Requirements:
• Bachelor’s or master’s degree in computer science, data engineering, or a related field.
• Minimum 5 years of experience in data engineering, with expertise in AWS or Azure services, Databricks, and/or
Informatica IDMC.
• Extensive experience with Informatica PowerCenter (CDI-PC) and IDMC
• Strong understanding of data integration concepts, ETL process and data quality management.
• Proficiency in designing and implementing data mappings, transformations, and workflows.
• Evaluate potential technical solutions and make recommendations to resolve data issues especially on
performance assessment for complex data transformations and long running data processes.
• Strong knowledge of SQL and NoSQL databases.
• Excellent problem-solving and analytical skills.
• Strong communication and collaboration skills.
• Databricks certifications, and Informatica certifications are a plus.
Preferred Skills:
• Experience with other cloud based ETL technologies.
• Experience in AWS Services focusing on ETL and data processing.
• Familiarity with data visualization tools like Tableau or Power BI.
• Understanding of DevOps principles for managing and deploying data pipelines.
• Experience with version control systems (e.g., Git) and CI/CD pipelines.
• Knowledge of data governance and data cataloguing tools, especially Informatica IDMC.