Job Description
We are seeking a skilled and motivated Intermediate Data Engineer to join our dynamic team. The successful candidate will play a crucial role in designing, developing, and maintaining our data infrastructure, ensuring efficient processing and analysis of large datasets. As an Intermediate Data Engineer, you will collaborate with cross-functional teams to implement data solutions that align with our business objectives.
Key Responsibilities:
Data Pipeline Development: Design, implement, and maintain scalable data pipelines for extracting, transforming, and loading large datasets.
Data Modelling: Develop and optimize data models to support analytical and reporting requirements, ensuring data accuracy and reliability.
Data Integration: Integrate diverse data sources and formats to enable seamless data flow across systems.
Data Quality Assurance:Implement and monitor data quality checks to ensure the integrity of data throughout the lifecycle.
Collaboration: Work closely with business analysts, colleagues, and stakeholders to understand data requirements and deliver effective solutions.
Performance Optimization: Identify and address performance bottlenecks in data processing and storage systems.
Documentation: Document data engineering processes, models, and configurations for knowledge sharing and future reference.
Adherence to Best Practices: Stay updated on industry trends and best practices in data engineering and promote their adoption within the team.
Requirements:
· Bachelor’s degree in computer science, Information Technology, or a related field.
· Minimum of 5 years of proven experience as a Data Engineer or in a similar role.
· Strong proficiency in programming languages like Spark SQL and Python, especially in a HIVE environment.
· Familiarity with cloud platforms (AWS), Big Data infrastructure, and tools such as Jira and Bitbucket.
· Experience with CI/CD pipelines and working in an Agile environment.
· Excellent problem-solving skills and attention to detail.
· Strong communication and collaboration skills.
· Knowledge of data warehousing concepts and technologies.
· Candidates with experience in the Banking Industry will have an added advantage.
· Ability to communicate effectively with multiple levels of management and summarize complex technical requests succinctly.
