Key Responsibilities:
- Data Pipeline Development: Design, build, and maintain data pipelines that extract, transform, and load (ETL) data from various sources into our data
- Data Modeling: Develop and maintain data models that optimize data storage and retrieval for analytical purposes.
- Data Integration: Collaborate with different departments to integrate data from various systems, ensuring data accuracy and consistency.
- Data Quality Assurance: Implement data quality checks and monitoring to ensure the accuracy and reliability of data.
- Data Security: Implement and maintain data security measures to protect sensitive information and comply with data privacy regulations.
- Data Documentation: Create and maintain documentation for data pipelines, schemas, and processes to facilitate knowledge sharing and transparency.
- Performance Optimization: Continuously optimize data pipelines and queries for better performance and scalability.
- Collaboration: Work closely with other stakeholders to understand their data requirements and provide the necessary support.
- Technology Stack: Stay up-to-date with the latest data engineering tools and technologies and make recommendations for their adoption when appropriate.
- Data Governance: Help establish data governance practices, including data cataloging, metadata management, and data lineage.
- Troubleshooting: Diagnose and resolve data-related issues and provide support to other teams when encountering data-related challenges.
Qualifications:
- Bachelor's degree in Computer Science, Data Engineering, or a related field
- Proven experience as a Data Engineer or similar role, preferably in a data-centric organization.
- Strong programming skills in languages such as Python, Java, or Scala.
- Proficiency in data warehousing, ETL tools.
- Experience with databases (SQL, NoSQL) and data modeling.
- Knowledge of data integration techniques and tools.
- Understanding of data privacy and security best practices.
- Excellent problem-solving and communication skills.
- Ability to work independently and collaboratively in a dynamic environment.
- Familiarity with data governance principles and data privacy regulations is a plus
