Job Overview
The Senior Data Engineer will be responsible for requirements gathering, solutioning, designing and building modern data platforms to support data-driven decision making. The Senior Data Engineer will execute technical
implementation of data engineering and visualization projects, and will be a hands-on role.
The Senior Data Engineer will help build a data and analytics consulting practice by taking part in recruiting efforts,
creating technical collateral, and staying on top of technology trends with ongoing training and certifications.
Responsibilities
● Pre-sales support: Support pre-sales activities, including whiteboard sessions, collaborating on solution
architecture design, and assisting in proposal and statement of work creation.
● Architect and build scalable data pipelines: Craft secure, efficient pipelines to ingest, transform, and load
data from diverse sources, ensuring quality and availability for downstream analysis.
● Design and optimize data infrastructure: Shape the future of our data platform, choosing and implementing
cutting-edge technologies like cloud storage, data lakes, and streaming platforms.
● Own data quality and governance: Champion data quality initiatives, establish data standards, and
implement monitoring and alerting systems to ensure data integrity.
● Collaborate with stakeholders: Bridge the gap between data and business, translating business needs into
technical solutions and working closely with data analysts, scientists, and business users.
● Automate and innovate: Drive automation initiatives to streamline data workflows and build reusable
components for greater efficiency and maintainability.
● Mentor and guide: Share your expertise by mentoring junior engineers and fostering a collaborative data
culture.
Qualifications
● 8+ years of experience in dealing with data i.e. Data Engineer or related role.
● Proven track record of designing, building, and maintaining data pipelines and infrastructure.
● Expertise in programming languages i.e. Python, Scala or Java.
● Expertise in SQL.
● Familiarity with multiple cloud platforms like AWS, Azure, or GCP and expertise in at least one.
● Expertise in multiple data processing frameworks e.g. Spark, Beam, Flink etc.
● Expertise in at least one workflow orchestration tool e.g. Airflow.
● Expertise in multiple messaging queues e.g. Kafka or PubSub.
● Expertise in at least one data warehouse e.g. Redshift, Bigquery, Snowflake, etc.
● Expertise in implementing data lake.
● Expertise in relational databases e.g. Postgres, MySQL, etc.
● Expertise in NoSQL databases e.g. DynamoDB, MongoDB, BigTable, etc.
● Familiarity with multiple data visualization tools e.g. Tableau, PowerBI, Superset, Quicksight, etc.
● Expertise in data catalog and governance tools e.g. Amundsen, Dataplex, Glue catalog, etc.
● Expertise in data validation framework(s) e.g. DVT, great expectations, etc.
● Strong understanding of Data warehouse, Data lake, Data mart, OLTP and OLAP concepts.
● Strong understanding of data modeling, ETL/ELT processes, and data quality principles.
● Excellent communication and collaboration skills, able to effectively translate technical concepts to
non-technical audiences.
Bonus Points
● Architecting and Implementing data platforms end-to-end.
● Experience with legacy data systems (e.g. Hadoop, Informatica).
● Experience with open source technologies i.e. Apache stack.
● Experience in DevOps, CI/CD and automation practices.
● Experience in networking and access controls.
● Any data engineering, visualization, or data science certifications on any of the clouds
