We are looking for a Senior Data Engineer to join our data engineering team. If you are passionate about solving problems and making improvements through data and technology, we are eager to talk to you. The ideal candidate has both a willingness and desire to work in a dynamic environment, is able to apply Agile methodologies in day-to-day activities, and is a self-motivated developer who also enjoys working in a team environment. Our people have a creative, innovative, fun, and collaborative attitude, and are dedicated to creating new and valuable solutions for HP.
The Senior Data Engineer applies advanced subject matter knowledge to solve complex business issues and is regarded as a subject matter expert. Frequently contributes to the development of new ideas and methods. Works on complex problems where analysis of situations or data requires an in-depth evaluation of multiple factors. Leads and/or provides expertise to functional project teams and may participate in cross-functional initiatives. Acts as an expert providing direction and guidance to process improvements and establishing policies. Frequently represents the organization to external customers/clients. Exercises significant independent judgment within broadly defined policies and practices to determine the best method for accomplishing work and achieving objectives. May provide mentoring and guidance to lower-level employees.
Responsibilities
You will design, develop, test, modify, deploy, and document solutions and services in both local and cloud-based deployments of our Big Data infrastructure. Create solutions using current programming languages and tools and interact with team members, customers, and partners to define requirements. Collaborate with stakeholders and data scientists to analyze, extract, and report meaningful insights from data collected.
A senior data engineer:
- Provides core contributions to multiple products, projects, solutions and processes through all stages of the development lifecycle including but not limited to design, analysis, coding, testing and integration.
- Is a highly skilled technical leader responsible for key system areas and ensures that changes are delivered with high quality and that best practices are used.
- May lead cross-team or cross-organization technical efforts related to the application design and development.
- Reviews and evaluates designs and project activities for compliance with applications design and development guidelines and standards; provides tangible feedback to improve product quality and mitigate failure risk.
- Provides domain-specific expertise and overall software leadership and perspective to cross-organization projects, programs, and activities.
- Drives innovation and integration of new technologies into projects and activities in the software applications design organization.
- Provides guidance and mentoring to less-experienced staff members.
- Participates in analyzing requirements and high-level designs.
- Implements, tests, and maintains software modules created.
- Participates in code reviews.
- Gives input to management on task duration estimates, technical information, and issue resolution.
- Provides domain-specific expertise and perspective to partners.
- Maintains awareness of new tools, environments, and methodologies being used in the Big Data space.
- Recommends ways to improve data reliability, efficiency, and quality.
- Develops processes for data modeling, mining, and reporting.
- Researches opportunities for acquiring new data and new uses for existing data.
Qualifications
- Typically 5+ years of experience in software or data engineering.
- Extensive experience in data modeling, data integration and processing of structured and unstructured data.
- Proficient in one or more programming languages (Python preferred).
- Strong SQL proficiency (experience with NoSQL – advantage).
- Experienced with Apache Spark
- Excellent communication skills; mastery in English and local language.
- Ability to effectively communicate product architectures, design and change proposals.
Additional preferred qualifications / advantages
- Databricks
- Pandas
- Dataiku
- Familiar with best practices of the data and software engineering lifecycle and/or best practices of the above platforms and tools.
- Scala
