We are seeking a talented Data Engineer experience in Quantexa with expertise in Hadoop, Scala, Spark, Elastic, Open Shift Container Platform (OCP) and DevOps practices.
As a Data Engineer, you will play a crucial role in designing, developing, and optimizing big data solutions using Apache Spark, Scala, and Elasticsearch.
You will collaborate with cross-functional teams to build scalable and efficient data processing pipelines and search applications.
Knowledge and experience in the Compliance / AML domain will be a plus.
Working experience with Quantexa tool is a must.
Requirements
- Bachelor's or master’s degree in computer science, Software Engineering, or a related discipline
- Possession of Quantexa certification as a Data Engineer or Data Architect, with proficiency in the tool
- Proficiency in programming languages commonly employed in data engineering, including Spark, Python, Scala
- Demonstrated experience as a Data Engineer, utilizing Hadoop, Spark, and data processing technologies in large-scale environments
- Expertise in the Scala programming language and familiarity with functional programming principles
- Prior experience with the Quantexa tool is highly desirable
- Comprehensive understanding of Apache Spark architecture, including RDDs, DataFrames, and Spark SQL
- Advanced proficiency in designing and developing data infrastructure utilizing Hadoop, Spark, and associated tools (HDFS, Hive, Pig, etc.)
- Experience with containerization platforms such as OpenShift Container Platform (OCP) and container orchestration via Kubernetes
- Knowledge of DevOps methodologies, CI/CD pipelines, and infrastructure automation tools (e.g., Docker, Jenkins, Ansible, BitBucket)
- Experience with Graphana, Prometheus, and Splunk will be considered an added advantage
- Background in integrating and utilizing Elasticsearch for data indexing and search applications
- Solid understanding of Elasticsearch data modeling, indexing strategies, and query optimization techniques
- Experience with distributed computing, parallel processing, and handling large datasets
- Proficient in performance tuning and optimization methods for Spark applications and Elasticsearch queries
- Strong problem-solving and analytical capabilities with the capacity to debug and resolve intricate issues
- Familiarity with version control systems (e.g., Git) and collaborative development environments
has context menu
