Job Description:
We are seeking a highly skilled and experienced PySpark Developer to join our dynamic team in the banking sector. The ideal candidate will have a strong background in big data technologies, particularly PySpark, and a proven track record of working in the banking domain.
Key Responsibilities:
- Develop and Maintain: Design, develop, and maintain scalable and efficient data pipelines using PySpark.
- Data Integration: Integrate large datasets from various sources, ensuring data quality and consistency.
- Performance Optimization: Optimize PySpark jobs for performance and scalability.
- Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
- Troubleshooting: Identify and resolve data-related issues and bottlenecks.
- Documentation: Create and maintain comprehensive documentation for all data processes and systems.
- Compliance: Ensure all data processes comply with industry standards and regulations, particularly those relevant to the banking sector.
Qualifications:
- Education: Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
- Experience: 6-9 years of experience in big data development, with a focus on PySpark.
- Domain Expertise: Preferably from the banking or financial services domain.
- Technical Skills:Proficiency in PySpark and Python.
Experience with big data technologies such as Hadoop, Hive, and Kafka.
Strong SQL skills and experience with relational databases.
Familiarity with cloud platforms like AWS, Azure, or Google Cloud.
Soft Skills:
- Excellent problem-solving and analytical skills.
- Strong communication and collaboration abilities.
- Ability to work in a fast-paced, dynamic environment.
Preferred Qualifications:
- Experience with machine learning frameworks and libraries.
- Knowledge of data warehousing concepts and ETL processes.
- Understanding of banking regulations and compliance requirements.
Benefits:
- Competitive salary and performance-based bonuses.
- Comprehensive health insurance and retirement plans.
- Opportunities for professional development and career growth.
- Flexible working hours and remote work options.
