Responsibilities
- Designing data processing pipelines: PySpark developers design, build, and unit test applications on the Spark framework using Python.
- Understanding business logic: PySpark developers understand the core business logic for bug fixes and maintenance.
- Interacting with clients: PySpark developers interact with clients to specify user requirements.
- Preparing test cases: PySpark developers prepare test cases and test case documents.
- Building applications: PySpark developers build applications for batch and streaming requirements.
- Using broadcast variables: PySpark developers use broadcast variables to share read-only data across nodes in the cluster.
- Identifying lookup tables: PySpark developers identify and broadcast frequently accessed lookup tables or reference data to improve performance.
- Developing data processing tasks: PySpark developers develop data processing tasks such as reading data from external sources, merging data, aggregating data, and performing data enrichment.
Skills/Requirement
Must Have Skills:
· Creating high performance PySpark applications which should be able to handle millions of rows from various file format
· Hands on experience in creating PySpark jobs for data transformation and aggregation
· Strong understanding of Hadoop architecture and design best practices
· Evaluate and implement innovative and continuous design improvements to PySpark framework
Good to Have Skills:
· Understanding of Financial Markets Asset Classes (FX, FI, Equities, Rates, Commodities & Credit), various trade types (OTC, exchange traded, Spot, Forward, Swap, Options) and related systems
· Surveillance domain knowledge, regulations (MAR, MiFID, CAT, Dodd Frank) and related Systems knowledge is certainly a plus
Interested candidates, who wish to apply for the above position, please send in your resume to [email protected]
PERSOLKELLY Singapore Pte Ltd EA License No. 01C4394 EA Reg No: R23118975 (ABDUL RAHMAN ANSARI RAJA)
We regret to inform you that only shortlisted candidates will be contacted.
**********************************
This is in partnership with Employment and Employability Institute Pte Ltd (“e2i”). e2i is the empowering network for workers and employers seeking employment and employability solutions. e2i serves as a bridge between workers and employers, connecting with workers to offer job security through job-matching, career guidance and skills upgrading services, and partnering employers to address their manpower needs through recruitment, training, and job redesign solutions. e2i is a tripartite initiative of the National Trades Union Congress set up to support nation-wide manpower and skills upgrading initiatives. By applying for this role, you consent to e2i’s PDPA.
By sending us your personal data and curriculum vitae (CV), you are deemed to consent to PERSOLKELLY Singapore Pte Ltd and its affiliates to collect, use and disclose your personal data for the purposes set out in the Privacy Policy available at https://www.persolkelly.com.sg/policies. You acknowledge that you have read, understood, and agree with the Privacy Policy.
**********************************
