[JR2023003186]
Job Description
Essential Duties and Responsibilities
- Design, implement, and optimize data pipelines to ensure efficient data ingestion, storage, and processing for training and fine-tuning LLMs, ensuring data quality, consistency, and relevance, via cloud computing technology.
- Ensure data quality and integrity throughout the pipeline, implementing appropriate data validation and cleansing techniques.
- Develop AI models using Large Language Models (LLMs) to build AI-driven applications such as chatbots, virtual assistants, and other natural language processing (NLP) solutions.
- Customize and fine-tune pre-trained LLMs foir specific tasks and domains, ensuring optimal model performance, while enhancing prompt engineering techniques at the same time.
- Continuously evaluate the performance of LLMs and fine-tuned models, implement optimization techniques to improve their efficiency.
- Evaluate model performance, perform relevant AB tests, and continuously improve accuracy, efficiency, and scalability.
- Collaborate with data engineers and analysts, product managers to understand project requirements, deliver AI solutions, and support the integration of AI models into scalable production systems.
- Stay up-to-date with industry trends and emerging technologies in data science, LLM research, and identify opportunities to apply these advances to improve existing projects.
Requirements
- Ability to work in fast paced, high pressure, agile environment.
- Ability and willingness to learn any new technologies and apply them at work in order to stay ahead.
- Strong in programming languages such as Python, SQL.
- Experience in developing and deploying applications running on cloud infrastructure such as AWS, Azure or Google Cloud Platform using Infrastructure as code tools such as Terraform, containerization tools like Dockers.
- Experience using orchestration tools like Airflow or Prefect, distributed computing framework like Spark or Dask.
- In-depth knowledge of machine learning techniques, Natural Language Processing (NLP), deep learning.
- Proficiency in working with large-scale datasets, data preprocessing, LLMs fine-tuning, model evaluation, and prompt engineering.
- Excellent problem-solving and analytical skills, with the ability to translate complex business problems into actionable insights.
- Excellent written and verbal communication skills for coordinating across teams.
- Has a Bachelor's, Master’s or PhD degree in computer science or similar discipline.
- 2+ years of experience in AI and data science, with a strong focus on LLMs and NLP.
