[1 year contract, renewable]
We are seeking a skilled Data Engineer with expertise in AI areas such as Machine Learning, Natural Language Processing (NLP), and Data Engineering. The ideal candidate will have a demonstrated track record of delivering data-driven projects and will play a critical role in developing and maintaining the data infrastructure that supports various AI-driven solutions for Intellectual Property (IP) relate.
Responsibilities:
· Design, develop, and deploy data tables, views, and marts in data warehouses, operational data stores, data lakes, and data virtualization.
· Perform data extraction, cleaning, transformation, and flow. Web scraping may also be part of the work scope in data extraction.
· Design, build, launch, and maintain efficient and reliable large-scale batch and real-time data pipelines with data processing frameworks.
· Integrate and collate data silos in a manner that is both scalable and compliant.
· Collaborate with Project Manager, Data Architect, Business Analysts, Frontend Developers, UX Designers, and Data Analysts to build scalable data-driven products.
· Be responsible for developing backend APIs and working on databases to support the applications.
· Work in an Agile environment that practices Continuous Integration and Delivery.
· Work closely with fellow developers through pair programming and code review processes.
Qualifications:
· Demonstrated experience in AI disciplines such as Machine Learning, Natural Language Processing (NLP), and data engineering, with a focus on applying these technologies in projects related to Trade Marks, Patents, and Designs.
· Proficient in general data cleaning and transformation (e.g., SQL, pandas, R, etc.).
· Proficient in building ETL pipelines (e.g., SQL Server Integration Services (SSIS), AWS Database Migration Services (DMS), Python, AWS Lambda, ECS Container Task, EventBridge, AWS Glue, Spring).
· Proficient in database design and various databases (e.g., SQL, PostgreSQL, AWS S3, Athena, MongoDB, Postgres/GIS, MySQL, SQLite, VoltDB, Cassandra, etc.).
· Experience in cloud technologies such as GPC, GCC (i.e., AWS, Azure, Google Cloud).
· Passion for data engineering in a big data environment using cloud platforms such as GPC, GCC (i.e., AWS, Azure, Google Cloud).
· Experience with building production-grade data pipelines, ETL/ELT data integration.
· Knowledge of system design, data structure, and algorithms.
· Familiar with data modeling, data access, and data storage infrastructure like Data Mart, Data Lake, Data Virtualization, and Data Warehouse.
· Familiar with REST API and web requests/protocols in general.
· Big Data Frameworks: Familiar with big data frameworks and tools (e.g., Hadoop, Spark, Kafka, RabbitMQ).
· Familiar with W3C Document Object Model and customized web scraping (e.g., BeautifulSoup, CasperJS, PhantomJS, Selenium, Node.js, etc.).
· Comfortable in at least one scripting language (e.g., SQL, Python).
· Comfortable in both Windows and Linux development environments.
· Interest in being the bridge between engineering and analytics, especially in contexts involving intellectual property (IP) projects like trademarks, patents, and designs.
· 1-3 years of hands-on experience in data engineering, with exposure to NLP.
We are looking for multiple Data Engineers to work on various projects. Specific stack and requirements will be made know to the short-listed applicants. Applicants need not to have all the qualifications.
