WHAT WE DO
We invite you to be part of our ambitious, close-knit team creating systems for large customers who need to crunch through Tbps of data in real-time. Our approach is relentless performance-oriented software engineering vs. server sprawl in our customers' data centres.
You will use the latest high-end hardware and continuously devise ways to push the envelope of software performance.
We build in-house systems if we must. We had to for indexing 1M 60-column rows/s, for aggregating high throughput event streams over hundreds of combinations of dimensions, and for pattern matching 5M patterns at 100Gb/s per 2RU. We use these to solve real customer problems.
You will experiment wildly. For example we implemented network monitoring using a GPU, and we tested 4-socket machines with 2T RAM. Our current favourite platform is a 2-socket system with Platinum 8180 CPUs (112 lcores in total), 4x40Gbps NICs and 1T RAM, which we use to process 160Gbps.
You will help us build a successful software platform for the long run. We invest a lot in flexibility, such as with our extensible rule engine and declarative aggregation system that empowers our analysts and helps us minimise the C code we have to write for supporting disparate use-cases.
We know the devil is in the details. You will improve performance through better memory allocation systems and better data structures, all while ensuring that they are integrated with Address Sanitizer and fully tested using unit tests and end-to-end regression tests.
We work end-to-end. You will implement data engineering solutions that are both efficient and secure for handling events from 500 million users, and to extract insight without leaking individual information.
We want to show off. To attract the best programmers we plan to showcase our technology. You can be part of our effort to open-source interesting pieces of our technology stack.
YOUR ROLE AS HIGH-PERFORMANCE DATA ENGINEER
As a High-Performance Data Engineer, you will create and maintain tools, mainly in C, for crunching large amounts of data in files or streams.
You will have to think both big, in terms of overall architecture, and small, in terms of low-level optimisations, to deliver solutions that are reusable, and match the performance of the best hardware.
Every capability you add directly translates to new offerings made possible. Every percent of performance improvement directly translates to large cost savings. At the same time, the correctness and reliability of your work will be the cornerstone to our customers’ trust.
WHAT WE VALUE
‣ Bachelor’s or Higher Degree in Computer Science or equivalent
‣ Software craftsmanship
‣ Attention to reliability and successful delivery
‣ Experience with large C code bases and high-performance C programming
‣ Familiarity with shared memory data structures and parallel algorithms
‣ Proficiency with Linux system & development tools
