Friday Hacks #227, September 09: Practical Challenges of Data Ingestion at Scale and From Thought to Launch
Posted on by Shen Yi Hong
Date/Time: Friday, September 09 at 7:00pm SGT
Venue: The Hangar @ i3 Building
Sign-up Link: NUSync Sign-up Link
Recording: https://youtu.be/s5Tm4XY8Zmc
Food 🍕 will be served!
If you are unable to come join us physically, you are welcome to join us online!
1) Practical Challenges of Data Ingestion at Scale
With data being at the core of what we do at Marshall Wace, this comes with the challenge of vertically and horizontally scaling data ingestion across thousands of pipelines and terabytes of data. In this session, Marshall Wace’s Head of Data Operations, Louis Ditzel, will discuss how his team tackles this through various in-house platforms.
Speaker Profile
Louis is the Head of Data Operations in Marshall Wace. He joined Marshall Wace in 2017 as a technology graduate and has since rotated through the Cybersecurity team as well as the Data Engineering team, eventually settling in the latter. He is currently focused on bringing scale to our data pipeline, improving the reliability of the platform and changing the way we monitor data ingestion.
2) From Thought to Launch
This talk will walk you through the lifecycle of a feature at GitHub and the processes involved in taking an idea to production leveraging GitHub features such as Discussions, Issues/Memex, Pull Requests, etc.
Speaker Profile
Yumin (@itsbagpack) is a Staff Software Engineer at Github under the Communities department. She is involved in the development of GitHub Sponsors, Discussions, Feed, Dashboard, Profile, Achievements, etc.
Supported by:
The HANGAR by NUS Enterprise — the campus hub for entrepreneurs.