Greetings! I am a computer scientist and engineer working at Google, and a founding PMC member of Apache Beam, providing a common programming model for batch and streaming dataflow DAG-based big data computations.
Do you love mathematics, design, code, and big data like I do? Here are highlights of my work on Beam:
- Apache Beam Pipeline Runner API
-
What is a Beam pipeline? What are the primitive transformations?
What are the user-defined functions? What is a Beam pipeline runner?
This document lays out the definition of the Beam programming model, along
with technical details of cross-runner support implemented in the Beam.
(read) - Lateness (and Panes) in Apache Beam
-
In a streaming computation, where data comes in telling you about events
that happened somewhere in the world, what does it mean for data to be "late"?
How should we manage computations over such data?
This document explores the concept of lateness and lays out the
mathematical description along with guidelines that steer the implementation
of Beam.
(read)
I also had a life before all that. You may also enjoy my other quasi-pedagogical writings on mathematics, computer science, and technology.