eBird | GitHub | Facebook | LinkedIn | Stack Overflow | Wikipedia | Lichess
As of 2024, I've lived in San Francisco since 2017. I've worked as a Software Engineer at Meta since 2016. I work on Cosco, a shuffle service for Apache Spark. Here is a talk that I gave about it. I call shuffle the "secret sauce" of MapReduce because it was hardly discussed in the original MapReduce paper, with later correspondence acknowledging that "careful attention must be paid to the implementation of the data-shuffling phase to avoid generating O(M*R) seeks."
Email: feldmando at gmail.com. If you really want, you can encrypt your messages with my PGP key.