Fault-tolerant and Communication Efficient Algorithms in Distributed Computing via by Coding Theory

The scalability of distributed computing is often limited by two aspects: (1) communication bottlenecks and (2) excessively slow stragglers or faulty nodes in the system. Coding Theory is the art and science of introducing redundancy into data despite deviations in ideal behavior from system components that interact with data. While coding theory has most conventionally been applied for dealing with errors and communications and data storage, this talk focuses on the application of distributed data processing. The talk will demonstrate recently developed code designs for both fault tolerance and communication efficiency in distributed computing. 
 
The talk will first showcase some recently developed codes for fault-tolerant matrix multiplications and compare their performance with code constructions in high performance computing from the 1980s. The developed codes will be measured in terms of the trade-off between the degree of fault-tolerance and the degree of redundancy. The code constructions will use polynomial evaluation and interpolation techniques. The talk will then demonstrate how the redundancy induced by code design can be used to reduce communications in distributed iterative linear processing systems. Finally, time-permitting,  the talk will present some very recent, promising results for non-linear data processing, specifically, showing the effect of data redundancy for improving the performance of stochastic gradient descent for optimization.
 No knowledge of coding theory will be required for the talk.


Biography: Viveck R. Cadambe is an Assistant Professor in the Electrical Engineering at Pennsylvania State University since August 2014. He received his Ph.D. from University of California Irvine in 2011, and has done postdoctoral stints at Boston University and MIT. His research interests are in information theory, coding theory and distributed systems theory. His research has studied applications to distributed data processing, geo-distributed key-value stores, cloud data storage systems and wireless communications. He has received the 2009 Information Theory Society Best Paper Award, and an NSF Career Award in 2016.