Prof. Siddhartha Banerjee
Operations Research and Information Engineering
Cornell University
Talk:
Compensated Coupling and the Bellman Inequalities: A User-Friendly Tutorial
Abstract:
In recent years, my co-authors and I have developed a simple paradigm for designing sequential decision-making policies based on sample-pathwise comparisons against a hindsight benchmark. Our approach generalizes many standard results used in studying MDPs and reinforcement learning, but also gives new policies which are much simpler and more effective than existing approaches. For a large class of widely-studied sequential decision-making problems -- including network revenue management, dynamic pricing, generalized assignment, online bin packing, online assortment optimization and bandits with knapsacks -- under natural conditions, our approach completely eliminates the curse of dimensionality and gets additive loss compared to the hindsight optimal which is independent of the state-space size.
In this talk, I will try and show some simple examples to illustrate the two main components of our paradigm -- the Bellman inequalities and the compensated coupling. Time permitting, I will try and describe how these ideas generalize to more complex problems, and also, how we can use this technique to incorporate historical data, as well as for studying multi-objective tradeoffs.
Biography:
Professor Siddhartha Banerjee is an associate professor in the School of Operations Research and Information Engineering (ORIE) at Cornell, as well as a field member in Departments of Computer Science, ECE, and the Center for Applied Mathematics.
Professor Siddhartha Banerjee works on topics at the intersection of data-driven decision-making and stochastic control, economics and computation, and large-scale network algorithms. Some of his current research interests include: new data-driven approaches for online decision-making, non-monetary mechanisms and information design, equity and fairness in online decision-making, stochastic coupling techniques in control.
