Network science is a fascinating field that is evolving rapidly across many domains. For long, classical system and learning theories have focused on designing and optimizing stand-alone systems or learners with great success. Nevertheless, progress in recent decades in the biological sciences, animal behavior studies, and the neuroscience of the brain, has revealed remarkable patterns of organization and structured complexity in the behavior of biological networks. These studies have brought forward notable examples of complex systems that derive their sophistication from coordination among simpler units and from the aggregation and processing of decentralized pieces of information. While each unit in these networks is not capable of sophisticated behavior on its own, it is the interaction among the constituents that leads to systems that are resilient to failure and that are capable of adjusting their behavior in response to changes in the environment. These discoveries have motivated diligent efforts towards a deeper understanding of information processing over graphs in several disciplines including signal processing, machine learning, optimization, control, and the social sciences.
This lecture surveys the field of adaptive networks and how collaboration among agents can lead to superior adaptation and learning performance over graphs. Adaptive networks consist of a collection of agents with learning abilities. The agents interact with each other on a local level and diffuse information across the network to solve inference and optimization tasks in a decentralized manner. Such networks are scalable, robust to node and link failures, and are particularly suitable for learning from big data sets by tapping into the power of collaboration among distributed agents. Still, some surprising phenomena arise when information is processed in a decentralized fashion over networked systems. For example, the addition of more informed agents does not necessarily lead to enhanced performance, and even minor variations in how information is processed by the agents can lead to failure. In other words, information processing over graphs gives rise to some revealing phenomena that deserve closer attention due to the coupling effect among the agents. We shall elaborate on such phenomena in the context of adaptive networks, and consider examples from a variety of areas including distributed sensing, intrusion detection, clustering, and machine learning.