Distributed processing deals with the extraction of information from data collected from multiple sources. For example, each node in a network of nodes distributed over a geographic area could collect noisy observations related to a certain parameter of interest. The nodes would then interact with their neighbors in a certain manner, as dictated by the network topology, in order to efficiently arrive at an estimate of the parameter. The objective is to derive an estimate that is as accurate as the one that would be obtained if each node had access to the information across the entire network.
In a traditional centralized solution, the nodes in the network collect observations and send them to a central location for processing. The central processor would then perform the required estimation tasks and broadcast the result back to the individual nodes. This mode of operation requires a powerful central processor, in addition to extensive amounts of communication between the nodes and the central processing unit. In comparison, in a distributed solution, the nodes rely solely on their local data and on interactions with their immediate neighbors, so that the global computational burden is shared among the nodes, and the amount of communications and the need for powerful processors are significantly reduced.
Naturally, the effectiveness of any distributed estimation implementation depends on the modes of cooperation established at the network level, as well as the processing strategies adopted at the node level. Regardless of the cooperative and distributed strategies adopted, it is an accepted fact that the processing has to be adaptive in order to cope with data statistical variations and network topology changes.
At the network level, this dissertation studies three such modes of cooperation: incremental, diffusion, and probabilistic diffusion. In the incremental mode of cooperation, information flows in a sequential manner from one node to the adjacent node. This mode of operation requires a cyclic pattern of collaboration among the nodes, and it tends to require the least amount of communications and power. In a diffusion implementation, each node communicates with all its immediate neighbors as dictated by the network topology. The amount of communication in this case is higher than in an incremental solution. Nevertheless, the nodes increase their cooperation with their neighbors. In the last mode of cooperation studied in this dissertation, the communications in the diffusion implementation can be relaxed by allowing each node to communicate only with a subset of its neighbors. The choice of which subset of neighbors to communicate with can be randomized according to some performance criterion.
At the node level, several learning rules are employed, such as the least-mean squares (LMS), recursive least-squares (RLS), and normalized LMS. Employing different learning rules at the node level, as well as different cooperation modes at the network level, gives rise to different adaptive network structures that inherit the tracking and processing abilities of standard adaptive systems.
Broadly, this dissertation develops distributed algorithms that enable a net- work of nodes to function as an adaptive entity in its own right. Specifically, the contributions of this work are threefold: 1) to motivate a family of adaptive algorithms for distributed estimation inspired by distributed optimization techniques; 2) to propose an adaptive network structure composed of an interconnected set of nodes that is able to respond to data in real time and to track variations in the statistical properties of the data; and 3) to analyze the performance of the resulting interconnected network of nodes. This task is challenging, since an adaptive network comprises a “system of systems” that processes data cooperatively in both time and space.
Acknowledgment This work was supported in part by the National Science Foundation under grants ECS-0601266 and ECS-0725441. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.