Fast Detection of Overlapping Communities via Online Tensor Methods

Notes, (Arxiv, 16 Oct 2013) Nov 5, 2013 https://sites.google.com/site/bigaidream/ Jie Fu [KEY] Stochastic Gradient Descent (SGD) for tensor decomposition; GPU for dense graph and CPU for sparse graph; Optimization of SGD by stacking them into matrix

Contents
Introduction ...................................................................................................................................... 1 Contribution ...................................................................................................................................... 1 Tricks: Efficient STGD via stacked vector operations: ........................................................................ 2

Introduction
The fast community detection methods, such as label propagation, local searches based on random walk are non-probabilistic and heuristic approaches. Existing probabilistic approaches, such as variational approach, are very slow and not easily parallelizable.

Contribution
GPU: carry out the decomposition of the smaller tensor implicitly, without forming it, using stochastic gradient descent. CPU: sparse graph GPU implementation is efficient for denser graphs, with larger number of communities, such as Facebook, our CPU implementation is efficient for sparse graphs, with large number of nodes, such as Yelp and DBLP which would not fit in the GPU memory wholly. To the best of our knowledge, while stochastic methods for matrix decomposition have been considered earlier, this is the first work incorporating stochastic optimization for tensor decomposition, and paves the way for further investigation on many theoretical and practical issues.

Tricks: Efficient STGD via stacked vector operations:

Although the updating equation for the stochastic gradient update is presented serially, we can update the k eigenvectors simultaneously in parallel. The basic idea is to stack the k eigenvectors into a matrix, then using the internal parallelism designed for BLAS III operations. Since the eigenvectors are just k length k vectors, we store them in the device memory and perform the iterations.