FENG Hui, TANG Xianding, YANG Tao, HU Bo. Split Method of Multipliers and Its Application to Parallel and Distributed Logistic Regression[J]. Chinese Journal of Electronics, 2014, 23(2): 305-310.
Citation: FENG Hui, TANG Xianding, YANG Tao, HU Bo. Split Method of Multipliers and Its Application to Parallel and Distributed Logistic Regression[J]. Chinese Journal of Electronics, 2014, 23(2): 305-310.

Split Method of Multipliers and Its Application to Parallel and Distributed Logistic Regression

Funds:  This work is supported by the National Science and Technology Major Project of China (No.2012ZX03001007-003).
  • Received Date: 2012-06-01
  • Rev Recd Date: 2013-05-01
  • Publish Date: 2014-04-05
  • We consider the scenario where two variables need to be optimized simultaneously. The minimization over one variable has an analytical solution, while it is intractable for the other. Under the Lagrangian dual framework, we propose two iterative optimization algorithms, which make partial minimization and gradient descent alternatingly over two variables. The first algorithm asserts that the iteration result converges to a KKT point under proper stepsize rules, which only needs the augmented Lagrangian function to be convex over partial variable. The second algorithm provides the local attraction property around the KKT point. Our algorithms provide a general solution to parallel and distributed optimization with summable objective functions. Simulation results on parallel and distributed logistic regression classification are present, which show faster convergence rate with less computational complexity compared with other methods.
  • loading
  • Y. Eldar, T. Davidson, A. B. Gershman, et al., "Special issue on convex optimization method for signal processing", IEEE J. Sel. Topics Signal Process., Vol.1, No.4, 2007.
    Y. Eldar, Z.-Q. Luo, W.-K. Ma, et al., "Special section on convex optimization in signal Processing", IEEE Singal Process. Mag., Vol.27, No.3, 2006.
    I. Schizas, A. Ribeiro, and G. Giannakis, "Consensus in Ad Hoc WSNs with noisy links-part I: Distributed estimation of deterministic signals, IEEE Trans. Signal Process., Vol.56, pp.350-364, 2008.
    I. Schizas, G. Giannakis, S. Roumeliotis, et al., "Consensus in Ad Hoc WSNs with noisy links-part II: Distributed estimation and smoothing of random signals, IEEE Trans. Signal Process., Vol.56, pp.1650-1666, 2008.
    D. P. Bertsekas, Nonlinear Programming, Belmont, Mass. U.S.A., Athena Scientific, 1999.
    D. P. Bertsekas, "Multiplier methods: A survey, Automatica, Vol.12, No.2, pp.133-145, 1976.
    D. G. Luenberger and Y. Ye, Linear and nonlinear programming, Springer, 2008.
    W. Yin, S. Osher, D. Goldfarb, and J. Darbon, "Bregman iterative algorithms for l1-minimization with applications to compressed sensing, SIAM J. Imaging Sci., Vol.1, No.1, pp.143-168, 2008.
    D. P. Bertsekas and J. N. Tsitsiklis, Parallel and distributed Computation: Numerical Methods, ch.3.4, New Jersey, Printice-Hall, 1989.
    S. Boyd, N. Parikh, E. Chu, et al. Distributed Optimization and Statistical Learning Via the Alternating Direction Method of Multipliers (Foundations and Trend), Now Publishers Inc, 2011.
    G. Mateos, I. Schizas, and G. Giannakis, "Distributed recursive least-squares for consensus-based in-network adaptive estimation, IEEE Trans. Signal Process., Vol.57, pp.4583-4588, 2009.
    I. Schizas, G. Mateos, and G. Giannakis, "Distributed LMS for consensus-based in-network adaptive processing, IEEE Trans. Signal Process., Vol.57, pp.2365-2382, 2009.
    P. A. Forero, A. Cano, G. B. Giannakis, "Consensus-based distributed support vector machines, J. Machine Learning Research, Vol.11 pp.1663-1707, 2010.
    G. Mateos, J. A. Bazerque, and G. B. Giannakis, "Distributed sparse linear regression, IEEE trans. Signal Process., Vol.58, No.10, pp.5262-5276, 2010.
    J. A. Bazerque, G. Mateos, and G. B. Giannakis, "Group-lasso on splines for spectrum cartography, IEEE trans. Signal Process., Vol.59, No.10, pp.4648-4663, 2011.
    J. F. C. Mota, J. Xavier, P. M. Q. Aguiar, and M. Puschel, "Distributed basis pursuit, IEEE trans. Signal Process., Vol.60, No.4, pp.1942-1956, 2012.
    T. Erseghe, D. Zennaro, E. Dall Anese, L. Vangelista, "Fast consensus by the alternating direction multipliers method, IEEE trans. Signal Process., Vol.59, No.11, pp.5523-5537, 2011.
    M. Jordan, J. Kleinberg, B. Scholkopf, Pattern Recognition and Machine Learning, Springer, 2006.
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Article Metrics

    Article views (226) PDF downloads(965) Cited by()
    Proportional views
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return