Very Sparse LSSVM Reductions for Large-Scale Data - implementation -

 

 

Very Sparse LSSVM Reductions for Large-Scale Data by Raghvendra Mall and Johan Suykens

 

Least squares support vector machines (LSSVMs) have been widely applied for classification and regression with comparable performance with SVMs. The LSSVM model lacks sparsity and is unable to handle large-scale data due to computational and memory constraints. A primal fixed-size LSSVM (PFS-LSSVM) introduce sparsity using Nyström approximation with a set of prototype vectors (PVs). The PFS-LSSVM model solves an overdetermined system of linear equations in the primal. However, this solution is not the sparsest. We investigate the sparsity-error tradeoff by introducing a second level of sparsity. This is done by means of -norm-based reductions by iteratively sparsifying LSSVM and PFS-LSSVM models. The exact choice of the cardinality for the initial PV set is not important then as the final model is highly sparse. The proposed method overcomes the problem of memory constraints and high computational costs resulting in highly sparse reductions to LSSVM models. The approximations of the two models allow to scale the models to large-scale datasets. Experiments on real-world classification and regression data sets from the UCI repository illustrate that these approaches achieve sparse models without a significant tradeoff in errors. 

 

An implementation is here. 

 

Some related presentation slides from Johan Suykens

 

• "Kernel methods for complex networks and big data": invited lecture at Statlearn 2015, Grenoble 2015: [pdf]


• "Fixed-size Kernel Models for Big Data": invited lectures at BigDat 2015, International Winter School on Big Data, Tarragona, Spain 2015:
  - Part I: Support vector machines and kernel methods: an introduction [pdf]
  - Part II: Fixed-size kernel models for mining big data [pdf] [video]
  - Part III: Kernel spectral clustering for community detection in big data networks [pdf]


• Dec 11, 2014: "Fixed-size kernel methods for data-driven modelling": plenary talk at ICLA 2014, International Conference on Learning and Approximation, Shanghai China 2014 [pdf]

 

W00092394.jpg was taken on May 03, 2015 and received on Earth May 04, 2015. The camera was pointing toward TITAN, and the image was taken using the MT2 and CL2 filters. Image Credit: NASA/JPL-Caltech/Space Science Institute

    Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.