Please use this identifier to cite or link to this item: http://localhost/handle/Hannan/580377
Title: Convergence of Gradient Descent for Minimum Error Entropy Principle in Linear Regression
Authors: Ting Hu;Qiang Wu;Ding-Xuan Zhou
subject: error analysis|gradient descent method|Minimum error entropy|error information|global convergence
Year: 2016
Publisher: IEEE
Abstract: We study the convergence of minimum error entropy (MEE) algorithms when they are implemented by gradient descent. This method has been used in practical applications for more than one decade, but there has been no consistency or rigorous error analysis. This paper gives the first rigorous proof for the convergence of the gradient descent method for MEE in a linear regression setting. The mean square error is proved to decay exponentially fast in terms of the iteration steps and of order O( 1) in terms of the sample size m. The mean square convergence is guaranteed when the step size is chosen appropriately and the scaling parameter is large enough.
URI: http://localhost/handle/Hannan/164563
http://localhost/handle/Hannan/580377
ISSN: 1053-587X
1941-0476
volume: 64
issue: 24
Appears in Collections:2016

Files in This Item:
File Description SizeFormat 
7572876.pdf569.69 kBAdobe PDFThumbnail
Preview File
Title: Convergence of Gradient Descent for Minimum Error Entropy Principle in Linear Regression
Authors: Ting Hu;Qiang Wu;Ding-Xuan Zhou
subject: error analysis|gradient descent method|Minimum error entropy|error information|global convergence
Year: 2016
Publisher: IEEE
Abstract: We study the convergence of minimum error entropy (MEE) algorithms when they are implemented by gradient descent. This method has been used in practical applications for more than one decade, but there has been no consistency or rigorous error analysis. This paper gives the first rigorous proof for the convergence of the gradient descent method for MEE in a linear regression setting. The mean square error is proved to decay exponentially fast in terms of the iteration steps and of order O( 1) in terms of the sample size m. The mean square convergence is guaranteed when the step size is chosen appropriately and the scaling parameter is large enough.
URI: http://localhost/handle/Hannan/164563
http://localhost/handle/Hannan/580377
ISSN: 1053-587X
1941-0476
volume: 64
issue: 24
Appears in Collections:2016

Files in This Item:
File Description SizeFormat 
7572876.pdf569.69 kBAdobe PDFThumbnail
Preview File
Title: Convergence of Gradient Descent for Minimum Error Entropy Principle in Linear Regression
Authors: Ting Hu;Qiang Wu;Ding-Xuan Zhou
subject: error analysis|gradient descent method|Minimum error entropy|error information|global convergence
Year: 2016
Publisher: IEEE
Abstract: We study the convergence of minimum error entropy (MEE) algorithms when they are implemented by gradient descent. This method has been used in practical applications for more than one decade, but there has been no consistency or rigorous error analysis. This paper gives the first rigorous proof for the convergence of the gradient descent method for MEE in a linear regression setting. The mean square error is proved to decay exponentially fast in terms of the iteration steps and of order O( 1) in terms of the sample size m. The mean square convergence is guaranteed when the step size is chosen appropriately and the scaling parameter is large enough.
URI: http://localhost/handle/Hannan/164563
http://localhost/handle/Hannan/580377
ISSN: 1053-587X
1941-0476
volume: 64
issue: 24
Appears in Collections:2016

Files in This Item:
File Description SizeFormat 
7572876.pdf569.69 kBAdobe PDFThumbnail
Preview File