MagicPlot Manual

Plotting and nonlinear fitting software

User Tools

Site Tools


fit_formulas

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
fit_formulas [Fri Jul 12 00:26:17 2013]
Alexander
fit_formulas [Tue May 30 16:28:13 2017] (current)
Alexander
Line 1: Line 1:
 ====== Fitting Algorithm and Computational Formulas ====== ====== Fitting Algorithm and Computational Formulas ======
  
-MagicPlot uses iterative [[wp>​Levenberg–Marquardt_algorithm|Levenberg–Marquardt]] [[wp>​Non-linear_least_squares|nonlinear least squares]] curve fitting algorithm which is widely used in most software.+MagicPlot uses iterative [[wp>​Levenberg–Marquardt_algorithm|Levenberg–Marquardt]] ​ [[wp>​Non-linear_least_squares|nonlinear least squares]] curve fitting algorithm which is widely used in most software.
  
-MagicPlot ​fitting procedure takes all advantages ​of multi-core processors.+MagicPlot ​implementation ​of Levenberg–Marquardt algorithm is optimised for using with multi-core processors. MagicPlot successfully passed testing with NIST Nonlinear Regression datasets (see our [[http://​magicplot.com/​downloads/​MagicPlot-NIST-Test.pdf|report]]).
  
 ===== Nonlinear Least Squares Algorithm ​ ===== ===== Nonlinear Least Squares Algorithm ​ =====
Line 15: Line 15:
   * //f//(//x, β<​sub>​1</​sub>,​...,​β<​sub>​p</​sub>//​) is the fit function which depends on value of //x// and fit parameters //​β<​sub>​k</​sub>//,​   * //f//(//x, β<​sub>​1</​sub>,​...,​β<​sub>​p</​sub>//​) is the fit function which depends on value of //x// and fit parameters //​β<​sub>​k</​sub>//,​
   * //p// is the number of fit parameters //​β<​sub>​k</​sub>//, ​   * //p// is the number of fit parameters //​β<​sub>​k</​sub>//, ​
-  * //​w<​sub>​i</​sub>//​ are normalized (Σ//​w<​sub>​i</​sub>//​ = 1) data weighting coefficients for each point (//​x<​sub>​i</​sub>,​ y<​sub>​i</​sub>//​).+  * //​w<​sub>​i</​sub>//​ are data weighting coefficients for each point (//​x<​sub>​i</​sub>,​ y<​sub>​i</​sub>//​).
  
-An initial guess for the parameters has to be provided to start minimization. Calculation of the new guess of parameters on each fit iteration is based on the fit function partial derivatives for current values of fit parameters ​and for each x value:+An initial guess for the parameters has to be provided to start minimization. Calculation of the new guess of parameters on each fit iteration is based on the fit function partial derivatives for current values of fit parameters for each //x// value:
  
 <​m>​{partial f}/{partial beta_m}(x_i,​ beta_1, ..., beta_p)</​m>​ <​m>​{partial f}/{partial beta_m}(x_i,​ beta_1, ..., beta_p)</​m>​
Line 25: Line 25:
 ===== Weighting of Data Points Using Y Errors ===== ===== Weighting of Data Points Using Y Errors =====
 MagicPlot can use weighting of //y// values based on y errors //​s<​sub>​i</​sub>//:​ MagicPlot can use weighting of //y// values based on y errors //​s<​sub>​i</​sub>//:​
-  * If standard //y// errors are **not** specified: all //​w<​sub>​i</​sub>//​=1 
-  * If standard //y// errors //​s<​sub>​i</​sub>//​ are specified: ​ 
  
-<m>w_i=1/{{s_i}^2}</m> +  * If standard //y// errors //s<sub>i</​sub>//​ are specified: //​w<​sub>​i</​sub>// ​= 1 / //​s<​sub>​i</​sub><​sup>​2</sup>// ((Corrected in MagicPlot 2.7: weights ​//​w<​sub>​i</​sub>// ​are not normalized anymore. In MagicPlot 2.5.1 and earlier the sum of weights was normalized ​to 1. This changing only affects the resulting Chi square value in the fit report if Y error column(sare set.)); 
- +  * Otherwise: all //w<sub>i</sub>// = 1.
-here //C// is normalizing coefficient ​(to make the sum of //​w<​sub>​i</​sub>// ​be equal to //N//)+
- +
-<m>C=N sum{i=1}{N}{{s_i}^2}</m>+
  
 In ''​Fit Plot Properties''​ dialog (''​Plot Data''​ tab) you can set one of the following methods to evaluate standard y errors //​s<​sub>​i</​sub>//:​ In ''​Fit Plot Properties''​ dialog (''​Plot Data''​ tab) you can set one of the following methods to evaluate standard y errors //​s<​sub>​i</​sub>//:​
Line 50: Line 45:
   * If the number of iterations exceeds maximum number of iterations, which is 100 by default   * If the number of iterations exceeds maximum number of iterations, which is 100 by default
  
-You can change the minimum allowable deviation decrement and maximum number of iterations in Fitting tab of MagicPlot Preferences.+You can change the minimum allowable deviation decrement and maximum number of iterations in ''​Fitting'' ​tab of ''​MagicPlot Preferences''​.
  
-===== Formulas =====+===== Formulas ​Used in Fit Report ​=====
 In the table below you can find the formulas which MagicPlot uses to calculate fit parameters and values in ''​Fit Report''​ tab. In the table below you can find the formulas which MagicPlot uses to calculate fit parameters and values in ''​Fit Report''​ tab.
  
 Because of some confusion in the names of the parameters in different sources (books and software), we also give many different names of same parameter in //note// column. Because of some confusion in the names of the parameters in different sources (books and software), we also give many different names of same parameter in //note// column.
  
 +|< 100% 15% 10% 45% 30% >|
 ^  Parameter Name  ^  Symbol ​ ^  Formula ​ ^  Note  ^ ^  Parameter Name  ^  Symbol ​ ^  Formula ​ ^  Note  ^
 ^ Original Data and Fit Model Properties ​ ^^^^ ^ Original Data and Fit Model Properties ​ ^^^^
-| Number of used data points ​ |  <​m>​N</​m> ​ |  ---  | This is the number of data points inside specified ​Fit Interval. ​ | +| Number of used data points ​ |  <​m>​N</​m> ​ |  ---  | This is the number of data points inside specified ​fit Interval. ​ | 
-| Fit parameters ​ |  //​β<​sub>​1</​sub>,​...,​β<​sub>​p</​sub>// ​ |  ---  | For peak-like functions (Gauss, Lorentz) these parameters are amplitude, position and half width at half maximum. ​ | +| Fit parameters ​ |  //​β<​sub>​1</​sub>,​...,​β<​sub>​p</​sub>// ​ |  ---  | For peak-like functions (Gauss, Lorentz) these parameters are amplitude, position and half width at half maximum. \\ Only parameters with unset ''​Lock''​ checkbox are taken into account.  | 
-| Number of fit function parameters //β// |  <​m>​p</​m> ​ |  ---  | This is the total number of parameters of all fit curves which are summarized to fit.  |+| Number of fit function parameters //β// |  <​m>​p</​m> ​ |  ---  | This is the total number of unlocked ​parameters of fit curves which are summarized to fit.  |
 | [[wp>​Degrees_of_freedom_(statistics)|Degrees of freedom]] ​ |  <​m>​nu</​m> ​ |  <​m>​N–p</​m> ​ | | | [[wp>​Degrees_of_freedom_(statistics)|Degrees of freedom]] ​ |  <​m>​nu</​m> ​ |  <​m>​N–p</​m> ​ | |
 | Estimated mean of data |  <​m>​overline{y}</​m> ​ |  <​m>​1/​N sum{i=1}{N}{y_i}</​m> ​ | | | Estimated mean of data |  <​m>​overline{y}</​m> ​ |  <​m>​1/​N sum{i=1}{N}{y_i}</​m> ​ | |
 | Estimated variance of data |  <​m>​s^2</​m> ​ |  <​m>​1/​{N–1} sum{i=1}{N}{(y_i – overline{y})^2}</​m> ​ | Not used by fit algorithm, only for comparison. | | Estimated variance of data |  <​m>​s^2</​m> ​ |  <​m>​1/​{N–1} sum{i=1}{N}{(y_i – overline{y})^2}</​m> ​ | Not used by fit algorithm, only for comparison. |
-| Data total sum of squaresTSS  |  TSS  |  <​m>​sum{i=1}{N}{w_i(y_i – overline{y})^2}</​m> ​ | **Also known as:** \\ • Sum of squares about the mean, \\ • SST.  |+| Data total sum of squares ​(TSS, SST)  ​| ​ TSS  |  <​m>​sum{i=1}{N}{w_i(y_i – overline{y})^2}</​m> ​ | **Also known as:** \\ • Sum of squares about the mean.  |
 ^ Fit Result ​ ^^^^ ^ Fit Result ​ ^^^^
-| Residual sum of squares<​nowiki>​RSS</​nowiki> ​ |  <​m>​chi^2</​m> ​ |  <​m>​sum{i=1}{N}{w_i(y_i – f(x_i,​beta_1,​...,​beta_p))^2}</​m> ​ | This value is minimized during the fit to find the optimal fit function parameters. \\ **Also known as:** \\ • 'Chi squared'​ \\ • Sum of squared residuals (SSR), \\ • Error sum of squares (ESS), \\ • Sum of squares due to error (SSE). ​ |+| Residual sum of squares ​(<​nowiki>​RSS</​nowiki>​ ​| ​ <​m>​chi^2</​m> ​ |  <​m>​sum{i=1}{N}{w_i(y_i – f(x_i,​beta_1,​...,​beta_p))^2}</​m> ​ | This value is minimized during the fit to find the optimal fit function parameters. \\ **Also known as:** \\ • 'Chi-squared'​ \\ • Sum of squared residuals (SSR), \\ • Error sum of squares (ESS), \\ • Sum of squares due to error (SSE). ​ |
 | Reduced //​χ//<​sup>​2</​sup> ​ |  <​m>​{{chi^2}_{red.}}</​m> ​ |  <​m>​{chi^2}/​nu = {chi^2}/​{N–p}</​m> ​ | The advantage of the reduced chi-squared is that it already normalizes for the number of data points and model (fit function) complexity. \\ **Also known as:** \\ • Mean square error (MSE), \\ • Residual mean square. ​ | | Reduced //​χ//<​sup>​2</​sup> ​ |  <​m>​{{chi^2}_{red.}}</​m> ​ |  <​m>​{chi^2}/​nu = {chi^2}/​{N–p}</​m> ​ | The advantage of the reduced chi-squared is that it already normalizes for the number of data points and model (fit function) complexity. \\ **Also known as:** \\ • Mean square error (MSE), \\ • Residual mean square. ​ |
 | Residual standard deviation ​ |  //s//  |  <​m>​sqrt({chi^2}_{red.})</​m> ​ | **Also known as:** \\ • Root mean square of the error (Root MSE) | | Residual standard deviation ​ |  //s//  |  <​m>​sqrt({chi^2}_{red.})</​m> ​ | **Also known as:** \\ • Root mean square of the error (Root MSE) |
 | [[wp>​Coefficient_of_determination|Coefficient of determination]] ​ |  <​m>​R^2</​m> ​ |  <m>1 – {chi^2}/​TSS</​m> ​ | //​R//<​sup>​2</​sup>​ will be equal to one if fit is perfect, and to zero otherwise. This is a biased estimate of the population //​R//<​sup>​2</​sup>,​ and will never decrease if additional fit parameters (fit curves) are added, even if they are irrelevant. | | [[wp>​Coefficient_of_determination|Coefficient of determination]] ​ |  <​m>​R^2</​m> ​ |  <m>1 – {chi^2}/​TSS</​m> ​ | //​R//<​sup>​2</​sup>​ will be equal to one if fit is perfect, and to zero otherwise. This is a biased estimate of the population //​R//<​sup>​2</​sup>,​ and will never decrease if additional fit parameters (fit curves) are added, even if they are irrelevant. |
-| Adjusted //​R//<​sup>​2</​sup> ​ |  <​m>​overline{R}^2</​m> ​ |  <m>1 – {N–1}/​{N–p–1}(1–R^2)</​m> ​ | Adjusted //​R//<​sup>​2</​sup>​ (or //degrees of freedom adjusted R-square//) is a slightly modified version of //​R//<​sup>​2</​sup>,​ designed to penalize for the excess number of fit parameters ​(fit curves) ​which do not add to the explanatory power of the regression. This statistic is always smaller than //​R//<​sup>​2</​sup>,​ can decrease as you add new fit curves, and even be negative for poorly fitting models. ​ | +| Adjusted //​R//<​sup>​2</​sup> ​ |  <​m>​overline{R}^2</​m> ​ |  <m>1 – {N–1}/​{N–p–1}(1–R^2)</​m> ​ | Adjusted //​R//<​sup>​2</​sup>​ (or //degrees of freedom adjusted R-square//) is a slightly modified version of //​R//<​sup>​2</​sup>,​ designed to penalize for the excess number of fit parameters which do not add to the explanatory power of the regression. This statistic is always smaller than //​R//<​sup>​2</​sup>,​ can decrease as you add new fit curves ​or introduce parameters, and even be negative for poorly fitting models. ​ | 
-| Covariance matrix of parameters //​β<​sub>​k</​sub>//​ |  <​m>​Sigma</​m> ​ |  <​m>​{chi^2}/​{N–p} delim{[}{alpha}{]}^{–1},​~where~delim{[}{alpha}{]}_{m,​n}=</​m>​ \\ <​m>​sum{i=1}{N}{w_i {{partial f}/{partial beta_m}(x_i,​ beta_1, ..., beta_p) {partial f}/{partial beta_n}}{(x_i,​ beta_1, ..., beta_p)}}</​m> ​ | Here α is the matrix of partial derivatives of fit function with respect to parameters //​β<​sub>​m</​sub>//​ and //​β<​sub>​n</​sub>//​ which is used for fitting. ​ |  +| Covariance matrix of parameters //​β<​sub>​k</​sub>//​ |  <​m>​Sigma</​m> ​ |  <​m>​{chi^2}/​{N–p} delim{[}{alpha}{]}^{–1},​~where~delim{[}{alpha}{]}_{m,​n}=</​m>​ \\ <​m>​sum{i=1}{N}{w_i {{partial f}/{partial beta_m}(x_i,​ beta_1, ..., beta_p) {partial f}/{partial beta_n}}{(x_i,​ beta_1, ..., beta_p)}}</​m> ​ | Here α is the matrix of partial derivatives of fit function with respect to parameters //​β<​sub>​m</​sub>//​ and //​β<​sub>​n</​sub>//​ which is also used by fitting ​algorithm to compute parameters for next iteration.  |  
-| Standard deviation of parameters //​β<​sub>​k</​sub>//​,\\ std. dev.  |  <​m>​s_k</​m> ​ |  <​m>​sqrt{delim{[}{Sigma}{]}_{k,​k}}</​m> ​ | These values are displayed in ''​Std. Dev.''​ column in parameters table. ​ |  +| Standard deviation of parameters //​β<​sub>​k</​sub>//​ \\ (std. dev. ​| ​ <​m>​s_k</​m> ​ |  <​m>​sqrt{delim{[}{Sigma}{]}_{k,​k}}</​m> ​ | These values are displayed in ''​Std. Dev.''​ column in parameters table. ​ |  
-| Correlation matrix of parameters //​β<​sub>​k</​sub>//​ |  <​m>​C</​m> ​ |  <​m>​delim{[}{C}{]}_{i,​j}=delim{[}{Sigma}{]}_{i,​j} / {s_i s_j}</​m> ​ | This matrix ​has 1's on diagonal. ​ | +| Correlation matrix of parameters //​β<​sub>​k</​sub>//​ |  <​m>​C</​m> ​ |  <​m>​delim{[}{C}{]}_{i,​j}=delim{[}{Sigma}{]}_{i,​j} / {s_i s_j}</​m> ​ | This matrix ​shows if the parameters are linked. The values lie in range -1...1, diagonal ​elements are always 1. If two parameters are linked the corresponding matrix value will be close to 1. It means that changing the first parameter compensates changing of the second one so that the fitting algorithm cannot select between them.  | 
  
 ===== See Also ===== ===== See Also =====
fit_formulas.1373574377.txt.gz · Last modified: Sun Nov 8 12:20:32 2015 (external edit)