Weighting options for fitting need to be added [edt by adm]

Post Reply
donovaly
Posts: 2
Joined: June 10th, 2020, 1:49 am

Weighting options for fitting need to be added [edt by adm]

Post by donovaly »

I encountered that since at least MP 2.9.2 fitting is broken because the calculation of the residual sum of squares (RSS) is wrong.

To reproduce:

- take the attached file. This is just a table containing values from a Gaussian function
- ij the "gaussfit" you can see that the fit results in a fairly poor result

but this is only the consequence of the wrong RSS calculation. To see this, look at the "Fermitfit". There I made a very good fit (to prevent MP from breaking it, I fixed 2 fit parameters). Renew the fit and check the output:

I get a RSS of ridiculously high 1.0941e6
The real RSS is 12.9436.

Since fitting minimizes the RSS, MP produces these bad fit results.
Attachments
Gaussfit.mppz
test file
(5.79 KiB) Downloaded 6 times

donovaly
Posts: 2
Joined: June 10th, 2020, 1:49 am

Re: Fitting broken since MP 2.9.x

Post by donovaly »

I found the bug:
MP's Wiki states in https://magicplot.com/wiki/fit_formulas#weighting_of_data_points_using_y_errors
MagicPlot can use weighting of y values based on y errors si:

If standard y errors si are specified: wi = 1 / si2 1);
Otherwise: all wi = 1.
I once came up with this because in MP 2.5.1 and earlier the weight was calculated incorrectly. However, the 1/sigma^2 weight fails with my example. The errors (column C in my example9 are calculated as 2% of the measurement values. So the error values are the result of a scaled Gaussian. In this case you cannot weight them as inverse squared variance but directly. (they are Gauss-distributed). So in this case the weight must be wi = si.

Therefore other programs like Origin or LabPlot offer so specify the weight. Here is a Screenshot from LabPlot:
Screenshot-LabPlot
Screenshot-LabPlot
Screenshot-LabPlot.jpg (28.79 KiB) Viewed 178 times
So despite for measurement values that are not specially distributed, the weight should be "Instrumental" while for a Gaussian as in my example, one must a "Direct" weighting. Doing so delivers the correct RSS and thus a perfect fit.

User avatar
MagicPlot
Site Admin
Posts: 60
Joined: December 27th, 2018, 5:24 pm
Location: St. Petersburg
Contact:

Weighting options for fitting need to be added

Post by MagicPlot »

Hello donovaly,

Thank you for the valuable feature request (and for the previous informative discussion if I'm not mistaken). We plan to add weighting options in the next releases.

I will rename this topic because chi square is computed as specified, but the weighting options do need to be implemented indeed.

jofrench
Posts: 5
Joined: July 6th, 2020, 3:34 pm

Re: Weighting options for fitting need to be added [edt by adm]

Post by jofrench »

Thanks. Very useful.

Post Reply