Anatella rounding
- Thread starter arthur2
- Start date
Inside Anatella, there is no rounding anywhere (unless you use the "round()" function of the calculator, but that's normal). Especially if you use .gel_anatella files to save your data, you are guaranteed that everything is perfect, without any rounding.
On the other hand, when you export to a format external to Anatella, then there can be a loss of precision due to rounding.
In your email, you show a screenshot of Excel. By default, the output to an excel file is done without any rounding: This is visible through the parameter here:
This parameter is documented in section 5.5.1.3. of the "AnatellaQuickUserGuide.pdf" (press F1 inside Anatella to read this PDF).
Also, by default, there is a little rounding when outputting to a text file (.csv or .txt) (because, otherwise, these files can get very large very quickly). You can see it here:
If you don't want any rounding in the .txt output, just choose %.16g (instead of %g) in this option.
Also since Anatella 2.76, the option to display in the "data preview" window all the columns with the "Float" meta-type with the maximum precision visible on the screen, I available. You have to choose the "%.16g" option here:
This way you can see that all calculations are done at maximum precision everywhere in Anatella (but it is generally less pleasant to read).
Cheers
On the other hand, when you export to a format external to Anatella, then there can be a loss of precision due to rounding.
In your email, you show a screenshot of Excel. By default, the output to an excel file is done without any rounding: This is visible through the parameter here:
This parameter is documented in section 5.5.1.3. of the "AnatellaQuickUserGuide.pdf" (press F1 inside Anatella to read this PDF).
Also, by default, there is a little rounding when outputting to a text file (.csv or .txt) (because, otherwise, these files can get very large very quickly). You can see it here:
If you don't want any rounding in the .txt output, just choose %.16g (instead of %g) in this option.
Also since Anatella 2.76, the option to display in the "data preview" window all the columns with the "Float" meta-type with the maximum precision visible on the screen, I available. You have to choose the "%.16g" option here:
This way you can see that all calculations are done at maximum precision everywhere in Anatella (but it is generally less pleasant to read).
Cheers