Reduced precision in output
using DEFAULT_NUM_FORMAT = "%f"
causes the output to have reduced precision. example using an f-string:
f"{0.0008534400105:f}"
Out[76]: '0.000853'
# vs.
f"{0.0008534400105}"
Out[77]: '0.0008534400105'
This applies similarly to numpy.savetxt(fname, X, fmt='%.18e' ...
; unfortunately, the default fmt also doesn't seem well-suited:
f"{0.0008534400105:.18e}"
Out[78]: '8.534400104999999726e-04'
- One solution is of course to provide a specific format for the output (or a sequence of formats), that matches that of the input.
- That still leaves unnecessary decimal places for VMISS, see #14 (closed)
- using
%g
%e
might be better-suited:
f"{0.0008534400105:e}"
Out[87]: '8.534400e-04'
f"{0.0008534400105:g}"
Out[88]: '0.00085344'
Edited by Florian Obersteiner