As the FPBench feature set grows, implementing FPBench can look more and more intimidating. But it shouldn't. These notes are our collective advice on implementing the FPBench standards, and should clarify the standards.
Not every feature of the standard needs to be implemented. For example,
:math-library, are optional.
PI, are optional
binary32, are optional
Focus on correctness for features you do implement, rather than the breadth of features you support. Plus, correctly implementing a feature usually only requires care, while implementing new features can require fundamental research.
Different interpretations are more harmful to interoperability
than missing features. Tools should, however, publish which
features they support. Users can then use tools like the
filter script to select the subsets of their
benchmarks supported by a tool.
Tools are free to reject any FPCore as unsupported. The FPCore standard is quite flexible, but this flexibility means that it is difficult for a tool to behave equally well on all inputs. For example, mixed-precision operations can be tedious to implement, while loops may be outside the purview of many techniques.
Tools should, however, make sure to raise an error when an FPCore is unsupported, instead of simply returning invalid results. For example, a tool that does not support mixed precision operations should raise an error instead of ignoring the precision annotations and producing incorrect results.
Tools are free to add features to the standard, as long as they namespace them to avoid conflicts. For example, tools can add:
(fixed 12 4)
Namespaced names, like
future-proof, while it is possible that future standards might
define standard functions like
median. Where multiple
plausible interpretations exist, namedspaced names are better.