FPBench Logo

FPBench Internal Tools

Interpreters and exporters for FPCore

FPBench includes a filtering tool, an FPCore reference interpreter, an accuracy measurement tool, and an exporter to C.

Filtering FPCore benchmarks

The filter script is located in the tools directory of the FPBench codebase.

The filter script reads a sequence of FPCore benchmarks from the standard input and writes to standard output those that satisfy a predicate:

racket tools/filter.rkt field value < file.fpcore

The -v or --invert flag can be passed before the field to invert the predicate.

The supported fields are:

Allows FPCore benchmarks whose :cite field is a list that includes the value.
Allows FPCore benchmarks that use the operator given as the value. Best used by inverting the predicate to filter out benchmarks that use unsupported operations. if, while, and let are considered operators. operation is a synonym for operator
Allows the FPCore benchmark to contain only operators from a list, given by providing multiple value arguments. Note that while both operator and operators examine the operators used in a benchmark, they filter by fairly different predicates.
Allows FPCore benchmarks whose property property is specified to be the given value, or, if no value is provided, that some value is specified.

The filter script's intended use is in selecting a supported subset of the FPBench benchmark suite. For example, to select all benchmarks that do not use loops or conditionals:

cat benchmarks/*.fpcore |
  racket tools/filter.rkt --invert operator while |
  racket tools/filter.rkt --invert operator if

The FPCore reference interpreter

The reference interpreter is located in the tools/ directory of the FPBench codebase, and can be run with:

racket tools/fpcore.rkt arguments ... < file.fpcore

The arguments in this case are real numbers using the standard floating-point number syntax. For example, running:

racket tools/fpcore.rkt 1.0 2.0
(FPCore (x y) (+ x y))

produces output 3.0.

The fma, expm1, log1p, isnormal, signbit, fmod, remainder, copysign, and nearbyint functions are not yet supported by the reference interpreter.

Statistical Accuracy Measurement

FPBench provides a tool to measure the accuracy of FPCore programs. This tool serves as a reference implementation for the Measures 1.0 standard, and uses statistical techniques to measure average forward error. Scaling and non-scaling error are both supported.

To measure the accuracy of an FPCore program, run:

racket tools/sample-accuracy.rkt < file.fpcore

The accuracy of each FPCore expression in the file will be printed to the screen. The accuracy measure and measurement technique can be configured:

Configures the pointwise error measure used; it may be abs for absolute error, rel for relative error, ulp for ULPs of error, or bit for bits of error (the default).
How many bits of precision to use to compute the ground truth (the default is 2048). Using more bits is slower but computes a more accurate ground truth and thus produces more-accurate results.