FPBench Logo

FPCore 1.0

FPBench 1.0 standards

FPBench is a standard benchmark suite for the floating point community to organize its efforts around. The benchmark suite is organized around a common format and a common set of measures of accuracy. FPBench 1.0 is the first version of this common format and set of measures.

FPBench 1.0 standardizes:

  1. The FPCore input format
  2. Metadata for FPCore benchmarks
  3. Standard measures of error
This is a draft specification. It is subject to unannounced changes.

FPCore benchmark format

FPCore is the format used for FPBench benchmarks. It is a simple functional programming language with conditionals and simple loops. The syntax is an easy-to-parse S-expression syntax.

(FPCore (x)
 :name "NMSE example 3.1"
 :cite (hamming-1987)
 :pre (>= x 0)
 (- (sqrt (+ x 1)) (sqrt x)))
An example FPCore program, from the hamming-ch3 suite.

Syntax

The syntax is a simple S-expression syntax with the following grammar.

FPCore
( FPCore (symbol)* property* expr )
expr
number
constant
symbol
( operation expr* )
( if expr expr expr )
( let ( [ symbol expr ]* ) expr )
( while expr ( [ symbol expr expr ]* ) expr )
property
:symbol expr
:symbol string
:symbol ( symbol* )
High-level grammar of FPCore. The tokens constant and operation, as well as base tokens like number, are defined below.

In this grammar, an FPCore term describes a single benchmark, with a set of free variables, a collection of metadata properties, and the floating-point expression defining the benchmark. White-space is ignored, and lines starting with the semicolon (;) are treated as comments and ignored. The basic tokens are defined as expected:

symbol
Any sequence of letters, digits, or characters from the set ~!@$%^&*_-+=<>.?/: not starting with a digit. The regular expression [a-zA-Z~!@$%^&*_\-+=<>.?/:][a-zA-Z0-9~!@$%^&*_\-+=<>.?/:]* implements this definition for ASCII alphabets.
number
An optional plus (+) or minus (-) sign followed by any sequence of digits. These may be optionally followed by a period (.) and another sequence of digits, which themselves may be optionally followed by an e, optional plus or minus sign, and a sequence of digits. The regular expression [-+]?[0-9]+(\.[0-9]+(e[-+]?[0-9]+)?)? implements this definition.
string
Any sequence of printable characters, spaces, tabs, or carriage returns, delimited by double quotes ("). Within the double quotes, backslashes (\) have special meaning. A backslash followed by a double quote represents a double quote and does not terminate the string; a backslash followed by a backslash represents a backslash; other escapes may also be supported by implementations, but their meaning is not defined in this standard. We recommend that implementations only use escapes defined in the C or Matlab standard libraries. The regular expression "([\x20-\x5b\x5d-\x7e]|\\["\\])+?" implements this definition for ASCII alphabets.

Supported operations

The following operations are supported:

Supported Mathematical Operations
+-*/fabs
fmaexpexp2expm1log
log10log2log1ppowsqrt
cbrthypotsincostan
asinacosatanatan2sinh
coshtanhasinhacoshatanh
erferfctgammalgammaceil
floorfmodremainderfmaxfmin
fdimcopysigntruncroundnearbyint
Supported Testing Operations
<><=>===
!=andornotisfinite
isinfisnanisnormalsignbit

except - which can be unary to represent negation. All operations have the same signature as the equivalent operations in C11. The arithmetic functions are all binary operators, The comparison operators and boolean operators and and or take an arbitrary number of arguments.

A comparison operator with more than two operators is interpreted as the conjuction of all ordered pairs of arguments. In other words, == tests that all its arguments are equal; != tests that all its arguments are distinct; and <, >, <=, and >= test that their arguments are sorted (in the appropriate order), with equal elements allowed for <= and >=, and disallowed for < and >.

Following IEEE-754 and common C and Fortran implementations, FPCore does not prescribe an accuracy to any mathematical functions except the arithmetic operators, sqrt, and fma. If the exact accuracy is important, we recommend that benchmark users declare the implementation used and its accuracy with the :math-library property.

Supported constants

The following constants are supported:

Supported Mathematical Constants
ELOG2ELOG10ELN2LN10
PIPI_2PI_41_PI2_PI
2_SQRTPISQRT2SQRT1_2INFINITYNAN
Supported Boolean Constants
TRUEFALSE

The floating-point constants defined just like their analogs in GNU libc. This gives FPCore support for all constants commonly provided by C and Fortran implementations. All constants are assumed to be the closest floating-point number to the mathematically-accurate value of the constant.

Semantics

FPCore expressions can describe concrete floating-point computations, abstract specifications of those computations, or intermediates between the two. The semantics of FPCore are correspondingly flexible.

FPCore does not restrict the representation used to represent values when evaluating FPCore expressions. Numeric values can be floating point values of various precision, fixed point values, or (for abstract uses) mathematical reals, though individual FPCore expressions can use the :precision property to clarify what representation is expected. Evaluation nonetheless proceeds by fixed rules.

Expressions return either numeric values or boolean values. Operations that receive values of mixed or incorrect types, such as (+ 1 TRUE) are illegal and the results of evaluating them undefined.

The semantics of function application are standard.

An if expression evaluates the conditional to a boolean and then returns the result of evaluating the branch chosen by the conditional.

Bindings in a let expression are evaluated simultaneously, as in a simultaneous substitution. Thus, (let ([a b] [b a]) (- a b)) is the same as (- b a) and (let ([a 1] [b a]) b) is illegal unless a is available in the context. For sequentially binding variables, use nested lets.

A while loop contains a conditional, a list of bound variables, and a return expression. Both the conditional and the return expression may refer to the bound variables. While loops are evaluated according the equality:

(while cond ([x x0 x*] ...) body)

(if (let ([x x0] ...) cond) (while cond ([x (let ([x x0] ...) x*) x*] ...) body) (let ([x x0] ...) body))

In other words, the bound variables are initialized by evaluating the initial expression in the surrounding context. On each iteration, the conditional is evaluated in the bound context. If the conditional is false, the return expression is evaluated (in the bound context) and its value is returned. If the condition is true, the update expressions for each variable are evaluated (in the bound context), and then each bound variable is updated to its new value.

If a loop has multiple variables, all of the update expressions are evaluated before any variable is re-bound; in other words, the old values of every bound variable is used to evaluate the update expression.

For example, (while (< i 1) ([i 0 (+ i 1)] [j 0 (+ i j)]) (+ j 1)) is evaluated by evaluating 0 to 0; binding i to 0 and j to 0; evaluating (< i 1) to truth; evaluating (+ i 1) to 1 and (+ i j) to 0; re-binding i to 1 and j to 0; evaluating (< i 1) to falsehood; and finally evaluating (+ j 1) to 1.