FPBench Logo

FPBench

Common standards for the floating-point research community

Benchmarks

FPBench was introduced at NSV’16. Since then, the benchmark suite has grown to 111 benchmarks, several implementations have appeared, and the standards have been deepened and improved.

Benchmark sources
Rosa34
Herbie31
Salsa26
FPTaylor17
Features used
Arithmetic118
Temporaries60
Comparison35
Loops30
Exponents19
Trigonometry15
Conditionals11
Domains
Textbooks28
Mathematics25
Controls10
Science10
Others45

FPBench also publishes tools for working with its standard formats, including an FPCore reference interpreter and an an FPCore exporter to C.

Tools

Standards

Current Status

FPBench was introduced at NSV’16. Since then, the benchmark suite has grown to 111 benchmarks, several implementations have appeared, and the standards have been deepened and improved.

Benchmark sources
Rosa37
Herbie28
Salsa25
FPTaylor21
Features used
Arithmetic111
Temporaries57
Comparison33
Loops28
Exponents16
Trigonometry15
Conditionals10
Domains
Textbooks28
Mathematics24
Controls10
Science10

FPBench also publishes tools for working with its standard formats, including an FPCore reference interpreter and an an FPCore exporter to C.

Papers

  1. Toward a Standard Benchmark Format and Suite for Floating-Point Analysis

    At NSV’16 by N. Damouche, M. Martel, P. Panchekha, C. Qiu, A. Sanchez-Stern, and Z. Tatlock
    Introducing FPBench and describing the standards.
  2. Combining Tools for Optimization and Analysis of Floating-Point Computations

    At FM’18 by H. Becker, P. Panchekha, E. Darulova, and Z. Tatlock
    Combining Daisy and Herbie using FPBench tools. All scripts are available in a branch.

News

  1. Zach gave a keynote about FPBench (and his projects Herbie, Herbgrind, and Titanic) at CoNGA’19. Let's hope for yet more benchmarks, tools, and standardization.
  2. Check out the new benchmarks browser! You can browse, search, and download the FPBench benchmarks from one convenient place.
  3. The FPBench 1.1 standards have been finalized! Check them out!
  4. A draft of the upcoming FPBench 1.1 standards has been released. To preview the changes and leave feedback, see the pull request on Github.
  5. The Daisy and Herbie have a paper at FM’18 comparing the two projects using the FPBench benchmarks, formats, and tools. Come talk to them at Oxford!
  6. FPBench was a hit at Dagstuhl 17352 (Analysis and Synthesis of Floating-point Programs), with a talk, a session, and discussions devoted to composing and comparing floating point tools. Thank you to all Dagstuhl participants for clarifying and prioritizing the challenges and for helping shape our next steps.
  7. Zach gave a talk at MPI-SWS (Saarbrüken) on FPBench and the need to create a common benchmark suite. Thank you Eva Darulova and her students for the invitation and the productive discussion on multiple precisions, benchmark sources, and community building that followed.
  8. The 1.0 versions of the FPCore, Metadata, and Measures standards have all been finalized. Build with them!
  9. Pavel Panchekha is giving a talk about FPBench today at the 9th NSV. Come learn about our effort to build common resources for the floating-point research community.