How to Validate Performance

Measure a published Signaloid benchmark on your account, retrieve its report, and resolve the issues that stop a run.

This guide covers the tasks you do before, during, and after a benchmark run on the Validate Performance page (). For a first run from start to finish, see the Validate Signaloid Performance on Your Account tutorial. For every field and message on the page, see the Validate Performance reference.

The Monte Carlo baseline runs on your machine. It needs git, make, a C compiler, and the GNU Scientific Library (GSL).

Run a benchmark

1. On the Validate Performance page, filter the list by industry and find the benchmark. Each benchmark has its own card, with its published speedup and a Run Benchmark button.
2. Click Run Benchmark, or Re-run Benchmark if you have run it before.
3. When the UxHw® run completes, click Next: Monte Carlo baseline.
4. Keep Local selected and click Continue to local run.
5. Run the clone and baseline commands the page shows, without changing them.
6. Import std.out, then data.out, from the root of the cloned repository.
7. Click See comparison.

Keep the page open until the comparison appears. If you leave, the task keeps running and appears on the Builds and Tasks page, but you must start a new benchmark run to get a comparison.

If you already connected the benchmark repository with your own branch or build directory, the run builds from src/ on main, then restores your branch and build directory after it submits the build. If the platform cannot restore them, a notification tells you. Restore them yourself under Connected Repositories.

Download an earlier report

Each run saves a PDF report to your Cloud Storage under Performance-Validation/Reports/. On the benchmark card, click Download report for the latest report of that benchmark.

A run on the same UTC day as an earlier run replaces that day's report. Download the earlier report first if you want to keep both.

Reports from earlier days stay in the same folder. Download them from Cloud Storage.

Fix a rejected result file

The page checks each file when you import it. Find the message in the table:

Message starts with	Cause	Fix
"std.out is too large to be the output of a completed run"	The file exceeds 5 MB, which means the run printed one line per sample instead of a summary	Run the baseline command exactly as the page shows it
"std.out is empty"	The run did not finish, or you redirected its output to another file	Run the command again and let it finish
"data.out holds no samples"	The run stopped before writing samples	Run the command again and let it finish
"data.out carries no execution time on its first line"	The first line is missing, or it disagrees by more than 5% with the CPU time used line in std.out	Import the std.out and data.out from the same run
"data.out holds N samples, and this benchmark's baseline writes M"	The command used a different -M value	Run the command exactly as the page shows it

After you have a new file, click Remove file on the rejected import and import the new file.

Run a benchmark your plan does not include

Signaloid sets the core for each benchmark. If your plan does not include that core, the card disables Run Benchmark and names the core. Click Compare plans to see which plans include it, or pick a benchmark on a core your plan includes.

To benchmark the same application with your own configuration, follow the benchmarking tool README in signaloid-python. You cannot compare results from another core with Signaloid's published figures.

Resolve a limit error

A benchmark run counts toward your plan's limits like any other build and task.

• Concurrent builds or tasks. Wait for your active runs to finish, or cancel one on the Builds and Tasks page, then start the benchmark again.
• Monthly compute allowance. The UxHw build or run stops when the allowance runs out. Check your remaining credits on the Usage page.

Remove the benchmark repository

The benchmark repository stays connected after the run, so you can build it on any core and branch. To remove it, follow Disconnect Repositories. The next benchmark run connects it again.