Validate Signaloid Performance on Your Account

Run a published Signaloid benchmark on your own account and measure the speedup of UxHw® over Monte Carlo.

In this tutorial we run the Individual Retirement Account Modeler benchmark. It compares the total returns of a tax-free retirement account with those of a taxable one. The application describes inputs such as the annual contribution and the tax rate as probability distributions.

On a C0Pro-L core, the application runs 466× faster than an optimized C Monte Carlo version that takes 17.3 million samples on an Amazon EC2 R7iz instance.

You run the application once on a UxHw core in the Signaloid Cloud Compute Engine, then run the Monte Carlo version of the same application source code on your machine. After you import the two resulting files, the platform compares the two execution times and saves a PDF report to your Cloud Storage.

Before you begin

The Monte Carlo baseline runs on your machine, so you need:

• git and make.
• A C compiler, such as GCC or Clang.
• The GNU Scientific Library (GSL).
• About 500 MB of free disk space. The baseline writes a 484 MB result file.

Keep the benchmark page open from start to finish. If you close the page, you lose the comparison.

Start the benchmark

Click Validate Performance () in the sidebar. The page shows each benchmark Signaloid publishes on its own card, and the buttons above the list filter the cards by industry.

Click All benchmarks, then find the Individual Retirement Account Modeler card in the Digital Banking category. The card names the C0Pro-L core and shows the Non-configurable badge. Signaloid sets the core and branch, and you cannot change them, so you can compare your result directly with Signaloid's.

Click Run Benchmark.

The run view opens and a notification confirms that the platform added the benchmark repository to your Connected Repositories.

Figure 1. Opening Validate Performance, choosing All benchmarks, and clicking Run Benchmark on the Individual Retirement Account Modeler card. A notification confirms that the platform added the benchmark repository to your Connected Repositories.

Watch the UxHw run

The first step, UxHw run, builds the application on C0Pro-L and runs it once. The status of the Build row, then the Task row, changes from Accepted to Initialising, In Progress, and Completed.

When the task completes, the first result card shows the C0 execution time and the program output with its distribution plot. This one execution produces the full output distribution.

Click Next: Monte Carlo baseline.

Run the Monte Carlo baseline

On Choose environment, keep Local selected and click Continue to local run.

The page now shows four numbered sub-steps. Open a terminal and clone the repository with the command from sub-step 1:

git clone --recurse-submodules --branch main https://github.com/signaloid/Signaloid-Demo-Finance-IndividualRetirementAccount

Then run the command from sub-step 2:

cd Signaloid-Demo-Finance-IndividualRetirementAccount && make local-build && ./demo-native-mc -S 0 -M 17300000 -T | tee std.out

The command builds the native Monte Carlo version of the application and runs it for 17.3 million Monte Carlo iterations. When it finishes, the command has written two files to the repository root: std.out and data.out.

Import the results

Drag std.out from the repository root onto Import std.out. The page shows the message "std.out imported".

Now drag data.out onto Import data.out. The page reads the file in your browser and shows a progress indicator. At about 484 MB, this import takes longer than the std.out import.

When the import finishes, click See comparison.

Figure 2. Choosing the Local environment, running the clone and baseline commands in a terminal, and importing std.out and data.out.

Read the comparison

The comparison shows the measured speedup at the top, then a bar chart of the two execution times. Your figure will differ from the published 466×, because your Monte Carlo baseline ran on your machine instead of an Amazon EC2 R7iz instance, and other tasks on the shared Signaloid Cloud Compute Engine can affect the UxHw timing.

Notice the two result cards. The Monte Carlo card shows a histogram of the 17.3 million samples you imported, on the same axes as the UxHw distribution. The distribution from one UxHw run closely resembles the histogram of 17.3 million Monte Carlo samples.

A notification confirms that the platform saved the report to your Cloud Storage at Performance-Validation/Reports/finance-individual-retirement-account-benchmark-report-<date>.pdf. Click Download report to keep a copy on your machine.

Figure 3. Opening the comparison, which shows the measured speedup, a bar chart of the two execution times, and the two result cards on the same axes, then downloading the report.

What you did

You ran a published Signaloid benchmark on your account, produced its Monte Carlo baseline on your own hardware, and measured the speedup of UxHw over that baseline. Back on the Validate Performance page, the card now shows your measured figure.

To learn how Signaloid measures its published figures, see How we measure speedup. To run other benchmarks or fix a rejected file, see How to Validate Performance. For every field on the page, see the Validate Performance reference.