Skip to main content

Running Repository Tasks

You can connect and run git repositories on the Signaloid Cloud Compute Engine. The Signaloid Cloud Compute Engine API currently only supports repositories hosted on GitHub. When you submit a repository task to the API, the Signaloid Cloud Compute Engine clones your repository (with --recurse-submodules), builds your code, and runs the application.

Note

To connect private repositories you need to authorise the Signaloid Cloud Compute Engine to access your GitHub account. You have to do this via the Signaloid Cloud Developer Platform. Refer to documentation on connecting repositories for more details.

This guide uses the Brown-Ham model GitHub repository from the Signaloid public demo repositories as an example. The following code examples show to connect and run the repository on the Signaloid Cloud Compute Engine API.

Note

The Task request communication and schema are different when running repository tasks compared to source code tasks.

For repository applications you need to connect the repository first and then provide the repository identifier (RepositoryID) to the Signaloid Cloud Compute Engine, instead of sending the entire application source with an API request. The repository is connected to your Signaloid Cloud account and any changes you make to the repository object are pulled each time you launch a repository task for this repository.

Prerequisites​

To run a repository task, you will need to have the following:

  • An API key to access the Signaloid Cloud API. Please refer to the Authentication guide for more details.
  • A GitHub repository containing the application source.
  • If the repository is private, authorization of the Signaloid Cloud Compute Enging to access your repositories.

Connecting a repository to your account​

To run a repository task you must first connect the repository to your Signaloid Cloud account. You can connect a repository using the /repositories endpoint of the API. Through this endpoint, you can also programmatically list your connected repositories and amend their configurations if needed.

To connect a repository to your account you need to send an HTTP POST request to the /repositories endpoint with a repository configuration object. The TypeScript type definition below shows the structure of the repository configuration object:

  type DataSource = {
    Location: string;
    Object: "DataSource";
    ResourceID: string;
    ResourceType: "Gateway" | "Bucket" | "Drive" | "SignaloidCloudStorage";
  }

  type TraceVariable = {
    File: string;
    LineNumber: number;
    Expression: string;
  }

  type RepositoryConfig = {
    RemoteURL: string;
    Commit: string;
    Branch: string;
    BuildDirectory: string;
    Arguments: string;
    Core?: string;
    DataSources?: DataSource[];
    TraceVariables?: TraceVariable[];
  };
tip

You can also use the Repositories page on the Signaloid Cloud Developer Platform to connect repositories to your account. Please refer to documentation on connecting repositories for more details.

The code below shows how to connect a repository to your account using the axios library. The code shows how to use the BuildDirectory key. We set the BuildDirectory to src/v3 to instruct the Signaloid Cloud Compute Engine to use the code src/v3 as the application source code and ignore the rest of the directories. The README in the GitHub repository explains the different versions of the Brown-Ham example.

  // API token generated from signaloid.io/settings/api
  const apiToken = "scce_yourSignaloidCloudApiKey";

  // Setup HTTP client
  const axios = require("axios");
  const signaloidClient = axios.create({
    baseURL: "https://api.signaloid.io",
  });

  // Setup request headers
  signaloidClient.defaults.headers["Authorization"] = `${apiToken}`;
  signaloidClient.defaults.headers["Content-Type"] = "application/json";

  const repositoryURL = "https://github.com/signaloid/Signaloid-Demo-Metallurgy-BrownHamModel";
  const repositoryConfig = {
    RemoteURL: repositoryURL,
    Commit: "42e4b5a6f5bc8641317f17d0662b54ae9fc587f9",
    Branch: "main",
    BuildDirectory: "src/v3",
    Arguments: "",
  };

  console.log(`Connecting ${repositoryURL} to your account via the API...`);
  let repositoryID;
  let connectRepositoryResponse;
  try {
    connectRepositoryResponse = await signaloidClient.post("/repositories", repositoryConfig);
    console.log("connectRepositoryResponse.status :>> ", connectRepositoryResponse.status);

    if (connectRepositoryResponse.status === 201) {
      repositoryID = connectRepositoryResponse.data.RepositoryID;
      console.log(`...repository successfully connected. Repository ID: ${repositoryID}`);
    }
  } catch (error) {
    console.log("Error :>> ", error); // [TODO]: handle errors
  }
Note

The Signaloid Cloud Compute Engine API allows you to connect the same repository multiple times to your account. This allows you to keep different configurations of the same GitHub repository. Each connected repository will have a different repository IDs even though they use the same GitHub repository as the base.

Setting up the task request​

The repository task request object has the following structure:

  type RepositoryTaskRequest = {
    Type: "Repository";
    Repository: string;
    Overrides?: {
      Arguments?: string;
      BuildDirectory?: string;
      Core?: string;
      DataSources?: DataSource[];
      TraceVariables?: TraceVariable[];
    };
  };

Task request object for an uncertainty-tracking (UT) C0 or C0 Pro core​

The snippet below shows how to initialise a task request object for a connected GitHub repository. You can override the Core field to choose one of the default Signaloid cores or one of your custom Signaloid cores that you want your application to run on. The Core field is optional and if you omit it, your task will run on the C0-XS+ core.

  // Core ID of the C0-S+ core
  const coreID = "cor_b21e4de9927158c1a5b603c2affb8a09";

  const taskRequest = {
    Type: "Repository",
    Repository: repositoryID, // your repository ID
    Overrides: {
      Core: coreID,
    },
  };

Task request object for a C0 or C0 Pro Reference core​

Note

You need to use the TraceVariables element in Overrides of the Task object to instruct the Signaloid Cloud Compute Engine about which variables it should trace. Your source code may contain multiple variables with the same name, defined in different scopes of your source code. Because of this, the Signaloid Cloud Compute Engine needs to know the file and line number of the declaration of the variable that you want to trace to be able to uniquely identify it.

The following code snippets show how to use the Signaloid API calls to instruct the Signaloid Cloud Compute Engine to run on C0-Reference core the source code shown at the top of the page. The TraceVariables element of the Task object, instructs the Signaloid Cloud Compute Engine to trace variable sigmaCMpa, in line 95 of file Brown-and-Ham-with-all-parameters-as-distributions.c as you can see in src/v3/src of the connected GitHub repository.

  // Core ID of the C0-Reference core
  const coreID = "cor_9a3efb0094405df5aeb61cf1f29606a0";

  const taskRequest = {
    Type: "Repository",
    Repository: repositoryID, // your repository ID
    Overrides: {
      Core: coreID,
      TraceVariables: [{ Expression: "sigmaCMpa", File: "src/v3/src/Brown-and-Ham-with-all-parameters-as-distributions.c", LineNumber: 95 }]
    },
  };

Submitting the task to the API​

As with the single-source application, once you have constructed the task request object, you can submit it to the Signaloid Cloud Compute Engine via a POST request to the /tasks endpoint.

  console.log("Submitting the task to the API...");
  let taskPostResponse;
  try {
    taskPostResponse = await signaloidClient.post("/tasks", taskRequest);

    if (taskPostResponse.data.TaskID) {
      console.log(`...task successfully created with ID: ${taskPostResponse.data.TaskID}`);
    }
  } catch (error) {
    console.log("Error :>> ", error); // [TODO]: handle errors
  }

The response from the API follows the following structure. You can find the full structure here by choosing the RepositoryTaskPost.

{
  "data": {
    "Object": "Task",
    "TaskID": "tsk_6dcae4d1c4f585901efa12a89a063996",
    "Owner": "usr_yourSignaloidCloudUserID",
    "Application":  {...}, // The application that was submitted to the API
    "Status": "Accepted",
    [...other response keys omitted]
  },
  "status": 202,
  "statusText": "Accepted",
  [...other response keys omitted]
}

You can now use the TaskID to retrieve the status and outputs of the task from the API.