Requesting Data

Gora smart contract call API

This document describes the API for making Gora oracle requests and receiving responses. It is intended to be definitive source for Gora Node Runner software behavior. The focus is on being complete, current and application-agnostic.

For getting started with Gora in smart contracts, developers are advised to use Gora-provided AlgoKit application template.

Requesting oracle data

Oracle requests are made by calling request method of the main Gora smart contract. This contract's application ID is a part of Gora network configuration and can be found using info command of the Gora CLI tool.

The request method accepts following arguments:

• spec - request specification; serialized data structure, describing the request. Its format depends on the request type and is explained in the next section.

• type - integer ID of the request type; the type determines encoding of the spec field.

• dest_app - Algorand application ID of the smart contract that will be called to return oracle response.

• dest_method - Name of the method to call in the smart contract specified by dest_id.

Encoding request specification

Request specification (spec) is defined and encoded as an Algorand ABI type. It is a structured type that holds properties of the request. Its structure depends on the request type which is specfied by the numeric type field. Currently there are two request types that both defined as ABI tuples with the following structure:

• tuple(sourceSpec[], aggregation, userData)

Where:

• sourceSpec[] - array of source specifications

• aggregation: uint32 - numeric ID of the aggegation method used for the request

• userData: byte[] - any data to attach to request and its response

A a source specification describes oracle sources queried in the request. It is an ABI tuple, its format depends on the request type and is described below.

Request type #1 - for predefined oracle sources

This is the original Gora request type which relies on oracle source definitions bundled with the Node Runner software. Source specifications for requests of this type contain the following fields:

• sourceId: uint32 - numeric ID of an oracle source

• sourceArgs[] - array of byte[] strings, arguments to the source

• maxAge: uint32 - maximum age of source data in seconds to be considered valid

Parametrized oracle sources

To add flexibility to requests of type #1, certain parameters of oracle source definitions can be specified at run time on per-request basis.

For example, if a source provides many pieces of data from the same endpoint, it is more convenient to let the requester specify the ones they want than to define a separate source for each. This is achieved by parametrizing valuePath property. Setting it to ##0 in the oracle source definition will make Gora nodes take its value from 0'th argument of the request being served. Parameter placeholders can just as well be placed inside strings where they will be substituted, e.g. http://example.com/##2&a=123.

The following oracle source definition properties can be parametrized: url, valuePath, timestampPath, valueType, value, roundTo, gateway. The substituted values are always treated as strings. For example, when supplying a parameter to set roundTo field to 5, string "5" must be used rather than the number.

Request type #2 - general URL requests

This type of oracle request was introduced for additional flexibility and security. It does not depend on a pre-configured list of oracle sources and allows authentication via third party without compromising decentralization. Source specifications for requests of this type contain the following fields:

• url: byte[] - source URL to query

• authUrl: byte[] - authenticator URL

• valueExpr: byte[] - expression to extract value from response

• timestampExpr: byte[] - expression to extract timestamp from response

• maxAge: uint32 - maximum age of data in seconds to be considered valid

• valueType: uint8 - return value type: 0 - string, 1 - number

• roundTo: uint8 - number of digits to round numeric value to

• gatewayUrl: byte[] - gateway URL (not for general use)

• reserved0: byte[] - reserved for future use

• reserved1: byte[] - reserved for future use

• reserved2: uint32 - reserved for future use

• reserved3: uint32 - reserved for future use

Third-party authentication

Requests of type 2 support using third party services to access sources that require authentication. For example, a price data feeds provider may protect their paid endpoints by requiring an access key (password) in URLs. Since everything stored by the blockchain is public, authentication keys cannot be held by smart contracts or included in oracle requests. Node operators may configure their own access keys for some sources, but not in the general case. Third-party authentication services that issue one-time authentication keys on per-request basis are designed to fill that gap. When authUrl field in the source specification is filled, Node Runner software will call this URL an receive a temporary auth key. The authenticator service will check that the node runner and the oracle request are both eligible to receive it.

Data extraction expressions

Expressions in valueExpr and timestampExpr fields follow the format: <type>:<expression>, e.g. jsonpath:$.data. The following expression types are supported:

• jsonpath: JSONPath expression, see: https://datatracker.ietf.org/doc/draft-ietf-jsonpath-base/

• xpath: XPath expression, see: https://www.w3.org/TR/2017/REC-xpath-31-20170321/

• regex: JavaScript regular expression, see: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions

• substr: substring specification, start and end offsets, e.g. substr:4,11

• bytes: same as substr, but operates on bytes rather than characters

Request type #3 - off-chain computation

For use cases that require even more flexibility, Gora supports oracle requests that execute user-supplied Web Assembly code. The code is executed off-chain by Gora network nodes and is subject to resource limits. Request specification ABI type for this kind of request has the following structure:

• execType: uint32 - executable type (0- binary WASM, 1- same, but gzipped)

• apiVersion: uint32 Gora API version the executable is written for

• execBody: byte[] - executable body

• userData: byte[] - user-supplied data, passed to destination app as is

• reserved0: uint32 - reserved for future use

• reserved1: uint32 - reserved for future use

• reserved2: byte[] - reserved for future use

• reserved3: byte[] - reserved for future use

To make use of this feature the developer must:

• Write their program using Gora Off-Chain API and any language that compiles to Web Assembly.

• Compress the resulting .wasm binary with gzip utility (optional).

• Populate execBody request specification field with the result of the above and make oracle request the usual way by calling Gora main smart contract.

Executable size is currently limited by maximum size of Algorand smart contract arguments to something under 2KB, depending on other arguments of the request smart contract call.

To get a grasp of Gora Off-Chain API and execution model, consider the example program: off_chain_example.cpp. It is supposed to be compiled with Clang:

clang off_chain_example.c -Os --target=wasm32-unknown-unknown-wasm -c -o off_chain_example.wasm

Working around Web Assembly's inability to make asynchronous calls, Gora off-chain programs are executed in steps. A step starts when the program's main function is called by the Node Runner and ends when it returns. During a step, the program can schedule HTTP requests. They are executed after the step and resulting data is made available to the program during the next step. The main function can access this and other node-provided data such as the number of step currently executing via the structure passed to it as goraCtx argument. The program signals to the Node Runner whether it wants to execute another step or terminate with the the main function return value.

Multi-value requests and responses

This feature allows requests of type 1 and 2 to fetch multiple pieces of data from the same source response. Normally, valuePath property contains a single expression, so just one value is returned by an oracle request. To return multiple values, it is possible to specify multiple expressions separated by tab character. For example: $.date\t$.time\t$.details.name. Since an oracle return value must be a single byte string for the consensus to work, returned pieces of data are packed into Algorand ABI type - an array of strings:

const multiResponse = new Algosdk.ABIArrayDynamicType(Algosdk.ABIType.from("byte[]"));

To access individual results, smart contract handling the oracle response must unpack this ABI type. Nth string in the array will correspond to the nth expression in the valuePath field. Important: all returned pieces of data in such responses are stringified, including numbers. For example, number 9183 will be returned as ASCII string "9183". Smart contract code handling the response must make the necessary conversions.

Rounding numeric response values

Certain kinds of data, such as cryptocurrency exchange rates, are so volatile that different Gora nodes are likely to get slightly different results despite querying them at almost the same time. To achieve consensus between nodes when using such sources, Gora can round queried values. A source that supports rounding will have "Round to digits" field when shown with gora sources --list command. Usually, the rounding setting will be parametrized, for example: "Round to digits: ##3". This means that the number of significant digits to round to is supplied in parameter with index 3. The number must be provided in string representation, like all parameters. Rounding will only affect the fractional part of the rounded number, all integer digits are always preserved. For example, if rounding parameter is set to "7", the number 123890.7251 will be rounded to 123890.7, but the number 98765430 will remain unaffected.

Example: generating an oracle request spec in Javascript

We start by building the request spec ABI type to encode our request. It can be accomplished in a single call, but will be done in steps here for clarity:

const Algosdk = require("algosdk");

const basicTypes = {
  sourceArgList: new Algosdk.ABIArrayDynamicType(Algosdk.ABIType.from("byte[]")),
  sourceId: Algosdk.ABIType.from("uint32"),
  maxAge: Algosdk.ABIType.from("uint32"),
  userData: Algosdk.ABIType.from("byte[]"),
  aggregation: Algosdk.ABIType.from("uint32"),
};

const sourceSpecType = new Algosdk.ABITupleType([
  basicTypes.sourceId,
  basicTypes.sourceArgList,
  basicTypes.maxAge
]);

const requestSpecType = new Algosdk.ABITupleType([
  new Algosdk.ABIArrayDynamicType(sourceSpecType),
  basicTypes.aggregation,
  basicTypes.userData
]);

Now we will use requestSpecType ABI type that we just created to encode a hypothetical Oracle request. We will query two sources for USD/EUR price pair and receive their average value. The data must be no more than an hour old in both cases. The sources are predefined in Gora with IDs 2 and 5, but one specifies currencies mnemonically while the other does it numerically:

const requestSpec = requestSpecType.encode([
  [
    [ 2, [ Buffer.from("usd"), Buffer.from("eur") ], 3600 ],
    [ 5, [ Buffer.from([ 12 ]), Buffer.from([ 44 ]) ], 3600 ],
  ],
  3, // average it
  Buffer.from("test") // let the receiving smart contract know it's a test
]);

Done. The requestSpec variable can now be used for spec argument when calling the request method for Gora main smart contract.

Decoding request responses

Results of an oracle request are returned by calling dest_method method of the smart contract specified in dest_id. The method gets passed the following two arguments:

• type: uint32 - response type; currently is always 1.

• body: byte[] - encoded body of the response (details below).

The body argument contains an ABI-encoded tuple of the following structure:

• byte[] - request ID. Currently the same as Algorand transaction ID of the request smart contract call that initiated the request.

• address - address of the account making the request

• byte[] - oracle return value, more details below

• byte[] - data specified in userData field of the request

• uint32 - result error code, see below

• uint64 - bit field with bits corresponding to the request sources; if n'th bit is set, the n'th source has failed to yield a valid value.

Result error codes

• 0 - normal result.

• 1 - result was truncated because it was over the allowed size. Result size limit is configured in Node Runner software and depends on maximum smart contract arguments size supported by Algorand.

Numeric oracle return values

When returned oracle value is a number, it is encoded into a 17-byte array. 0's byte encodes value type:

• 0 - empty value (not-a-number, NaN)

• 1 - positive number

• 2 - negative number

Bytes 1 - 8 contain the integer part, 9 - 17 - the decimal fraction part, as big endian uint64's.

For example, 0x021000000000000000ff00000000000000 in memory order (first byte has 0 offset) decodes as -16.255