The Macrobond client Web API Series Provider

Introduction

The Macrobond client Web API Series Provider series provider allows the Macrobond application to use any web service that implements the HTTP and JSON based web API as described below. It offers similar features to the SQL Database Connector, but uses a predefined API contract instead of SQL to retrieve the data. It also offers the option of enabling creation and deletion of series as well-structured lists of series in the data tree with indentations, emphasized rows and groups of series.

Technical details

The API uses JSON in responses and POST requests.

In POST requests, you should set Content-Type: application/json; charset=utf-8.

In all responses you should set the Cache-Control header. In most cases you want to set this to Cache-Control: no-cache, private.

Sample server

There is a Sample server implemented in C#/.NET Core at GitHub.

Implementation

The API must implement the loadseries and getcapabilities methods. The other methods are optional and will only be called if the server reports that it supports those features via the getcapabilities method.

Method Capability Comment
loadseries Load one or more series given the names of the series
getcapabilties Return a list of capabilities implemented by the server
searchseries search Search for series using keywords entered by the user
loadtree browse Return a tree structure that the user can browse to navigate the database
listseries browse List series in a location of the database tree
createseries editSeries Create or update a series
removeseries editSeries Remove a series
loadmeta meta Load just the metadata of a series (MB app 1.27)
loadvintage revisions Load a vintage series (MB app 1.27)
loadvintagetimestamps revisions Load the vintages timestamps of a series (MB app 1.27)
loadrelease revisions+revisionsRelease Load a release series (MB app 1.27)
loadcompletehistory revisions+revisionsCompleteHistory Loads all vintages of a series (MB app 1.27)

For communication, both http and https are supported. For authentication, basic http authentication is supported. In this case, https is strongly recommended.

There are some requirements on the database imposed by the Macrobond application:

  • Each time series must have a unique identifier that can be used for retrieving the time series values and metadata. Any lower-case characters except double quote, colon, and any other white space than ordinary space (ASCII 32) is allowed, but it is best if the identifier begins with a-z and continues with a-z, 0-9 or underscore since such identifiers can be used unquoted in the Macrobond application.
  • A time series cannot start before year 1600 or end after year 2499.
  • The time series frequencies are limited to: day, week, month, quarter, 4 months, half year and year.

There is a swagger file for the API here: https://schemas.macrobond.com/webclientapi.swagger.json

Working with the Web API Series Provider

Configuration

To define what database server to talk to, you define a simple XML file like below:

<?xml version="1.0" encoding="utf-8"?>
<web:WebApiSeriesProvider xmlns:web="http://schemas.mbnd.eu/2019/webapiseriesprovider">
  <web:Host>http://ourdbserver:5555</web:Host>
</web:WebApiSeriesProvider>

In the Macrobond application, in the Settings dialog (Configuration > Settings > My series (for MB pre-1.28: Edit > Settings > My series)) you specify the path to this configuration file:See here pre-1.27 version view

 

See here pre-1.27 version view

 

During development and troubleshooting, it can be helpful to enable the log console.

See here pre-1.27 version view

 

This will cause a log window to be displayed as soon as there is any communication via the Web API. The log contains information about what queries that are sent, what the response is and some diagnostic messages that can be helpful for troubleshooting the service.

Metadata

Each series in the server must contain meta data in the form of a dictionary of string+object. The meta data gives different information about how the series should be handled by the application. Every property in a Meta data dictionary is case sensitive, and they should be written with pascal case. Some meta data properties are optional but there are some that are crucial to make a series work.

The property PrimName is required as a unique identifier of the series. Letters of the identifier must be lower case.

It is recommended to specify the frequency using the Frequency attribute. This can have one of the values annual, semiannual, quadmonthly, quarterly, bimonthly, monthly, weekly or daily.

The StartDate is required when no list of dates is provided for the series. In most cases you also want to include a Description attribute too.

The Class attribute influences how automatic frequency conversion is done and is useful to set. Here are a list of possible values: Class attribute info

The Region metadata specifies what region a data series refers to. For countries, the values is the ISO 3166 code of the region. See this list for possible values: Region attribute info

The Currencymetadata attribute is the ISO 4217 currency code. This is used for automatic currency conversion in the Macrobond application. See this list for possible values: Currency attribute info

The IHInfofree-text attributes allows for providing verbose description of the time series. It can be accessed from within the Macrobond application in the Time series information view.

For more information about metadata see Commonly used metadata.

Commands

getcapabilities

Tells the Macrobond application whether this server support browse, search and edit capabilities.

This method must be implemented by the server.

Response

One or more of these boolean values:

  • browse
  • search
  • editSeries
  • allowMultipleSeriesPerRequest
  • meta
  • revisions
  • revisionsRelease
  • revisionsCompleteHistory

If browse is false or not present, no tree will be shown in the application. This should be true if loadtree and listseries have been implemented.

If search is false or not present, no search bar will be available in the application. This should be true if searchseries has been implemented.

If editSeries is false or not present, the option to create or remove series will not be available to the user. Set to true if the createseries and removeseries have been implemented.

If allowMultipleSeriesPerRequest is set to false, then there will never be more than one series requested in loadseries. This is less efficient, but can simplify the implementation in some scenarios. By default, this is set to true. This setting was introduced in Macrobond application 1.26.

If meta is set to true, then loadmeta might be called by the application to retrieve the just the metadata of a series in some situations. This is useful since there is less data to return than the complete series data. This setting was introduced in Macrobond application 1.27.

If revisions is set to true, the provider must implement loadvintage and loadvintagedates. Series that has revisions are expected to include the metadata StoresRevisionHistory, FirstRevisionTimeStamp and LastRevisionTimeStamp. The last two are set only if there actually are any revisions for the series. This enables series vintages in the application. This setting was introduced in Macrobond application 1.27.

If revisions and revisionsRelease are  true, the provider must implement loadrelease. This enables series releases in the application. This setting was introduced in Macrobond application 1.27.

If revisions and revisionsCompleteHistoryare  true, the provider must implement completehistory. This enables the "Show revision history" feature in the application. This setting was introduced in Macrobond application 1.27.

Example

URL:

/getcapabilities

Response:

{
"browse": true,
"search": true
}

loadseries

Returns a list of series from the server by name.

This method must be implemented by the server.

Parameters

  • n : Array of strings. The identifier of a series. The parameter can be included several times in order to request several series simultaneously. (It is possible to turn off the option to send multiple series in one request. See allowMultipleSeriesPerRequest in getcapabilities above.)

Response

A list of responses corresponding to each series requested in the same order as in the request. Each element in the list is either a data element with

  • values
  • metaData

or an error element with an error text explaining why the series could not be loaded. You must not include both data and error elements for a series.

There are three options for specifying the calendar:

  • Specify the frequency without list of dates. For daily frequencies you may also specify the DayMask attribute if you do not want the default of Monday-Friday. In this case the calendar will be created from Frequency, StartDate and DayMask.
  • Specify the frequency attribute and a list of dates. For daily frequencies you may also specify the DayMask attribute if you do not want the default of Monday-Friday. In this case the calendar will be created from Frequency, StartDate and DayMask and values will be made available for the periods corresponding to the specified dates. Any 'gaps' in the set of dates will be gaps in the resulting calendar. The StartDate metadata must reference the first observation.
  • Include a list of dates, but do not specify the frequency. The application will guess the frequency based on the provided set of observations. It will pick the lowest frequency to cover the specified dates without duplicates. The StartDate metadata must reference the first observation. This option is not allowed if if meta is returned from getcapabilities.

The DayMask metadata is used for daily series. It is a bitmask as an integer where Sunday is the bit with value 1. A mask for Monday-Friday is thus 62. A string can be passed instead of a bitmask and should then have the weekdays in English separated with space. For instance Monday Tuesday Wednesday Thursday Friday. Such a string will be converted to the corresponding bitmask.

Example

URL:

/loadseries?n=name1&n=name2&n=name3

Response:

[
    {
        "data": {
            "metaData": {
                "Description": "0-7 Years",
                "Frequency": "annual",
                "PrimName": "name1",
                "StartDate": "1990-01-01T00:00:00"
            },
            "values": [
                600.0,
                570.0,
                630.0,
                450.0,
                200.0
            ]
        }
    },
    {
        "data": {
            "dates": [
                "1990-01-01T00:00:00",
                "1990-02-01T00:00:00",
                "1990-04-01T00:00:00",
                "1990-05-01T00:00:00"
            ],
            "metaData": {
                "Description": "8-12 Years",
                "Frequency": "monthly",
                "PrimName": "name2"
            },
            "values": [
                10,
                11,
                12,
                13
            ]
        }
    },
    {
        "error": "Series name3 not found"
    }
]

loadmeta

Returns a list of series metadata from the server by name.

This method is optional to implemented by the server. It is called only if meta is returned from getcapabilities. The application will call this method instead of loadseries in some cases when it needs only the metadata and not the full series data. This can be useful if it is a lot of work for the server to retrieve the full series data.

Parameters

  • n : Array of strings. The identifier of a series. The parameter can be included several times in order to request several series simultaneously. (It is possible to turn off the option to send multiple series in one request. See allowMultipleSeriesPerRequest in getcapabilities above.)

Response

A list of responses corresponding to each series requested in the same order as in the request. Each element in the list is either a data element with

  • metaData

or an error element with an error text explaining why the series could not be loaded. You must not include both data and error elements for a series.

One difference compared to loadseries is that the metadata fields Frequency and StartDate must be specified. For daily series, it is recommended to also set DayMask. If not set Monday-Friday will be assumed.

Example

URL:

/loadmeta?n=name1&n=name2&n=name3

Response:

[
    {
        "data": {
            "metaData": {
                "Description": "0-7 Years",
                "Frequency": "annual",
                "PrimName": "name1",
                "StartDate": "1990-01-01T00:00:00"
            }
        }
    },
    {
        "data": {
            "metaData": {
                "Description": "8-12 Years",
                "Frequency": "monthly",
                "PrimName": "name2",
                "StartDate": "1997-01-01T00:00:00"
            }
        }
    },
    {
        "error": "Series name3 not found"
    }
]

searchseries

Gets a list of meta data of series from given string.

Parameters

  • query: The text entered by the user

Response

The response is a list of metadata sets. The metadata must contain Description and PrimName.

The Macrobond application can show some additional attributes as columns in the list of a search result. These are optional. Frequency, StartDate, EndDate, Currency, DisplayUnit, Region, LastValue, PreviousLastValue, Class

Example

URL:

/searchseries?query=US

Response:

[
    {
        "Description": "US Main Regressions",
        "Frequency": "annual",
        "PrimName": "series1"
    },
    {
        "Description": "US Main Regressions",
        "PrimName": "series2"
    }
]

loadtree

Returns a tree or subtree from the database. If the tree is large or costly to load from the database, it can be split up into sub trees that will be loaded when the user expands them.

Parameter

  • reference: Optional. If specified, load the referenced sub tree. If not specified, load the root of the tree.

Response

The returned data forms a nested tree which is used to create a database tree in the application UI.

Each node in the tree can contain either Children, ChildrenReference or SeriesReference.

A Children element contains a nested list of the tree.

A ChildrenReference contains a deferred nested list of the tree. A request for the branch will be sent as a loadtree request to the server and the specified reference will be included in the query.

The leaves of the tree contain an element called SeriesReference. When the user clicks on such a branch in the tree a listseries request, including the specified reference, is sent to the server to retrieve the list of series to present to the user.

Example

URLs:

/loadtree
/loadtree?reference=treeNodeReference

Response:

[
    {
        "Description": "First folder",
        "SeriesReference": "ref1"
    },
    {
        "Children": [
            {
                "Description": "First sub folder",
                "SeriesReference": "ref2"
            },
            {
                "ChildrenReference": "sub2",
                "Description": "Second sub folder"
            }
        ],
        "Description": "Second folder"
    }
]

listseries

Lists all series for a leaf in the data tree.

The application will call this method when the user selects a leaf in the data tree.

Parameters

  • reference: The reference from the leaf in the data tree.

Response

The following elements are required:

  • groups
  • entityMeta
  • names

The list of series will be presented in the Macrobond application when a leaf in the data tree is selected.

Aspects

The result can optionally contain 'aspects' will be presented as tabs in the UI. For example 'NSA, Monthly' in the image above.

The presented list below the aspect tabs is the same for all tabs but refers to different series.

If there is only one aspect, then aspects can omitted.

For every aspect there must be as many metadata collections in each row of series. If there are two aspects then every row of series must contain two collections, if there are three aspects then every row must contain three collections etc.

If there are no aspects, the entityMeta element contains only one collection.

The response is a list of metadata sets. The metadata must contain Description and PrimName.

The Macrobond application can show some additional attributes as columns in the list of a search result. These are optional. FrequencyStartDateEndDateCurrencyDisplayUnitRegionLastValuePreviousLastValueClass

Groups

The list may be divided into groups. If there is only one group, then the name property of that group can be omitted.

Formatting

The entries in the list can be indented by specifying an indentation level.

Entries can be emphasized like 'Food & Beverages' in the image above.

Additional space can be added above an entry, like for 'Food' in the image, but specifying spaceAbove as true.

Example

URL:

/listseries?reference=serieslistreference

Response:

{
    "aspects": [
        {
            "description": "Tooltip of tab a",
            "name": "Tab 1"
        },
        {
            "description": "Tooltip of tab b",
            "name": "Tab 2"
        }
    ],
    "groups": [
        {
            "name": "The first group",
            "series": [
                {
                    "description": "Variable 1",
                    "emphasized": false,
                    "entityMeta": [
                        {
                            "Description": "Total, Goods",
                            "PrimName": "series1a",
                            "Region": "se"
                        },
                        {
                            "Description": "Total, Goods",
                            "PrimName": "series1b",
                            "Region": "se"
                        }
                    ],
                    "indentation": 3,
                    "spaceAbove": false
                }
            ]
        }
    ]
}

createseries

Creates a new in-house series.

Parameters

  • lastModified : Timestamp. When replacing an existing series, this value reflects the LastModifiedTimeStamp metadata, if present, of the series edited by the user. It will not be included when creating a new series. It is included when overwriting an existing series unless the forceReplace parameter is true.
  • forceReplace : Boolean. If this parameter is included and set to true, then the existing series should be overwritten. If the series does not already exist, HTTP status 404 Not Found should be returned.
Response

Success (200): Must return new lastModified time for the series.

Not Found (404): If lastModified has a value or forceReplace is true but the series could not be found.

Conflict (409): If the lastModified parameter does not match the actual timestamp of the stored series. This means that someone else might be editing the same series.

Example

URL:

/createseries

Post data:

{
    "dates": [
        "2016-01-01T00:00:00",
        "2017-01-01T00:00:00",
        "2018-01-01T00:00:00",
        "2019-01-01T00:00:00"
    ],
    "metaData": {
        "Description": "Total",
        "Frequency": "monthly",
        "LastModifiedTimeStamp": "2019-12-04T00:00:00",
        "PrimName": "myseries1",
        "StartDate": "2016-01-01T00:00:00"
    },
    "values": [
        "767161, 782911, 866119, 929586"
    ]
}

Response:

2019-12-04T00:00:00

removeseries

Removes a series.

Parameters

  • name : The name of the series to be removed

Response

Success (200): The series was removed.

Not Found (404): Series with given name parameter could not be found.

Example

URL:

/removeseries?name=myseries1

loadvintage

Load a specific vintage of a series as it looked at the specified time.

This method can be used by Macrobond application 1.27 and later.

The returned series is expected to have the metadata RevisionSeriesType=vintage and RevisionTimeStamp set to the time of the vintage, if it is known. RevisionLabel can optionally be set if there is a descriptive text of the vintage.

Please note that the actual vintage of the series, as specified in the RevisionTimeStamp metadata should be at or before the requested time.

When a time before the first known vintage is requested, there are two scenarios:

  1. For some series, the complete revision history might not be known and there is a version of the series as it looked before revisions were collected. In this case this original series is returned without any RevisionTimeStamp metadata.
  2. If all revision of the series is known, an empty series is returned without the RevisionTimeStamp metadata.

Parameters

  • n: The name of the series
  • timestamp: The time of the vintage. This will be in a ISO 8601 format with a time zone offset or Z for UTC.

Response

Success (200): The vintage series is returned. This is the same format as the data element in the return from the loadseries.

Not Found (404): Series with given name could not be found.

Example

URL:

/loadvintage?n=myseries1&timestamp=2022-06-27T14%3A00%3A00Z

Response:

{
    "dates": [
        "2022-01-01T00:00:00",
        "2022-02-01T00:00:00",
        "2022-04-01T00:00:00",
        "2022-05-01T00:00:00"
    ],
    "metaData": {
        "Description": "8-12 Years",
        "Frequency": "monthly",
        "PrimName": "myseries1",
        "RevisionSeriesType": "vintage",
        "RevisionTimeStamp": "2022-05-15T10%3A00%3A00Z"
    },
    "values": [
        10,
        11,
        12,
        13
    ]
}

loadvintagetimestamps

Load the vintage timestamps available for a series. This is used by the application to populate lists in the GUI where the user can pick a vintage.

This method can be used by Macrobond application 1.27 and later.

Parameters

  • n: The name of the series

Response

Success (200): The list of timestamps + optional label name is returned.

Not Found (404): Series with given name could not be found.

Example

URL:

/loadvintagetimestamps?n=myseries1

Response:

[
  {"timeStamp": "2022-05-13T10%3A00%3A00Z", "label": "May 2022 first release" },
  {"timeStamp": "2022-05-15T10%3A00%3A00Z" }
]

loadrelease

Load a specific release of a series. Typically release 0 corresponds to the first value of each observation, release 1 is the first revision of each value and so on.

This method can be used by Macrobond application 1.27 and later.

The returned series is expected to have the metadata RevisionSeriesType=nth and RevisionSeriesNth set to the ordinal of the revision. RevisionLabel can optionally be set if there is a descriptive text of the release.

Parameters

  • n: The name of the series
  • nthrelease: The ordinal of the release. It can be 0 or greater.

Response

Success (200): The series is returned. This is the same format as the data element in the return from the loadseries.

Not Found (404): Series with given name could not be found.

Example

URL:

/loadrelease?n=myseries1&nthrelease=1

Response:

{
  "values": [
    10, 11, 12, 13],
  "dates": [
    "2022-01-01T00:00:00", "2022-02-01T00:00:00", "2022-04-01T00:00:00", "2022-05-01T00:00:00"],
  "metaData": {
    "Description": "8-12 Years",
    "PrimName": "myseries1",
    "Frequency": "monthly",
    "RevisionSeriesType": "nth",
    "RevisionSeriesNth": 1
    }
}

loadcompletehistory

Load all vintages of a series. This is used by the Macrobond application to show a complete view of all historical vintages of a series.

This method can be used by Macrobond application 1.27 and later.

The returned series are expected to have the metadata RevisionSeriesType=vintage and RevisionTimeStamp set to the time of the vintage. The first vintage may omit the RevisionTimeStamp if it is not known. RevisionLabel can optionally be set if there is a descriptive text of the vintage.

Parameters

  • n: The name of the series

Response

Success (200): The complete history is returned.

Not Found (404): Series with given name could not be found.

Example

URL:

/loadcompletehistory?n=myseries1

Response:

[
    {
        "dates": [
            "2022-01-01T00:00:00",
            "2022-02-01T00:00:00",
            "2022-04-01T00:00:00"
        ],
        "metaData": {
            "Description": "8-12 Years",
            "Frequency": "monthly",
            "PrimName": "myseries1",
            "RevisionSeriesType": "vintage"
        },
        "values": [
            10,
            11,
            12
        ]
    },
    {
        "dates": [
            "2022-01-01T00:00:00",
            "2022-02-01T00:00:00",
            "2022-04-01T00:00:00",
            "2022-05-01T00:00:00"
        ],
        "metaData": {
            "Description": "8-12 Years",
            "Frequency": "monthly",
            "PrimName": "myseries1",
            "RevisionLabel": "May 2022 first release",
            "RevisionSeriesType": "vintage",
            "RevisionTimeStamp": "2022-05-13T10%3A00%3A00Z"
        },
        "values": [
            10,
            11,
            12,
            13.5
        ]
    },
    {
        "dates": [
            "2022-01-01T00:00:00",
            "2022-02-01T00:00:00",
            "2022-04-01T00:00:00",
            "2022-05-01T00:00:00"
        ],
        "metaData": {
            "Description": "8-12 Years",
            "Frequency": "monthly",
            "PrimName": "myseries1",
            "RevisionSeriesType": "vintage",
            "RevisionTimeStamp": "2022-05-15T10%3A00%3A00Z"
        },
        "values": [
            10,
            11,
            12,
            13.7
        ]
    }
]