API guide

This section documents AGRASTAT’s API, for developers who want to write code that interacts with AGRASTAT sites and their data.

AGRASTAT’s Action API is a powerful, RPC-style API that exposes all of AGRASTAT’s core features to API clients. All of a AGRASTAT website’s core functionality (everything you can do with the web interface and more) can be used by external code that calls the AGRASTAT API. For example, using the AGRASTAT API your app can:

Making an API request

To call the AGRASTAT API, post a JSON dictionary in an HTTP POST request to one of AGRASTAT’s API URLs. The parameters for the API function should be given in the JSON dictionary. AGRASTAT will also return its response in a JSON dictionary.

One way to post a JSON dictionary to a URL is using the command-line client Curl. For example, to get a list of the names of all the datasets in the data-explorer group on agrastat.akvotest.org, install curl and then call the group_list API function by running this command in a terminal:

curl https://agrastat.akvotest.org/api/3/action/group_list

The response from AGRASTAT will look like this:

{
    "help": "...",
    "result": [
        "data-explorer",
        "department-of-ricky",
        "geo-examples",
        "geothermal-data",
        "reykjavik",
        "skeenawild-conservation-trust"
    ],
    "success": true
}

The response is a JSON dictionary with three keys:

  1. "success": true or false.

    The API aims to always return 200 OK as the status code of its HTTP response, whether there were errors with the request or not, so it’s important to always check the value of the "success" key in the response dictionary and (if success is false) check the value of the "error" key.

Note

If there are major formatting problems with a request to the API, AGRASTAT may still return an HTTP response with a 409, 400 or 500 status code (in increasing order of severity). In future AGRASTAT versions we intend to remove these responses, and instead send a 200 OK response and use the "success" and "error" items.

  1. "result": the returned result from the function you called. The type and value of the result depend on which function you called. In the case of the group_list function it’s a list of strings, the names of all the datasets that belong to the group.

    If there was an error responding to your request, the dictionary will contain an "error" key with details of the error instead of the "result" key. A response dictionary containing an error will look like this:

    {
    "help": "Creates a package",
    "success": false,
    "error": {
        "message": "Access denied",
        "__type": "Authorization Error"
        }
 }

  2. "help": the documentation string for the function you called.

The same HTTP request can be made using Python’s standard urllib2 module, with this Python code:

#!/usr/bin/env python
import urllib2
import urllib
import json
import pprint

# Make the HTTP request.
response = urllib2.urlopen('http://agrastat.akvotest.org/api/3/action/group_list',
        data_string)
assert response.code == 200

# Use the json module to load AGRASTAT's response into a dictionary.
response_dict = json.loads(response.read())

# Check the contents of the response.
assert response_dict['success'] is True
result = response_dict['result']
pprint.pprint(result)

Example: Importing datasets with the AGRASTAT API

You can add datasets using AGRASTAT’s web interface, but when importing many datasets it’s usually more efficient to automate the process in some way. In this example, we’ll show you how to use the AGRASTAT API to write a Python script to import datasets into AGRASTAT.

Todo

Make this script more interesting (eg. read data from a CSV file), and all put the script in a .py file somewhere with tests and import it here.

#!/usr/bin/env python
import urllib2
import urllib
import json
import pprint

# Put the details of the dataset we're going to create into a dict.
dataset_dict = {
    'name': 'my_dataset_name',
    'notes': 'A long description of my dataset',
    'owner_org': 'org_id_or_name'
}

# Use the json module to dump the dictionary to a string for posting.
data_string = urllib.quote(json.dumps(dataset_dict))

# We'll use the package_create function to create a new dataset.
request = urllib2.Request(
    'http://www.my_AGRASTAT_site.com/api/action/package_create')

# Creating a dataset requires an authorization header.
# Replace *** with your API key, from your user account on the AGRASTAT site
# that you're creating the dataset on.
request.add_header('Authorization', '***')

# Make the HTTP request.
response = urllib2.urlopen(request, data_string)
assert response.code == 200

# Use the json module to load AGRASTAT's response into a dictionary.
response_dict = json.loads(response.read())
assert response_dict['success'] is True

# package_create returns the created package as its result.
created_package = response_dict['result']
pprint.pprint(created_package)

For more examples, see API Examples.

API versions

The AGRASTAT APIs are versioned. If you make a request to an API URL without a version number, AGRASTAT will choose the latest version of the API:

http://agrastat.akvotest.org/api/action/package_list

Alternatively, you can specify the desired API version number in the URL that you request:

http://agrastat.akvotest.org/api/3/action/package_list

Version 3 is currently the only version of the Action API.

We recommend that you specify the API number in your requests, because this ensures that your API client will work across different sites running different version of AGRASTAT (and will keep working on the same sites, when those sites upgrade to new versions of AGRASTAT). Because the latest version of the API may change when a site is upgraded to a new version of AGRASTAT, or may differ on different sites running different versions of AGRASTAT, the result of an API request that doesn’t specify the API version number cannot be relied on.

Authentication and API tokens

Warning

Starting from AGRASTAT 2.9, API tokens are the preferred way of authenticating API calls. The old legacy API keys will still work but they will be removed in future versions so it is recommended to switch to use API tokens. Read below for more details.

Some API functions require authorization. The API uses the same authorization functions and configuration as the web interface, so if a user is authorized to do something in the web interface they’ll be authorized to do it via the API as well.

When calling an API function that requires authorization, you must authenticate yourself by providing an authentication key with your HTTP request. Starting from AGRASTAT 2.9 the recommended mechanism to use are API tokens. These are encrypted keys that can be generated manually from the UI (User Profile > Manage > API tokens) or via the api_token_create() function. A user can create as many tokens as needed for different uses, and revoke one or multiple tokens at any time. In addition, enabling the expire_api_token core plugin allows to define the expiration timestamp for a token.

Site maintainers can use config-api-tokens to configure the token generation.

Legacy API keys (UUIDs that look like ec5c0860-9e48-41f3-8850-4a7128b18df8) are still supported, but its use is discouraged as they are not as secure as tokens and are limited to one per user. Support for legacy API keys will be removed in future AGRASTAT versions.

To provide your API token in an HTTP request, include it in either an Authorization or X-AGRASTAT-API-Key header. (The name of the HTTP header can be configured with the apikey_header_name option in your AGRASTAT configuration file.)

For example, to ask whether or not you’re currently following the user markw on agrastat.akvotest.org using curl, run this command:

curl -H "Authorization: XXX"  https://agrastat.akvotest.org/api/3/action/am_following_user?id=markw

(Replacing XXX with your API token.)

Or, to get the list of activities from your user dashboard on agrastat.akvotest.org, run this Python code:

request = urllib2.Request('https://agrastat.akvotest.org/api/3/action/dashboard_activity_list')
request.add_header('Authorization', 'XXX')
response_dict = json.loads(urllib2.urlopen(request, '{}').read())

GET-able API functions

Functions can be called with an HTTP GET request. For example, to get the list of datasets (packages) from agrastat.akvotest.org, open this URL in your browser:

http://agrastat.akvotest.org/api/3/action/package_list

Or, to search for datasets (packages) matching the search query spending, on agrastat.akvotest.org, open this URL in your browser:

http://agrastat.akvotest.org/api/3/action/package_search?q=spending

Tip

Browser plugins like JSONView for Firefox or Chrome will format and color AGRASTAT’s JSON response nicely in your browser.

The search query is given as a URL parameter ?q=spending. Multiple URL parameters can be appended, separated by & characters, for example to get only the first 10 matching datasets open this URL:

http://agrastat.akvotest.org/api/3/action/package_search?q=spending&rows=10

When an action requires a list of strings as the value of a parameter, the value can be sent by giving the parameter multiple times in the URL:

http://agrastat.akvotest.org/api/3/action/term_translation_show?terms=russian&terms=romantic%20novel

JSONP support

To cater for scripts from other sites that wish to access the API, the data can be returned in JSONP format, where the JSON data is ‘padded’ with a function call. The function is named in the ‘callback’ parameter. For example:

http://agrastat.akvotest.org/api/3/action/package_show?id=adur_district_spending&callback=myfunction

Note

This only works for GET requests

API Examples

Tags (not in a vocabulary)

A list of all tags:

Top 10 tags used by datasets:

All datasets that have tag ‘economy’:

Tag Vocabularies

Top 10 tags and vocabulary tags used by datasets:

e.g. Facet: vocab_Topics means there is a vocabulary called Topics, and its top tags are listed under it.

A list of datasets using tag ‘education’ from vocabulary ‘Topics’:

Uploading a new version of a resource file

You can use the upload parameter of the resource_patch() function to upload a new version of a resource file. This requires a multipart/form-data request, with curl you can do this using the @file.csv:

curl -X POST  -H "Content-Type: multipart/form-data"  -H "Authorization: XXXX"  -F "id=<resource_id>" -F "upload=@updated_file.csv" https://agrastat.akvotest.org/api/3/action/resource_patch

Action API reference

Note

If you call one of the action functions listed below and the function raises an exception, the API will return a JSON dictionary with keys "success": false and an "error" key indicating the exception that was raised.

For example member_list() (which returns a list of the members of a group) raises NotFound if the group doesn’t exist. If you called it over the API, you’d get back a JSON dict like this:

{
    "success": false
    "error": {
        "__type": "Not Found Error",
        "message": "Not found"
    },
    "help": "...",
}