JSON API Documentation

If you make anything utilizing our API, please let us know about it - we would love to see!

Except where noted otherwise, all queries require the following header to receive a JSON response:

accept: application/json

Workflow

Add a new Workflow

POST

/workflows

Parameters

Field	Type	Description
url	String	The URL of the repository. Either a gitlab.com or github.com URL, or any Git repository URL. If a gitlab.com or github.com repository URL is used, the following fields will take priority over those parsed from the URL if provided.
branch	String	The branch of the repository. Required if the URL is not from gitlab.com or github.com.
path	String	The path within the repository to the workflow file. Required if the URL is not from gitlab.com or github.com.
packedId	String	The ID of the workflow, to be provided if the workflow file is packed (contains multiple descriptions with $graph).

Input

{
    "url": "https://github.com/common-workflow-language/workflows/tree/master/workflows/compile/compile1.cwl"
}

{
    "url": "https://bitbucket.org/markrobinson96/workflows.git",
    "branch": "master",
    "path": "/workflows/make-to-cwl/dna.cwl"
}

Success

HTTP/1.1 202 Accepted
Location: /queue/:queueid

Already Exists

HTTP/1.1 303 See Other
Location: /workflows/:URL

Packed Workflow with Multiple Choices

HTTP/1.1 422 Unprocessable Entity
{
    "message": "This workflow file is packed and contains multiple workflow descriptions. Please provide a packedId parameter with one of the following values"
    "packedId": [
        "workflow.cwl",
        "main",
        "workflow_exome.cwl",
        "align.cwl",
        "workflow.cwl_2"
    ]
}

Error

HTTP/1.1 400 Bad Request

Get Workflow Details

GET

/workflows/:url

Parameters

Field	Type	Description
url	String	Partial URL to the workflow eg "github.com/owner/repo/tree/branch/workflow.cwl" or "bitbucket.org/owner/repo.git/master/workflow.cwl

Success 200

Field	Type	Description
retrievedFrom.repoUrl	String	URL of the Github repository
retrievedFrom.branch	String	Github branch name or commit ID
retrievedFrom.path	String	Path to the workflow within the repository
retrievedFrom.packedId	String	The ID of the workflow within the file, if this is a packed CWL file (contains a document graph).
retrievedFrom.url	URL	The URL to the page on the site if type is not GENERIC, otherwise the same as repoUrl
retrievedFrom.rawUrl	URL	The URL directly to the file if type is not GENERIC, otherwise the same as repoUrl
retrievedFrom.type	Enum	One of GENERIC, GITHUB, GITLAB, BITBUCKET
retrievedOn	Date	The last time this workflow was updated from the branch
lastCommit	String	The last commit ID on the branch
label	String	The label string for the workflow from the CWL description
doc	String	The doc string for the workflow from the CWL description
inputs
outputs
steps
dockerLink	String	The URL to dockerhub or "true" if an unrecognised image is used in a docker hint or requirement
cwltoolVersion	String	The version of cwltool used to parse the workflow
visualisationDot	String	The DOT source used to generate the visualisation image
visualisationXdot	URL	A link to the xdot source code for the visualisation image
visualisationPng	URL	A link to the png format visualisation image
visualisationSvg	URL	A link to the svg format visualisation image
robundle	URL	A link to the Research Object Bundle download

Example URL

/workflows/github.com/common-workflow-language/workflows/tree/master/workflows/lobSTR/lobSTR-workflow.cwl

Example Response

{
    "retrievedFrom": {
        "repoUrl": "https://github.com/common-workflow-language/workflows.git",
        "branch": "master",
        "path": "workflows/lobSTR/lobSTR-workflow.cwl",
        "url": "https://github.com/common-workflow-language/workflows/blob/master/workflows/lobSTR/lobSTR-workflow.cwl",
        "rawUrl": "https://raw.githubusercontent.com/common-workflow-language/workflows/master/workflows/lobSTR/lobSTR-workflow.cwl",
        "type": "GITHUB"
    },
    "retrievedOn": 1502714218616,
    "lastCommit": "17b65ea19d81527090fded62ffa0e1ba3b25d561",
    "label": "lobSTR-workflow.cwl",
    "inputs": {
        "reference": {
            "type": "File"
        },
        "rg-sample": {
            "type": "String"
        },
        "p1": {
            "type": "ee54066e73d068edfbf9ca5059e14acd"
        },
        "p2": {
            "type": "79ab371b6d0fdbadbf9a4a89ef393353"
        },
        "output_prefix": {
            "type": "String"
        },
        "rg-lib": {
            "type": "String"
        },
        "strinfo": {
            "type": "File"
        },
        "noise_model": {
            "type": "File"
        }
    },
    "outputs": {
        "vcf": {
            "type": "File"
        },
        "vcf_stats": {
            "type": "File"
        },
        "bam_stats": {
            "type": "File"
        },
        "bam": {
            "type": "File"
        }
    },
    "steps": {
        "allelotype": {
            "run": "allelotype.cwl",
            "runType": "COMMANDLINETOOL",
            "sources": {
                "file:///data/git/3ec0d19076841a1140e71cc3a1b330a56099aac2/workflows/lobSTR/lobSTR-workflow.cwl#allelotype/bam": {
                    "sourceIDs": [
                        "samindex"
                    ]
                },
                "allelotype": {
                    "sourceIDs": [
                        "strinfo"
                    ]
                },
                "file:///data/git/3ec0d19076841a1140e71cc3a1b330a56099aac2/workflows/lobSTR/lobSTR-workflow.cwl#allelotype/noise_model": {
                    "sourceIDs": [
                        "noise_model"
                    ]
                },
                "file:///data/git/3ec0d19076841a1140e71cc3a1b330a56099aac2/workflows/lobSTR/lobSTR-workflow.cwl#allelotype/reference": {
                    "sourceIDs": [
                        "reference"
                    ]
                },
                "file:///data/git/3ec0d19076841a1140e71cc3a1b330a56099aac2/workflows/lobSTR/lobSTR-workflow.cwl#allelotype/output_prefix": {
                    "sourceIDs": [
                        "output_prefix"
                    ]
                }
            }
        },
        "samsort": {
            "run": "samtools-sort.cwl",
            "runType": "COMMANDLINETOOL",
            "sources": {
                "samsort": {
                    "sourceIDs": [
                        "lobSTR"
                    ]
                },
                "file:///data/git/3ec0d19076841a1140e71cc3a1b330a56099aac2/workflows/lobSTR/lobSTR-workflow.cwl#samsort/output_name": {
                    "defaultVal": "\\\"aligned.sorted.bam\\\""
                }
            }
        },
        "lobSTR": {
            "label": "lobSTR",
            "run": "lobSTR-tool.cwl",
            "runType": "COMMANDLINETOOL",
            "sources": {
                "lobSTR": {
                    "sourceIDs": [
                        "rg-lib"
                    ]
                },
                "file:///data/git/3ec0d19076841a1140e71cc3a1b330a56099aac2/workflows/lobSTR/lobSTR-workflow.cwl#lobSTR/p1": {
                    "sourceIDs": [
                        "p1"
                    ]
                },
                "file:///data/git/3ec0d19076841a1140e71cc3a1b330a56099aac2/workflows/lobSTR/lobSTR-workflow.cwl#lobSTR/p2": {
                    "sourceIDs": [
                        "p2"
                    ]
                },
                "file:///data/git/3ec0d19076841a1140e71cc3a1b330a56099aac2/workflows/lobSTR/lobSTR-workflow.cwl#lobSTR/output_prefix": {
                    "sourceIDs": [
                        "output_prefix"
                    ]
                },
                "file:///data/git/3ec0d19076841a1140e71cc3a1b330a56099aac2/workflows/lobSTR/lobSTR-workflow.cwl#lobSTR/rg-sample": {
                    "sourceIDs": [
                        "rg-sample"
                    ]
                },
                "file:///data/git/3ec0d19076841a1140e71cc3a1b330a56099aac2/workflows/lobSTR/lobSTR-workflow.cwl#lobSTR/reference": {
                    "sourceIDs": [
                        "reference"
                    ]
                }
            }
        },
        "samindex": {
            "run": "samtools-index.cwl",
            "runType": "COMMANDLINETOOL",
            "sources": {
                "samindex": {
                    "sourceIDs": [
                        "samsort"
                    ]
                }
            }
        }
    },
    "dockerLink": "https://hub.docker.com/r/rabix/lobstr",
    "cwltoolVersion": "1.0.20170810192106",
    "visualisationDot": "digraph workflow {\n  graph [\n    bgcolor = \"#eeeeee\"\n    color = \"black\"\n    fontsize = \"10\"\n    labeljust = \"left\"\n    clusterrank = \"local\"\n    ranksep = \"0.22\"\n    nodesep = \"0.05\"\n  ]\n  node [\n    fontname = \"Helvetica\"\n    fontsize = \"10\"\n    fontcolor = \"black\"\n    shape = \"record\"\n    height = \"0\"\n    width = \"0\"\n    color = \"black\"\n    fillcolor = \"lightgoldenrodyellow\"\n    style = \"filled\"\n  ];\n  edge [\n    fontname=\"Helvetica\"\n    fontsize=\"8\"\n    fontcolor=\"black\"\n    color=\"black\"\n    arrowsize=\"0.7\"\n  ];\n  subgraph cluster_inputs {\n    rank = \"same\";\n    style = \"dashed\";\n    label = \"Workflow Inputs\";\n    \"p2\" [fillcolor=\"#94DDF4\",label=\"p2\"];\n    \"p2\" [fillcolor=\"#94DDF4\",label=\"p2\"];\n    \"output_prefix\" [fillcolor=\"#94DDF4\",label=\"output_prefix\"];\n    \"reference\" [fillcolor=\"#94DDF4\",label=\"reference\"];\n    \"noise_model\" [fillcolor=\"#94DDF4\",label=\"noise_model\"];\n    \"strinfo\" [fillcolor=\"#94DDF4\",label=\"strinfo\"];\n    \"p1\" [fillcolor=\"#94DDF4\",label=\"p1\"];\n    \"p1\" [fillcolor=\"#94DDF4\",label=\"p1\"];\n    \"rg-lib\" [fillcolor=\"#94DDF4\",label=\"rg-lib\"];\n    \"rg-sample\" [fillcolor=\"#94DDF4\",label=\"rg-sample\"];\n  }\n  subgraph cluster_outputs {\n    rank = \"same\";\n    style = \"dashed\";\n    labelloc = \"b\";\n    label = \"Workflow Outputs\";\n    \"bam\" [fillcolor=\"#94DDF4\",label=\"bam\"];\n    \"vcf\" [fillcolor=\"#94DDF4\",label=\"vcf\"];\n    \"bam_stats\" [fillcolor=\"#94DDF4\",label=\"bam_stats\"];\n    \"vcf_stats\" [fillcolor=\"#94DDF4\",label=\"vcf_stats\"];\n  }\n  \"allelotype\" [label=\"allelotype\"];\n  \"samsort\" [label=\"samsort\"];\n  \"lobSTR\" [label=\"lobSTR\"];\n  \"samindex\" [label=\"samindex\"];\n  \"strinfo\" -> \"allelotype\" [label=\"strinfo\"];\n  \"reference\" -> \"allelotype\" [label=\"reference\"];\n  \"output_prefix\" -> \"allelotype\" [label=\"output_prefix\"];\n  \"noise_model\" -> \"allelotype\" [label=\"noise_model\"];\n  \"samindex\" -> \"allelotype\" [label=\"bam\"];\n  \"lobSTR\" -> \"samsort\" [label=\"input\"];\n  \"default1\" -> \"samsort\" [label=\"output_name\"];\n  \"default1\" [label=\"\\\"aligned.sorted.bam\\\"\", fillcolor=\"#D5AEFC\"];\n  \"rg-lib\" -> \"lobSTR\" [label=\"rg-lib\"];\n  \"rg-sample\" -> \"lobSTR\" [label=\"rg-sample\"];\n  \"p2\" -> \"lobSTR\" [label=\"p2\"];\n  \"p1\" -> \"lobSTR\" [label=\"p1\"];\n  \"reference\" -> \"lobSTR\" [label=\"reference\"];\n  \"output_prefix\" -> \"lobSTR\" [label=\"output_prefix\"];\n  \"samsort\" -> \"samindex\" [label=\"input\"];\n  \"samindex\" -> \"bam\";\n  \"allelotype\" -> \"vcf\";\n  \"lobSTR\" -> \"bam_stats\";\n  \"allelotype\" -> \"vcf_stats\";\n}\n",
    "visualisationPng": "/graph/png/github.com/common-workflow-language/workflows/blob/master/workflows/lobSTR/lobSTR-workflow.cwl",
    "visualisationSvg": "/graph/svg/github.com/common-workflow-language/workflows/blob/master/workflows/lobSTR/lobSTR-workflow.cwl",
    "roBundle": "/robundle/github.com/common-workflow-language/workflows/blob/master/workflows/lobSTR/lobSTR-workflow.cwl",
    "visualisationXdot": "/graph/xdot/github.com/common-workflow-language/workflows/blob/master/workflows/lobSTR/lobSTR-workflow.cwl"
}

View Workflows

GET

/workflows

Parameters

Field	Type	Description
search	String	Optional: Limits the workflows received to those containing this string in the label or doc
page	Integer	The page number to access (0 indexed, defaults to 0)
size	Integer	The page size requested (defaults to 10)

Success 200

Field	Type	Description
content	List<Workflow>	The workflows returned as per Get Workflow Details
last	Boolean	Whether this page is the last one
totalElements	Integer	The total number of workflows
totalPages	Integer	The total number of pages of this size
first	Boolean	Whether this page is the first one
numberOfElements	Integer	The number of workflows on this page
size	Integer	The page size requested
number	Integer	The current page number

Example Response

{
    "content": [
        {

        },
        {

        }
    ],
    "last": true,
    "totalElements": 2,
    "totalPages": 1,
    "sort": null,
    "first": true,
    "numberOfElements": 2,
    "size": 10,
    "number": 0
}

Queue

Check Queue Element

GET

/queue/:id

Parameters

Field	Type	Description
id	String	The ID of the queue element from the Location header of Add a new Workflow

Pending 200

Field	Type	Description
cwltoolStatus	enum	One of RUNNING, ERROR
cwltoolVersion	String	The version of cwltool being used to parse the workflow
message	String	Error message if cwltoolStatus is ERROR

Example Responses

{
    "cwltoolStatus": "RUNNING",
    "cwltoolVersion": "1.0.20170622090721",
}

{
    "cwltoolStatus": "ERROR",
    "cwltoolVersion": "1.0.20170622090721",
    "message": "Tool definition failed initialization:\n('https://sparql-test.commonwl.org/schema.rdf', SSLError(SSLError(CertificateError(\"hostname 'sparql-test.commonwl.org' doesn't match either of 'biowardrobe.com', 'demo.biowardrobe.com', 'www.biowardrobe.com'\",),),))\n"
}

Finished 303

HTTP/1.1 303 See Other
Location: /workflows/:url

Examples

Python - Build script for Travis

The API can be utilised as a build step in continuous integration such as in the following example

travis.yml

The python script can be included in the 'script' or 'after_success' areas of travis.yml depending on your usage

language: python

python:
  - "2.7"

install:
  - pip install requests

script:
  - ...

after_success:
  - python cwlviewer.py

cwlviewer.py

# Must list 'pip install requests' in travis.yml
import requests
import time
import os

# You can alternatively define these in travis.yml as env vars or arguments
BASE_URL = 'https://view.commonwl.org'
WORKFLOW_PATH = '/workflows/workflow.cwl'

# Travis environment variables
TRAVIS_COMMIT = os.getenv('TRAVIS_COMMIT')
TRAVIS_REPO_SLUG = os.getenv('TRAVIS_REPO_SLUG')

# Whole workflow URL on github
workflowURL = 'https://github.com/' + TRAVIS_REPO_SLUG + '/blob/' + TRAVIS_COMMIT + WORKFLOW_PATH

# Headers
HEADERS = {
    'user-agent': 'my-app/0.0.1',
    'accept': 'application/json'
}

# Add new workflow with the specific commit ID of this build
addResponse = requests.post(BASE_URL + '/workflows',
                            data={'url': workflowURL},
                            headers=HEADERS)

if addResponse.status_code == requests.codes.accepted:
    qLocation = addResponse.headers['location']

    # Get the queue item until success
    qResponse = requests.get(BASE_URL + qLocation,
                             headers=HEADERS,
                             allow_redirects=False)
    maxAttempts = 5
    while qResponse.status_code == requests.codes.ok and qResponse.json()['cwltoolStatus'] == 'RUNNING' and maxAttempts > 0:
        time.sleep(5)
        qResponse = requests.get(BASE_URL + qLocation,
                                 headers=HEADERS,
                                 allow_redirects=False)
        maxAttempts -= 1

    if qResponse.headers['location']:
        # Success, get the workflow
        workflowResponse = requests.get(BASE_URL + qResponse.headers['location'], headers=HEADERS)
        if (workflowResponse.status_code == requests.codes.ok):
            workflowJson = workflowResponse.json()
            # Do what you want with the workflow JSON
            # Include details in documentation files etc
            print(BASE_URL + workflowJson['visualisationSvg'])
            print('Verified with cwltool version ' + workflowJson['cwltoolVersion'])
            # etc...
        else:
            print('Could not get returned workflow')
    elif qResponse.json()['cwltoolStatus'] == 'ERROR':
        # Cwltool failed to run here
        print(qResponse.json()['message'])
    elif maxAttempts == 0:
        print('Timeout: Cwltool did not finish')

else:
    print('Error adding workflow')

SVG sketch of CWL workflow

POST

/graph/svg

Input

Content-Type: text/yaml

#!/usr/bin/env cwl-runner
  
cwlVersion: v1.0
class: Workflow

inputs:
  usermessage: string

outputs:
  response:
    outputSource: step0/response
    type: File

steps:
  step0:
    run:
      class: CommandLineTool
      inputs:
        message:
          type: string
          inputBinding:
            position: 1
      baseCommand: echo
      outputs:
        response:
          type: stdout
    in:
      message: usermessage
    out: [response]

Output

Content-Type: image/svg+xml

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg width="450pt" height="270pt"
     viewBox="0.00 0.00 463.00 278.00" xmlns="http://www.w3.org/2000/svg">
	<!-- ... -->
</svg>

Success

HTTP/1.1 200 OK
Content-Type: image/svg+xml

Error

HTTP/1.1 400 Bad Request

PNG sketch of CWL workflow

POST

/graph/png

Input

Content-Type: text/yaml

#!/usr/bin/env cwl-runner
  
cwlVersion: v1.0
class: Workflow

inputs:
  usermessage: string

outputs:
  response:
    outputSource: step0/response
    type: File

steps:
  step0:
    run:
      class: CommandLineTool
      inputs:
        message:
          type: string
          inputBinding:
            position: 1
      baseCommand: echo
      outputs:
        response:
          type: stdout
    in:
      message: usermessage
    out: [response]

Output

\x89PNG\n\x1a\n\x00\x00 (binary)

Success

HTTP/1.1 200 OK
Content-Type: image/png

Error

HTTP/1.1 400 Bad Request