How to use an API call to download FastQ files from BaseSpace

BaseSpace API calls can be used to download FastQ files from BaseSpace. This approach can be adopted to download other file types as well.

The first step in using BaseSpace API is to generate an access token - the easiest way to do thisis using BaseSpace CLI.

1. Authenticate BaseSpace CLI.

2. Obtain the authorization/access token from the CLI config file - located in ~/.basespace/{config name}.cfg - default.cfg is the default config.

The following API call is used to download files from BaseSpace.

curl -X 'GET' -H 'Authorization: Bearer {Access Token}' 'https://api.basespace.illumina.com/v2/files/{File ID}/content' -LOJ

The File ID for FastQ files is obtained from the dataset ID using the following endpoint. GET /datasets/{Dataset ID}/files.

Now, these API calls can be tied together using a JSON-parsing tool such as jq to download FastQ files associated with projects, runs.

For example, below is a command that can be used to download all FastQs associated with a run.

curl -X 'GET' -H 'Authorization: Bearer {Access Token}' -H 'Content-Type: application/json' 'https://api.basespace.illumina.com/v2/datasets?InputRuns={Run ID}&Limit=200&Offset=0' | jq '.Items | .[] | select(.DatasetType.Id | test("fastq")) | .HrefFiles' | xargs -I @ curl -X 'GET' -H 'Authorization: Bearer {Access Token}' -H 'Content-Type: application/json' @ | jq '.Items | .[].HrefContent' | xargs -I $ curl -X 'GET' -H 'Authorization: Bearer {Access Token}' $ -LOJ

Similarly, the following command can be used to download all FastQs from a project.

curl -X 'GET' -H 'Authorization: Bearer {Access Token}' -H 'Content-Type: application/json' 'https://api.basespace.illumina.com/v2/datasets?ProjectID={Project ID}&Limit=200&Offset=0' | jq '.Items | .[] | select(.DatasetType.Id | test("fastq")) | .HrefFiles' | xargs -I @ curl -X 'GET' -H 'Authorization: Bearer {Access Token}' -H 'Content-Type: application/json' @ | jq '.Items | .[].HrefContent' | xargs -I $ curl -X 'GET' -H 'Authorization: Bearer {Access Token}' $ -LOJ

Further filtering can be implemented using different features of jq. Alternatively, use BaseSpace CLI to download FastQ files in a customized fashion.

More information about BaseSpace API endpoints can be found here.

For any feedback or questions regarding this article (Illumina Knowledge Article #7506), contact Illumina Technical Support [e-mail].