How does BaseSpace Command Line Interface (CLI) check the integrity of file uploads?

BaseSpace CLI is a command line tool available for Linux, Windows, and Mac that can be used to interact with BaseSpace Sequence Hub for a variety of tasks, including file uploads. To verify the integrity of files uploaded to BaseSpace Sequence Hub, BaseSpace CLI uses a version of the open source program s3gof3r.

s3gof3r verifies file integrity as described in the s3gof3r README file:

• s3gof3r calculates the md5 hash of the stream in parallel while uploading and downloading. On upload, a file containing the md5 hash is saved in s3. This is checked against the calculated md5 on download. On upload, the content-md5 of each part is calculated and sent with the header to be checked by AWS. s3gof3r also checks the 'hash of hashes' returned by S3 in the Etag field on completion of a multipart upload.

For any feedback or questions regarding this article (Illumina Knowledge Article #8437), contact Illumina Technical Support [e-mail].