Bulk Deletion of Runs and Projects Using BaseSpace CLI with Windows PowerShell
Note
The information in the article is for Windows users. It does not apply to CLI usage in Linux or Mac OS.
Background
Sometimes users may need to delete runs or projects that date back several months or years, and additionally free up storage space in a BaseSpace account. In cases where many items must be removed at once, performing this manually through the Guided User Interface of BaseSpace or via the BaseSpace Command-Line Interface (BaseSpace CLI) can be time‑consuming and impractical. To streamline this process, the BaseSpace CLI can be combined with Windows PowerShell scripting to efficiently perform bulk deletion of multiple runs or projects.
Requirements
BaseSpace CLI installed. Detailed installation instructions on the CLI Overview webpage.
Windows PowerShell
Launching the Windows PowerShell
Bulk deletion via this approach can only be executed through Windows PowerShell.
To open a Windows PowerShell session:
Open the Windows Start menu
Type “Windows PowerShell” into the search bar
Select Windows PowerShell
A PowerShell terminal Window will open
Authenticate to BaseSpace Account
Before performing any data deletion, ensure to authenticate both the BaseSpace instance and the account that contains the data intended to be removed.
Instructions for performing authentication are provided in the CLI Overview. If needed to authenticate to a workgroup, navigate to How to Authenticate BaseSpace CLI to a Workgroup for detailed steps.
For example, authentication to theEUC1 BaseSpace instance can be done using:
To confirm authentication to the correct instance and account, run the following command:
This command will output account username, and BaseSpace instance under the Host section. Ensure this matches the environment where data deletion will be intended.
I. For Bulk Deletion of Runs
The following example demonstrates a bulk deletion operation that removes all Runs older than 50 days. This command can be customized based on the specific criteria required for data cleanup:
In this example:
--older-than=50d #Filter for Runs older than 50 days.
--terse #Outputs only the Run IDs.
Each ID is piped “ | ” into bs delete run -i for deletion.
-v/--verbose (optional) #Enables verbose output during deletion.
--preserve-metadata (optional) #Enables metadata for runs to be retained while deleting more space-consuming files such as BCL-files.
Details about the available arguments are included in the CLI Overview and CLI Examples webpages. Moreover, additional details are provided via the --help flag in BaseSpace CLI such as bs list runs --help.
Excluding Specific Runs From Bulk Deletion
During bulk deletion, certain Runs may need to be excluded from the operation. This can be achieved by filtering out specific Run IDs before piping the list into the deletion command.
To exclude multiple Run IDs, Select-String -NotMatch can be used with a regular expression pattern. Separate each Run ID in the exclusion list using the pipe symbol “ | ”.
The following example excludes Run IDs 27812785 and 27830803 from the deletion list:
Additional Filtering Options and Details
The bs list runs and bs list projects commands supports additional filtering parameters that can help narrow down the list of Runs targeted for deletion including:
--newer-than #Filter for items that are newer than the given duration, for example 5m (5 minutes) or 12h (12 hours). Permitted suffixes are s, m, h, d, w, y
--older-than #Filter for items that are older than the given duration, for example 10d (10 days) or 2y (2 years). Permitted suffixes are s, m, h, d, w, y
--filter-field #Enter the field to use when for filtering (default: Name)
--filter-term #Filter by regular expression
The --filter-field and --filter-term options can be used together to filter Runs or Projects based on specific metadata fields. This is useful when items share identifiable naming patterns or attributes.
For example, when working with Runs from a MiSeq i100 instrument whose serial number SH09999 is included in the Run name, all associated Runs newer than 5 days can be filtered and deleted using:
Another example, to identify and delete Runs that are newer than 20 days, have a status of Failed, and without preserving any data or metadate, the following command can be used:
The --preserve-metadata argument has been intentionally omitted from the bs delete run -i command in this example. As a result, both data and metadata associated with each Run will be deleted.
II. For Bulk Deletion of Projects
The same approach used for bulk deletion of Runs also applies to Projects. The key difference is that the deletion command must be updated to use bs delete project -i instead. As with Runs, filters and exclusion patterns can be combined to specify which Projects are deleted.
Excluding Specific Projects From Bulk Deletion
Specific Projects IDs can likewise be excluded in the same way as Runs by using the -NotMatch filter. The following example excludes Projects IDs 9686677, 9685676 and 9684677.
III. Complete Bulk Deletion Without Filters
Filter options are optional and do not need to be applied in every scenario. If the intenstion is to delete all Runs or all Projects associated with an account, filtering parameters can be omitted entirely. The following command retrieves all Run IDs or Project IDs and deletes them without applying any filters:
For Runs:
For Projects:
IV. Complete Bulk Deletion Process
After deletion, data or projects are moved to Trash and still count toward total storage. To finalize the deletion process, navigate to the Trash in BaseSpace and empty it to permanently remove the runs and/or projects from BaseSpace.
For any feedback or questions regarding this article (Illumina Knowledge Article #10078), contact Illumina Technical Support [email protected].
Last updated
Was this helpful?