Running XCP-D

Warning

XCP-D may not run correctly on M1 chips.

Execution and Input Formats

The XCP-D workflow takes fMRIPrep, Nibabies, HCP Pipelines, ABCD-BIDS, and UK Biobank outputs in the form of BIDS derivatives. In these examples, we use an fmriprep output directory.

The outputs are required to include at least anatomical and functional outputs with at least one preprocessed BOLD image. Additionally, each of these should be in directories that can be parsed by the BIDS online validator (even if it is not BIDS valid - we do not require BIDS valid directories). The directories must also include a valid dataset_description.json.

The exact command to run in XCP-D depends on the Installation method and data that needs to be processed. We start first with the bare-metal Manually Prepared Environment (Python 3.10) installation, as the command line is simpler. XCP-D can be executed on the command line, processesing fMRIPrep outputs, using the following command-line structure, for example:

xcp_d /path/to/fmriprep_dir \
   /path/to/output_dir \
   participant \ # analysis_level
   --mode <mode> \ # required
   --participant-label <label> # optional

However, we strongly recommend using Container Technologies. Here, the command-line will be composed of a preamble to configure the container execution, followed by the XCP-D command-line options as if you were running it on a bare-metal installation.

Command-Line Arguments

Minimal Inputs

The minimal inputs required to run XCP-D are:

  • A native-space preprocessed T1w or T2w image.

  • A preprocessed BOLD image in MNI152NLin6Asym, MNI152NLin2009cAsym, MNIInfant (nibabies derivatives), or fsLR (CIFTI processing) space.

  • The functional brain mask and boldref image in the same space as the preprocessed BOLD data.

  • The confounds associated with the BOLD image, along with the associated JSON file.

  • The anatomical brain mask in the same space as the preprocessed BOLD data.

  • The transform from the native anatomical space to the standard space the BOLD image is in, and its inverse.

Surface files, such as the pial and white matter GIFTI files, may be required depending on the settings you use.

Below are an example lists of inputs.

Warning

Please note that the filenames may differ based on the pipeline, or even version of the pipeline, used for preprocessing.

The specific files required by XCP-D may also vary slightly depending on the settings you use.

For NIfTI processing:

dataset_description.json
sub-x/
   anat/
      sub-x_desc-preproc_T1w.nii.gz  # Can be T1w or T2w. Note that this is native anatomical space.
      sub-x_desc-preproc_T1w.json
      sub-x_space-MNI152NLin6Asym_desc-brain_mask.nii.gz
      sub-x_space-MNI152NLin6Asym_desc-brain_mask.json
      sub-x_from-MNI152NLin6Asym_to-T1w_mode-image_xfm.h5
      sub-x_from-T1w_to-MNI152NLin6Asym_mode-image_xfm.h5
   func/
      sub-x_task-rest_desc-confounds_timeseries.tsv
      sub-x_task-rest_desc-confounds_timeseries.json
      sub-x_task-rest_space-MNI152NLin6Asym_desc-preproc_bold.nii.gz
      sub-x_task-rest_space-MNI152NLin6Asym_desc-preproc_bold.json
      sub-x_task-rest_space-MNI152NLin6Asym_boldref.nii.gz
      sub-x_task-rest_space-MNI152NLin6Asym_boldref.json
      sub-x_task-rest_space-MNI152NLin6Asym_desc-brain_mask.nii.gz
      sub-x_task-rest_space-MNI152NLin6Asym_desc-brain_mask.json

For CIFTI processing:

dataset_description.json
sub-x/
   anat/
      sub-x_desc-preproc_T1w.nii.gz  # Can be T1w or T2w. Note that this is native anatomical space.
      sub-x_desc-preproc_T1w.json
      sub-x_space-MNI152NLin6Asym_desc-brain_mask.nii.gz
      sub-x_space-MNI152NLin6Asym_desc-brain_mask.json
      sub-x_from-MNI152NLin6Asym_to-T1w_mode-image_xfm.h5
      sub-x_from-T1w_to-MNI152NLin6Asym_mode-image_xfm.h5
   func/
      sub-x_task-rest_desc-confounds_timeseries.tsv
      sub-x_task-rest_desc-confounds_timeseries.json
      sub-x_task-rest_space-fsLR_den-91k_bold.dtseries.nii
      sub-x_task-rest_space-fsLR_den-91k_bold.json
      sub-x_task-rest_space-MNI152NLin6Asym_desc-preproc_bold.nii.gz  # Needed for QC figures
      sub-x_task-rest_space-MNI152NLin6Asym_boldref.nii.gz
      sub-x_task-rest_space-MNI152NLin6Asym_boldref.json

Surface files:

dataset_description.json
sub-x/
   anat/
      # Mesh files in fsnative space, to be warped to fsLR space
      sub-x_hemi-L_pial.surf.gii
      sub-x_hemi-R_pial.surf.gii
      sub-x_hemi-L_white.surf.gii
      sub-x_hemi-R_white.surf.gii

      # Sphere files for registration
      sub-x_hemi-L_space-fsaverage_desc-reg_sphere.surf.gii
      sub-x_hemi-R_space-fsaverage_desc-reg_sphere.surf.gii

      # Morphometry files in fsLR space, to be parcellated
      sub-x_hemi-L_space-fsLR_den-91k_curv.dscalar.nii
      sub-x_hemi-L_space-fsLR_den-91k_sulc.dscalar.nii
      sub-x_hemi-L_space-fsLR_den-91k_thickness.dscalar.nii
      sub-x_hemi-L_space-fsLR_den-91k_myelinw.dscalar.nii
      sub-x_hemi-L_space-fsLR_den-91k_desc-smoothed_myelinw.dscalar.nii

Filtering Inputs with BIDS Filter Files

XCP-D allows users to choose which preprocessed files will be post-processed with the --bids-filter-file parameter. This argument must point to a JSON file, containing filters that will be fed into PyBIDS.

The keys in this JSON file are unique to XCP-D. They are our internal terms for different inputs that will be selected from the preprocessed dataset. The full list of keys can be found in the file xcp_d/data/io_spec.yaml.

"bold" determines which preprocessed BOLD files will be chosen. You can set a number of entities here, including “session”, “task”, “space”, “resolution”, and “density”. We recommend NOT setting the datatype, suffix, or file extension in the filter file.

Warning

We do not recommend applying additional filters to any of the following fields. We have documented them here, for edge cases where they might be useful, but the only field that most users should filter is "bold".

"t1w" selects a native T1w-space, preprocessed T1w file.

"t2w" selects a native T1w-space, preprocessed T2w file. If a T1w file is not available, this file will be in T2w space.

"anat_brainmask" selects an anatomically-derived brain mask in the same space as the BOLD data. This file is used (1) to estimate head radius for FD calculation (after warping to native space) and (2) to calculate coregistration quality metrics.

"anat_to_template_xfm" selects a transform from T1w (or T2w, if no T1w image is available) space to standard space. The standard space that will be used depends on the "bold" files that are selected.

"template_to_anat_xfm" selects a transform from standard space to T1w/T2w space. Again, the standard space is determined based on other files.

There are additional keys that control how mesh and morphometry files are selected. Please refer to io_spec.yaml for more information on them.

atlas selects atlases for parcellation. This is primarily useful for specifying spaces or resolutions. This field is not reflected in the io_spec.yaml file.

Example bids-filter-file

In this example file, we only run XCP-D on resting-state preprocessed BOLD runs from session “01”.

{
   "bold": {
      "session": ["01"],
      "task": ["rest"]
   }
}

Running XCP-D via containers

Apptainer

If you are computing on an HPC, we recommend using Apptainer. See Container Technologies for installation instructions.

Once a user specifies the container options and the image to be run, the command line options are the same as the bare-metal installation.

apptainer run --cleanenv xcp_d-<version>.simg \ # container args
   /path/to/fmriprep_dir \ #xcpd args
   /path/to/output_dir \
   participant \ # analysis_level
   --mode <mode> \ # required
   --participant-label <label> # optional

By default, Apptainer will mount (make accessible) your current working directory inside the container. If you need to access files from other locations on your system, you’ll need to explicitly bind those directories using the -B flag. For example:

apptainer run -B /home/user/data \ # Mount data directory
   --cleanenv xcp_d-<version>.simg \
   /home/user/data/fmriprep \
   /home/user/data/xcpd_output \
   participant \
   --mode <mode>

Docker

If you are running XCP-D locally, we recommend Docker. See Container Technologies for installation instructions.

In order to run Docker smoothly, it is best to prevent permissions issues associated with the root file system. Running Docker as user on the host will ensure the ownership of files written during the container execution.

A Docker container can be created using the following command:

docker run --rm -it \ # docker args
   -v /home/user/data/fmriprep \
   -v /home/user/data/wkdir \
   -v /home/user/data/xcpd_output \
   pennlinc/xcp_d:<version> \
   /home/user/data/fmriprep \ #xcpd args
   /home/user/data/xcpd_output \
   participant \ # analysis_level
   --mode <mode> \ # required
   --participant-label <label> # optional

Custom Confounds

If you would like to denoise your data with confounds that are not included in the built-in confound strategies, you can create your own. Please see The confound configuration file format for more info on the configuration file format.

If you have a set of custom confounds that you would like to use, you need to organize them into a BIDS dataset. The dataset should only require the actual confound files, preferably with associated sidecar JSONs, and a dataset_description.json file. Using the same filenames as the fMRIPrep confound files is an easy way to organize this dataset.

Including Signal Regressors

Warning

Signal regressors are not currently supported in combination with voxel-wise regressors.

Warning

Please be careful when including signal regressors. Most of the time this is probably a bad idea. For example, if you run tedana, only noise components from tedana should be orthogonalized with respect to signal components. You wouldn’t want to orthogonalize other noise signals (e.g., motion parameters) with respect to the signal components from tedana.

Let’s say you have some nuisance regressors that are not necessarily orthogonal to some associated regressors that are ostensibly signal. For example, if you want to denoise your data while still retaining task signal, you could include the predicted task signals in your confounds.

For more information about different types of denoising, see tedana’s documentation, this NeuroStars topic, and/or Pruim et al. (2015).

So how do we implement this in XCP-D? In order to define regressors that should be treated as signal, and thus orthogonalize the noise regressors with respect to known signals instead of regressing them without modification, you should include those regressors in your custom confounds file, with column names starting with signal__ (lower-case “signal”, followed by two underscores).

Task Regression

If you want to regress task-related signals out of your data, you can use a custom confound configuration.

Here we document how to include task effects as confounds.

Tip

The basic approach to task regression is to convolve your task regressors with an HRF, then save those regressors to a confounds file.

Warning

This method is still under development.

We recommend using a tool like Nilearn to generate convolved regressors from BIDS events files. See this example.

import json
import os

import numpy as np
import pandas as pd
from nilearn.glm.first_level import make_first_level_design_matrix

N_VOLUMES = 200
TR = 0.8
frame_times = np.arange(N_VOLUMES) * TR
events_df = pd.read_table("sub-X_task-Z_events.tsv")

task_confounds = make_first_level_design_matrix(
   frame_times,
   events_df,
   drift_model=None,
   add_regs=None,
   hrf_model="spm",
)

# The design matrix will include a constant column, which we should drop
task_confounds = task_confounds.drop(columns="constant")

# Prepare the derivative dataset
os.makedirs("/my/project/directory/custom_confounds/sub-X/func", exist_ok=True)
# Include a dataset_description.json file
with open("/my/project/directory/custom_confounds/dataset_description.json", "w") as fo:
   json.dump(
      {
         "Name": "Custom Confounds",
         "BIDSVersion": "1.6.0",
         "DatasetType": "derivative"
      },
      fo,
   )

# Assuming that the fMRIPrep confounds file is named
# "sub-X_task-Z_desc-confounds_timeseries.tsv",
# we will name the custom confounds file the same thing, in a separate folder.
task_confounds.to_csv(
   "/my/project/directory/custom_confounds/sub-X/func/sub-X_task-Z_desc-confounds_timeseries.tsv",
   sep="\t",
   index=False,
)

Then, create a confounds config file to include derivatives from custom_confounds. Something like this should work:

name: my_custom_confounds
description: |
   Nuisance regressors were task regressors convolved with an HRF and motion parameters.
confounds:
   motion:
      dataset: preprocessed
      query:
         space: null
         cohort: null
         res: null
         den: null
         desc: confounds
         extension: .tsv
         suffix: timeseries
      columns:
      - trans_x
      - trans_y
      - trans_z
      - rot_x
      - rot_y
      - rot_z
   task:
      dataset: custom
      query:
         space: null
         cohort: null
         res: null
         den: null
         desc: confounds
         extension: .tsv
         suffix: timeseries
      columns:  # Assume the task regressors are called "condition1" and "condition2"
      - condition1
      - condition2

Command Line XCP-D with Custom Confounds

Last, run XCP-D with your custom configuration file and the path to the custom derivatives dataset.

apptainer run -B /home/user/data \
   --cleanenv xcpd_<version>.simg \
   /home/user/data/path/to/fmriprep_dir \
   /home/user/data/path/to/output_dir \
   participant \ # analysis_level
   --mode <mode> \ # required
   --participant-label <label> # optional
   --datasets custom=/home/user/data/path/to/custom_confounds \
   --nuisance-regressors /home/user/data/path/to/custom_config.yaml

External Atlases

While XCP-D comes with many built-in parcellations, we understand that many users will want to use different ones.

As long as the parcellation is organized in a BIDS-Atlas dataset and is in fsLR-32k space (for CIFTI processing) or MNIInfant, MNI152NLin6Asym, or MNI152NLin2009cAsym space (for NIfTI processing), you can use it with XCP-D.

Warning

BIDS Extension Proposal 38 (Atlas Specification) has not been integrated in BIDS yet, so the organization and naming for atlas datasets may change in the future.

We have attempted to follow the proposed structure in XCP-D, but we cannot guarantee that this will not change.

Tip

The main elements from the BIDS-Atlas dataset that XCP-D uses are:

  1. There must be a dataset_description.json file with DatasetType set to “atlas”.

  2. The atlas metadata files must have the same entities as the atlas image files, as PyBIDS does not support the inheritance principle when querying BIDS-Atlas datasets (yet).

  3. There must be a TSV file for the atlas, with “index” and “label” columns.

To do this, use the --datasets and --atlases parameters. The --datasets parameter should point to the directory containing the BIDS-Atlas dataset, and the --atlases parameter should include the names of the atlases in the dataset to use.

For example, consider a scenario where you have two BIDS-Atlas datasets, one containing all of the Schaefer 2018 resolutions and one containing the AAL atlas. These datasets are in /data/atlases/schaefer and /data/atlases/aal, respectively. The file structure for these two datasets might look like this:

/data/atlases/
   schaefer/
      dataset_description.json
      atlas-Schaefer100/
         atlas-Schaefer100_dseg.tsv
         atlas-Schaefer100_space-fsLR_den-32k_dseg.dlabel.nii
         atlas-Schaefer100_space-fsLR_den-32k_dseg.json
      atlas-Schaefer200/
         atlas-Schaefer200_dseg.tsv
         atlas-Schaefer200_space-fsLR_den-32k_dseg.dlabel.nii
         atlas-Schaefer200_space-fsLR_den-32k_dseg.json
      ...
      atlas-Schaefer1000/
         atlas-Schaefer1000_dseg.tsv
         atlas-Schaefer1000_space-fsLR_den-32k_dseg.dlabel.nii
         atlas-Schaefer1000_space-fsLR_den-32k_dseg.json
   aal/
      dataset_description.json
      atlas-AAL/
         atlas-AAL_dseg.tsv
         atlas-AAL_space-fsLR_den-32k_dseg.dlabel.nii
         atlas-AAL_space-fsLR_den-32k_dseg.json

You may want to only apply the Schaefer100 atlas from the schaefer dataset and the AAL atlas from the aal dataset, along with one of XCP-D’s built-in atlases (4S156Parcels). Here’s what the XCP-D call might look like:

apptainer run -B /home/user/data \
   --cleanenv xcpd_<version>.simg \
   /home/user/data/path/to/fmriprep_dir \
   /home/user/data/path/to/output_dir \
   participant \ # analysis_level
   --mode <mode> \ # required
   --datasets schaefer=/home/user/data/path/to/schaefer_atlas aal==/home/user/data/path/to/aal_atlas \
   --atlases Schaefer100 AAL 4S156Parcels

XCP-D will search for atlas-Schaefer100, atlas-AAL, and atlas-4S156Parcels across the schaefer, aal, and XCP-D’s built-in atlas datasets. If the atlases are found, then they will be used for parcellation.

Important

Atlas names must be unique across BIDS-Atlas datasets. If two atlases have the same name, XCP-D will raise an error.

Advanced Applications

XCP-D can be used in conjunction with other tools, such as tedana and phys2denoise. We have attempted to document these applications with working code in PennLINC/xcp_d-examples. If there is an application you think would be useful to document, please open an issue in that repository.

Preprocessing Requirements for XCP-D

XCP-D is designed to ingest data from a variety of different preprocessing pipelines. However, each supported pipeline must be explicitly supported within XCP-D in order for the workflow to select the correct files.

Additionally, XCP-D may require files that are only created with specific settings in the preprocessing pipelines.

fMRIPrep/Nibabies

In order to work on fMRIPrep or Nibabies derivatives, XCP-D needs derivatives in one of a few template spaces, including “MNI152NLin6Asym”, “MNI152NLin2009cAsym”, “MNIInfant”, and “fsLR”. We may add support for additional templates in the future, but currently you must have at least one of these among your output spaces. XCP-D does not have any specific requirements for resolution of volumetric derivatives, but we do require fsLR-space CIFTIs be outputted in 91k density.

Troubleshooting

Logs and crashfiles are outputted into the <output dir>/xcp_d/sub-<participant_label>/log directory. Information on how to customize and understand these files can be found on the nipype debugging page.

Support and communication

All bugs, concerns and enhancement requests for this software can be submitted here: https://github.com/PennLINC/xcp_d/issues.

If you have a question about using XCP-D, please create a new topic on NeuroStars with the “Software Support” category and the “xcp_d” tag. The XCP-D developers follow NeuroStars, and will be able to answer your question there.