.. include:: links.rst ############### Running *XCP-D* ############### .. warning:: *XCP-D* may not run correctly on **M1 chips**. .. _usage_inputs: *************************** Execution and Input Formats *************************** The *XCP-D* workflow takes `fMRIPrep`_, `Nibabies`_, `HCP Pipelines`_, `ABCD-BIDS`_, and `UK Biobank` outputs in the form of BIDS derivatives. In these examples, we use an fmriprep output directory. The outputs are required to include at least anatomical and functional outputs with at least one preprocessed BOLD image. Additionally, each of these should be in directories that can be parsed by the BIDS online validator (even if it is not BIDS valid - we do not require BIDS valid directories). The directories must also include a valid `dataset_description.json`. The exact command to run in *XCP-D* depends on the :doc:`installation` method and data that needs to be processed. We start first with the *bare-metal* :ref:`installation_manually_prepared_environment` installation, as the command line is simpler. *XCP-D* can be executed on the command line, processesing fMRIPrep outputs, using the following command-line structure, for example: .. code-block:: bash xcp_d --file-format cifti --despike --head_radius 40 -w /wkdir --smoothing 6 However, we strongly recommend using :ref:`installation_container_technologies`. Here, the command-line will be composed of a preamble to configure the container execution, followed by the *XCP-D* command-line options as if you were running it on a *bare-metal* installation. .. _usage_cli: ********************** Command-Line Arguments ********************** .. argparse:: :ref: xcp_d.cli.parser._build_parser :prog: xcp_d :func: _build_parser --band-stop-min : @after For the "notch" filter option, we recommend the following values. .. list-table:: Recommended values, based on participant age :align: left :header-rows: 1 :stub-columns: 1 * - Age Range (years) - Recommended Value (bpm) * - < 1 - 30 * - 1 - 2 - 25 * - 2 - 6 - 20 * - 6 - 12 - 15 * - 12 - 18 - 12 * - 19 - 65 - 12 * - 65 - 80 - 12 * - > 80 - 10 --band-stop-max : @after For the "notch" filter option, we recommend the following values. .. list-table:: Recommended values, based on participant age :align: left :header-rows: 1 :stub-columns: 1 * - Age Range (years) - Recommended Value (bpm) * - < 1 - 60 * - 1 - 2 - 50 * - 2 - 6 - 35 * - 6 - 12 - 25 * - 12 - 18 - 20 * - 19 - 65 - 18 * - 65 - 80 - 28 * - > 80 - 30 --warp-surfaces-native2std : @after .. list-table:: The surface files that are generated by the workflow :align: left :header-rows: 1 :stub-columns: 1 * - Filename - Description * - ``_hemi-_space-fsLR_den-32k_pial.surf.gii`` - The gray matter / pial matter border. * - ``_hemi-_space-fsLR_den-32k_white.surf.gii`` - The smoothed gray matter / white matter border for the cortex. * - ``_hemi-_space-fsLR_den-32k_midthickness.surf.gii`` - The midpoints between wm and pial surfaces. This is derived from the FreeSurfer graymid (``mris_expand`` with distance=0.5 applied to the WM surfs). * - ``_hemi-_space-fsLR_den-32k_inflated.surf.gii`` - An inflation of the midthickness surface (useful for visualization). This file is only created if the input type is "hcp" or "dcan". * - ``_hemi-_space-fsLR_den-32k_desc-hcp_midthickness.surf.gii`` - The midpoints between wm and pial surfaces. This is created by averaging the coordinates from the wm and pial surfaces. * - ``_hemi-_space-fsLR_den-32k_desc-hcp_inflated.surf.gii`` - An inflation of the midthickness surface (useful for visualization). This is derived from the HCP midthickness file. This file is only created if the input type is "fmriprep" or "nibabies". * - ``_hemi-_space-fsLR_den-32k_desc-hcp_vinflated.surf.gii`` - A very-inflated midthicknesss surface (also for visualization). This is derived from the HCP midthickness file. This file is only created if the input type is "fmriprep" or "nibabies". ************** Minimal Inputs ************** The minimal inputs required to run *XCP-D* are: - A native-space preprocessed T1w or T2w image. - A preprocessed BOLD image in MNI152NLin6Asym, MNI152NLin2009cAsym, MNIInfant (nibabies derivatives), or fsLR (CIFTI processing) space. - The functional brain mask and boldref image in the same space as the preprocessed BOLD data. - The confounds associated with the BOLD image, along with the associated JSON file. - The anatomical brain mask in the same space as the preprocessed BOLD data. - The transform from the native anatomical space to the standard space the BOLD image is in, and its inverse. Surface files, such as the pial and white matter GIFTI files, may be required depending on the settings you use. Below are an example lists of inputs. .. warning:: Please note that the filenames may differ based on the pipeline, or even version of the pipeline, used for preprocessing. The specific files required by *XCP-D* may also vary slightly depending on the settings you use. For NIfTI processing: .. code-block:: dataset_description.json sub-x/ anat/ sub-x_desc-preproc_T1w.nii.gz # Can be T1w or T2w. Note that this is native anatomical space. sub-x_desc-preproc_T1w.json sub-x_space-MNI152NLin6Asym_desc-brain_mask.nii.gz sub-x_space-MNI152NLin6Asym_desc-brain_mask.json sub-x_from-MNI152NLin6Asym_to-T1w_mode-image_xfm.h5 sub-x_from-T1w_to-MNI152NLin6Asym_mode-image_xfm.h5 func/ sub-x_task-rest_desc-confounds_timeseries.tsv sub-x_task-rest_desc-confounds_timeseries.json sub-x_task-rest_space-MNI152NLin6Asym_desc-preproc_bold.nii.gz sub-x_task-rest_space-MNI152NLin6Asym_desc-preproc_bold.json sub-x_task-rest_space-MNI152NLin6Asym_boldref.nii.gz sub-x_task-rest_space-MNI152NLin6Asym_boldref.json sub-x_task-rest_space-MNI152NLin6Asym_desc-brain_mask.nii.gz sub-x_task-rest_space-MNI152NLin6Asym_desc-brain_mask.json For CIFTI processing: .. code-block:: dataset_description.json sub-x/ anat/ sub-x_desc-preproc_T1w.nii.gz # Can be T1w or T2w. Note that this is native anatomical space. sub-x_desc-preproc_T1w.json sub-x_space-MNI152NLin6Asym_desc-brain_mask.nii.gz sub-x_space-MNI152NLin6Asym_desc-brain_mask.json sub-x_from-MNI152NLin6Asym_to-T1w_mode-image_xfm.h5 sub-x_from-T1w_to-MNI152NLin6Asym_mode-image_xfm.h5 func/ sub-x_task-rest_desc-confounds_timeseries.tsv sub-x_task-rest_desc-confounds_timeseries.json sub-x_task-rest_space-fsLR_den-91k_bold.dtseries.nii sub-x_task-rest_space-fsLR_den-91k_bold.json sub-x_task-rest_space-MNI152NLin6Asym_desc-preproc_bold.nii.gz # Needed for QC figures sub-x_task-rest_space-MNI152NLin6Asym_boldref.nii.gz sub-x_task-rest_space-MNI152NLin6Asym_boldref.json Surface files: .. code-block:: dataset_description.json sub-x/ anat/ # Mesh files in fsnative space, to be warped to fsLR space sub-x_hemi-L_pial.surf.gii sub-x_hemi-R_pial.surf.gii sub-x_hemi-L_white.surf.gii sub-x_hemi-R_white.surf.gii # Sphere files for registration sub-x_hemi-L_space-fsaverage_desc-reg_sphere.surf.gii sub-x_hemi-R_space-fsaverage_desc-reg_sphere.surf.gii # Morphometry files in fsLR space, to be parcellated sub-x_hemi-L_space-fsLR_den-91k_curv.dscalar.nii sub-x_hemi-L_space-fsLR_den-91k_sulc.dscalar.nii sub-x_hemi-L_space-fsLR_den-91k_thickness.dscalar.nii sub-x_hemi-L_space-fsLR_den-91k_myelinw.dscalar.nii sub-x_hemi-L_space-fsLR_den-91k_desc-smoothed_myelinw.dscalar.nii .. _filter_files: *************************************** Filtering Inputs with BIDS Filter Files *************************************** *XCP-D* allows users to choose which preprocessed files will be post-processed with the ``--bids-filter-file`` parameter. This argument must point to a JSON file, containing filters that will be fed into PyBIDS. The keys in this JSON file are unique to *XCP-D*. They are our internal terms for different inputs that will be selected from the preprocessed dataset. The full list of keys can be found in the file ``xcp_d/data/io_spec.yaml``. ``"bold"`` determines which preprocessed BOLD files will be chosen. You can set a number of entities here, including "session", "task", "space", "resolution", and "density". We recommend NOT setting the datatype, suffix, or file extension in the filter file. .. warning:: We do not recommend applying additional filters to any of the following fields. We have documented them here, for edge cases where they might be useful, but the only field that most users should filter is ``"bold"``. ``"t1w"`` selects a native T1w-space, preprocessed T1w file. ``"t2w"`` selects a native T1w-space, preprocessed T2w file. If a T1w file is not available, this file will be in T2w space. ``"anat_brainmask"`` selects an anatomically-derived brain mask in the same space as the BOLD data. This file is used (1) to estimate head radius for FD calculation (after warping to native space) and (2) to calculate coregistration quality metrics. ``"anat_to_template_xfm"`` selects a transform from T1w (or T2w, if no T1w image is available) space to standard space. The standard space that will be used depends on the ``"bold"`` files that are selected. ``"template_to_anat_xfm"`` selects a transform from standard space to T1w/T2w space. Again, the standard space is determined based on other files. There are additional keys that control how mesh and morphometry files are selected. Please refer to ``io_spec.yaml`` for more information on them. ``atlas`` selects atlases for parcellation. This is primarily useful for specifying spaces or resolutions. This field is not reflected in the ``io_spec.yaml`` file. Example bids-filter-file ======================== In this example file, we only run *XCP-D* on resting-state preprocessed BOLD runs from session "01". .. code-block:: json { "bold": { "session": ["01"], "task": ["rest"] } } ****************************** Running *XCP-D* via containers ****************************** .. _run_docker: Docker ====== If you are running *XCP-D* locally, we recommend Docker. See :ref:`installation_container_technologies` for installation instructions. In order to run Docker smoothly, it is best to prevent permissions issues associated with the root file system. Running Docker as user on the host will ensure the ownership of files written during the container execution. A Docker container can be created using the following command: .. code-block:: bash docker run --rm -it \ -v /dset/derivatives/fmriprep:/fmriprep:ro \ -v /tmp/wkdir:/work:rw \ -v /dset/derivatives/xcp_d:/out:rw \ pennlinc/xcp_d:latest \ /fmriprep /out participant \ --file-format cifti --despike --head_radius 40 -w /work --smoothing 6 .. _run_apptainer: Apptainer ========= If you are computing on an :abbr:`HPC (High-Performance Computing)`, we recommend using Apptainer. See :ref:`installation_container_technologies` for installation instructions. If the data to be preprocessed is also on the HPC or a personal computer, you are ready to run *XCP-D*. .. code-block:: bash apptainer run --cleanenv xcp_d.sif \ /dset/derivatives/fmriprep \ /dset/derivatives/xcp_d \ --participant-label label Relevant aspects of the ``$HOME`` directory within the container ================================================================ By default, Apptainer will bind the user's ``$HOME`` directory on the host into the ``/home/$USER`` directory (or equivalent) in the container. Most of the time, it will also redefine the ``$HOME`` environment variable and update it to point to the corresponding mount point in ``/home/$USER``. However, these defaults can be overwritten in your system. It is recommended that you check your settings with your system's administrator. If your Apptainer installation allows it, you can work around the ``$HOME`` specification, combining the bind mounts argument (``-B``) with the home overwrite argument (``--home``) as follows: .. code-block:: bash apptainer run -B $HOME:/home/xcp \ --home /home/xcp \ --cleanenv xcp_d.simg \ Therefore, once a user specifies the container options and the image to be run, the command line options are the same as the *bare-metal* installation. **************** Custom Confounds **************** If you would like to denoise your data with confounds that are not included in the built-in confound strategies, you can create your own. Please see :ref:`confound_config` for more info on the configuration file format. If you have a set of custom confounds that you would like to use, you need to organize them into a BIDS dataset. The dataset should only require the actual confound files, preferably with associated sidecar JSONs, and a ``dataset_description.json`` file. Using the same filenames as the fMRIPrep confound files is an easy way to organize this dataset. Including Signal Regressors =========================== .. warning:: Signal regressors are not currently supported in combination with voxel-wise regressors. .. warning:: Please be careful when including signal regressors. Most of the time this is probably a bad idea. For example, if you run tedana, only noise components from tedana should be orthogonalized with respect to signal components. You wouldn't want to orthogonalize other noise signals (e.g., motion parameters) with respect to the signal components from tedana. Let's say you have some nuisance regressors that are not necessarily orthogonal to some associated regressors that are ostensibly signal. For example, if you want to denoise your data while still retaining task signal, you could include the predicted task signals in your confounds. For more information about different types of denoising, see `tedana's documentation `_, `this NeuroStars topic `_, and/or `Pruim et al. (2015) `_. So how do we implement this in *XCP-D*? In order to define regressors that should be treated as signal, and thus orthogonalize the noise regressors with respect to known signals instead of regressing them without modification, you should include those regressors in your custom confounds file, with column names starting with ``signal__`` (lower-case "signal", followed by two underscores). Task Regression =============== If you want to regress task-related signals out of your data, you can use a custom confound configuration. Here we document how to include task effects as confounds. .. tip:: The basic approach to task regression is to convolve your task regressors with an HRF, then save those regressors to a confounds file. .. warning:: This method is still under development. We recommend using a tool like Nilearn to generate convolved regressors from BIDS events files. See `this example `_. .. code-block:: python import json import os import numpy as np import pandas as pd from nilearn.glm.first_level import make_first_level_design_matrix N_VOLUMES = 200 TR = 0.8 frame_times = np.arange(N_VOLUMES) * TR events_df = pd.read_table("sub-X_task-Z_events.tsv") task_confounds = make_first_level_design_matrix( frame_times, events_df, drift_model=None, add_regs=None, hrf_model="spm", ) # The design matrix will include a constant column, which we should drop task_confounds = task_confounds.drop(columns="constant") # Prepare the derivative dataset os.makedirs("/my/project/directory/custom_confounds/sub-X/func", exist_ok=True) # Include a dataset_description.json file with open("/my/project/directory/custom_confounds/dataset_description.json", "w") as fo: json.dump( { "Name": "Custom Confounds", "BIDSVersion": "1.6.0", "DatasetType": "derivative" }, fo, ) # Assuming that the fMRIPrep confounds file is named # "sub-X_task-Z_desc-confounds_timeseries.tsv", # we will name the custom confounds file the same thing, in a separate folder. task_confounds.to_csv( "/my/project/directory/custom_confounds/sub-X/func/sub-X_task-Z_desc-confounds_timeseries.tsv", sep="\t", index=False, ) Then, create a confounds config file to include derivatives from ``custom_confounds``. Something like this should work: .. code-block:: yaml name: my_custom_confounds description: | Nuisance regressors were task regressors convolved with an HRF and motion parameters. confounds: motion: dataset: preprocessed query: space: null cohort: null res: null den: null desc: confounds extension: .tsv suffix: timeseries columns: - trans_x - trans_y - trans_z - rot_x - rot_y - rot_z task: dataset: custom query: space: null cohort: null res: null den: null desc: confounds extension: .tsv suffix: timeseries columns: # Assume the task regressors are called "condition1" and "condition2" - condition1 - condition2 Command Line *XCP-D* with Custom Confounds ========================================== Last, run *XCP-D* with your custom configuration file and the path to the custom derivatives dataset. .. code-block:: bash apptainer run --cleanenv -B /my/project/directory:/mnt xcpd_latest.sif \ /mnt/input/fmriprep \ /mnt/output/directory \ participant \ --participant_label X \ --datasets custom=/mnt/custom_confounds \ --nuisance-regressors /mnt/custom_config.yaml **************** External Atlases **************** While *XCP-D* comes with many built-in parcellations, we understand that many users will want to use different ones. As long as the parcellation is organized in a BIDS-Atlas dataset and is in fsLR-32k space (for CIFTI processing) or MNIInfant, MNI152NLin6Asym, or MNI152NLin2009cAsym space (for NIfTI processing), you can use it with *XCP-D*. .. warning:: BIDS Extension Proposal 38 (Atlas Specification) has not been integrated in BIDS yet, so the organization and naming for atlas datasets may change in the future. We have attempted to follow the proposed structure in *XCP-D*, but we cannot guarantee that this will not change. .. tip:: The main elements from the BIDS-Atlas dataset that *XCP-D* uses are: 1. There must be a dataset_description.json file with DatasetType set to "atlas". 2. The atlas metadata files must have the same entities as the atlas image files, as PyBIDS does not support the inheritance principle when querying BIDS-Atlas datasets (yet). 3. There must be a TSV file for the atlas, with "index" and "label" columns. To do this, use the ``--datasets`` and ``--atlases`` parameters. The ``--datasets`` parameter should point to the directory containing the BIDS-Atlas dataset, and the ``--atlases`` parameter should include the names of the atlases in the dataset to use. For example, consider a scenario where you have two BIDS-Atlas datasets, one containing all of the Schaefer 2018 resolutions and one containing the AAL atlas. These datasets are in ``/data/atlases/schaefer`` and ``/data/atlases/aal``, respectively. The file structure for these two datasets might look like this: .. code-block:: /data/atlases/ schaefer/ dataset_description.json atlas-Schaefer100/ atlas-Schaefer100_dseg.tsv atlas-Schaefer100_space-fsLR_den-32k_dseg.dlabel.nii atlas-Schaefer100_space-fsLR_den-32k_dseg.json atlas-Schaefer200/ atlas-Schaefer200_dseg.tsv atlas-Schaefer200_space-fsLR_den-32k_dseg.dlabel.nii atlas-Schaefer200_space-fsLR_den-32k_dseg.json ... atlas-Schaefer1000/ atlas-Schaefer1000_dseg.tsv atlas-Schaefer1000_space-fsLR_den-32k_dseg.dlabel.nii atlas-Schaefer1000_space-fsLR_den-32k_dseg.json aal/ dataset_description.json atlas-AAL/ atlas-AAL_dseg.tsv atlas-AAL_space-fsLR_den-32k_dseg.dlabel.nii atlas-AAL_space-fsLR_den-32k_dseg.json You may want to only apply the Schaefer100 atlas from the ``schaefer`` dataset and the AAL atlas from the ``aal`` dataset, along with one of *XCP-D*'s built-in atlases (``4S156Parcels``). Here's what the *XCP-D* call might look like: .. code-block:: bash apptainer run --cleanenv -B /data:/data xcpd_latest.sif \ /data/dataset/derivatives/fmriprep \ /data/dataset/derivatives/xcp_d \ participant \ --mode linc \ --datasets schaefer=/data/atlases/schaefer aal==/data/atlases/aal \ --atlases Schaefer100 AAL 4S156Parcels *XCP-D* will search for ``atlas-Schaefer100``, ``atlas-AAL``, and ``atlas-4S156Parcels`` across the ``schaefer``, ``aal``, and *XCP-D*'s built-in atlas datasets. If the atlases are found, then they will be used for parcellation. .. important:: Atlas names must be unique across BIDS-Atlas datasets. If two atlases have the same name, *XCP-D* will raise an error. ********************* Advanced Applications ********************* *XCP-D* can be used in conjunction with other tools, such as ``tedana`` and ``phys2denoise``. We have attempted to document these applications with working code in `PennLINC/xcp_d-examples `_. If there is an application you think would be useful to document, please open an issue in that repository. ************************************** Preprocessing Requirements for *XCP-D* ************************************** *XCP-D* is designed to ingest data from a variety of different preprocessing pipelines. However, each supported pipeline must be explicitly supported within *XCP-D* in order for the workflow to select the correct files. Additionally, *XCP-D* may require files that are only created with specific settings in the preprocessing pipelines. fMRIPrep/Nibabies ================= In order to work on fMRIPrep or Nibabies derivatives, *XCP-D* needs derivatives in one of a few template spaces, including "MNI152NLin6Asym", "MNI152NLin2009cAsym", "MNIInfant", and "fsLR". We may add support for additional templates in the future, but currently you must have at least one of these among your output spaces. *XCP-D* does not have any specific requirements for resolution of volumetric derivatives, but we do require fsLR-space CIFTIs be outputted in 91k density. *************** Troubleshooting *************** Logs and crashfiles are outputted into the ``/xcp_d/sub-/log`` directory. Information on how to customize and understand these files can be found on the `nipype debugging `_ page. ************************* Support and communication ************************* All bugs, concerns and enhancement requests for this software can be submitted here: https://github.com/PennLINC/xcp_d/issues. If you have a question about using *XCP-D*, please create a new topic on `NeuroStars `_ with `the "Software Support" category and the "xcp_d" tag `_. The *XCP-D* developers follow NeuroStars, and will be able to answer your question there.