xcp_d.config module
A Python module to maintain unique, run-wide XCP-D settings.
This module implements the memory structures to keep a consistent, singleton config.
Settings are passed across processes via filesystem, and a copy of the settings for
each run and subject is left under
<xcp_d_dir>/sub-<participant_id>/log/<run_unique_id>/xcp_d.toml
.
Settings are stored using ToML.
The module has a to_filename()
function to allow writing out
the settings to hard disk in ToML format, which looks like:
This config file is used to pass the settings across processes,
using the load()
function.
Configuration sections
- class xcp_d.config.environment[source]
Read-only options regarding the platform and environment.
Crawls runtime descriptive settings (e.g., default FreeSurfer license, execution environment, nipype and XCP-D versions, etc.). The
environment
section is not loaded in from file, only written out when settings are exported. This config section is useful when reporting issues, and these variables are tracked whenever the user does not opt-out using the--notrack
argument.- cpu_count = 2
Number of available CPUs.
- exec_docker_version = None
Version of Docker Engine.
- exec_env = 'posix'
A string representing the execution platform.
- free_mem = 6.2
Free memory at start.
- nipype_version = '1.8.6'
Nipype’s current version.
- overcommit_limit = '50%'
Linux’s kernel virtual memory overcommit limits.
- overcommit_policy = 'heuristic'
Linux’s kernel virtual memory overcommit policy.
- templateflow_version = '24.2.0'
The TemplateFlow client version installed.
- version = '0.7.4.dev4+g2160bea'
XCP-D’s version.
- class xcp_d.config.execution[source]
Configure run-level settings.
- atlases = []
Selection of atlases to apply to the data.
- bids_database_dir = None
Path to the directory containing SQLite database indices for the input BIDS dataset.
- bids_description_hash = None
Checksum (SHA256) of the
dataset_description.json
of the BIDS dataset.
- bids_filters = None
A dictionary of BIDS selection filters.
- boilerplate_only = False
Only generate a boilerplate.
- custom_confounds = None
A path to a folder containing custom confounds to include in the postprocessing.
- debug = []
Debug mode(s).
- fmri_dir = None
An existing path to the preprocessing derivatives dataset, which must be BIDS-compliant.
- fs_license_file = None
An existing file containing a FreeSurfer license.
- log_dir = None
The path to a directory that contains execution logs.
- log_level = 25
Output verbosity.
- low_mem = None
Utilize uncompressed NIfTIs and other tricks to minimize memory allocation.
- md_only_boilerplate = False
Do not convert boilerplate from MarkDown to LaTex and HTML.
- notrack = False
Do not collect telemetry information for XCP-D.
- output_dir = None
Folder where derivatives will be stored.
- participant_label = None
List of participant identifiers that are to be preprocessed.
- reports_only = False
Only build the reports, based on the reportlets found in a cached working directory.
- run_uuid = '20240513-160406_19ddd88a-12c6-4e64-b87e-4aff1cab830b'
Unique identifier of this particular run.
- task_id = None
Select a particular task from all available in the dataset.
- templateflow_home = PosixPath('/home/docs/.cache/templateflow')[source]
The root folder of the TemplateFlow client.
- work_dir = PosixPath('/home/docs/checkouts/readthedocs.org/user_builds/xcp-d/checkouts/latest/docs/work')[source]
Path to a working directory where intermediate results will be available.
- write_graph = False
Write out the computational graph corresponding to the planned preprocessing.
- xcp_d_dir = None
Root of XCP-D BIDS Derivatives dataset.
- class xcp_d.config.workflow[source]
Configure the particular execution graph of this workflow.
- band_stop_max = None
High cutoff frequency for the band-stop filter.
- band_stop_min = None
Low cutoff frequency for the band-stop filter.
- bandpass_filter = True
Apply a band-pass filter to the data.
- bpf_order = 2
Order of the band-pass filter.
- cifti = False
Postprocess CIFTI inputs instead of NIfTIs.
- combineruns = False
Combine runs of the same task.
- dcan_qc = True
Run DCAN QC.
- despike = False
Despike the BOLD data before postprocessing.
- dummy_scans = 0
Number of label-control volume pairs to delete before CBF computation.
- exact_time = []
Produce correlation matrices limited to each requested amount of time.
- fd_thresh = 0.3
Framewise displacement threshold for censoring.
- head_radius = 50
Radius of the head in mm.
- high_pass = 0.01
Lower bound of the band-pass filter.
- input_type = 'fmriprep'
Postprocessing pipeline type.
- low_pass = 0.1
Upper bound of the band-pass filter.
- min_coverage = 0.5
Coverage threshold to apply to parcels in each atlas.
- min_time = 240
Post-scrubbing threshold to apply to individual runs in the dataset.
- motion_filter_order = 4
Order of the filter to apply to the motion regressors.
- motion_filter_type = None
Type of filter to apply to the motion regressors.
- params = '36P'
Nuisance regressors to include in the postprocessing.
- process_surfaces = False
Warp FreeSurfer’s surfaces to the MNI space.
- smoothing = 6
Full-width at half-maximum (FWHM) of the smoothing kernel.
- class xcp_d.config.nipype[source]
Nipype settings.
- crashfile_format = 'txt'
The file format for crashfiles, either text or pickle.
- get_linked_libs = False
Run NiPype’s tool to enlist linked libraries for every interface.
- memory_gb = None
Estimation in GB of the RAM this workflow can allocate at any given time.
- nprocs = 2
Number of processes (compute tasks) that can be run in parallel (multiprocessing only).
- omp_nthreads = None
Number of CPUs a single process can access for multithreaded execution.
- plugin = 'MultiProc'
NiPype’s execution plugin.
- plugin_args = {'maxtasksperchild': 1, 'raise_insufficient': False}
Settings for NiPype’s execution plugin.
- resource_monitor = False
Enable resource monitor.
- stop_on_first_crash = True
Whether the workflow should stop or continue after the first error.
Usage
A config file is used to pass settings and collect information as the execution graph is built across processes.
from xcp_d import config
config_file = config.execution.work_dir / '.xcp_d.toml'
config.to_filename(config_file)
# Call build_workflow(config_file, retval) in a subprocess
with Manager() as mgr:
from xcp_d.cli.workflow import build_workflow
retval = mgr.dict()
p = Process(target=build_workflow, args=(str(config_file), retval))
p.start()
p.join()
config.load(config_file)
# Access configs from any code section as:
value = config.section.setting
Logging
Other responsibilities
The config
is responsible for other conveniency actions.
Switching Python’s
multiprocessing
to forkserver mode.Set up a filter for warnings as early as possible.
Automated I/O magic operations. Some conversions need to happen in the store/load processes (e.g., from/to
Path
<->str
,BIDSLayout
, etc.)
- class xcp_d.config.environment[source]
Bases:
_Config
Read-only options regarding the platform and environment.
Crawls runtime descriptive settings (e.g., default FreeSurfer license, execution environment, nipype and XCP-D versions, etc.). The
environment
section is not loaded in from file, only written out when settings are exported. This config section is useful when reporting issues, and these variables are tracked whenever the user does not opt-out using the--notrack
argument.- cpu_count = 2
Number of available CPUs.
- exec_docker_version = None
Version of Docker Engine.
- exec_env = 'posix'
A string representing the execution platform.
- free_mem = 6.2
Free memory at start.
- nipype_version = '1.8.6'
Nipype’s current version.
- overcommit_limit = '50%'
Linux’s kernel virtual memory overcommit limits.
- overcommit_policy = 'heuristic'
Linux’s kernel virtual memory overcommit policy.
- templateflow_version = '24.2.0'
The TemplateFlow client version installed.
- version = '0.7.4.dev4+g2160bea'
XCP-D’s version.
- class xcp_d.config.execution[source]
Bases:
_Config
Configure run-level settings.
- atlases = []
Selection of atlases to apply to the data.
- bids_database_dir = None
Path to the directory containing SQLite database indices for the input BIDS dataset.
- bids_description_hash = None
Checksum (SHA256) of the
dataset_description.json
of the BIDS dataset.
- bids_filters = None
A dictionary of BIDS selection filters.
- boilerplate_only = False
Only generate a boilerplate.
- custom_confounds = None
A path to a folder containing custom confounds to include in the postprocessing.
- debug = []
Debug mode(s).
- fmri_dir = None
An existing path to the preprocessing derivatives dataset, which must be BIDS-compliant.
- fs_license_file = None
An existing file containing a FreeSurfer license.
- log_dir = None
The path to a directory that contains execution logs.
- log_level = 25
Output verbosity.
- low_mem = None
Utilize uncompressed NIfTIs and other tricks to minimize memory allocation.
- md_only_boilerplate = False
Do not convert boilerplate from MarkDown to LaTex and HTML.
- notrack = False
Do not collect telemetry information for XCP-D.
- output_dir = None
Folder where derivatives will be stored.
- participant_label = None
List of participant identifiers that are to be preprocessed.
- reports_only = False
Only build the reports, based on the reportlets found in a cached working directory.
- run_uuid = '20240513-160406_19ddd88a-12c6-4e64-b87e-4aff1cab830b'
Unique identifier of this particular run.
- task_id = None
Select a particular task from all available in the dataset.
- templateflow_home = PosixPath('/home/docs/.cache/templateflow')[source]
The root folder of the TemplateFlow client.
- work_dir = PosixPath('/home/docs/checkouts/readthedocs.org/user_builds/xcp-d/checkouts/latest/docs/work')[source]
Path to a working directory where intermediate results will be available.
- write_graph = False
Write out the computational graph corresponding to the planned preprocessing.
- xcp_d_dir = None
Root of XCP-D BIDS Derivatives dataset.
- xcp_d.config.from_dict(settings, init=True, ignore=None)[source]
Read settings from a flat dictionary.
- xcp_d.config.load(filename, skip=None, init=True)[source]
Load settings from file.
- Parameters:
filename (
os.PathLike
) – TOML file containing XCP-D configuration.skip (dict or None) – Sets of values to ignore during load, keyed by section name
init (bool or
Container
) – Initialize all, none, or a subset of configurations.
- class xcp_d.config.loggers[source]
Bases:
object
Keep loggers easily accessible (see
init()
).
- class xcp_d.config.nipype[source]
Bases:
_Config
Nipype settings.
- crashfile_format = 'txt'
The file format for crashfiles, either text or pickle.
- get_linked_libs = False
Run NiPype’s tool to enlist linked libraries for every interface.
- memory_gb = None
Estimation in GB of the RAM this workflow can allocate at any given time.
- nprocs = 2
Number of processes (compute tasks) that can be run in parallel (multiprocessing only).
- omp_nthreads = None
Number of CPUs a single process can access for multithreaded execution.
- plugin = 'MultiProc'
NiPype’s execution plugin.
- plugin_args = {'maxtasksperchild': 1, 'raise_insufficient': False}
Settings for NiPype’s execution plugin.
- resource_monitor = False
Enable resource monitor.
- stop_on_first_crash = True
Whether the workflow should stop or continue after the first error.
- class xcp_d.config.seeds[source]
Bases:
_Config
Initialize the PRNG and track random seed assignments.
- master = None
Master random seed to initialize the Pseudorandom Number Generator (PRNG).
- class xcp_d.config.workflow[source]
Bases:
_Config
Configure the particular execution graph of this workflow.
- band_stop_max = None
High cutoff frequency for the band-stop filter.
- band_stop_min = None
Low cutoff frequency for the band-stop filter.
- bandpass_filter = True
Apply a band-pass filter to the data.
- bpf_order = 2
Order of the band-pass filter.
- cifti = False
Postprocess CIFTI inputs instead of NIfTIs.
- combineruns = False
Combine runs of the same task.
- dcan_qc = True
Run DCAN QC.
- despike = False
Despike the BOLD data before postprocessing.
- dummy_scans = 0
Number of label-control volume pairs to delete before CBF computation.
- exact_time = []
Produce correlation matrices limited to each requested amount of time.
- fd_thresh = 0.3
Framewise displacement threshold for censoring.
- head_radius = 50
Radius of the head in mm.
- high_pass = 0.01
Lower bound of the band-pass filter.
- input_type = 'fmriprep'
Postprocessing pipeline type.
- low_pass = 0.1
Upper bound of the band-pass filter.
- min_coverage = 0.5
Coverage threshold to apply to parcels in each atlas.
- min_time = 240
Post-scrubbing threshold to apply to individual runs in the dataset.
- motion_filter_order = 4
Order of the filter to apply to the motion regressors.
- motion_filter_type = None
Type of filter to apply to the motion regressors.
- params = '36P'
Nuisance regressors to include in the postprocessing.
- process_surfaces = False
Warp FreeSurfer’s surfaces to the MNI space.
- smoothing = 6
Full-width at half-maximum (FWHM) of the smoothing kernel.