pipeline¶

`pipeline` Package¶

This pipeline module includes the following workflows:

qipipe.pipeline.qipipeline: the soup-to-nuts pipeline to stage, mask, register and model new images
qipipe.pipeline.staging: executes the staging workflow to detect new images, group them by volume, import them into XNAT and prepare them for TCIA import
qipipe.pipeline.mask: creates a mask to subtract extraneous tissue from the input images
qipipe.pipeline.registration: masks the target tissue and corrects motion artifacts
qipipe.pipeline.modeling: performs pharmokinetic modeling

`mask`¶

class qipipe.pipeline.mask.MaskWorkflow(**opts)¶

Bases: qipipe.pipeline.workflow_base.WorkflowBase

The MaskWorkflow class builds and executes the mask workflow.

The workflow creates a mask to subtract extraneous tissue for a given input session 4D NIfTI time series. The new mask is uploaded to XNAT as a session resource named mask.

The mask workflow input is the input_spec node consisting of the following input fields:

subject: the XNAT subject name

session: the XNAT session name

scan: the XNAT scan number

time_series: the 4D NIfTI series image file

The mask workflow output is the output_spec node consisting of the following output field:

mask: the mask file

The optional workflow configuration file can contain the following sections:

fsl.MriVolCluster: the

qipipe.interfaces.mri_volcluster.MriVolCluster interface options

If the optional configuration file is specified, then the workflow settings in that file override the default settings.

Option crop_posterior:
Parameters:	opts – the `qipipe.pipeline.workflow_base.WorkflowBase` initializer keyword arguments, as well as the following keyword arguments:
	crop posterior to the center of gravity, e.g. for a breast tumor

__init__(**opts)¶

If the optional configuration file is specified, then the workflow settings in that file override the default settings.

Option crop_posterior:
Parameters:	opts – the `qipipe.pipeline.workflow_base.WorkflowBase` initializer keyword arguments, as well as the following keyword arguments:
	crop posterior to the center of gravity, e.g. for a breast tumor

run(subject, session, scan, time_series)¶

Runs the mask workflow on the scan NIfTI files for the given time series.

Parameters:	subject – the input subject session – the input session scan – the input scan number time_series – the input 3D NIfTI time series to mask
Returns:	the mask file location

workflow = None¶: The mask creation workflow.

qipipe.pipeline.mask.run(subject, session, scan, time_series, **opts)¶

Creates a qipipe.pipeline.mask.MaskWorkflow and runs it on the given inputs.

Parameters:	subject – the input subject session – the input session scan – the input scan number time_series – the input 4D NIfTI time series to mask opts – additional `MaskWorkflow` initialization parameters
Returns:	the mask file location

`modeling`¶

qipipe.pipeline.modeling.FASTFIT_CONF_PROPS = ['model_name', 'optimization_params', 'optional_outs']¶: The Fastfit configuration property names.

qipipe.pipeline.modeling.FASTFIT_PARAMS_FILE = 'params.csv'¶: The Fastfit parameters CSV file name.

qipipe.pipeline.modeling.FXL_MODEL_PREFIX = 'ext_tofts.'¶: The Fastfit Standard TOFTS model prefix.

qipipe.pipeline.modeling.MODELING_CONF_FILE = 'modeling.cfg'¶: The modeling workflow configuration.

qipipe.pipeline.modeling.MODELING_PREFIX = 'pk_'¶: The modeling XNAT object label prefix.

class qipipe.pipeline.modeling.ModelingWorkflow(**opts)¶

Bases: qipipe.pipeline.workflow_base.WorkflowBase

The ModelingWorkflow builds and executes the Nipype pharmacokinetic mapping workflow.

The workflow calculates the modeling parameters for an input 4D time series NIfTI image file as follows:

Compute the R1₀ value, if it is not given in the options
Convert the DCE time series to a R1 map series
Determine the AIF and R1 fit parameters from the time series
Optimize the OHSU pharmacokinetic model
Upload the modeling result to XNAT

The modeling workflow input is the input_spec node consisting of the following input fields:

subject: the subject name
session: the session name
mask: the mask to apply to the images
time_series: the 4D time series NIfTI file to model
bolus_arrival_index: the bolus uptake volume index
the R1 modeling parameters described below

If an input field is defined in the configuration file R1 section, then the input field is set to that value.

If the R1₀ option is not set, then it is computed from the proton density weighted scans and DCE series baseline image.

The outputs are collected in the output_spec node for the FXL (Tofts standard) model and the FXR (shutter speed) model with the following fields:

r1_series: the R1 series files
pk_params: the AIF and R1 parameter CSV file
fxr_k_trans, fxl_k_trans: the K^trans vascular permeability

transfer constant
delta_k_trans: the FXR-FXL K^trans difference
fxr_v_e, fxl_v_e: the v_e extravascular extracellular volume

fraction
fxr_tau_i: the τ_i intracellular H₂O mean lifetime
fxr_chi_sq, fxl_chi_sq: the χ² intensity goodness of fit

In addition, if R1₀ is computed, then the output includes the following fields:

pdw_file: the proton density weighted image
dce_baseline: the DCE series baseline image
r1_0: the computed R1₀ value

This workflow is adapted from the AIRC DCE implementation.

Note

This workflow uses proprietary OHSU AIRC software, notably the OHSU implementation of the shutter speed model.

The modeling parameters can be defined in either the options or the configuration as follows:

The parameters can be defined in the configuration R1 section.
The keyword arguments take precedence over the configuration settings.
The r1_0_val takes precedence over the R1_0 computation fields pd_dir and max_r1_0. If r1_0_val is set in the input options, then pd_dir and max_r1_0 are not included from the result.
If pd_dir and max_r1_0 are set in the input options and r1_0_val is not set in the input options, then a r1_0_val configuration setting is ignored.
The base_end defaults to 1 if it is not set in either the input options or the configuration.

Parameters:

opts – the qipipe.pipeline.workflow_base.WorkflowBase initializer keyword arguments, as well as the following keyword arguments:
r1_0_val – the optional fixed R1₀ value
max_r1_0 – the maximum computed R1₀ value, if the fixed R1₀ option is not set
pd_dir – the proton density files parent directory, if the fixed R1₀ option is not set
base_end – the number of volumes to merge into a R1 series baseline image (default is 1)

__init__(**opts)¶

The modeling parameters can be defined in either the options or the configuration as follows:

The parameters can be defined in the configuration R1 section.
The keyword arguments take precedence over the configuration settings.
The r1_0_val takes precedence over the R1_0 computation fields pd_dir and max_r1_0. If r1_0_val is set in the input options, then pd_dir and max_r1_0 are not included from the result.
If pd_dir and max_r1_0 are set in the input options and r1_0_val is not set in the input options, then a r1_0_val configuration setting is ignored.
The base_end defaults to 1 if it is not set in either the input options or the configuration.

Parameters:

opts – the qipipe.pipeline.workflow_base.WorkflowBase initializer keyword arguments, as well as the following keyword arguments:
r1_0_val – the optional fixed R1₀ value
max_r1_0 – the maximum computed R1₀ value, if the fixed R1₀ option is not set
pd_dir – the proton density files parent directory, if the fixed R1₀ option is not set
base_end – the number of volumes to merge into a R1 series baseline image (default is 1)

resource = None¶: The XNAT resource name for all executions of this qipipe.pipeline.modeling.ModelingWorkflow instance. The name is unique, which permits more than one model to be stored for each input volume without a name conflict.

run(subject, session, scan, time_series, **opts)¶

Executes the modeling workflow described in qipipe.pipeline.modeling.ModelingWorkflow on the given input time series resource. The time series can be the merged scan NIFTI files or merged registration files.

This run method connects the given inputs to the modeling workflow inputs. The execution workflow is then executed, resulting in a new uploaded XNAT resource.

Option bolus_arrival_index:
Parameters:	subject – the subject name session – the session name scan – the scan number time_series – the 4D modeling input time series file location opts – the following keyword parameters:
	the bolus uptake volume index
Option mask:	the XNAT mask resource name
Returns:	the modeling result dictionary

technique = None¶: The modeling technique. Built-in techniques include mock.

workflow = None¶: The modeling workflow described in qipipe.pipeline.modeling.ModelingWorkflow.

qipipe.pipeline.modeling.OHSU_CONF_SECTIONS = ['Fastfit', 'R1', 'AIF']¶: The OHSU AIRC modeling configuration sections.

qipipe.pipeline.modeling.associate(names, values)¶

Captures the synchronized names and values in a dictionary.

Parameters:	names – the field names values – the field values
Returns:	the target {name: value} dictionary

qipipe.pipeline.modeling.create_profile(technique, time_series, configuration, sections, dest)¶

qipipe.helpers.metadata.create_profile() wrapper.

Parameters:	technique – the modeling technique time_series – the modeling input time series file path configuration – the modeling workflow interface settings sections – the profile sections dest – the output profile file path

qipipe.pipeline.modeling.get_aif_shift(time_series, bolus_arrival_index)¶

Calculates the arterial input function offset as:

t_arrival - t₀

where t₀ is the first slice acquisition time and t_arrival averages the acquisition times at and immediately following bolus arrival.

Parameters:	time_series – the modeling input 4D NIfTI image file path bolus_arrival_index – the bolus uptake series index
Returns:	the parameter CSV file path

qipipe.pipeline.modeling.get_fit_params(cfg_file, aif_shift)¶

Makes the CSV file containing the following modeling fit parameters:

aif_shift: arterial input function parameter array
aif_delta_t: acquisition time deltas
aif_shift: acquisition time shift
r1_cr: contrast R1
r1_b_pre: pre-contrast R1

The aif_shift is calculated by get_aif_shift() and passed to this function. The remaining parameters are read from the MODELING_CONF_FILE.

Parameters:	cfg_file – the modeling configuration file
Returns:	the parameter CSV file path

qipipe.pipeline.modeling.get_r1_0(pdw_file, t1w_file, max_r1_0, mask=None)¶

Returns the R1_0 map NIfTI file from the given proton density and T1-weighted images. The R1_0 map is computed using the pdw_t1w_to_r1 function. The pdw_t1w_to_r1 module must be in the Python path.

Parameters:	pdw_file – the proton density NIfTI image file path t1w_file – the T1-weighted image file path max_r1_0 – the R1_0 range maximum mask – the optional mask image file path to use
Returns:	the R1_0 map NIfTI image file path

qipipe.pipeline.modeling.make_baseline(time_series, base_end)¶

Makes the R1_0 computation baseline NIfTI file.

Parameters:	time_series – the modeling input 4D NIfTI image file path base_end – the exclusive limit of the baseline computation input series
Returns:	the baseline NIfTI file name
Raises:	ModelingError – if the end index is a negative number

qipipe.pipeline.modeling.run(subject, session, scan, time_series, **opts)¶

Creates a qipipe.pipeline.modeling.ModelingWorkflow and runs it on the given inputs.

Parameters:	subject – the input subject session – the input session scan – input scan time_series – the input 4D NIfTI time series opts – the `qipipe.pipeline.modeling.ModelingWorkflow` initializer and run options
Returns:	the `qipipe.pipeline.modeling.ModelingWorkflow.run()` result

`pipeline_error`¶

exception qipipe.pipeline.pipeline_error.PipelineError¶

Bases: exceptions.Exception

The common pipeline error class.

`qipipeline`¶

qipipe.pipeline.qipipeline.MULTI_VOLUME_ACTIONS = ['stage', 'roi', 'register', 'model']¶: The workflow actions which apply to a multi-volume scan.

class qipipe.pipeline.qipipeline.QIPipelineWorkflow(project, scan_input, actions, **opts)¶

Bases: qipipe.pipeline.workflow_base.WorkflowBase

QIPipeline builds and executes the imaging workflows. The pipeline builds a composite workflow which stitches together the following constituent workflows:

staging: Prepare the new DICOM visits, as described in qipipe.pipeline.staging.StagingWorkflow
mask: Create the mask from the staged images, as described in qipipe.pipeline.mask.MaskWorkflow
registration: Mask, register and realign the staged images, as described in qipipe.pipeline.registration.RegistrationWorkflow
modeling: Perform PK modeling as described in qipipe.pipeline.modeling.ModelingWorkflow

The constituent workflows are determined by the initialization options stage, register and model. The default is to perform each of these subworkflows.

The workflow steps are determined by the input options as follows:

If staging is enabled, then the DICOM files are staged for the subject directory inputs. Otherwise, staging is not performed. In that case, if registration is enabled as described below, then the previously staged volume scan stack images are downloaded.
If modeling is enabled and the registration resource option is set, then the previously realigned images with the given resource name are downloaded.
If registration or modeling is enabled and the XNAT mask resource is found, then that resource file is downloaded. Otherwise, the mask is created from the staged images.

The workflow input node is input_spec with the following fields:

subject: the subject name
session: the session name
scan: the scan number

The constituent workflows are combined as follows:

The staging workflow input is the workflow input.
The mask workflow input is the newly created or previously staged scan NIfTI image files.
The modeling workflow input is the combination of the previously uploaded and newly realigned image files.

The pipeline workflow is available as the qipipe.pipeline.qipipeline.QIPipelineWorkflow.workflow instance variable.

Parameters:

project – the XNAT project name
scan_input – the qipipe.staging.iterator.iter_stage() scan input
actions – the actions to perform
opts – the qipipe.staging.WorkflowBase initialization options as well as the following keyword arguments:
dest – the staging destination directory
collection – the image collection name
registration_resource – the XNAT registration resource name
registration_technique – the class:qipipe.pipeline.registration.RegistrationWorkflow technique
modeling_resource – the modeling resource name
modeling_technique – the class:qipipe.pipeline.modeling.ModelingWorkflow technique
scan_time_series – the scan time series resource name
realigned_time_series – the registered time series resource name

__init__(project, scan_input, actions, **opts)¶

Parameters:

project – the XNAT project name
scan_input – the qipipe.staging.iterator.iter_stage() scan input
actions – the actions to perform
opts – the qipipe.staging.WorkflowBase initialization options as well as the following keyword arguments:
dest – the staging destination directory
collection – the image collection name
registration_resource – the XNAT registration resource name
registration_technique – the class:qipipe.pipeline.registration.RegistrationWorkflow technique
modeling_resource – the modeling resource name
modeling_technique – the class:qipipe.pipeline.modeling.ModelingWorkflow technique
scan_time_series – the scan time series resource name
realigned_time_series – the registered time series resource name

modeling_resource = None¶: The modeling XNAT resource name.

modeling_technique = None¶: The modeling technique.

registration_resource = None¶: The registration resource name.

registration_technique = None¶: The registration technique.

run_with_dicom_input(actions, scan_input)¶

Parameters:	actions – the workflow actions to perform scan_input – the `qipipe.staging.iterator.iter_stage()` scan input dest – the TCIA staging destination directory (default is the current working directory)

run_with_scan_download(project, scan_input, actions)¶

Runs the execution workflow on downloaded scan image files.

Parameters:	project – the project name scan_input – the {project, subject, session} object actions – the workflow actions

workflow = None¶: The pipeline execution workflow. The execution workflow is executed by calling the run_with_dicom_input() or run_with_scan_download() method.

qipipe.pipeline.qipipeline.SINGLE_VOLUME_ACTIONS = ['stage']¶: The workflow actions which apply to a single-volume scan.

qipipe.pipeline.qipipeline.exclude_files(in_files, exclusions)¶

Parameters:	in_files – the input file paths exclusions – the file names to exclude
Returns:	the filtered input file paths

qipipe.pipeline.qipipeline.run(*inputs, **opts)¶

Creates a qipipe.pipeline.qipipeline.QIPipelineWorkflow and runs it on the given inputs. The pipeline execution depends on the actions option, as follows:

If the workflow actions includes stage or roi, then the input is the QIPipelineWorkflow.run_with_dicom_input() DICOM subject directories input.
Otherwise, the input is the QIPipelineWorkflow.run_with_scan_download() XNAT session labels input.

Parameters:	inputs – the input directories or XNAT session labels to process opts – the `qipipe.staging.iterator.iter_stage()` and `QIPipelineWorkflow` initializer options, as well as the following keyword options: project – the XNAT project name collection – the image collection name actions – the workflow actions to perform (default `MULTI_VOLUME_ACTIONS`)

`registration`¶

qipipe.pipeline.registration.ANTS_CONF_SECTIONS = ['ants.Registration']¶: The common ANTs registration configuration sections.

qipipe.pipeline.registration.ANTS_INITIALIZER_CONF_SECTION = 'ants.AffineInitializer'¶: The initializer ANTs registration configuration sections.

qipipe.pipeline.registration.FSL_CONF_SECTIONS = ['fsl.FLIRT', 'fsl.FNIRT']¶: The FSL registration configuration sections.

qipipe.pipeline.registration.REG_PREFIX = 'reg_'¶: The XNAT registration resource name prefix.

class qipipe.pipeline.registration.RegisterImageWorkflow(technique, **opts)¶

Bases: qipipe.pipeline.workflow_base.WorkflowBase

The RegisterImageWorkflow registers an input NIfTI scan image against a reference image.

Three registration techniques are supported:

ants: ANTS SyN symmetric normalization diffeomorphic registration (default)
fsl: FSL FNIRT non-linear registration
mock: Test technique which copies each input scan image to the output image file

The optional workflow configuration file can contain overrides for the Nipype interface inputs in the following sections:

AffineInitializer: the

qipipe.interfaces.ants.utils.AffineInitializer options
ants.Registration: the ANTs Registration interface options
ants.ApplyTransforms: the ANTs ApplyTransform interface options
fsl.FNIRT: the FSL FNIRT interface options

Note

Since the XNAT resource name is unique, a qipipe.pipeline.registration.RegisterScanWorkflow instance can be used for only one registration workflow. Different registration inputs require different qipipe.pipeline.registration.RegisterScanWorkflow instances.

If the optional configuration file is specified, then the workflow settings in that file override the default settings.

Parameters:	technique – the required registration `technique` opts – the `qipipe.pipeline.workflow_base.WorkflowBase` initializer options, as well as the following keyword arguments: initialize – flag indicating whether to create an initial affine transform (ANTs only, default false)

__init__(technique, **opts)¶

If the optional configuration file is specified, then the workflow settings in that file override the default settings.

Parameters:	technique – the required registration `technique` opts – the `qipipe.pipeline.workflow_base.WorkflowBase` initializer options, as well as the following keyword arguments: initialize – flag indicating whether to create an initial affine transform (ANTs only, default false)

run(in_file, reference, **opts)¶

Runs the realignment workflow on the given session scan image.

Parameters:	reference – the volume to register against in_file – the input session scan volume image file opts – the following keyword arguments:
Option mask:	the image mask file path
Returns:	the realigned output file paths

technique = None¶: The lower-case XNAT registration technique. The built-in techniques include ants, fnirt` and mock.

workflow = None¶: The realignment workflow.

class qipipe.pipeline.registration.RegisterScanWorkflow(reference, **opts)¶

Bases: qipipe.pipeline.workflow_base.WorkflowBase

The RegistrationWorkflow registers input NIfTI scan images against a reference image.

The mask can be obtained by running the qipipe.pipeline.mask.MaskWorkflow workflow.

Three registration techniques are supported:

ants: ANTS SyN symmetric normalization diffeomorphic registration (default)
fsl: FSL FNIRT non-linear registration
mock: Test technique which copies each input scan image to the output image file

The optional workflow configuration file can contain overrides for the Nipype interface inputs in the following sections:

AffineInitializer: the

qipipe.interfaces.ants.utils.AffineInitializer options
ants.Registration: the ANTs Registration interface options
ants.ApplyTransforms: the ANTs ApplyTransform interface options
fsl.FNIRT: the FSL FNIRT interface options

Note

Since the XNAT resource name is unique, a qipipe.pipeline.registration.RegisterScanWorkflow instance can be used for only one registration workflow. Different registration inputs require different qipipe.pipeline.registration.RegisterScanWorkflow instances.

If the optional configuration file is specified, then the workflow settings in that file override the default settings.

Parameters:	reference – the volume to register against opts – the `qipipe.pipeline.workflow_base.WorkflowBase` and `RegisterImageWorkflow` options, as well as the following keyword arguments: technique – the optional registration `technique` (default `DEF_TECHNIQUE`)

__init__(reference, **opts)¶

If the optional configuration file is specified, then the workflow settings in that file override the default settings.

Parameters:	reference – the volume to register against opts – the `qipipe.pipeline.workflow_base.WorkflowBase` and `RegisterImageWorkflow` options, as well as the following keyword arguments: technique – the optional registration `technique` (default `DEF_TECHNIQUE`)

resource = None¶: The unique XNAT registration resource name. Uniqueness permits more than one registration to be stored for a given session without a name conflict.

run(subject, session, scan, in_files, mask=None)¶

Runs the registration workflow on the given session scan images.

Parameters:	subject – the subject name session – the session name scan – the scan number in_files – the input session scan volume image files mask – the optional image mask file path
Returns:	the realigned 4D time series file path

technique = None¶: The registration technique (default DEF_TECHNIQUE).

workflow = None¶: The registration workflow.

qipipe.pipeline.registration.run(subject, session, scan, in_files, **opts)¶

Runs the registration workflow on the given session scan images.

Parameters:	subject – the subject name session – the session name scan – the scan number in_files – the input session scan 3D NIfTI images opts – the `RegisterScanWorkflow` initializer and `RegisterScanWorkflow.run()` options as well as the following keyword option: reference – the volume number of the image to register against (default is the first image)
Returns:	the 4D registration time series

`roi`¶

The proprietary OHSU mask conversion workflow.

class qipipe.pipeline.roi.ROIWorkflow(**kwargs)¶

Bases: qipipe.pipeline.workflow_base.WorkflowBase

The ROIWorkflow class builds and executes the ROI workflow which converts the BOLERO mask .bqf files to NIfTI.

The ROI workflow input consists of the input_spec and iter_slice nodes. The input_spec contains the following input fields:

subject: the subject name
session: the session name
scan: the scan number
time_series: the 4D time series file path
lesion: the lesion number

The iter_slice contains the following input fields:

slice_sequence_number: the one-based slice sequence number
in_file: the ROI mask``.bqf`` file to convert

The output is the 3D mask NIfTI file location. The file name is lesion.nii.gz.

If the optional configuration file is specified, then the workflow settings in that file override the default settings.

Parameters:	kwargs – the `qipipe.pipeline.workflow_base.WorkflowBase` initializer keyword arguments

__init__(**kwargs)¶

If the optional configuration file is specified, then the workflow settings in that file override the default settings.

Parameters:	kwargs – the `qipipe.pipeline.workflow_base.WorkflowBase` initializer keyword arguments

run(subject, session, scan, time_series, *inputs)¶

Runs the ROI workflow on the given session scan images.

Parameters:	subject – the subject name session – the session name scan – the scan number time_series – the 4D scan time series file path inputs – the input (lesion number, slice sequence number, in_file) tuples to convert
Returns:	the XNAT converted ROI resource name, or None if there were no inputs

workflow = None¶: The ROI workflow.

qipipe.pipeline.roi.ROI_FNAME_PAT = 'lesion%d'¶: The ROI file name pattern.

qipipe.pipeline.roi.ROI_RESOURCE = 'roi'¶: The XNAT ROI resource name.

qipipe.pipeline.roi.base_name(lesion)¶

Parameters:	lesion – the lesion number
Returns:	the base name to use

qipipe.pipeline.roi.run(subject, session, scan, time_series, *inputs, **opts)¶

Runs the ROI workflow on the given session ROI mask files.

Parameters:	subject – the subject name session – the session name scan – the scan number time_series – the 4D scan time series inputs – the `ROIWorkflow.run()` (lesion number, slice sequence number, in_file) inputs opts – the `ROIWorkflow` initializer options
Returns:	the `ROIWorkflow.run()` result

`staging`¶

qipipe.pipeline.staging.SCAN_CONF_FILE = 'scan.cfg'¶: The XNAT scan configuration file name.

qipipe.pipeline.staging.SCAN_METADATA_RESOURCE = 'metadata'¶: The label of the XNAT resource holding the scan configuration.

class qipipe.pipeline.staging.ScanStagingWorkflow(is_multi_volume=True, **opts)¶

Bases: qipipe.pipeline.workflow_base.WorkflowBase

The ScanStagingWorkflow class builds and executes the scan staging supervisory Nipype workflow. This workflow delegates to qipipe.pipeline.staging.stage_volume() for each iterated scan volume.

The scan staging workflow input is the input_spec node consisting of the following input fields:

collection: the collection name
subject: the subject name
session: the session name
scan: the scan number

The scan staging workflow has one iterable:

the iter_volume node with input fields volume and in_files

This iterable must be set prior to workflow execution.

The staging workflow output is the output_spec node consisting of the following output field:

out_file: the 3D volume stack NIfTI image file

Parameters:	is_multi_volume – flag indicating whether to include volume merge tasks opts – the `qipipe.pipeline.workflow_base.WorkflowBase` initializer keyword arguments

__init__(is_multi_volume=True, **opts)¶

Parameters:	is_multi_volume – flag indicating whether to include volume merge tasks opts – the `qipipe.pipeline.workflow_base.WorkflowBase` initializer keyword arguments

run(collection, subject, session, scan, vol_dcm_dict, dest)¶

Executes this scan staging workflow.

Parameters:	collection – the collection name subject – the subject name session – the session name scan – the scan number vol_dcm_dict – the input {volume: DICOM files} dictionary dest – the destination directory
Returns:	the (time series, volume files) tuple

workflow = None¶: The scan staging workflow sequence described in qipipe.pipeline.staging.StagingWorkflow.

class qipipe.pipeline.staging.VolumeStagingWorkflow(**opts)¶

Bases: qipipe.pipeline.workflow_base.WorkflowBase

The StagingWorkflow class builds and executes the staging Nipype workflow. The staging workflow includes the following steps:

Group the input DICOM images into volume.
Fix each input DICOM file header using the qipipe.interfaces.fix_dicom.FixDicom interface.
Compress each corrected DICOM file.
Upload each compressed DICOM file into XNAT.
Stack each new volume’s 2-D DICOM files into a 3-D volume NIfTI file using the DcmStack interface.
Upload each new volume stack into XNAT.
Make the CTP QIN-to-TCIA subject id map.
Collect the id map and the compressed DICOM images into a target directory in collection/subject/session/volume format for TCIA upload.

The staging workflow input is the input_spec node consisting of the following input fields:

collection: the collection name
subject: the subject name
session: the session name
scan: the scan number

The staging workflow has two iterables:

the iter_volume node with input fields volume and dest
the iter_dicom node with input fields volume and dicom_file

These iterables must be set prior to workflow execution. The iter_volume dest input is the destination directory for the iter_volume volume.

The iter_dicom node itersource is the iter_volume.volume field. The iter_dicom.dicom_file iterables is set to the {volume: [DICOM files]} dictionary.

The DICOM files to upload to TCIA are placed in the destination directory in the following hierarchy:

/path/to/dest/

subject/

session/

volumevolume number/

file ...

where:

subject is the subject name, e.g. Breast011
session is the session name, e.g. Session03
volume number is determined by the qipipe.staging.image_collection.Collection.patterns volume DICOM tag
file is the DICOM file name

The staging workflow output is the output_spec node consisting of the following output field:

image: the 3D volume stack NIfTI image file

Note

Concurrent XNAT upload fails unpredictably due to one of

the causes described in the qixnat.facade.XNAT.find method documentation.

The errors are addressed by the following measures:

setting an isolated pyxnat cache_dir for each execution node

serializing the XNAT find-or-create access points with ``JoinNode``s

increasing the SGE submission resource parameters as shown in the conf/staging.cfg [upload] section

If the optional configuration file is specified, then the workflow settings in that file override the default settings.

Parameters:	opts – the `qipipe.pipeline.workflow_base.WorkflowBase` initializer keyword arguments

__init__(**opts)¶

If the optional configuration file is specified, then the workflow settings in that file override the default settings.

Parameters:	opts – the `qipipe.pipeline.workflow_base.WorkflowBase` initializer keyword arguments

run(collection, subject, session, scan, volume, dest, *in_files)¶

Executes this volume staging workflow.

Parameters:	collection – the collection name subject – the subject name session – the session name scan – the scan number volume – the volume number dest – the destination directory in_files – the input DICOM files
Returns:	the output 3D NIfTI volume file path

workflow = None¶: The staging workflow sequence described in qipipe.pipeline.staging.StagingWorkflow.

qipipe.pipeline.staging.run(subject, session, scan, *in_dirs, **opts)¶

Runs the staging workflow on the given DICOM input directory. The return value is a {volume: file} dictionary, where volume is the volume number and file is the 3D NIfTI volume file.

Parameters:	subject – the subject name session – the session name scan – the scan number in_dirs – the input DICOM file directories opts – the `ScanStagingWorkflow` initializer options
Returns:	the `ScanStagingWorkflow.run()` result

qipipe.pipeline.staging.stage_volume(collection, subject, session, scan, volume, in_files, dest, opts)¶

Stages the given volume. The processed DICOM .dcm.gz files are placed in the dest/volume subdirectory. The child VolumeStagingWorkflow runs in the _parent_/volume_volume_ directory, where:

_parent_ is the parent base directory specified in the options (default current directory)
_volume_ is the volume argument

Parameters:	collection – the collection name subject – the subject name session – the session name scan – the scan number volume – the volume number in_files – the input DICOM files dest – the parent destination directory opts – the `VolumeStagingWorkflow` initializer options
Returns:	the 3D NIfTI volume file

qipipe.pipeline.staging.upload_dicom(project, subject, session, scan, dcm_dir)¶

Uploads the staged .dcm.gz files in dcm_dir to the XNAT scan DICOM resource

Parameters:	project – the project name subject – the subject name session – the session name scan – the scan number dcm_dir – the input staged directory

qipipe.pipeline.staging.upload_nifti(project, subject, session, scan, files)¶

Uploads the staged NIfTI files to the XNAT scan NIFTI resource.

Parameters:	project – the project name subject – the subject name session – the session name scan – the scan number files – the NIfTI files to upload

qipipe.pipeline.staging.volume_format(collection)¶

The DcmStack format for making a file name from the DICOM volume tag.

Example:

>> volume_format('Sarcoma')
"volume%(AcquisitionNumber)03d"

Parameters:	collection – the collection name
Returns:	the volume file name format

`workflow_base`¶

class qipipe.pipeline.workflow_base.WorkflowBase(name, **opts)¶

Bases: object

The WorkflowBase class is the base class for the qipipe workflow wrapper classes.

If the distributable flag is set, then the execution is distributed using the Nipype plug-in specified in the configuration plug_in parameter.

The workflow plug-in arguments and node inputs can be specified in a qiutil.ast_config.ASTConfig file. The configuration directory order consist of the order consist of the search locations in low-to-high precedence order consist of the following:

the qipipe module conf directory
the config_dir initialization keyword option

The common configuration is loaded from the default.cfg file or in the directory locations. The workflow-specific configuration file name is the lower-case name of the WorkflowBase subclass with .cfg extension, e.g. registration.cfg for qipipe.workflow.registration.RegistrationWorkflow. The configuration settings are then loaded from the common configuration files followed by the workflow-specific configuration files.

Initializes this workflow wrapper object. The parent option obviates the other options.

Parameters:	name – the module name opts – the following keyword arguments: project – the `project` parent – the parent workflow for a child workflow base_dir – the `base_dir` config_dir – the optional workflow node `configuration` file location or dictionary dry_run – the `dry_run` flag distributable – the `distributable` flag
Raises:	PipelineError – if there is neither a project nor a parent argument

INTERFACE_PREFIX_PAT = <_sre.SRE_Pattern object>¶

Regexp matcher for an interface module.

Example:

>>> from qipipe.pipeline.workflow_base import WorkflowBase
>>> WorkflowBase.INTERFACE_PREFIX_PAT.match('nipype.interfaces.ants.util.AverageImages').groups()
('nipype.',)

MODULE_PREFIX_PAT = <_sre.SRE_Pattern object>¶

Regexp matcher for a module prefix.

Example:

>>> from qipipe.pipeline.workflow_base import WorkflowBase
>>> WorkflowBase.MODULE_PREFIX_PAT.match('ants.util.AverageImages').groups()
('ants.', 'ants.', 'util.', 'AverageImages')
>>> WorkflowBase.MODULE_PREFIX_PAT.match('AverageImages')
None

__init__(name, **opts)¶

Initializes this workflow wrapper object. The parent option obviates the other options.

Parameters:	name – the module name opts – the following keyword arguments: project – the `project` parent – the parent workflow for a child workflow base_dir – the `base_dir` config_dir – the optional workflow node `configuration` file location or dictionary dry_run – the `dry_run` flag distributable – the `distributable` flag
Raises:	PipelineError – if there is neither a project nor a parent argument

base_dir = None¶: The workflow execution directory (default a new temp directory).

config_dir = None¶: The workflow node inputs configuration directory.

configuration = None¶: The workflow node inputs configuration.

depict_workflow(workflow)¶

Diagrams the given workflow graph. The diagram is written to the name.dot.png in the workflow base directory.

:param workflow the workflow to diagram

dry_run = None¶: Flag indicating whether to prepare but not run the workflow.

is_distributable = None¶: Flag indicating whether to submit jobs to a cluster.

logger = None¶: This workflow’s logger.

project = None¶: The XNAT project name.

pipeline¶

pipeline Package¶

mask¶

modeling¶

pipeline_error¶

qipipeline¶

registration¶

roi¶

staging¶

workflow_base¶

`pipeline` Package¶

`mask`¶

`modeling`¶

`pipeline_error`¶

`qipipeline`¶

`registration`¶

`roi`¶

`staging`¶

`workflow_base`¶