Calculation pipeline

UML diagrams presenting the flow of the analysis for each module are available here: https://github.com/ANCPLabOldenburg/MEG-QC-code/tree/main/diagrams

meg_qc.calculation.meg_qc_pipeline.add_raw_to_config_json(derivative, reuse_config_file_path: str, all_taken_raw_files: List[str])

Add the list of all taken raw files to the existing list of used settings in the config file.

Expects that the config file .ini and the .json file (with the same name) are already saved as derivatives.

To get corresponding json here use the easy way: just exchange ini to json in reuse file path (not using ANCPbids for it). The ‘proper’ way would be to: - query the desc entitiy of the reused config file - get the json file with the same desc entity This way will still assume that desc are exactly the same, so we use the easy way without ANCPbids d-tour.

The function will also output the updated list of all taken raw files for this ds based on the users choice: rewrite or not the subjects that have already been processed with this config file.

Parameters:

• derivative (ancpbids.Derivative) – Derivative object to save the config file.

• reuse_config_file_path (str) – Path to the config file used for this ds conversion before.

• all_taken_raw_files (list) – List of all the raw files processed in this run, for this ds.

Returns:

all_taken_raw_files – Updated list of all the raw files processed in this run, for this ds.

Return type:

list

meg_qc.calculation.meg_qc_pipeline.ask_user_rerun_subs(reuse_config_file_path: str, sub_list: List[str])

Ask the user if he wants to rerun the same subjects again or skip them.

Parameters:

• reuse_config_file_path (str) – Path to the config file used for this ds conversion before.

• sub_list (list) – List of subjects to run the QC on.

Returns:

sub_list – Updated list of subjects to run the QC on.

Return type:

list

meg_qc.calculation.meg_qc_pipeline.check_config_saved_ask_user(dataset)

Check if there is already config file used for this ds: If yes - ask the user if he wants to use it again. If not - use default one. When no config found or user doesnt want to reuse - will return None. otherwise will return the path to one config file used for this ds before to reuse now.

Parameters:

dataset (ancpbids.Dataset) – Dataset object to work with.

Returns:

config_file_path – Path to the config file used for this ds conversion.

Return type:

str

meg_qc.calculation.meg_qc_pipeline.check_ds_paths(ds_paths: List[str] | str)

Check if the given paths to the data sets exist.

Parameters:

ds_paths (list or str) – List of paths to the BIDS-conform data sets to run the QC on.

Returns:

ds_paths – List of paths to the BIDS-conform data sets to run the QC on.

Return type:

list

meg_qc.calculation.meg_qc_pipeline.check_sub_list(sub_list: List[str] | str, dataset)

Check if the given subjects are in the data set.

Parameters:

• sub_list (list or str) – List of subjects to run the QC on.

• dataset (ancpbids.Dataset) – Dataset object to work with.

Returns:

sub_list – Updated list of subjects to run the QC on.

Return type:

list

meg_qc.calculation.meg_qc_pipeline.create_config_artifact(derivative, config_file_path: str, f_name_to_save: str, all_taken_raw_files: List[str])

Save the config file used for this run as a derivative.

Note: it is important the config and json to it have the exact same name except the extention! The code relies on it later on in add_raw_to_config_json() function.

Parameters:

• derivative (ancpbids.Derivative) – Derivative object to save the config file.

• config_file_path (str) – Path to the config file used for this ds conversion

• f_name_to_save (str) – Name of the config file to save.

• all_taken_raw_files (list) – List of all the raw files processed in this run, for this ds.

meg_qc.calculation.meg_qc_pipeline.get_files_list(sid: str, dataset_path: str, dataset)

Different ways for fif, ctf, etc… Using ancpbids to get the list of files for each subject in ds.

Parameters:

• sid (str) – Subject ID to get the files for.

• dataset_path (str) – Path to the BIDS-conform data set to run the QC on.

• dataset (ancpbids.Dataset) – Dataset object to work with.

Returns:

• list_of_files (list) – List of paths to the .fif files for each subject.

• entities_per_file (list) – List of entities for each file in list_of_files.

meg_qc.calculation.meg_qc_pipeline.get_list_of_raws_for_config(reuse_config_file_path: str)

Get the list of all raw files processed with the config file used before.

Parameters:

reuse_config_file_path (str) – Path to the config file used for this ds conversion before.

Returns:

• list_of_files (list) – List of all the raw files processed in this run, for this ds.

• config_desc (str) – Description entity of the config file used before.

meg_qc.calculation.meg_qc_pipeline.make_derivative_meg_qc(default_config_file_path: str, internal_config_file_path: str, ds_paths: List[str] | str, sub_list: List[str] | str = 'all')

Main function of MEG QC:

• Parse parameters from config: user config + internal config

• Get the data .fif file for each subject using ancpbids

• Run initial processing (filtering, epoching, resampling)

• Run whole QC analysis for every subject, every fif (only chosen metrics from config)

• Save derivatives (csvs, html reports) into the file system using ancpbids.

Parameters:

• default_config_file_path (str) – Path the config file with all the parameters for the QC analysis - default. later the function will ask the user if he wants to use the same config file again or use another one.

• internal_config_file_path (str) – Path the config file with all the parameters for the QC analysis preset - not to be changed by the user.

• ds_paths (list or str) – List of paths to the BIDS-conform data sets to run the QC on. Has to be list even if there is just one path.

• sub_list (list or str) – List of subjects to run the QC on. Can be ‘all’ or 1 subj like ‘009’ or list of several subjects like [‘009’, ‘012’].