General information about plugins

Plugins are the object which in bmiptools allow to apply the image-processing transformations. They are designed so that certain operations can be executed in a standardized way, as showed in the Advanced API usage section. In particular, they can be alla initialized in the same way (via the transformation dictionary), they can be fitted all in the same way (if optimization is possible), applied to a stack in the same way, and finally the actual configuration of the plugin can be obtained all in the same way.

What is plugin optimization?

The word ‘optimization’ deserve some explanation in the context of bmiptools plugins. Plugins implement some image processing transformation, and many of them have some optimization procedure which need to be executed in order to produce the transformed image. Is that the meaning of the words ‘plugin optimization’, in the context of bmiptools? No. In bmiptools, the words ‘plugin optimization’ are referred to the automatic selection of the parameters (if any) of the image processing transformation.

An example may clarify better this point. An effective denoising technique is the so called total variational denoising. In this denoising technique, given a noisy image as input, the output is produced by finding the minimum of a suitable mathematical function (loss function) which depends on a series of parameters the user need to specify. No general analytical solution for this kind of minimization problem is possible, but it is possible to find algorithms that allow to find the minimum (or an output image close to it). Clearly, the minimum found depends not only on the input image, but also on the parameters specified in the loss function. From the point of view of the mathematical terminology, the minimization of the loss function is an optimization problem. By the way optimization of the Denoiser plugin in bmiptool (allowing to do also total variational denoising) is not referred to the solution of the minimization/optimization problem just explained. Rather ‘plugin optimization’ has to be understood as the mathematical procedure which selects the best parameter of the total variational denoising filter (in this case, this can be done by using the J-invariance criteria).

Attention

The plugin optimization is done using (eventually part of) the stack in order to estimate the plugin parameters. Once that these parameters are found, the plugin are initialized with them and then applied to all the slices with the same parameters. In this sense the plugin optimization in bmiptools is global for a stack and is not done slice by

slice.

Transformation dictionary

All the plugins are equipped with a dictionary which contains all the parameters that the user may set. This dictionary, called transformation dictionary is the one that appears after the corresponding plugin name in the pipeline dictionary generated by pipeline.Pipeline when a pipeline is created (see here and here).

The transformation dictionary has a common structure for all the plugins having an optimization method, which is briefly described below.

auto_optimize: The input is a boolean, i.e. true or false (in the json file, corresponding to True or False in python ). When true, the auto-optimization routine of the plugin is executed, according to the content found in the optimization_setting field.
optimization_setting: It contains a dictionary with all the setting associated to the optimization process (typically the definition of the parameter space, in which the optimal parameters are searched). Many times it makes sense to reduce the optimization time, it can be useful to restrict a the plugin optimization routine to a smaller region of the input stack, called bounding box. The bounding box is specified using the two objects below
- opt_bounding_box: It contains a dictionary which specify the region in the YX-plane used for the plugin optimization routine. Typically is a dictionary with the following structure
  {'use_bounding_box': True, 'y_limits_bbox': [None,None], 'x_limits_bbox': [None,None] },
  The meaning should be evident.
  - use_bounding_box: if True the bounding box is used, otherwise not;
  - y_limits_bbox: list used to express the limits in the Y-direction;
  - x_limits_bbox: list used to express the limits in the X-direction.
  The convention used to express the Y/X-direction limits are discussed here.
- fit_step: It is an integer number expressing the interval between two slices of the stack that are used during the optimization. If 1 all the stack is used, for n>1 only the slices having a distance of n on the 0-axis (i.e. the Z-direction). This parameter therefore determine the number of slices used during the optimization.
Attention

When there is the need to restrict some internal mathematical procedure of the plugin to just a subregion of the stack, the opt_bounding_box and fit_step fields can be found also outside the optimization_setting section of the transformation dictionary.

This is the case of the Registrator plugin, which has no plugin optimization in the sense discussed above. However this plugin still perform a proper mathematical optimization, in order to estimate the necessary registration parameters. This operation can be speed up if just a smaller portion of the stack in the YX-plane is used. For this reason the opt_bounding_box field is present in the transformation dictionary of this plugin.

In addition to the options discussed above the transformation dictionary of a plugin, after the optimization_setting field, contains all the plugin specific parameters. Below the general structure of a possible optimization plugin is sketched. Concrete examples of transformation plugins are shown in the documentation of plugin, where all the plugin specific parameters are explained.

{
'auto_optimize': True,                              # enable/disable plugin optimization
'optimization_setting':{'opt_param1': ...
                        ...                         # plugin optimization specific parameters
                        'opt_paramM': ...
                        'opt_bounding_box':{'use_bounding_box': True,
                                            'y_limits_bbox': [None,None],   # bounding box parameters
                                            'x_limits_bbox': [None,None]
                                            },
                        'fit_step':10
                        },
'param1': ...
...                                                 # plugin specific parameters
'paramN': ...
}

Bounding box specification convention

For the definition of the bounding box a numpy compatible convention is used. In particular the fields y_limits_bbox and x_limits_bbox are two lists with just two elements corresponding to to the left and the right element of the numpy sliceing for the correspoing axis. More precisely, given

y_limits_bbox: [a,b],
x_limits_bbox: [c,d]

where a, b, c, and d are integer number or None, this corresponds to the following selection in the (at least) 3-dimensional array stack containing the stack

stack[:,a:b,c:d]

Keep in mind that for a numpy array arr, one has

arr[None:b] = arr[:b]
arr[a:None] = arr[a:]
arr[None:None] = arr[:] = arr

For the user which are not familiar with numpy, the examples below can be helpful. They show a stack of 500x800x1200 voxels and a bounding box on YX-plane of 300x300 in different places. Note that the origin of the reference frame is in the front-top-left corner of the stack.

Attention

When the bounding box parameters are specified in the pipeline json, keep in mind the simple rules explained here about main the differences between the python notation and the json notation.

GUI Plugins setting

The transformation dictionary is not only useful for the python API. Indeed the transformation dictionary of each plugin turns out to be also the model used to construct the plugin GUI via GuiPI (see GuiPI: automatic GUI generation for plugins). As such the explanation of the parameters given for each transformation dictionary the plugins, is also the explanation of the possible setting the user may chose using the GUI. There is just on difference in the name of the parameter: in the python API the variables does not have blank space in their name, while in the corresponding GUI parameters every underscore is replaced with a blank space, e.g. auto_optimize in the Python API, becomes auto optimize in the bmiptools GUI.

Available plugins

The currently available plugins are: