This article covers the Spine Tracer pipeline operation and how it can be used in conjunction with Neurite/Neuron Tracer operations.
TL;DR
- Start by using either the Neurite Tracer or Neuron Tracer operations to create neurites
- Use the Spine Tracer with existing neurites to detect and quantify spines
Introduction
The Spine Tracer module was introduced in arivis Pro 4.3 as a complement to the Neuron Tracer functionality introduced in arivis Vision4D 4.0. It is designed to identify and quantify neuronal spines. It requires that neurites have already been segmented in the current pipeline as a prerequisite.
The Spine tracer is then used to detect and quantify dendritic spines in the immediate vicinity of the detected neurites.
Adding Spine Detection to a Pipeline
As mentioned above, the first prerequisite for spine detection is having already detected neurites. We can use either the Neurite or Neuron Tracer for this task. The Spine Tracing operation can be added immediately after the neurite tracer in the pipeline, or after some filtering operations.
The arivis Pro Spine Tracer can use 3 different methods to detect spines:
- AI-Assisted - This method uses a built in DL model for spine detection. This model automatically identifies spines around the neurites with little to no image pre-processing required and can be used in a majority of cases.
- Use Segments - In this case another pipeline operations has already previously segmented the spines and the Spine Tracer is only responsible for assigning the spines to their respective neurites. The spines can be segmented in any suitable method.
- Use Probability Map - Much like for the Use Segments option, we can use any suitable pipeline operation to isolate and enhance the spines from the rest of the image data, but now the segmentation itself is carried out by the Spine Tracer.
These options make it easy for the majority of users to carry out spine quantification with little additional effort while providing the flexibility to address more challenging images.
In every case below the first step is to select the input tag for the trace objects. Most pipelines of this type will only have one tag for the traces, but it is possible to have several tracing operations in the same pipeline, we therefore need to take care to select the correct tag for the traces at this stage.
AI Assisted Spine Detection
The AI Assisted Spine detection is the simplest way to use the Spine Tracer, using a built-in Deep Learning model to automatically detect the spines. It is the easiest way to configure this operation since requires only 1 additional parameter.
Note that, as the name implies, this method uses an AI Deep Learning model to automatically detect the spines. While this model can run on the CPU this model benefits greatly from GPU acceleration. Please see this article for more information on installing the GPU acceleration package if necessary.
After selecting the input tag for the trace as mentioned above, using the AI-assisted spine detection all we need to do is select the maximum spine length. This can be done by entering the desired value manually, if known, or by using the ruler to measure the length of spines on a 2D plane (note that the ruler is not available while in the 4D Viewer).
Once the max spine length is set we can run the operation to see the results. As always, we can revert the operation and change the parameters as needed if we are unhappy with the results, or try a different spine detection method.
Assigning previously segmented objects as spines
Using the built in model for spine detection is clearly very easy and the results are generally very good, however, in some cases better results can be obtained through other segmentation methods. We can then use the Spine Tracer to assign these previously segmented objects to neurites if preferred.
How the spine head segments are segmented isn't important, but they must belong to the current pipeline. This can be done by either segmenting them de novo or by importing the results of previous pipelines. In this example we've segmented the spines using a custom DL model via the Deep Learning Segmenter:
Selecting the Use Segment method we have a couple of new options:
First, we should note that only the head of the spines needs to be segmented. The neck of the spine is detected automatically from the Trace channel as selected in the operation parameters. Then, we must also select the Trace objects and the Spine segments that we want to link together. The selection is based on the tags created for these objects in the pipeline.
As before, we can set a maximum spine length to reduce occurrences of incorrectly segmented objects being included as spines. In this case, any object whose center is further than the selected value is not included in the final result.
Because the neurite and spines are both segmented independently the segments could conceivably overlap and could also include portions of the trace segment. Therefore, the Segment based spine tracer allows users to set a limit for the maximum percentage volume of the spine segment that is allowed to overlap the neurite.
When we run the operation the segments are converted to spines and connected to the neurite:
Using a probability map
The integration of AI ML and DL algorithms in arivis has hugely facilitated the segmentation of complex structures from noisy images while requiring little by way of image processing experience for the users. Because of this, and thanks to significant improvement in GPU computing, it is now possible to use AI models that can be significantly better and overall faster than traditional algorithms in all aspects of the image analysis workflow. Indeed the AI Assisted method mentioned above uses a custom spine detection model to facilitate this type of analysis. However, it is also possible to use AI models for both neurite enhancement and spine detection if the included algorithms do not provide adequate results.
This complete pipeline uses a custom DL model to enhance both the neurites and the spine heads as separate classes, then uses the resulting probability maps to trace the neurites and segment the spines:
Note that the model must have been created in advance of the pipeline execution. These models can be created using arivis Cloud, and model creation is included in the arivis AI Toolkit module, but arivis supports any DL model that can be saved as an ONNX file for this purpose as well.
The first step in this case is to use the Deep Learning Reconstruction operation to create the probability maps. In our example the model has 2 classes, but it is also possible to run 2 separate DL reconstruction operations, each with its own single class to enhance the spines and neurites separately, or use DL to enhance the spines alone if the arivis tracing algorithms suffice for trace detection.
These probability maps are stored as temporary new channels that are available to the pipeline but automatically deleted upon pipeline completion. Please consult this article on configure the Result Storage operation to learn about storing the probability map permanently if needed.
In our example, the Neurite Tracer uses a threshold based algorithm on the Neurites probability map to detect the neurite. The Spine tracer then uses both probability maps for the spine detection. The Trace channel is used to enhance the neck detection while the Probability map channel is used to detect the spines.
As before we still have the Max. spine length parameter, but we also have a Head threshold parameter to optimise the spine detection. Since probability maps aren't necessarily binary and the results will depend on the quality of the model, some experimentation by the user with this parameter may be required to obtain the best results.
Reviewing spine detection results
Once we have executed the Spine Tracer operation the results will be displayed in the image and the Objects table.
As always, we can select which features to display in the objects table and these features can also be exported from the pipeline to facilitate batch processing of multiple images.
Details of the various features are available from the Help files, and so are further details on the neurite and spine tracer operations.