Creating movies in arivis

This article explains how to export movies from images in arivis Pro

TL;DR

• Use the Movie export option for simple stacks or fast exports
• Use the Storyboard to export animations with 3/4D datasets and advanced effects and transitions
• Use VR Stories in arivis Pro VR to create animations using virtual reality
• Neither the storyboard nor VR stories record the creation/editing of objects. Screen recording software can also be used as an alternative

Introduction

Movies can be a great way to communicate the content of images and the results of analysis. In this article, we look at two ways one can export movies from images in arivis Pro.

Shallow, or single-plane time series can easily be exported as a simple movie showing the time progressions:

For samples with depth, an animation can tell a more compelling story:

Exporting time/plane progressions

If we have a simple time-lapse or plane stack, the simplest movie we can create is a simple playback through the available dimension. To create such movies we can use the Export function.

From here we can export a simple progression through the planes and/or time points.

Depending on the dataset different settings may be more appropriate.

If the dataset is a simple time series with a single plane, then it will not matter whether we select the whole stack or use only the current plane.

If we are exporting a Z stack with a single time point then using the whole stack is important as it would only export a single plane otherwise. Whether we want to do this or not depends somewhat on the data in the set. For a sample with a relatively stable Z-positioning, where a single plane shows all the information we want to communicate this may be fine. For images where maybe the focus can vary and the sample is relatively shallow, creating a maximum-intensity projection from which we can extract a movie may be preferable.

It is possible to export a movie where we play back sequentially, but in most cases involving a 4D dataset, an animation tells a more compelling story.

Various video export formats are available. MPG4 or H264 are recommended for their compatibility with a large array of video editing software.

The video resolution can be chosen from the list of available presets, which includes a range of sizes and aspect ratios, and for images with a very large XY field of view downscaling to one of these resolutions may be the most practical. Note that if the image XY resolution is not the same as the video resolution, the rescale settings will either resize the image to fit the output resolution, or add black edges to fill in the gaps. 

Bear in mind the expected display medium when considering what resolution to use. There is little point in exporting a video at a much larger resolution than the display on which you expect to play it back on. Videos for presentations rarely need to be larger than 1080p (1920x1080 pixels), especially since higher resolution videos are also much larger files and require a lot more memory to play back. 

The Video overlay tab allows us to add various elements on top of the image data. for example, we can add a time stamp or segmented objects, and choose where on the video frame this information should be displayed.

Finally, for multi-channel datasets, we have the choice of producing a movie with all channels overlaid on top of each other or side by side using the "Use Channel Split" option.

 

Using the Storyboard panel to create animations in the 4D Viewer

As we can see above, the Movie Export function can be a fast and effective way to create an animation. However, it is very limited by the types of transition and effects we can apply to images. Also, for datasets with a lot of depth a volumetric rendering animation can be much more effective at communicating spatial relationships, and using clipping planes and object display transitions can make it easier to understand the relationships between objects. As a whole, volumetric rendering animations can create a much more powerful impression of the image and analysis.

Getting started with the Storyboard

To get started with the Storyboard, first we must switch to the 4D Viewer.

Then, we can open the Storyboard panel from the Shortcut toolbar.

The Storyboard panel opens at the bottom of the Viewer.

From here, creating a movie is a relatively simple process, though some practice and skill can significantly improve the results.

The Storyboard works on the concept of transitioning between various Keyframes. A Keyframe is a record of a multitude of 4D viewer parameters, including:

• Volume position and orientation
• Zoom level
• Opacity settings
• Clipping plane parameters
• Current time point being displayed
• Object visibility settings

... and others.

Creating an animation is as simple as:

1. configuring the viewer to show a specific viewpoint, including all the parameters mentioned above
2. Adding a keyframe
3. Repeat 1. and 2. above until the animation is complete
4. Export the animation as a video

The process is relatively simple because all that's needed is to set up the various parameters we want to see in our video and then record these as keyframes. The software then automatically handles all the transitions.

However, while it is super simple to animate simple zooms, pans and rotations, adding elements like changes in transparency, time points, appearance of objects, clipping planes and all the other aspects that can be included in a storyboard animation, can takes some practice. For example, consider the difference between these 2 movies of the same dataset:

First, a simple 360 degree rotation.

And now the same dataset, but with zooms, clipping planes and object transitions.

And here is how we created these animations.

Throughout the process of creating a storyboard we can review changes using the playback buttons in the storyboard toolbar.

Once the storyboard is complete we can export our animation as a movie using the Export button.

Exporting Movies

Clicking the Export button above opens the Storyboard Movie Export window where we can configure our export options.

Movie saving options

Unlike snapshots and HD image exports that can normally be stored in memory for easy copy & paste into an output destination (e.g. PowerPoint presentation), movies are generally much larger and therefore need to be written to the hard disk and so the first options we find relate to the save location and name. The save location and name are completely up to you (the default is to save the movie with the same name as the original file in the same folder as the dataset). For the video format we need to consider where the video will be shown. Generally most modern devices support the H264 format and it is our recommendation. It is also a good format for importing into video editing software if we want to splice our arivis video with other footage, but some older systems may not support it and MPEG4 may be more suitable in such cases. 

Video Resolution and Framerate

After the file saving settings we have the movie video output options. These can make a significant impact on the quality of the video output and the time required to create it.

Generally speaking, the higher the quality of the video the longer the movie export will take. 

The Video Resolution setting affects both the size of the output file and the quality of the output. Here the the way the video will be presented will be quite important. If we include the video in a PowerPoint presentation, using the XGA or HD (720p) options are usually fine, especially if the video does not take the entire slide. If we export it for full screen viewing then Full HD (1080p) up to Ultra HD (4K) may be preferrable, but we should remember that the time it takes to render the video will increase significantly, and so will the output file size, for what may appear to be only a marginal gain in image quality in many cases. For example, going from 1080p to 4K will typically double the rendering time. arivis also support 360 video outputs which can be quite interesting in some cases, but requires viewers that support this format and some user interactivity to set the view angle when playing back the video.

The frame rate affects how smooth the video playback can be. Low frame rates (below 24FPS) can appear a little jerky, while 25-30FPS is the standard for most videos. Using 60FPS can appear smoother, and is recommended for 360 videos, but some video players won't be able to play back these high framerates smoothly resulting in noticeable jumps as the media player lags and then catches up to the required framerate. The most important thing to note is that going from 30FPS to 60FPS will require at least twice as much time to process the video and will result in a file that is about twice as large. 

Generally, we recommend using a low video resolution and framerate (720p at 25FPS) to produce a video quickly, and switch to high resolution and higher framerates only on occasions to really showcase data only as needed.

Data Resolution

This setting is mostly related to how the software handles large datasets. By large datasets we mean datasets for which a single timepoint cannot easily fit in the video memory of the GPU.

Viewing data in 3D and rendering videos are both highly dependent on the GPU. We cover the topic of how arivis handles large data in the 4D viewer in this article, but in short, since most GPUs have a finite amount of video memory available and a finite ability to render 3D datasets within a given amount of time, arivis typically down-samples data to make possible to render quickly in 3D. This down-sampling can lead to a noticeable drop in in the level of detail we can render and can also lead to noticeable down-sampling artefacts. As we explained in this article about rendering HD screenshots, it is always possible to render both screenshots and videos with the highest level of details, but for large datasets this comes at the cost of high processing times.

The data resolution slider adapts both to the PC configuration and to the dataset.

The scale goes from 64MB of VRAM usage on the left to the full resolution of the dataset on the right, and each graduation mark typically represents a doubling of the memory usage.

Note that since the 3D viewer renders 8bit version of the images the VRAM requirement may be significantly smaller than the actual dataset. Also, since we can only render one timepoint in any given frame, the amount of memory required is limited by the amount of data in a single timepoint.

The colour coding reflects the hardware configuration. The green part of the scale represents the amount of VRAM available on the GPU. Sticking to the green part of the scale reduces the loading times and also leads to much faster renders.

Since most computers typically have more system memory that video memory, we can use this as temporary storage for the image data instead, up to the amount of system memory available. The loading time is usually comparatively faster because loading 1MB from the disk in the RAM is usually faster than loading that data in the VRAM, but the rendering time suffers because the GPU is now reading data from the RAM rather than the faster and closer VRAM.

But since some datasets can be larger than even the RAM available, arivis also allows creating high resolution videos even at native resolution by using a hybrid rendering approach, though this is done at the cost of rendering time. 

If we are rendering a very large dataset, it is recommended the minimum amount of data resolution required to see the detail that we need to see. Some experimentation using HD screenshots may be worthwhile to find out what those optimal settings might be. It may also be worth considering splitting the animation into several movies and using volume clipping to narrow down the rendering to specific regions of interest for the high level of detail required and then, if necessary, merging the movies into a single file using video editing software.

Creating movies for video tutorials or to show arivis processes

None of the options above can be used for recording application processes like object creation, pipeline execution, and other image or object modifications. However, as mentioned above a picture is worth a thousand words and a movie even more so. Sometimes it is easier to create movies to explain the process. In such cases it is better to use screen recording software to create these movies. Some basic screen recording software do not feature video editing tools but the movies can be imported into video editing tools. Several screen recording software exist that include movie editing options with advanced features. An example of this can be seen above where we used screen recording software to record the process of generating and exporting the storyboard.