VideoStitch Tutorial

VideoStitch brings your  360 ideas to life!

Produce stunning 360° and high resolution panoramic videos. VideoStitch is not only amazingly fast, but it is packed with features to get the best performance out of your Freedom360. Features include: real-time, interactive preview, flexible calibration workflow, H264 & industry standard encoding and automatic synchronization.

Buy your VideoStitch license
Download VideoStitch demo

 

VideoStitch Workflow and Tips

System requirements:

  • NVIDIA graphics card: CUDA compatible, compute capability 1.3 or better, 2GB minimum, 4GB recommended (more CUDA cores are better, more GPU memory lets you render larger frames)
  • PTGUI pro (or Hugin) for camera calibration (and stitching templates)

Minimum requirement to achieve a good stitch:

  • Use the same frame rate with all GoPro cameras.
  • Use a strong audio signal (i.e. clapping hands) at the beginning of the video.

Before opening VideoStitch:

  • Shoot video: learn more about camera settings and how to connect and use the wifi remote in Tutorials.
  • Prepare your videos for stitching:
    • Transfer files from the micro SD card to your hard drive.
    • Make sure there is one file per camera and name the videos i.e. cam1.mp4 , cam2.mp4. If there are several movie files please see Remark 1 below for more info.
    • Place all 6 video files in the same folder.
  • Download the PTGui template Hero3 black (if you are using the black edition) from our Samples and Template section for free.
    • The template is optimized for the Freedom360 mount with GoPro HD Hero3 black cameras.
    • It contains all basic settings and image parameters to create a first stitch. It gives you a good starting point, but you still need to optimize it.
    • Use this template to start building your own calibration for VideoStitch.

     

The 360º video stitching workflow with VideoStitch using PTGui as calibration tool:

The VideoStitch workflow consists of three parts presented in the tabs below. The process starts and ends with the VideoStitch interface: load the images, synchronize, create a calibration, apply the calibration, adjust exposure, preview the result as equi-rectangular or interactive movie, choose output parameters and render. The middle part is about editing the calibration in PTGui. A calibration is a stitching template for VideoStitch. Therefore great care should be applied to this part to achieve the best possible result. However, if you already familiar with PTGui and its panorama stitching capabilities feel free to skip the middle tab.

[tabs]

[tab title=”1. VideoStitch: Source”]

  1. Open VideoStitch. Here is a picture of the interface: VS_splash_screen    The workspace consists of 4 panels which are stacked behind each other (upper right corner).VideoStitch opens onto the Source panel.
    workspace_source Note: The Output, Interactive and Process panels become available after a calibration has been imported and a video is ready to stitch.First time using VideoStitch:

    • Go to Edit and click on Preferences.
    • Choose your CUDA Device (only necessary if you have more than NVIDIA card in your system).
    • Calibration Tool: Browse to the folder that contains PTGui. (Windows: PTGui.exe).
    • Choose your Language: English or French.
    • Click OK.

    preferences_panel

  2. Drag and drop your six source videos into VideoStitch.
  3. Synchronization: Creating a good calibration and therefore a good final video requires that the source videos are properly synchronized. For this part of the tutorial we assume there was a loud sound pattern (like a clap) at the start of the video. If there is no audio signal for synchronization please see Remark 2 on the bottom of the page.
      • Click VS_syncbutton
      • Click on Compute offsets

    VS_sync

    • Check that the offsets are reasonable, i.e. if you started the cameras with the Wifi remote the offsets shouldn’t be more than a couple of frames. If you started the cameras manually one by one the offsets can be larger since it takes a couple of seconds to turn all cameras on. Also if you use a higher frame rate i.e. 100 fps the offsets will be larger than with 24 fps.
    • If all fails and the synchronization does seem way off proceed to Remark 2.
  4. Create a calibration:
    • Choose a point on the timeline that represents the video as whole in regards to distance to objects. Avoid featureless parts where the video only shows blue sky or white walls. Without features in the video the calibration will be difficult (or even useless).
    • Click Edit and then Extract stills
      VS_extractstills_advt
    • Images will be extracted from the videos to the project folder,
    • When VideoStitch extracts images, it names them based on the input indexes : input-0.jpg, input-1.jpg, … 
    • If you checked the Open calibration tool option, PTGui will be launched automatically with these images.

 

[/tab]

[tab title=”2. PTGui: Edit Calibration”]PTGui has a main interface window and the Panorama Editor window. You will do changes/optimizations in the interface window. The ‘Panorama Editor’ shows your stitched panorama. You will need to use it a lot to check for stitching errors.

  1.  When images are loaded in PTGui a window, Camera / lens data (EXIF), pops up. Click Cancel.PTGui_load_images_small
  2. Go to File -> Apply Template -> Other… (or press Ctrl+ T)PTGui_ApplyTemplate
  3. Browse to the folder where you saved the PTGui template for the Hero3 black, select the file Freedom360_V506_template.pts and click Open.
  4. The Panorama Editor shows now a rough stitch.PTGui_PE_aftertemplate
  5. Save (Ctrl+ S).
  6. Time to optimize! For optimization you will need the Control Points and Optimizer tabs. After each optimization you need to check in the Panorama Editor for stitching errors and if the scene is coming together.
  7. Starting with Control Points:
    • Control points are matching points on two overlapping pictures. They are needed to get an optimal stitch. Image pairs 0/1, 2/3 and 4/5 have no overlapping areas since they are captured by cameras on the two opposite sides of the cube.
    • Click on the Control Points tab.PTGui_ControlPoints_tab
    • The interface shows a pair of images. Clicking on the tabs PTGui_ControlPoints_tab_2 let’s you choose the pair of images you want to work on. You need to generate control points for each pair of overlapping images.
    • Start by selecting image 0 on the left and image 2 on the right i.e. the image pair 0/2.
    • There are four different ways to generate control points and you will eventually need to use all of them. Always start with step 1 and work your way through step 4 if necessary until you have enough control points:
        1. Click on Control Points in the menu bar and then click on Generate Control Points for all images.
        2. Click on Control Points in the menu bar and then click on Generate Control Points for image 0 and 2.
          PTGui_generate_CP
        3. While holding the “Shift” key down, click & drag your mouse to define a selection within one of the images. Then right click on this selection, and chose ‘add control points here’ to generate more control points within this area.
        4. Manual: Click on easy to identifiable point in one of the images (left or right). Then click on the corresponding feature in the other image. (Sometimes the software already does this for you.)
    • Save (Ctrl+ S).
    • After 0/2 select 0/3 etc. until you get to 0/5. Then select image 1 on the left and image 2 on the right. Check and generate control points for all pairs until 3/5. (Remember 4/5 doesn’t have overlap.)PTGui_CP_02
    • Save (Ctrl+ S).
    • Note 1: Make sure you have at several control points for each pair of overlapping images spread out over the overlapping area. Control points enable image features to match later in the finished video.
    • Note 2: Rotate each image to make it easier to see how the two images fit together. Use zoom to better see details!
    • VERY IMPORTANT! Make sure that you have GOOD control points. Success of the next part depends on this.
  8. After each optimization (next steps) you will need the Panorama Editor to check the image. Here are two important tools to make your life easier:
    • Click  PTGui_PE_switchtoimagenumbers to see which image tiles have the most problems. PTGui_PanoramaEditor_ImageNumbers
    • Select Detail Viewer PTGui_select_DetailViewer to search for any stitching errors. Move the magnifying glass in the Panorama Editor window and watch the magnified view in the Detail Viewer window. PTGui_Detail_Viewer
  9. Optimization:
    • Click on the Optimizer tab.PTGui_Optimizer_tab
    • IMPORTANT:
      • If at any point during optimization you think you need more control points in some areas, go back and add more control points.
      • If at any point during optimization you think the last step made the stitch worse UNDO (Ctrl +Z) it and don’t use this optimization.
      • If at any point during optimization you think you achieved a GOOD stitch you are done with optimizing. Move on to the next step.
    • First Optimization: Yaw, Pitch and Roll (YPR) are already selected for the six images.PTGui_Optimizer_initialsettingsClick  PTGui_RunOptimizer The first optimization is likely to have the following result. PTGui_Optimizer_bad Click OK and you can see the changes applied in the Panorama Editor. Save (Ctrl+ S).
    • Second Optimization: Optimize globally the Field of View. Check the Field of View box. PTGui_YPR_FOVRun Optimizer. Check Panorama Editor if this improved the stitch. If it made it worse, undo (Ctrl+ z).  Save (Ctrl+ S).
    • Third Optimization: Hshift/Vshift. For all six tiles: Check all the boxes for Hshift and Vshift. PTGui_YPR_FOV_HVshiftRun Optimizer. Got to the Image Parameter tab and check that the Hshift and Vshift values are less than 50. If not, undo. Check Panorama Editor if this improved the stitch. If it made it worse, undo (Ctrl+ z). Save (Ctrl+ S).
    • Fourth Optimization: Viewpoint correction. Select the tile(s) that still have the most problems such the tiles covering the floor (0, 3, 4) or tiles that contain nearby objects. Check those tiles. (Never check all tiles!)PTGui_YPR_FOV_HVshift_VP Run Optimizer. Check Panorama Editor if this improved the stitch. If it made it worse, undo (Ctrl+ z). Save (Ctrl+ S).
    • Last resort optimization: abc lens parameters. Check the b (lens distortion) box. Run Optimizer. Check Panorama Editor if this improved the stitch. If it made it worse, undo (Ctrl+ z). Save (Ctrl+ S). If it still needs more work (only if you are really sure your control points are good control points) check the a (lens distortion)  and c (lens distortion) boxes. Run Optimizer. Check Panorama Editor if this improved the stitch. If it made it worse, undo (Ctrl+ z). Save (Ctrl+ S). PTGui_YPR_FOV_HVshift_VP_abc
    • Note 1: Don’t use Horizontal or Vertical shear. It is not supported by VideoStitch.
    • Note 2: Sometimes it helps to start over for a particular pair of images i.e. delete all or most control points and generate new ones.
    • Note 3: Disabling yaw, pitch, & roll optimization on one of the images ensures the panorama global orientation is preserved during optimization. (This is how it should be set initially.)
  10. After you are done optimizing, click on Control Points in the top menu and select Delete the worst control points.  Run Optimizer again, and check the Panorama Editor to see if the scene is coming together. If the last step made the stitch worse UNDO (Ctrl +Z) it and move to the next step.
  11. Level the horizon:
    • In the Panorama Editor window click & drag your mouse up (start on the lower edge) until you can see the tripod/pole etc. where your camera is mounted in the center of the panorama. PTGui_levelling
    • Click on the nadir point (i.e. the point where the pole connects to the tripod or the point where pole reaches your hand) and drag it down to the edge and center of the window. The image is now leveled. (You can also use the ‘numerical transform’ from the tools menu to transform your panorama via pitch=-90.)
    • Save (Ctrl+ S).
    • Note: If you used ‘Field of View’ in the Optimizer you need to do some more corrections. You need to add vertical control points which is explained in the next step.
  12. Vertical control points:
    • Go to the Control Point tab.
    • Select the same image on the left and on the right starting with image 0.
    • Look for vertical features such as the side of a building etc.
    • In the left image click with the mouse on the top of the feature, in the right image click the bottom of the feature. PTGui_cp_vertical
    • Save (Ctrl+ S).
    • Repeat for the remaining five images.
  13. Last time to use the Optimizer:
    • Go to the Optimizer tab.
    • Optimize only YPR. Uncheck everything else.
    • Note: Everything else is already optimized and doesn’t need to run again. Changes only affect YPR.
    • Save (Ctrl+ S).
  14. Mask out the pole:
    • Go to the Mask tab.PTGui_Mask_tab
    • The pole is visible in images 0, 3 and 4. This is where you want to apply a mask so the pole disappears from the final result.
    • Select image 0.
    • If there is already a mask applied (red pattern covering part of the image) make sure it is where you want it to be or click Clear Mask in the top menu.
    • Choose the red mask tool (red circle below the image) and the appropriate pencil size.
    • Apply mask by clicking on the feature you want to mask out, holding the mouse button down and painting over the feature. To create a straight line click where you want the line to start, then hold the “Shift” key down, and click where you want the line to stop.
    • To remove part of the mask, choose the eraser (white circle below the image) and the appropriate pencil size and click on the area.
    • Save (Ctrl+ S).PTGui_mask
    • Repeat on images 3 and 4.
    • Note 1: Watch the Panorama Editor (better: choose Detail Viewer) so you don’t mask out everything/all pixels. If multiple masks are overlapping, no image data will appear in the output. With other words you will have a black hole.
    • Note 2: It is not possible to remove the tripod legs in this example because we don’t have an image showing the area under the legs.
    • Save (Ctrl+ S).
  15. Exposure/HDR:
    • Go to the Exposure / HDR tab.PTGui_Exposure_tab
    • Click on settings. PTGui_Exposure_setting
    • Optimize Exposure -> Enabled (full)
    • Optimize White Balance -> Disabled (if you set a white balance on your GoPro cameras.)/ Enabled (if you used Auto white balance);
    • Five image boxes checked, one unchecked. The unchecked image should be the one where you like the exposure.
    • Click on PTGui_Exposure_optimizeNow
    • Save (Ctrl+ S)
  16. Save (Ctrl+ S) and return to VideoStitch.

 

[/tab]

[tab title=”3. VideoStitch: Output, Interactive, Process”]

  1. Back in VideoStitch: Apply Calibration
    • Got to Edit and choose Apply Calibration. VS_Apply_Calibration
    • Choose the PTGui file that you just created/edited and click Open. VS_calibration_file
    • You will be prompted to create an output file for your VideoStitch project. Choose a name and click Save.VS_create_output_file
    • Note: Once a calibration has been loaded, it is not possible to change input footage directly in the GUI.
  2. After the calibration is applied the stitched video can be previewed in the Output and Interactive workspaces.
    • Click on Image Number to see video tiles.
    • Output: The preview is played back in one of the chosen projections i.e. Equirectangular
    • Interactive: The preview is played back as interactive video.
    • Note: Speed up the play-back by reducing the size i.e. Width of the output and switching from Multiband (Quality) to Linear (Fast) blending in the Process panel. VS_Process_Blender
  3. Exposure Compensation
    • Click on Edit and select Exposure Compensation. VS_Op_SelectExposure
    • Choose over which range the Exposure Compensation should be applied, i.e. First frame/Last frame.
    • Choose how often the exposure should be corrected, Adjust every: … frames. If light conditions changed a lot during the shoot you want to adjust more often. So choose a smaller number of frames.
    • If you used Auto White Balance while shooting, you can Compensate white balance here (if you have not done so in PTGui). Just  check the box.
    • Click Compensate and wait for Exposure Compensation to finish. VS_Exposure_progress
    • Close the window.
    • Save (Ctrl+ S)
  4. Render the final video
    • Got to the Process panel. VS_Process_settings
    • Choose the project settings : output width / height, sequence’s duration : start time & end time, fps, Blender (back to Multiband (Quality)). VS_fps
    • Set the desired video encoding. For more about encoding read Remark 3 below.VS_output_format VS_codec
    • Select the soundtrack that should be copied from one input video to the output. VS_Audio_source
    • Set the output file name.
    • Set the device(s) to stitch with. Multiple GPU is available with VideoStitch Extended only.
    • Hit Process to start rendering the video. VS_progress
    • Save (Ctrl+ S)

 

[/tab]

[/tabs]

 

[button link=”https://freedom360.us/shop/videostitch-ptgui/” color=”#39b54a”]Buy a license now![/button]

[button link=”https://freedom360.us/software-download/”]VideoStitch Software Download[/button]

For additional support and to learn more about computer hardware considerations visit video-stitch.com.

Remark 1: More than 1 file per camera for one continuous shoot.

In video mode, GoPro cameras have a file size limit of ~4GB. This means: If you are shooting with a GoPro Hero 3 black edition with a resolution of 1440p, a frame rate of 48fps and protunes ON the 4GB translate to about 15 minutes of footage. If you are shooting longer than that, the camera will start a new file until it reaches 4GB and start another file.
Before you start a stitch with VideoStitch, you should concatenate the video files, i.e create one continuous file, from each camera. You can find the instructions how to do it here: http://www.video-stitch.com/forum/index.php?page=Thread&postID=61

Remark 2: Synchronization and Calibration

A good synchronization between the six video sources is the most important part for a good final result. If you can’t use Audio synchronization (no distinct audio signal or audio and video are out of sync) the synchronization tool offers direct access to the synchronization settings. This means you can change the frame offsets for each individual source video by clicking the up/down marker. Checking the boxes next to say two of the videos you can increase/decrease offset of these two videos together.
Drawback of this method: You need to apply a calibration in order to preview the video and be able to see the effect of changing the frame offsets.

Tip 1: Start creating a set of quality calibrations. A quality calibration can easily be re-used to bootstrap new VideoStitch projects.
Tip 2: Record with a high fps when possible, this gives you finer ‘grain’ when fine tuning synchronization.
Tip 3: Double-check your synchronization by using a second application i.e. Kolor Autopano Video and synchronize. If you get about the same values (maybe off by ±1 frame) in both apps you can be pretty sure your source videos are properly synchronized.

Remark 3: Codecs

VideoStitch offers three codecs for video encoding, H264 (default), MPEG4 and MPEG2. However for very high resolution output you might want to export the video as a image sequence such as *.jpg or *.tiff.

H264: up to 4096 pixels. Doesn’t support resolutions larger than 4096 pixels.
Default encoding, recommended for direct web publishing.

MPEG4: output size must be a multiple of 8. Mpeg4 doesn’t support videos that exceed 8192 pixels dimensions. VideoStitch outputs Mpeg4 part2 (not AVC) encoded video.

MPEG2: doesn’t support resolutions that are multiple of 4096.
The MPEG2 codec is widely supported by video players. It provides an acceptable quality at the price of a high bitrate.
Very high resolution videos (over 8192 pixels) won’t be decoded properly by most video players and editing suite as such high resolution videos are not common in the industry yet.

Remark 4: Process Settings

It is always best practice to render your video as a master with the highest possible resolution and later adjust it for particular publishing option. This will give you the most flexibility and control over your material. It also allows you to create video files for specific purposes without rendering again. However, sometimes this is not important and it is more desirable to publish fast. Adjust your settings accordingly.

Framerate: Render the video with the same frame rate as the input. Adjust frame rate in an optimized compressor such as Quicktime, After Effects or Premiere. For immediate publishing, you can adjust the frame rate down right away; i.e. from 100fps to 25fps, from 48fps to 24fps, if you have 30 fps (29.97) keep it that way.
Bitrate: Master file: 15000 kbps. For immediate publishing: 2000- 5000 kbps (2-5 Mbps)
Settings for immediate IPad publishing: Width x Height: 1920 x 960; Output format: mp4; Codec: h264; Bitrate: VBR; Bitrate (kbps): 5000;