Lightfield Studio conversion SBS to 2x2 start converting with NO Progress

Lightfield Studio conversion SBS to 2x2 start converting but there is NO Progression. Even I left the process running for several hours, the progression wheel doesnt move, there is no calculation of time needed for conversion etc.

On the same computer i can render 3x3 - 9 views to preview live 3D content to china 50 inch 3D no glasses monitor. Any video can be played via player 123MPC Media Player Classic - Download (softonic.com) with a good result. I wonder how this can be used with Lumepad…

Maybe there is an option to speed up the way of conversion on different way
or to play this process live or streaming or just play 2D movies as 3D:

2D ORBIT or LINE records including formats 16:9, 1:1, 180 and 360 to 2x2 LIVE conversion:
frameN _ frameN+1x gap
frameN+2x gap_ frameN+3x gap…etc repeating

where gap is adjustable by number of frames between - this will lead to depth adjustment.

SBS records including formats 16:9, 1:1, 180 and 360 to _2x2 LIVE conversion:
LLRR-LLRR-LLRR…etc repeating
where depth adjustment need to saved to each scene marked within the video

4V picture + depthmap records including formats 16:9, 1:1, 180 and 360 to _2x2 or even to 4x4 LIVE conversion:
1l2L3R4r-1l2L3R4r-1l2L3R4r-…etc repeating

Hi Johan - can you share a screen capture of this instance to help@leiainc.com? We can then check if there is anything of issue.

Can you tell us more details about the computer? Operating system, CPU, storage, RAM? Certain features like the SBS to Lightfield feature are CPU-bound and a GPU will not accelerate the progress.

We are looking for more methods of editing in future versions, including allowing you to import and paint depth maps per-frame in videos. We also want to add options for VR180/360 video and others. However for now your best bet is to use other tools for the majority of your editing and use Lightfield Studio just for the final leg from either 2D to Lightfield or 3D SBS to Lightfield. The only exception is for orbital shots which you may want to start with Orbital to Lightfield before doing other editing.

3 Likes

Hi Joe, thanks for the reply… I try with short video and figure it out … Light field studio create three temp subfolders… first one to extract all frames to png and during this step it looks like no progression… maybe a prediction calculation would be welcome to estimate time needed based on the speed of generating frames and length of the movie…

1 Like

Thanks Nima, appreciate your work…

I also find stereo content can play without conversion as stereo or 4v. Ixs there any benefit to make conversion? It would be great to do the same way with orbital or linear or 2d direct on the lumepad. The mark in , mark out and the frame gap adjustment would be needed and everything could be done on lumepad than… no need for time consuming process on windows comp…

@johaninpika I totally see what you’re saying, but it’s really not possible right now.

We use a technology called Go4V to do SBS to Lightfield conversion on the fly on the Lume Pad. It works great in a pinch for watching Blu-Ray rips and previewing video shot on Hydrogen or Lume Pad, but ultimately for professional-level quality for creators you’ll really want to author your content using Lightfield Studio to do expert-level SBS to Lightfield conversion to ensure that the video is as sharp as possible and the depth is as accurate as possible.

For Orbital Content Conversion and 2D Content Conversion, we’re actually doing very intensive machine learning on those video files to do the most accurate depth estimation. Unfortunately, there’s no way to run those ML models on a Lume Pad for now. Our technology is improving every day and maybe in just a couple of years we’ll be able to offer good 2D to Lightfield video conversion on the fly on Lume Pad, but for now it’s even difficult for powerful computers, let alone a tablet like Lume Pad.

3 Likes

Noted and shared with the team. Thanks, Johan!

1 Like

I understand important part to offer professional solution with Light field studio and great algorithms you had created for depth map estimation. The result is really amazing I was surprised how good you can manage with grass or trees or even hairs…

2 Likes

What is crossing my mind to have very the same solution like Go4v to do orbital or linear on the fly on lumepad. So there wouldn’t be any depth map estimation and no hard processing. Just the adjustment of gap frames between 1234 would be good enough to create 4v video out of orbital or linear 2d video. Of course user must take care about constant speed during recording but this is not so critical when using drone or 360 camera on car or bike or boat… Many times I just like to create short movies, cut out few seconds sweep and than missing just this type of conversion…

1 Like

Hey @johaninpika,

We do have a solution for that, but not on Lume Pad. This uses FFMPEG on PC. It will work very, very fast and can work well, but there are a LOT of caveats to that. That’s why we strongly recommend just using the Lightfield Studio tool for the best quality. Either way, I figure I should share this script with you since it might be exactly what you’re looking for.

Video Requirements

The requirements for compatible video are as follows:

  1. The video must be shot with the camera moving horizontally at a consistent speed with little to no vertical or zooming motion.
  2. The subjects in the video should not move or move as little as possible.

Description

These instructions will use FFMPEG to duplicate a video, set a time delay, clip the ends of the two video clips, then generate a 3D SBS video.

Instructions

Install FFMPEG using Brew (MacOS/Linux), Chocolatey (Windows), your favorite package manager, or from ffmpeg.org

You can create two time delayed clips(Left Eye & Right Eye) by doing the following:

  1. Duplicate/copy the 2D clip you want to convert.
  2. Choose your baseline, which becomes your time delay value(t). Larger time delay is a larger baseline.
  3. Delete (t) from the end of one clip(making it your Left Eye clip)
  4. Delete (t) from the beginning of the other clip(making it your Right Eye clip)
NOTE: The above instructions assume your video was shot from Left to Right. If your video was shot from Right to Left, then the eye values in step 3 and 4 are reversed.

That’s it, you now have a left and a right eye clip, so you just need to stitch them together.

The FFMPEG script required to create a 3D video from two pre-time delayed clips is as follows:

ffmpeg -i Right_Input.mp4 -vf "movie=Left_Input.mp4 [in1]; [in]pad=iw*2:ih:iw:0[in0]; [in0][in1] overlay=0:0 [out]" SBS_Output.mp4

How to Choose a Good Time Delay:

If you’d like to identify the perfect time delay for your particular clip, you can export frames from the video at regular intervals and create SBS photos to preview on Lume Pad or other SBS viewer. You should probably choose a small slice of your total footage to capture the frames from as more than 5 or 6 per output are unneeded. The following FFMPEG script will output frames, where “n” should be replaced with the number of frames per second you’d like to output. e.g. using the value 4 on a 60fps video will output the first frame, 15th frame, 30th frame, and 45th frame in the first second, and another sequence of four starting with the 60th frame the next second.

ffmpeg -i LinearFootageSlice.mp4 -r n output_%04d.jpg

This will be a fast way for you to generate 3D video from your orbital and horizontally shot content using a rig, slider, drone, or other source, but if there are variations like lots of movement of subjects, vertical motion of the camera, or zooms in or out or focus changes, this will have clear visual issues.

Running the script above on Lume Pad is impossible in realtime especially if you’re talking about duplicating and stitching 2D video of 4K resolution and similar! I think it’s better we leave things like this to Creators on PC who can easily share it with Lume Pad after running the script.

6 Likes