Introducing Movie Player 2.0

The new Movie Player actor in version 2.0 offers more efficient video playback of H264 and Apple Pro Res codecs using Apple's most recent video playback technology. Under Windows, you will also experience performance gains by using Windows native formats such as WMV.

Important Note for Isadora v1 Users: when you open a file created with Isadora v1 in Isadora 2.0, the Movie Player actors will be automatically be renamed to Classic Movie Player. The Classic Movie Player still uses the old playback engine to maintain compatibility with older patches. You will not see the performance gains listed above if you use this the classic version of this actor! Also take note of the new names for the various kinds of video streams below.

Unlike the classic Movie Player in Isadora v1, Movie Player 2.0 has a mutable output; it can produce any of the three possible video formats supported by Isadora 2.0:

  • vid-gpu Video in this cross-platform format is processed by your graphics card (GPU). This is now the preferred format because it is much faster than the older 'video' format, which was processed by your main processor (CPU). To render this format to the stage, you will need to use a Texture Projector actor. (This actor will be replaced by a new Projector actor, also with a mutable input, before the public beta phase comes to a conclusion; please stay tuned for more info.) The vid-gpu format is compatible with all FreeFrameGL plugins (also new in 2.0) which are blazingly fast compared to older Isadora video effects.
  • vid-cpu Video in this cross-platform format (called as 'video' in Isadora 1.0) is  processed by your computer's main processor (CPU). This is the classic Isadora video format, but is no longer preferred because it is much slower than the new vid-gpu format. 
  • vid-ci This Mac OS X only format (called 'image' in Isadora v1) is usually processed by your graphics card (GPU). This video format is also very efficient. It is compatible with Apple's Core Image plugins, and also any Quartz Composer plugins that have been loaded into Isadora. The only real disadvantage when compared to vid-gpu is that is it not cross-platform: a patch using vid-ci cannot be moved to Windows computer.

The mutable output of the new Movie Player makes it easy to work with Isadora's other video related actors. If you want to process the image using a FreeFrame GL effect the output format is already vid-gpu. But, if you need to use a classic Isadora effect which requires vid-cpu, just connect the Movie Player's output to the effect's input: it will automatically change to vid-cpu. The same goes for any Core Image related plugins; just connect and the output of Movie Player 2.0 becomes vid-ci.

However, there is a now slight performance penalty when using vid-cpu with Movie Player 2.0. Movie Player 2.0 always decodes the video stream to the graphics card/GPU first, regardless of the output setting. Isadora must move the image from the GPU to the main processor's RAM before it can output a vid-cpu video stream. For this reason, you should always choose vid-gpu for new patches unless you absolutely require vid-cpu. (One obvious place where you must use vid-cpu is if you wish to do interactive tracking using the Eyes or Eyes++ actors.)

Understanding the new Optimize input

Isadora v1 used Apple's QuickTime to play video on both Macintosh and Windows. Now, with Movie Player 2.0, the default playback engine will be AVFoundation on Mac OS X, and DirectShow on Windows. The advantage of using these system is speed: you can play more high-resolution clips at the same time than you can with QuickTime.

But, AVFoundation and DirectShow do not shine when it comes to interactive control: they respond poorly when you do things like "scrubbing" a video by sending values into the 'position' input of the Movie Player. But interactive control was the reason Isadora was invented! So, we have made sure you can access QuickTime as a playback engine. Even though it is slow compared to the AVFoundation and DirectShow it has numerous advantages when it comes to interactive control.

That's where the optimize input comes in. If you leave it set to performance (the default) Isadora will attempt to use AVFoundation or DirectShow if possible. But if you set the 'optimize' input to interaction then Isadora will use QuickTime if possible.

Why do we say "if possible"? Because AVFoundation and DirectShow cannot play movie files compressed with certain video codecs. QuickTime's Animation codec is a good example: neither AVFoundation not DirectShow can play this format. So, even when you've set the optimize mode to performance, playback will revert to QuickTime if neither AVFoundation or DirectShow cannot play that type of movie.

Similarly, if you set the optimize input to interaction and QuickTime cannot play the movie, it will switch to AVFoundation or DirectShow. (Right now, this would only happen if you set optimize to interaction and tried to play a WMV or AVI file, which only DirectShow can play.)

So how can you tell which system is playing your movie? As of Isadora 2.0.0b3, Movie Player 2.0 has a new output called 'pb engine'. If it says AV, then AVFoundation is playing your movie; DS means DirectShow, and QT means QuickTime.

SUPPORTED FILE FORMATS

Here is a list of file formats supported by each playback system:

  • AVFoundation: Apple Pro Res (all flavors), H264, MP4, Photo JPEG, DV NTSC/PAL
  • DirectShow: AVI and WMV movies.
  • QuickTime: There are many, but Photo JPEG, DV NTSC/PAL, HAP, and Animation are probably the most important. (Apple's complete list can be found here.) QuickTime also supports H264 and MP4, but playback is very inefficient when compared to AVFoundation.

PLAYBACK ENGINE PROS + CONS

 
PROS
CONS
AVFoundation
• Smoothest Playback
• Excellent performance with H264 and AppleProRes movies
• H264 playback is hardware (GPU) accelerated.
• Also plays MP4, Photo JPEG and NTSC/PAL DV movies.
• Poor interactive control*
• Noticeable glitch when looping movies.
• Slower preload times**
• No support for HAP, WMV, or AVI codecs. No "legacy" codecs like Animation.
• No Text Track or Frequency Band support
• No audio "pan" support
• No audio output routing support
• Available on Mac OS X only
• Pitch correction algorithm cannot be disabled on Mac OS X 10.8***
DirectShow
• Smoothest Playback• Excellent performance with WMV and AVI movies
• Supports the highly efficient HAP codec
• Poor interactive control*
• No Text Track or Frequency Band support
• Available on Windows Only
QuickTime
• Excellent Interactive Control
• Nearly seamless looping
• Good performance with Photo JPEG, DV, Animation
• Supports the highly efficient HAP codec
• Poor performance, especially with high resolution H264, MP4 or Apple Pro Res clips
• Does not support Windows WMV or AVI Files.

* Poor interactive control means this: rapidly changing the speed, play start, play length, or position inputs may result in "lumpy" playback.)

** When you transition from one scene to the next, Isadora must preload all of the movies in the next scene before it can actually tell them to play and jump to the next scene. Preloading has never been instantaneous. But AVFoundation performs less well than it's counterparts in this regard; this is something we are very actively seeking to improve. However, as with Isadora v1, you may still use the Preload Scene actor to preload the scene before you need to jump. If you do this, the jump will be nearly instantaneous.

*** AVFoundation has a feature that attempts to correct the pitch of a movie when you played it slower or faster than normal speed. QuickTime does not uses this algorithm, so in Isadora v1, when you changed the speed of the movie, the pitch of the sound changed too. Unfortunately, the ability to disable the pitch correction feature was added only in Mac OS 10.9. So, when changing the speed of movies in 10.8, you will hear the pitch corrected result, not the changing pitch as found in Isadora v1.