I have two other methods for creating video masks that I wanted to share. The first one is purely for experimental purposes only because it is a processor hog, and probably wouldn’t be very realistic for actual use (though it is exactly what I was trying to achieve visually).

While thinking about my first attempt I realized that by using threshold to split up the video I was essentially creating a very hard edged mask with no gradation to it at all. This is fine, and as I said, you could always apply a blur filter or drop shadow to the bitmap to help soften up the edge, but it wasn’t truly a gradient video mask.

For the new method I did almost exactly what I did in my last attempt, but instead of using threshold to break the video up, I took the color value of each pixel (getPixel), turned that number into something ranging from 0-255 (the full range alpha has), and then reset each pixel in the bitmap including an alpha value this time (setPixel32). In other words darker pixels from the video were less visible, lighter pixels were more visible. The only problem with this is that it is on a pixel by pixel basis, so every single frame of the video you are reading, processing and rewriting each pixel during the transition.

For anyone who thinks their computer is strong enough: Gradient Video Mask Demo 2 (the first time you will see the ink, then if you click it again you will see a random video of a car being used as the mask.)

For my next attempt…

This sort of goes back to one of the original ideas I was working on, but my friend Jay (a motion graphics artist) helped to revive the concept by suggesting rendering the alpha channel from After Effects and just setting that as a mask. Unfortunately, that alone does not work which is why I had originally veered away from this method. If you set a video player, alpha channel or not, as a mask, flash just sees it as a rectangle the size of the video player. Not good.

Then I started to wonder if it would maintain the alpha channel when writing each frame of the video (with alpha) to a bitmap. Bingo! This is probably the most processor friendly method out of the three, but there is one drawback. You need two videos in order to execute a full transition with this method. You need one video with the ink keyed out for the image you want to disappear, and then you need another copy of that video with the area around the ink keyed out for the stuff you want to come in. This could cause issues if you don’t make sure the videos are synced up before they play.

Either way, this seems like it may be the way to go if you have the option of keying out your videos before you import them. Check it out: Video Mask Demo 3

AS3 Video Mask v1 Source