The Ups and Downs of Automated Captioning for Video

On November 19, Google announced that they would begin using their proprietary automatic speech recognition (ACR) technology with the YouTube caption system to create automatic captions for videos uploaded to the site.

This could be very good, or very bad.

Let's start with the good. Whether you realize it or not, there are a great number of internet users who are deaf, or have hearing difficulties - while the exact number may be unknown, it's a safe bet to assume between 20 and 40 million in the United States alone, or between 5-10% of the population.  Video for this segment of internet users has been largely inaccessible until last year, when YouTube finally started offering the ability to add captions. Note: I can't really blame Google for this as the flash technology used for encoding the videos didn't offer a simple way to apply cue points which are critical for keeping the captions with the proper video timeline.

Now, all our deaf friends will finally get to read the mostly pointless dialogue that exists on the hundreds of millions of videos on the internet's most popular video site. They may find, however, that the videos are more enjoyable when you CAN'T hear or read what they're saying. I mean, how do you caption a cat running into a wall? "mmmeeow.....meeeooowww....[cat running]....thud!....meeOOWWW!"

And now, the dark side of automated captioning.

One word. Spam.

With all the newly created text that will start appearing from these ACR videos, the content scrapers are drooling. Instead of spending hours crawling and pulling text from commercial and private websites, news sites, blogs and wikipedia to use in their spam sites, they can get all the keyword-rich copy right from YouTube's captioned videos.

As Andrew Shotland from Local SEO Guide explains in this excellent MediaPost article,

"Having a transcript in the video is huge for SEO...having targeted text on a page helps the video rank in search engines for specific searches."

but, he continues...

"if the video transcripts work similar to the embeds, the tag that lets people add the video to their Web site, anyone pulling the YouTube video onto their site can also rank for the text. ...It will be a spammer's wonderland."

Interested to see how this works in action? Here's a sample.


It's still in its' early stages, but the accuracy is sure to improve over time - anyone who uses the mobile Google voice search knows just how powerful this feature really is.

If you're not incorporating video optimization into your website marketing, it's time to start - now.