Android 10 introduces live-captions for all videos and works completely offline

Google has released a new feature called Live Caption with the Pixel 4. Now users will be able to automatically add captions to any video with spoken audio.
Google Play App Store on Pixel 3 XL Android Phone
Live Caption will roll out to older models, like the Google Pixel 3 XL, later in the year. Photo: Tony Webster via Flickr

Google unveiled a host of new tech at its hardware event in New York on Tuesday. As expected, the star of the show was the new Google Pixel 4.

But there was also a number of other interesting new Google products and software on show.

One of the many interesting new software features which will be released on the Google Pixel 4 is Live Caption.

The new Live Caption feature will allow users to automatically add captions to video which contains speech.

At first the feature will be exclusive to the Google Pixel 4 but, good news for older Pixel owners, it will roll out to the Pixel 3, Pixel 3 XL, Pixel 3a and Pixel 3a XL later in the year.

What exactly does Live Caption do?

Live Caption will allow users to automatically caption any video on their phone. It won’t matter whether they have an internet connection or not, Live Caption can work fully-offline.

Once the captions have been added to the video, users can move them around the screen so that they don’t get in the way of the video content itself. The caption text can also be expanded with a simple double-tap.

Google HQ
Google’s new Live Caption feature works completely offline. Photo: Robbie Shade via Flickr

The only content Live Caption can’t add captions to is phone and video calls. Google has also kept privacy in mind. According to the Live Caption summary on Google’s Android blog, “the captions always stay private and never leave your phone.”

Live Caption might sound like a bit of a gimmicky feature, but it could actually come in very handy. If you need to decipher speech in a video and you’re in a loud environment without your headphones, Live Caption is the perfect solution.

What inspired Google to create Live Caption?

It’s been a decade since Google first added the automatic caption feature to Youtube. The Youtube system is still a bit hit-and-miss, and it can come up with some down right hilarious captions if used on someone with a strong or unusual accent.

Now Google is bringing the same captioning ability to smartphones, and it has the potential to help a lot of people.

As Google mentions in its summary of Live Caption, there are 466 million people in the world who are deaf or hard of hearing.

Gallaudet University
Google worked with Gallaudet University when developing Live Caption. Photo: Mr.TinDC via Flickr

That’s a lot of people who miss out on enjoying non-captioned video content every day.

Google says it turned to the deaf and hard of hearing community during the development of Live Caption.

In collaboration with various deaf organisations and Gallaudet University, the world’s leading deaf and hard of hearing university, Google received input from the people its new software would help the most.

Although Live Caption is only available in English for now, Google says it plans to develop the system to work with additional languages in the future.

Notify of
You May Also Like