Facebook Inc. Mark Zuckerberg, Chief Executive Officer and founder of the House of Representatives, speaks at a House Energy and Commerce Commission hearing on Wednesday, April 11, 2018 in Washington, DC, USA.
Andrew Haller | Bloomberg | Getty Images
Facebook on Thursday announced a research project to collect 2,200 hours of first-person footage from around the world to train the next generation of artificial intelligence models.
The project is called Ego4D and could prove important to Facebook’s Reality Labs division. The department is working on a number of projects that could benefit from AI models trained using video footage shot from a human perspective. This includes smart glasses and virtual reality, such as the Ray-Ban stories that Facebook released last month, which Facebook has invested heavily in since acquiring Oculus for $2 billion in 2014. This will happen.
This video can teach artificial intelligence to understand or recognize something in the real or virtual world that can be viewed from a first-person perspective through glasses or an Oculus headset.
Facebook announced in November that it would release its Ego4D dataset to researchers.
“This release is an open dataset, has a research agenda, and facilitates progress both internally and externally in academia. [allow] Facebook senior researcher Kristen Grauman told CNBC.
According to Grauman, the dataset can be deployed in AI models used to train technologies like robots to understand the world more quickly.
“Traditionally, robots learn by doing something in the world, or by showing them how to do things literally on hand,” Glauman said. “From our own experience, there is an opportunity for them to learn from the video.”
A consortium of 13 partners from Facebook and the university relied on more than 700 participants from nine countries to capture first-person footage. According to Facebook, Ego4D has 20 times longer videos than other datasets of this type.
Facebook’s university partners include Carnegie Mellon University in the United States, the University of Bristol in the United Kingdom, the National University of Singapore, the University of Tokyo in Japan, and the International Institute of Information Technology in India.
The footage was shot in the United States, United Kingdom, Italy, India, Japan, Singapore and Saudi Arabia. Facebook said it hopes to expand the project to more countries, including Colombia and Rwanda.
“The key design decisions of this project are, in the first place, leaders in the field, interested in these issues, willing to pursue them, and with geographic diversity. We wanted a partner to stick with,” said Gaumann. said.
Raven Story Glass
Sal Rodriguez | CNBC
The Ego4D announcement will come at an interesting time for Facebook.
The company has continuously stepped up its hardware efforts. Last month, we released our first smartglass, the $299 Ray-Ban Stories. And in July, Facebook announced the formation of a product team specializing in “metaverse,” a concept that involves creating a digital world where multiple people can live at the same time.
But over the past month, a plethora of internal investigations leaked at Facebook by Franshausen, a former Facebook product manager who turned whistleblower, has been flooded with news articles. There was a slide in the published studies showing that Instagram is harmful to the mental health of teenagers.
Footage was captured using off-the-shelf devices such as GoPro cameras and Vuzix smart glasses.
Facebook said it has instructed participants not to capture personally identifiable symptoms when collecting footage indoors to protect their privacy. This includes people’s faces, conversations, tattoos and jewelry. Facebook said it removed personally identifiable information from the video, which blurred viewers’ faces and vehicle license plate numbers. The company said that audio has also been removed from many videos.
“Phase 1 of all the university partners that collected this video was a very focused and important process in developing the right archive policy,” Grauman said.