Google I/O 2022: Tech giant introduces AI tool for real-world searches, 10-shade skin tone scale, more

Google laid out plans to bridge the real world and its digital universe of search, Maps and other services using new advances in artificial intelligence.

Alphabet Inc`s Google on Wednesday laid out plans to bridge the real world and its digital universe of search, Maps and other services using new advances in artificial intelligence.

At its annual Google I/O developer conference, the company previewed a feature that would let users take video of shelves of wine bottles at a store and ask its system to automatically identify options from Black-owned wineries.

"This is like having a supercharged Ctrl+F for the world around you," said Prabhakar Raghavan, a Google senior vice president, referring to the keyboard shortcut for finding something in a document. "You can search your whole world, asking questions any way and anywhere."

Later this year, Maps will launch an immersive view for some big cities that fuses Street View and aerial images "to create a rich, digital model of the world," Google said.

Google also unveiled a palette of 10 skin tones that it described as a step forward in making gadgets and apps that better serve people of color.

The company said its new Monk Skin Tone Scale replaces a flawed standard of six colors known as the Fitzpatrick Skin Type, which had become popular in the tech industry to assess whether smartwatch heart-rate sensors, artificial intelligence systems including facial recognition and other offerings show color bias.

Tech researchers acknowledged that Fitzpatrick underrepresented people with darker skin. Reuters exclusively reported last year that Google was developing an alternative.

The company partnered with Harvard University sociologist Ellis Monk, who studies colorism and had felt dehumanized by cameras that failed to detect his face and reflect his skin tone.

Monk said Fitzpatrick is great for classifying differences among lighter skin. But most people are darker, so he wanted a scale that "does better job for the majority of world," he said.

Monk through Photoshop and other digital art tools curated 10 tones - a manageable number for people who help train and assess AI systems. He and Google surveyed around 3,000 people across the United States and found that a significant number said a 10-point scale matched their skin as well as a 40-shade palette did.

Tulsee Doshi, head of product for Google`s responsible AI team, called the Monk scale "a good balance between being representative and being tractable."

Google is already applying it. Beauty-related Google Images searches such as "bridal makeup looks" now allow filtering results based on Monk. Image searches such as "cute babies" now show photos with varying skin tones.

The Monk scale also is being deployed to ensure a range of people are satisfied with filter options in Google Photos and that the company`s face-matching software is not biased.

Still, Doshi said problems could seep into products if companies do not have enough data on each of the tones, or if the people or tools used to classify others` skin are biased by lighting differences or personal perceptions.