Later, I tried putting similar photos through Google Lens, a Pinterest Lens competitor that launched earlier this year, in beta, on the Pixel phones. Google’s version doesn’t understand that it sees a chair, or even furniture, and offers me an apology. It mistakes the pillow for a quilt. The only thing it does match correctly is that Rothko print, though it’s worth noting that identifying 2D art is widely considered to be among the simplest challenges for visual-search tools.
Google Lens just isn’t very good. At least not yet. But you can see how visual AI ties into the company’s larger business imperative, as well as that of other tech giants. Google is in the business of indexing, so it makes sense that the company would want to help users visually identify the world around them. Facebook has its social graph, focused on connecting users to friends, and a vested interest in using AI to identify faces. Amazon has e-commerce. For it, visual search could be a bridge between the digital and physical worlds–for instance, letting you photograph a pair of shoes to look for similar, perhaps cheaper, ones on Amazon. Each company might approach visual AI differently, but the implications are the same: There’s money at stake in this burgeoning field, even if it’s too soon to articulate just how much. “Think about the possibility of taking a picture to search for something you can’t even describe–it’s pretty powerful stuff,” says Forrester’s search analyst Collin Colburn. “It might be the most immature [search], but it probably has the most potential.”
Google has hundreds of employees working on visual AI alone. Facebook has 20,000 employees and 300 AI researchers, plus it operates 1.2 million visual AI experiments on the social network at any given moment. Amazon has over 500,000 employees with 5,000 working on Alexa–its new Echo Show features not just a microphone, but a camera to interact with Alexa, too–a camera gives Amazon the view of a whole room inside your home. Pinterest? It has just 12 employees devoted to visual search.
But Pinterest is mightier than it seems. To start with, it has a massive data set on which to train its visual AI. The more images you have, the smarter the algorithm will get, and the better it will be at serving recommendations users actually want. The largest public data set used by many researchers, Image Net, consists of 14 million crowdsourced photos of everyday objects. Pinterest has billions–uploaded by eager pinners, skimmed from blogs, and posted by corporations themselves–most of which are immaculately staged and lit because they’re official product photography. Computers see perfect images with more ease. Just as significant: These photos have been hand-tagged and labeled by Pinterest’s own loyal users for years.
“You want to have samples of everything that can happen and everything that can be seen. The larger the data set, the larger that probability that you’re not going to be surprised,” says Manuela Veloso, head of machine learning at Carnegie Mellon University. “What’s interesting about [Pinterest’s] billions is they’re going to cover extremes.”
It also helps that Pinterest, by design, offers somewhat fuzzy results for any search. Queries about denim jackets will elicit results with denim jackets. But if one image in the feed has black denim rather than blue, or perhaps a blue denim purse, it doesn’t look like a mistake. That’s the lesson Pinterest learned from avocados. Exact matches are the specialty of Google search, which has been optimized to respond to specific questions–like, “How do you grill fish?”–with the perfect link. Pinterest users tend to pose vaguer queries: They might search for “seafood dinner ideas” several times a week. For them, a non-exact match is not a error. It’s inspiration.
Source : https://www.fastcodesign.com/90152812/pinterest-sees-the-future744