We're less than 24 hours removed from Facebook's big announcement of Graph Search, and despite all the laudatory praise for the idea of the thing, early reviews have pointed to one very big issue: Graph Search doesn't always work. It's not glitchy, per se; it handles natural language well, says The Verge's Thomas Houston, and people seem to like the "amazing user interface and flexibility," too. But the "third pillar" of Facebook just doesn't yet have enough data to offer relevant answers to social-search questions. The results are "lacking" says Engadget, and Gizmodo calls this "hollow" experience an "authority problem." Or more bluntly, as writes Slate's Farhad Manjoo put is, "The search results aren't that good." That's right, for all the users and user information the social network has compiled in its nine-year history, Facebook search has a data problem. So, despite looking like a "visually rich beast" with the potential to kill Google, Yelp, OkCupid, and LinkedIn, Graph Search doesn't sound like a replacement for any of those services because users haven't handed over quite enough intel for all the right answers — at least not in the first 24 hours.
Specifically, Graph Search's early beta users and reviewers found that the proprietary Facebook algorithm didn't offer useful results. For example, when Manjoo searched "Restaurants liked by people who live in Palo Alto, California," Facebook recommended the Facebook Culinary Team. "Now, I’ve had a couple good sandwiches at Facebook’s café, but I don’t remember ever being knocked off my feet," he explains. After a series of successful searches, SearchEngineLand's Danny Sullivan encountered a similar problem while searching for restaurants in Newport Beach: "Trust me, Newport Dunes isn't a hot restaurant."
Before you even get to answers, Graph Search only has a limited number of search queries available at this early stage in beta testing. Users still can't search for music or get too specific with questions they're asking — "running shoes liked by people who have run marathons" didn't work for Manjoo. Lars Rasmussen, the lead engineer on the project, told Manjoo that expanding the pool of nouns and verbs is a longer-term goal for the Graph Search team. (Then again, Facebook said that about Open Graph, too. And we're still waiting for the search term "watching," which seems pretty important.)
Part of the current relevancy problem has to do with the limited number of users who have access to the Graph Search right now. Facebook will do a very slow roll-out of the product, meaning right now the data set is very limited. Algorithms improve with more use, as GigaOm's Om Malik explains: "With as many as a billion searches on Facebook every day, even few million queries are going to be enough to help fine tune this ranking algorithm."
But the problem goes beyond usage. The early Graph Search complaints also have to do with the type of information people give Facebook. To deliver results, Graph Search draws from a data set that includes "likes," check-ins, engagement, photos, and personal information, all of which we've poured into the social network — but to differing degrees, since we didn't know Graph Search was ever really coming along. For some queries like "friends that work at The Verge that are younger than 25" or "engineers at Google who are friends of engineers at Facebook," Facebook will have all the data it needs to give the right answers. That requires a little "About Me" info, and that's that. But when it comes to recommending places, books, or movies, Graph Search either needs "likes" or check-ins, something that requires constant updating. Users have to actively "like" a plumber, update their favorites, or check in to their favorite Indian restaurant with Facebook on a regular basis. And right now Yelp and other service-oriented sites might have a leg up on user comfort there.
The preliminary round of Graph Search testing suggests that people simply aren't used to using Facebook in a way that Graph Search requires, at least not enough to make the search engine something revolutionary right now. As CNET's Jennifer Van Grove put it: "The 'liked by' filter for queries is a poor replacement for actual affinity, but it's currently the primary filter for plucking recommended content from friends out of Facebook's search experience." That doesn't mean Graph Search doesn't have potential. That the tool exists — and that Facebook will no doubt push for it hard as the third head, along with News Feed and Timeline, of its information monster to "make the world more open and connected" — might get people to start liking and checking in more often. That's Facebook's hope, at least.
There's always the downside: Without total utility at the outset, Graph Search could languish and go the way of Google Wave. (Rasmussen worked on that doomed Google project, by the way.) It doesn't sound that hopeless this early. The bad results "aren't fatal," explains Manjoo, and besides, it's really early.