Andrew Hutchinson
Source: www.socialmediatoday.com, October 2024
Could Meta build its own search engine, and if it did, what would that look like?
Reports have emerged that Meta’s looking to build its own search tool for Meta AI, in order to reduce its reliance on Google and Microsoft’s Bing for web searches within its AI chatbot.
When Meta first announced its Meta AI chatbot in September last year, it explained how it would be using web search results from Bing to help expand its responses:
“In text-based chats, Meta AI has access to real-time information through our search partnership with Bing and offers a tool for image generation.”
When Meta AI responses utilize Bing for information for context, Meta provides a web link to the search query.
Then in April this year, Meta announced an additional partnership with Google to expand its web search responses even further, providing a similar web link on Google-powered replies.
And now, according The Information, Meta’s looking to build its own web crawler to power search queries independently, in order to reduce its reliance on its search partners for Meta AI.
As per The Information:
“As Meta Platforms tries to keep up with OpenAI in developing artificial intelligence, the Facebook owner is working on a search engine that crawls the web to provide conversational answers about current events to people using its Meta AI chatbot.”
Conceptually, that would give Meta an alternative, in case Microsoft or Google, both of whom are competing with Meta on generative AI development, decided to end their search deals.
But that would also mean that Meta would need to build a complex search system, which crawls the web in a similar way to both Google and Bing. Which is a big undertaking, and why Meta went with these partnerships in the first place.
But could Meta actually create a web crawler that functions in a similar way, and would that be good enough to provide adequate answers in Meta’s chatbot?
Meta already has various web crawlers that gather information from external websites, and those crawlers have been ramping up activity in recent months, as Meta looks to gather more data for its AI projects. They could also already be gathering data for this new search engine, but a dedicated Facebook search engine, again, would be a big project.
Though it is a logical one in the broader information gathering sense, and it would also give Meta more data for its AI language models moving forward, and maybe, that’s actually the bigger picture here, as opposed to simply powering in-stream search.
It’s more likely that Meta’s shoring up its data sources to power its AI tools, and utilizing its existing crawlers to scrape whatever data it can from external providers who haven’t updated their robots.txt protections. That would help Meta build upon its already vast data stores, and if it’s already gathering that info, it makes sense to also reduce its reliance on external search providers where it can.
So maybe it’s likely less about building a competitor to these providers, and more about making best use of its own data gathering processes. But even so, it’s a significant undertaking, which could provide expanded opportunities for Zuck and Co., if they can get it right.
Really, though, it’s not an unexpected move, given the competition in the space. Again, Meta is battling with these two providers for AI supremacy, and as the race continues to heat up, it’ll come as no surprise if they stop helping Meta in this regard.
So there are a couple of angles to it, but it seems like Meta’s looking to expand its data-gathering process in order to continue to build more powerful, real-time AI data tools. Which could result in a broader Meta search tool, powering real-time insights for its apps. Which could have particular value for Threads, in providing up-to-the-minute notes on evolving news stories to complement engagement in the app.
