The tech industry loves garage startup stories. From Hewlett-Packard to Google, the stories of self-made giants have inspired generations of entrepreneurs.
However, a start-up that wants to succeed with today’s hottest technology, the artificial intelligence used in chatbots like ChatGPT and Google Bard, will need huge amounts of money and computing power to make such an impressive move. tales may be becoming a thing of the past.
In 2019, Aiden Gomez and Nick Frost left Google to found an AI startup in Toronto called Cohere that could compete with their former employers. A few months later, they returned to Google, asking if they would sell the vast amount of computing power they would need to build their own AI technology. After Google CEO Sundar Pichai personally approved the deal, the tech giant gave them what they wanted.
“It’s ‘Game of Thrones.’ That’s what it is,” said David Katz, a partner at Cohere’s first investor, Radical Ventures. Big companies like Google, Microsoft and Amazon control the chip, he added. “They control computing power,” he says. “They are choosing who gets it.”
It’s hard to build a groundbreaking AI startup without the backing of a “hyperscaler” that manages the vast data centers where AI systems can run. And it puts the industry giant back in the driver’s seat of what many expect to be the most significant change for the tech industry in decades.
OpenAI, the startup behind ChatGPT, recently raised $10 billion from Microsoft. Most of that money will go to Microsoft in exchange for time on massive clusters of computer servers run by larger companies. Spanning thousands of specialized computer chips, these machines are essential to improving and expanding your skills in ChatGPT and similar technologies.
Competitors won’t be able to catch up to OpenAI unless they get comparable computing power. Cohere recently raised $270 million, bringing the total raised to over $440 million. Much of that money will be used to buy computing power from Google and others.
Other startups have made similar arrangements, most notably a Silicon Valley company called Anthropic, founded in 2021 by a group of former OpenAI researchers. Character.AI was founded by his two leading researchers at Google. Inflection AI founded by former Google executives. Inflation raised $1.3 billion in a funding round last week, bringing the total to $1.5 billion.
At Google, Gomez said, transformersthe underlying technology used to create chatbots such as ChatGPT and Google Bard.
Transformer is a powerful example of what scientists call neural networks, mathematical systems that can learn skills by analyzing data. Neural networks have been around for years, helping power everything from conversational digital assistants like Siri to instant translation services like Google Translate.
Transformers took the idea into new territory. Running hundreds or even thousands of computer chips allows us to analyze more data faster.
Using this technology, companies such as Google and OpenAI have started building systems that learn from vast amounts of digital text such as Wikipedia articles, digital books, and chat logs. As these systems have more data to analyze, they can now generate their own texts, such as regular reports, blog posts, poems, and computer code.
These systems, called large scale language models, currently power chatbots such as Google Bard and ChatGPT.
Long before ChatGPT came out, Gomez left Google to start his own company with Frost and fellow Toronto entrepreneur Ivan Chan. The goal was to build a large-scale language model comparable to Google’s.
At Google, he and his fellow researchers had access to nearly unlimited computing power. After I left the company, I needed something similar. So he and his co-founder bought it from his Google, which sells access to the same chip through his cloud computing services.
Over the next three years, Cohere built a large language model. Comparable to almost any other rival. The company is now selling the technology to other companies. The idea is to give every company the technology they need to build and run their own AI applications, from chatbots to search engines to tutors.
“The strategy is to build a platform that other people can build from and experiment with,” Gomez said.
OpenAI offers a service along the same lines called GPT-4, which many companies are already using to build chatbots and other applications. This new technology can analyze, generate and edit text. But soon it will be able to process images and sounds as well. OpenAI is preparing a version of GPT-4 that can inspect photos, instantly describe them, and even answer questions about them.
Microsoft CEO Satya Nadella said the company’s deal with OpenAI is like a reciprocal relationship the company has developed over the years with its smaller competitors. “I grew up in a company that always did these kinds of deals with other companies,” he told The New York Times earlier this year.
As the industry races to take on GPT-4, entrepreneurs, investors and experts are debating who will be the ultimate winner. Most agree that OpenAI is leading the way in this area. But Cohere and a handful of other companies are building similar technology.
Tech giants are in a strong position because they have the vast resources needed to push these systems ahead of everyone else.google too Patented for transformeris the underlying technology behind the AI systems that Cohere and many other companies are building.
But there is a wildcard: open source software.
Meta, another giant with the computing power needed to build the next wave of AI, recently open-sourced its latest large-scale language model. This means anyone can reuse it and build on top of it. Many in the field believe that with this kind of freely available software anyone can compete.
“If you put the brainpower of every researcher on the planet together, you could beat any company,” said Amr Awadala, chief executive of AI startup Vectara and a former Google executive. But you still have to pay to access the data centers of much larger competitors.
