#MS MONOPOLY FLOPS UPDATE#
#MS MONOPOLY FLOPS SOFTWARE#

In reality they probably batch during high volume, but have GPUs sitting fallow during low volume.- Tom Goldstein December 6, 2022 I assume nodes are always in use with a batch size of 1. This is a back-of-the-envelope calculation. I estimate the cost of running ChatGPT is $100K per day, or $3M per month.

A back-of-the-envelope estimation shows that with one million users, ChatGPT costs $100,000 per day and around $3 million per month. By controlling the entire market, it has managed to create a self-reinforcing loop in which it collects more data, improves its search results, and serves more relevant ads.Īs a potential search engine, ChatGPT doesn’t have a business model yet. And its ad network also expands to websites and other mediums.īasically, Google controls two sides of a market: The content seekers and the advertisers. And let’s not forget that Google has many other products that enhance the digital profiles it creates for its users, including YouTube, Gmail, Chrome, and Android. This makes its business even more efficient and profitable. Google also has the power to personalize search results and ads based on the data it collects from users. But Google’s share of the online search market is so large that even with very low click-through rates, it makes billions of dollars yearly. You rarely click on the ads that increasingly show up above the fold on the search engine results page. Google Search is not a perfect business model. Google has built an advertising empire on its search engine. The business model for ChatGPTīut perhaps the biggest challenge of an LLM-based search engine is the business model. And as the LLM’s training corpus grows, the model will also have to become larger to generalize well across its knowledge base. Given the nonlinear nature of deep neural networks, there are limits to how much you can parallelize the inference operation. But the amount of computation is still a lot more than querying indexes. Granted, the size of the neural network is not comparable to search engine databases. LLMs, on the other hand, run information through the entire neural network every time they receive a prompt. Therefore, even though the body of online information is growing, search engine speed is not dropping. They have indexing, sorting, and search algorithms that can pinpoint the right records at very fast speeds. Search engines don’t need to browse their entire dataset for each query. LLMs such as ChatGPT take several seconds to compose their responses. Companies like Google have created highly optimized database infrastructure that can find millions of answers in less than a second. LLMs also have an inference speed problem. Again, we see the traces of search engine components. The operators of the LLM search engine also need mechanisms and tools to determine which web sources are reliable sources of knowledge and should be given priority. Setting up and parallelizing these processors for training and running the models is both a technical and financial challenge. Since there is no single piece of hardware that can fit the model, it has to be broken up and spread across several processors, such as A100 GPUs. And it has to be done multiple times per day if it is to stay up to date with the latest news.īased on GPT 3.5, ChatGPT probably has at least 175 billion parameters.

#MS MONOPOLY FLOPS UPDATE#

Maybe not every update will require full retraining, but it will nonetheless be much more expensive than adding and modifying records in a search engine database. Updating the search engine database is also a very cost-effective operation.īut for large language models, adding new knowledge requires retraining the model.

#MS MONOPOLY FLOPS SOFTWARE#

Search engines have the tools and software to constantly index new and modified pages. Another challenge ChatGPT and other LLMs face is updating their knowledge base.

Ms monopoly flops