{"id":43,"date":"2021-10-14T11:00:00","date_gmt":"2021-10-14T11:00:00","guid":{"rendered":"http:\/\/pc-keeper.tech\/index.php\/2021\/10\/14\/ais-smarts-now-come-with-a-big-price-tag\/"},"modified":"2021-10-14T11:00:00","modified_gmt":"2021-10-14T11:00:00","slug":"ais-smarts-now-come-with-a-big-price-tag","status":"publish","type":"post","link":"https:\/\/pc-keeper.tech\/index.php\/2021\/10\/14\/ais-smarts-now-come-with-a-big-price-tag\/","title":{"rendered":"AI\u2019s Smarts Now Come With a Big Price Tag"},"content":{"rendered":"<p> [ad_1]<br \/>\n<\/p>\n<div>\n<p><span class=\"lead-in-text-callout\">Calvin Qi, who<\/span> works at a search startup called Glean, would love to use the latest artificial intelligence algorithms to improve his company\u2019s products.<\/p>\n<p class=\"paywall\">Glean provides tools for searching through applications like Gmail, Slack, and Salesforce. Qi says new AI techniques for parsing language would help Glean\u2019s customers unearth the right file or conversation a lot faster.<\/p>\n<p class=\"paywall\">But training such a cutting-edge AI algorithm costs several million dollars. So Glean uses smaller, less capable AI models that can\u2019t extract as much meaning from text.<\/p>\n<p class=\"paywall\">\u201cIt is hard for smaller places with smaller budgets to get the same level of results\u201d as companies like Google or Amazon, Qi says. The most powerful AI models are \u201cout of the question,\u201d he says.<\/p>\n<p class=\"paywall\">AI has spawned exciting breakthroughs in the past decade\u2014programs that can beat humans at complex games, steer cars through city streets under certain conditions, respond to spoken commands, and write coherent text based on a short prompt. Writing in particular relies on recent advances in computers\u2019 ability to parse and manipulate language.<\/p>\n<p class=\"paywall\">Those advances are largely the result of feeding the algorithms more text as examples to learn from, and giving them more chips with which to digest it. And that costs money.<\/p>\n<p class=\"paywall\">Consider OpenAI\u2019s language model GPT-3, a large, mathematically simulated neural network that was fed reams of text scraped from the web. GPT-3 can find statistical patterns that predict, with striking coherence, which words should follow others. Out of the box, GPT-3 is significantly better than previous AI models at tasks such as answering questions, summarizing text, and correcting grammatical errors. By one measure, it is 1,000 times more capable than its predecessor, GPT-2. But training GPT-3 cost, by some estimates, almost $5 million.<\/p>\n<p class=\"paywall\">\u201cIf GPT-3 were accessible and cheap, it would totally supercharge our search engine,\u201d Qi says. \u201cThat would be really, really powerful.\u201d<\/p>\n<aside aria-hidden=\"true\" class=\"PullQuoteEmbedWrapper-sc-ehaOzL rhrog\">\n<div class=\"PullQuoteEmbedContent-fPpgtc izZYPW\">\n<p class=\"paywall\">\u201cIt is hard for smaller places with smaller budgets to get the same level of results\u201d as companies like Google or Amazon.<\/p>\n<p class=\"paywall\">Calvin Qi, Glean<\/p>\n<\/div>\n<\/aside>\n<p class=\"paywall\">The spiraling cost of training advanced AI is also a problem for established companies looking to build their AI capabilities.<\/p>\n<p class=\"paywall\">Dan McCreary leads a team within one division of Optum, a health IT company, that uses language models to analyze transcripts of calls in order to identify higher-risk patients or recommend referrals. He says even training a language model that is one-thousandth the size of GPT-3 can quickly eat up the team\u2019s budget. Models need to be trained for specific tasks and can cost more than $50,000, paid to cloud computing companies to rent their computers and programs.<\/p>\n<p class=\"paywall\">McCreary says cloud computing providers have little reason to lower the cost. \u201cWe cannot trust that cloud providers are working to lower the costs for us building our AI models,\u201d he says. He is looking into buying specialized chips designed to speed up AI training.<\/p>\n<p class=\"paywall\">Part of why AI has progressed so rapidly recently is because many academic labs and startups could download and use the newest ideas and techniques. Algorithms that produced breakthroughs in image processing, for instance, emerged from academic labs and were developed using off-the-shelf hardware and openly shared data sets.<\/p>\n<p class=\"paywall\">Over time, though, it has become increasingly clear that progress in AI is tied to an exponential increase in the underlying computer power.<\/p>\n<p class=\"paywall\">Big companies have, of course, always had advantages in terms of budget, scale, and reach. And large amounts of computer power are table stakes in industries like drug discovery.<\/p>\n<aside aria-hidden=\"true\" class=\"PullQuoteEmbedWrapper-sc-ehaOzL rhrog\">\n<div class=\"PullQuoteEmbedContent-fPpgtc izZYPW\">\n<p class=\"paywall\">\u201cI think it does cut down innovation.\u201d<\/p>\n<p class=\"paywall\">Chris Manning, professor in machine learning, Stanford\u00a0<\/p>\n<\/div>\n<\/aside>\n<p class=\"paywall\">Now, some are pushing to scale things up further still. Microsoft said this week that, with Nvidia, it had built a language model more than twice as large as GPT-3. Researchers in China say they\u2019ve built a language model that is four times larger than that.<\/p>\n<p class=\"paywall\">\u201cThe cost of training AI is absolutely going up,\u201d says David Kanter, executive director of MLCommons, an organization that tracks the performance of chips designed for AI. The idea that larger models can unlock valuable new capabilities can be seen in many areas of the tech industry, he says. It may explain why Tesla is designing its own chips just to train AI models for autonomous driving.<\/p>\n<p class=\"paywall\">Some worry that the rising cost of tapping the latest and greatest tech could slow the pace of innovation by reserving it for the biggest companies, and those that lease their tools.<\/p>\n<p class=\"paywall\">\u201cI think it does cut down innovation,\u201d says Chris Manning, a Stanford professor who specializes in AI and language. \u201cWhen we have only a handful of places where people can play with the innards of these models of that scale, that has to massively reduce the amount of creative exploration that happens.\u201d<\/p>\n<\/div>\n<p>[ad_2]<br \/>\n<br \/><a href=\"https:\/\/www.wired.com\/story\/ai-smarts-big-price-tag\">Source link <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>[ad_1] Calvin Qi, who works at a search startup called Glean, would love to use the latest artificial intelligence algorithms&hellip;<\/p>\n","protected":false},"author":1,"featured_media":44,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[42,43,44,45,46,47,48,49,50,16,41,51],"tags":[],"class_list":["post-43","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-hub","category-ai-hub-application-text-generation","category-ai-hub-company-open-ai","category-ai-hub-end-user-big-company","category-ai-hub-end-user-small-company","category-ai-hub-sector-health-care","category-ai-hub-sector-it","category-ai-hub-source-data-text","category-ai-hub-technology-natural-language-processing","category-business","category-business-artificial-intelligence","category-machine-earning"],"_links":{"self":[{"href":"https:\/\/pc-keeper.tech\/index.php\/wp-json\/wp\/v2\/posts\/43","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pc-keeper.tech\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/pc-keeper.tech\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/pc-keeper.tech\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/pc-keeper.tech\/index.php\/wp-json\/wp\/v2\/comments?post=43"}],"version-history":[{"count":0,"href":"https:\/\/pc-keeper.tech\/index.php\/wp-json\/wp\/v2\/posts\/43\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/pc-keeper.tech\/index.php\/wp-json\/wp\/v2\/media\/44"}],"wp:attachment":[{"href":"https:\/\/pc-keeper.tech\/index.php\/wp-json\/wp\/v2\/media?parent=43"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/pc-keeper.tech\/index.php\/wp-json\/wp\/v2\/categories?post=43"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/pc-keeper.tech\/index.php\/wp-json\/wp\/v2\/tags?post=43"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}