{"id":373,"date":"2021-07-04T16:14:35","date_gmt":"2021-07-04T15:14:35","guid":{"rendered":"https:\/\/toshareproject.it\/artmakerblog\/?p=373"},"modified":"2021-07-04T16:14:35","modified_gmt":"2021-07-04T15:14:35","slug":"gpt-j-an-open-source-mimic-of-the-gpt-3-neural-net","status":"publish","type":"post","link":"https:\/\/toshareproject.it\/artmakerblog\/gpt-j-an-open-source-mimic-of-the-gpt-3-neural-net\/","title":{"rendered":"GPT-J, an open-source mimic of the GPT-3 neural net"},"content":{"rendered":"

*This seems like it might be significant.<\/p>\n

 <\/p>\n

https:\/\/towardsdatascience.com\/cant-access-gpt-3-here-s-gpt-j-its-open-source-cousin-8af86a638b11<\/a><\/p>\n

(…)<\/p>\n

EleutherAI project: Open-sourcing AI researchthe project was born in July 2020 as a quest to replicate OpenAI GPT-family models. A group of researchers and engineers decided to give OpenAI a \u201crun for their money\u201d and so the project began. <\/strong><\/p>\n

Their ultimate goal is to replicate GPT-3-175B to \u201cbreak OpenAI-Microsoft monopoly\u201d on transformer-based language models.<\/strong><\/p>\n

Since the transformer was invented in 2017, we\u2019ve seen increased effort in creating powerful language models. GPT-3 is the one that became a superstar, but all over the world companies and institutions are competing to find an edge that allows them to take a breath at a hegemonic position. In the words of Alexander Rush, a computer science professor at Cornell University, \u201cThere is something akin to an NLP space race going on.\u201d<\/strong><\/p>\n

Because powerful language models need huge amounts of computing power, big tech companies are best prepared to tackle the challenges. But, ahead of their interest in advancing science and helping humanity towards a better future, they put their need for profit. OpenAI started as a non-profit organization but soon realized they\u2019d need to change the approach to fund their projects. As a result, they partnered with Microsoft and received $1 billion. <\/strong><\/p>\n

Now, OpenAI has to move in between the commercial requirements imposed by Microsoft and its original mission.<\/strong><\/p>\n

EleutherAI is trying to compete with these two \u2014 and other \u2014 AI giants with help from Google and CoreWeave, their cloud computing providers. OpenAI\u2019s models and their specific characteristics aren\u2019t public, so EleutherAI\u2019s researchers are trying to solve the puzzle by combining their extensive knowledge with the sparse bits of info OpenAI has been publishing in their papers (GPT-1, GPT-2, GPT-3, and others)….<\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"

*This seems like it might be significant.   https:\/\/towardsdatascience.com\/cant-access-gpt-3-here-s-gpt-j-its-open-source-cousin-8af86a638b11 (…) EleutherAI project: Open-sourcing AI researchthe project was born in July 2020 as a quest to replicate OpenAI GPT-family models. A group of researchers and engineers decided to give OpenAI a \u201crun for their money\u201d and so the project began. Their ultimate goal is to replicate […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-373","post","type-post","status-publish","format-standard","hentry","category-uncategorised"],"yoast_head":"\nGPT-J, an open-source mimic of the GPT-3 neural net | Artmaker Blog<\/title>\n<meta name=\"description\" content=\"GPT-J, an open-source mimic of the GPT-3 neural net | Artmaker Blog\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/toshareproject.it\/artmakerblog\/gpt-j-an-open-source-mimic-of-the-gpt-3-neural-net\/\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"GPT-J, an open-source mimic of the GPT-3 neural net | Artmaker Blog\" \/>\n<meta property=\"og:description\" content=\"GPT-J, an open-source mimic of the GPT-3 neural net | Artmaker Blog\" \/>\n<meta property=\"og:url\" content=\"https:\/\/toshareproject.it\/artmakerblog\/gpt-j-an-open-source-mimic-of-the-gpt-3-neural-net\/\" \/>\n<meta property=\"og:site_name\" content=\"Artmaker Blog\" \/>\n<meta property=\"article:published_time\" content=\"2021-07-04T15:14:35+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Bruce Sterling\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebSite\",\"@id\":\"https:\/\/toshareproject.it\/artmakerblog\/#website\",\"url\":\"https:\/\/toshareproject.it\/artmakerblog\/\",\"name\":\"Artmaker Blog\",\"description\":\"on Toshareproject.it - curated by Bruce Sterling\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/toshareproject.it\/artmakerblog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-GB\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/toshareproject.it\/artmakerblog\/gpt-j-an-open-source-mimic-of-the-gpt-3-neural-net\/#webpage\",\"url\":\"https:\/\/toshareproject.it\/artmakerblog\/gpt-j-an-open-source-mimic-of-the-gpt-3-neural-net\/\",\"name\":\"GPT-J, an open-source mimic of the GPT-3 neural net | Artmaker Blog\",\"isPartOf\":{\"@id\":\"https:\/\/toshareproject.it\/artmakerblog\/#website\"},\"datePublished\":\"2021-07-04T15:14:35+00:00\",\"dateModified\":\"2021-07-04T15:14:35+00:00\",\"author\":{\"@id\":\"https:\/\/toshareproject.it\/artmakerblog\/#\/schema\/person\/6f20726ed2761431f3e0ff4e096c3085\"},\"description\":\"GPT-J, an open-source mimic of the GPT-3 neural net | Artmaker Blog\",\"breadcrumb\":{\"@id\":\"https:\/\/toshareproject.it\/artmakerblog\/gpt-j-an-open-source-mimic-of-the-gpt-3-neural-net\/#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/toshareproject.it\/artmakerblog\/gpt-j-an-open-source-mimic-of-the-gpt-3-neural-net\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/toshareproject.it\/artmakerblog\/gpt-j-an-open-source-mimic-of-the-gpt-3-neural-net\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/toshareproject.it\/artmakerblog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"GPT-J, an open-source mimic of the GPT-3 neural net\"}]},{\"@type\":\"Person\",\"@id\":\"https:\/\/toshareproject.it\/artmakerblog\/#\/schema\/person\/6f20726ed2761431f3e0ff4e096c3085\",\"name\":\"Bruce Sterling\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/toshareproject.it\/artmakerblog\/#personlogo\",\"inLanguage\":\"en-GB\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/c390e8ed4db57a34278dcf667f928a643cf769a865c8a8632dcd310412bb9a99?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/c390e8ed4db57a34278dcf667f928a643cf769a865c8a8632dcd310412bb9a99?s=96&d=mm&r=g\",\"caption\":\"Bruce Sterling\"},\"description\":\"Art director at Share Festival, author and journalist\",\"sameAs\":[\"http:\/\/toshareproject.it\/tomorrowart\"],\"url\":\"https:\/\/toshareproject.it\/artmakerblog\/author\/brucesterling\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"GPT-J, an open-source mimic of the GPT-3 neural net | Artmaker Blog","description":"GPT-J, an open-source mimic of the GPT-3 neural net | Artmaker Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/toshareproject.it\/artmakerblog\/gpt-j-an-open-source-mimic-of-the-gpt-3-neural-net\/","og_locale":"en_GB","og_type":"article","og_title":"GPT-J, an open-source mimic of the GPT-3 neural net | Artmaker Blog","og_description":"GPT-J, an open-source mimic of the GPT-3 neural net | Artmaker Blog","og_url":"https:\/\/toshareproject.it\/artmakerblog\/gpt-j-an-open-source-mimic-of-the-gpt-3-neural-net\/","og_site_name":"Artmaker Blog","article_published_time":"2021-07-04T15:14:35+00:00","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Bruce Sterling","Estimated reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebSite","@id":"https:\/\/toshareproject.it\/artmakerblog\/#website","url":"https:\/\/toshareproject.it\/artmakerblog\/","name":"Artmaker Blog","description":"on Toshareproject.it - curated by Bruce Sterling","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/toshareproject.it\/artmakerblog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-GB"},{"@type":"WebPage","@id":"https:\/\/toshareproject.it\/artmakerblog\/gpt-j-an-open-source-mimic-of-the-gpt-3-neural-net\/#webpage","url":"https:\/\/toshareproject.it\/artmakerblog\/gpt-j-an-open-source-mimic-of-the-gpt-3-neural-net\/","name":"GPT-J, an open-source mimic of the GPT-3 neural net | Artmaker Blog","isPartOf":{"@id":"https:\/\/toshareproject.it\/artmakerblog\/#website"},"datePublished":"2021-07-04T15:14:35+00:00","dateModified":"2021-07-04T15:14:35+00:00","author":{"@id":"https:\/\/toshareproject.it\/artmakerblog\/#\/schema\/person\/6f20726ed2761431f3e0ff4e096c3085"},"description":"GPT-J, an open-source mimic of the GPT-3 neural net | Artmaker Blog","breadcrumb":{"@id":"https:\/\/toshareproject.it\/artmakerblog\/gpt-j-an-open-source-mimic-of-the-gpt-3-neural-net\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/toshareproject.it\/artmakerblog\/gpt-j-an-open-source-mimic-of-the-gpt-3-neural-net\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/toshareproject.it\/artmakerblog\/gpt-j-an-open-source-mimic-of-the-gpt-3-neural-net\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/toshareproject.it\/artmakerblog\/"},{"@type":"ListItem","position":2,"name":"GPT-J, an open-source mimic of the GPT-3 neural net"}]},{"@type":"Person","@id":"https:\/\/toshareproject.it\/artmakerblog\/#\/schema\/person\/6f20726ed2761431f3e0ff4e096c3085","name":"Bruce Sterling","image":{"@type":"ImageObject","@id":"https:\/\/toshareproject.it\/artmakerblog\/#personlogo","inLanguage":"en-GB","url":"https:\/\/secure.gravatar.com\/avatar\/c390e8ed4db57a34278dcf667f928a643cf769a865c8a8632dcd310412bb9a99?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/c390e8ed4db57a34278dcf667f928a643cf769a865c8a8632dcd310412bb9a99?s=96&d=mm&r=g","caption":"Bruce Sterling"},"description":"Art director at Share Festival, author and journalist","sameAs":["http:\/\/toshareproject.it\/tomorrowart"],"url":"https:\/\/toshareproject.it\/artmakerblog\/author\/brucesterling\/"}]}},"_links":{"self":[{"href":"https:\/\/toshareproject.it\/artmakerblog\/wp-json\/wp\/v2\/posts\/373","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/toshareproject.it\/artmakerblog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/toshareproject.it\/artmakerblog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/toshareproject.it\/artmakerblog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/toshareproject.it\/artmakerblog\/wp-json\/wp\/v2\/comments?post=373"}],"version-history":[{"count":1,"href":"https:\/\/toshareproject.it\/artmakerblog\/wp-json\/wp\/v2\/posts\/373\/revisions"}],"predecessor-version":[{"id":374,"href":"https:\/\/toshareproject.it\/artmakerblog\/wp-json\/wp\/v2\/posts\/373\/revisions\/374"}],"wp:attachment":[{"href":"https:\/\/toshareproject.it\/artmakerblog\/wp-json\/wp\/v2\/media?parent=373"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/toshareproject.it\/artmakerblog\/wp-json\/wp\/v2\/categories?post=373"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/toshareproject.it\/artmakerblog\/wp-json\/wp\/v2\/tags?post=373"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}