With the Groq AI model making ChatGPT appear weak and drawing parallels with Elon Musk’s model, also named Grok, social media users are beginning to notice.
Groq is the latest artificial intelligence (AI) model that is causing quite a stir on social media thanks to its response speed and innovative technologies that may eliminate the need for GPUs.
Also read: AI is Making Phishing More Convincing Than Ever
The AI model became an overnight sensation after its public benchmark tests went viral on the social media platform X, demonstrating that Grog outperformed the popular AI chatbot ChatGPT.
The first public demo using Groq: a lightning-fast AI Answers Engine.
It writes factual, cited answers with hundreds of words in less than a second.
More than 3/4 of the time is spent searching, not generating!
The LLM runs in a fraction of a second.https://t.co/dVUPyh3XGV https://t.co/mNV78XkoVB pic.twitter.com/QaDXixgSzp
— Matt Shumer (@mattshumer_) February 19, 2024
Groq using language processing unit (LPU)
Groq’s response speed is due to the team behind the AI model developing a unique application-specific integrated circuit (ASIC) chip for large language models (LLMs), which allows it to produce 500 tokens per second. The publicly available version of the model, ChatGPT-3.5, can generate around 40 tokens per second.
Rather than using the expensive and rare graphics processing units (GPUs) usually used to run AI models, the developer of this model, Groq Inc., claims to have created the first language processing unit (LPU) through which it runs its model.
Wow, that's a lot of tweets tonight! FAQs responses.
• We're faster because we designed our chip & systems
• It's an LPU, Language Processing Unit (not a GPU)
• We use open-source models, but we don't train them
• We are increasing access capacity weekly, stay tuned pic.twitter.com/nFlFXETKUP— Groq Inc (@GroqInc) February 19, 2024
According to Groq, LPUs are also more energy efficient. LPUs can do more computations per watt thanks to their ability to prevent underutilization of cores and reduce the effort required to manage multiple threads.
Additionally, multiple TSPs can be connected thanks to Groq’s chip design, which eliminates the traditional bottlenecks associated with GPU clusters. According to Groq, this minimizes the hardware requirements for large AI models and makes the system scalable.
Groq challenges Grok.
However, the company behind Groq was founded in 2016, and Groq was registered as a trademark. The creators of the Groq model released a blog post last November, calling out Elon Musk for his choice of moniker, Grok (but spelled with a “k”), as the AI model was beginning to gain more popularity.
Ohhh and one more thing, Groq is way older than Elons Grok, and in November when Elon launched Grok, Groq replied.
So I can sense a lawsuit coming soon. https://t.co/BQLtDvjQOS
— Linus (●ᴗ●) (@LinusEkenstam) February 19, 2024
In the blog post, Groq said they can see why Musk might want to adopt their name. Musk likes fast things (rockets, hyperloops, one-letter company names), and their product, the Groq LPU Inference Engine, is the quickest way to run large language models (LLMs) and other generative AI applications. However, they must ask Musk to please choose another name and fast.
Neither Grok nor Musk has commented on X (formerly Twitter) on the similarity between the names of the two models since Groq went viral on social media.
Users react
However, several users on the platform have started comparing the LPU model and other popular GPU-based models.
According to one user who works in AI development, Groq is a “game changer” for products that require low latency (which refers to the time it takes to execute a request and get a response).
side by side Groq vs. GPT-3.5, completely different user experience, a game changer for products that require low latency pic.twitter.com/sADBrMKXqm
— Dina Yerlan (@dina_yrl) February 19, 2024
Another user said that Groq’s LPUs might be a good substitute for the “high-performing hardware” of the highly sought-after Nvidia A100 and H100 chips, as well as a “massive improvement” over GPUs in the future when it comes to meeting the demands of AI applications.
This occurs when leading AI developers are working to create their chips internally to avoid depending solely on Nvidia’s models.
and then