Groq: The Fastest AI in the World

Welcome Renaissance Creator,

While most of our recent updates have been in new models and software releases, today we will focus on the hardware side of things.

Groq, a company started by former Google TPU (Tensor Processing Unit) team members, just announced their new product, the fastest and cheapest access to AI processing on the market.

This edition will feature a lot more jargon than normal, because hardware is generally less discussed within the industry. Today, we will get into GPUs, LPUs (??), TaaS, and the backbone of what makes this industry run.

Let’s dive in.

The One Big Thing

When most people talk about “using AI”, they are referring to ChatGPT or a similar consumer facing application that they access over the internet to perform some task.

What is really happening is they are connecting to OpenAI’s servers, who are connecting to their private data center that houses large amounts of GPUs (Graphics Processing Units, usually NVIDIA’s), where Large Language Models (LLMs) run and AI processing is performed, a query is processed, and a result is then passed back to your computer.

Some people, generally hobbyists or power users, take this burden on themselves and host the LLMs on their own computers or custom hardware, taking on the AI processing themselves (and the costs that come with it). There are now plenty of open source LLMs you can simply download and run on your own hardware stack.

This last part, however, is more cumbersome and costly than most people can afford.

Enter Groq.

Groq is a service that runs these foundational models on their own custom hardware and allows users to access them over the internet (for free, for now). Based on early results, the answers they provide to user requests are much faster than those provided by ChatGPT. As a fun test, I asked Groq if it was a better AI than ChatGPT, and it refused to confirm or deny (see above photo).

The way Groq does this is by using LPUs (Language Processing Units) rather than GPUs as the hardware layer these models run on. Their team is made up of the initial Google group that worked on TPUs (Tensor Processing Units, different custom hardware for AI models), and has been building this for the past 8 years.

While many results so far, some are still a bit skeptical about how these TaaS (Token as a Service) costs scale over the long term.

Whether or not Groq and LPUs create a new standard for hardware and AI processing generally remains to be seen, but this is the first step in many people’s projections that compute will only get cheaper as time goes on, further democratizing access to this technology.

Artists’ Gallery

Open Source Text to Video Models

Text to 3D Model, coming soon to the Apple Vision Pro

Tools

Must have tools for every Renaissance creator to add to their toolkit:

Deep Tech

The newest and coolest in the research world that you need to know about:

Closing Thought

Work With Us!

The AI Renaissance is coming and we are building the best community of the people making it happen.

Contact us to sponsor your product or brand and reach the exact audience for your needs across our newsletter and podcast network.