Google's Gemini is polarizing opinions

Read time: 2.5 minutes

Welcome back, Superhuman

Google’s launch of Gemini, its competitor to GPT-4, has been nothing short of dramatic. The launch came with big claims of Gemini beating GPT-4 and several performance benchmarks. But as the dust settled on the announcement, many experts and media outlets began casting doubts over some of the claims and marketing material.

TODAY’S MENU

• Gemini is courting controversy. We evaluate the claims.

• How to create QR codes hidden in art

• Infographic: Top AI apps by Discord invite traffic

• Friday Laughs: GPT-4 vs Gemini vs Your Dad

• 5 new AI tools to boost your productivity

• AI Generated Images: Cosmic Santa

NEWS

• Soaring High: Time Magazine names OpenAI’s Sam Altman their CEO of the Year.

• Earned It: Wikipedia names ChatGPT as the most viewed article this year with 49.5 million page views.

• Getting Complacent? OpenAI admits ChatGPT is getting lazier and says they’re looking into fixing it.

• The Competition: X is rolling out its ChatGPT competitor Grok to users.

• Across the Pond: Google’s Gemini won’t be available yet in Europe and the UK due to regulatory hurdles.

INSIGHT

Google’s Gemini launch was widely applauded. Now, some of the claims are drawing controversy.

“Google, this is embarrassing“ tweets Machine Learning engineer Santiago, describing one of Google’s demo videos for its new AI model Gemini which has generated millions of impressions across different social media platforms.

The video in question shows Gemini seamlessly answering questions about several images that are being shown to it. However, there’s one big problem with this video: it’s not happening in real-time like it’s being shown. According to a Bloomberg article, the video demo “wasn’t carried out in real time or in voice.“.

This information has cast some doubt over the model’s features and its performance. Many social media accounts and some media outlets have called the video ‘fake.‘

Another point of debate is how well Gemini performed on the MMLU, a popular benchmark used to evaluate the knowledge and problem-solving ability of AI models.

Google claimed that Gemini was the first AI model to outperform human experts on the test. However, Brett Winton from ArkInvest and others pointed out that the results were achieved by deploying certain prompting techniques, and that Gemini is likely behind both human experts and GPT-4 on the benchmark.

While some of the frustrations and criticisms leveled at Google are understandable, accusing Google of ‘lying’ or ‘faking’ might be a bit of a stretch. The YouTube video description of the demo mentioned earlier states the following: “For the purposes of this demo, latency has been reduced and Gemini outputs have been shortened for brevity.“ As for the MMLU claim, Google DeepMind’s website states that different prompting techniques were used.

While both sides have valid arguments, a tweet from the CEO of Perplexity AI Aravind Sriniva takes a balanced view: “Reality: Gemini is cool. The first model that genuinely is comparable to GPT 4. Real accomplishment. Especially that it was just a dense model. Marketing was overboard, but Deepmind is known for aggressive PR. Demos like the multimodal video in reality will be possible in less than a year.”

TOGETHER WITH AE STUDIO

Hire a world class AI team for 80% less

Trusted by leading startups and Fortune 500 companies

Building an AI product is hard. Engineers who understand AI are expensive and hard to find. And there's no way of telling who's legit and who's not.

That's why companies around the world trust AE Studio. We help you craft and implement the optimal AI solution for your business with our team of world class AI experts from Harvard, Stanford and Princeton.

Our development, design, and data science teams work closely with founders and executives to create custom software and AI solutions that get the job done for a fraction of the cost.

Book a free consultation session now

p.s. mention you came from Superhuman to get an exclusive $10,000 discount on your first project.

AI AT WORK

How to create QR codes hidden in art

From restaurant menus to product discounts and hidden features, scannable QR codes have seen a resurgence in recent years. But most QR codes are pretty bland. Here’s how to create a QR code that stands out and gets the attention of customers:

• Go to the OpenArt QR generator website here 

• Sign up for free to get access

• Then enter the website you want to create a QR code for

• Pick the style of image you want to generate for the QR code

• Click generate and scroll down to see the results

• Download the image

The whole process takes about 2 minutes. You’ll be able to scan the QR code image with your phone and get to the website you want.

INFOGRAPHIC

Source

TOGETHER WITH DEEPGRAM

There’s a new text-to-speech (TTS) API in town. Introducing Aura, a powerful real-time text-to-speech API designed for conversational voice applications. Compared to alternatives, Aura produces human-like speech more quickly and efficiently.

Learn more about Deepgram Aura, or be the first to access the new API by joining the waitlist here →

5 AI Tools to Supercharge Your Productivity

Parsio: Extract structured data from your PDFs, emails and other documents, automatically.

Strut: Capture projects, notes, drafts, and more in collaborative workspaces using AI.

Innovating with AI (sponsored): Elevate your workflow with the No-Code AI Toolkit. Designed to help you save hours with automation tools and streamlining tasks without code. Transform how you work — get instant access now.

Dumbbell: Upgrade your workout experience with motion tracking fitness using just your phone camera. Enables you to automatically log workouts, and count reps/sets.

Kommunicate: Supercharge your customer support with AI-powered chatbot. Reduce support costs, elevate customer experience and grow your business.

FRIDAY LAUGHS

A lighthearted moment to kick start your weekend

AI-GENERATED IMAGES

Cosmic Santa

ADVERTISE WITH US

Acquire new customers and drive revenue by partnering with us

Superhuman is the world’s biggest AI newsletter for businesses and professionals with 500,000+ readers working at the world’s leading startups and enterprises. Companies like Amazon, Calendly, and Notion feature their products in Superhuman. Main ads are typically sold out 4 weeks in advance. You can book future ad spots here.  

🧞 Your wish is my command

Reviews of the day

Thanks for reading.

Until next time!

Zain

p.s. if you want to sign up for this newsletter or share it with a friend or colleague, you can find us here