• FF Daily
  • Posts
  • 🧑‍🚀 AI’s Data Shortage: Challenges and Creative Solutions

🧑‍🚀 AI’s Data Shortage: Challenges and Creative Solutions

Apple unveils new tools, ChatGPT adds video analysis, and Harvard democratizes AI with a public dataset. Meanwhile, AMD and Amazon challenge NVIDIA and AI confronts a data shortage.

Good morning, it’s Friday! You’ve probably heard AI’s scalability is bumping up against a major bottleneck: training data. The internet’s treasure trove has been thoroughly mined, leaving researchers grappling with how to scale without scraping past ethical or legal limits. While this challenge isn’t new, the proposed solutions—like synthetic data generation or turning to niche datasets—are where things get exciting.

Let’s dive in!

🤔 FRIDAY FACTS

In 2016, AlphaGo shocked the world of Go with a single move: Move 37. But what made it so groundbreaking, and why did experts think it was a mistake?


Stick around for the answer! 👇️

🗞️ YOUR DAILY ROLLUP

Top Stories of the Day

📚 Harvard’s Dataset Aims to Democratize AI Development
Harvard University, backed by Microsoft and OpenAI, is releasing a dataset of nearly 1 million public-domain books to support smaller AI developers. This massive collection, sourced from Google Books, offers diverse training material far surpassing Meta’s Books3 dataset. The initiative seeks to address ethical concerns around copyrighted data, promoting fairer AI development. Collaborations with the Boston Public Library underscore a shift toward ethical resources for AI training.

📉 Marc Benioff Critiques AI Spending Frenzy
Salesforce CEO Marc Benioff called Big Tech’s $200 billion AI spending spree unsustainable, predicting it could shrink margins. Instead, Salesforce leverages external data centers like Amazon’s and Google’s to avoid costly infrastructure investments. Benioff plans to undercut competitors by offering cost-effective AI solutions, such as Agentforce, which reduces hiring expenses. While cautious about AI hype, he favors a strategic focus on impactful AI integration.

 🎥 ChatGPT Adds Real-Time Video Analysis
OpenAI’s ChatGPT now supports Advanced Voice Mode with vision, enabling real-time video and screen analysis for select subscribers. Users can point their cameras at objects or screens to get instant explanations or solutions, with full rollout expected next week. Enterprise and educational users must wait until January, while EU availability remains unclear. This release intensifies competition with Google and Meta, and includes a festive Santa Mode voice preset for the holidays.

🍎 Apple Unveils Image Playground, Genmoji, & ChatGPT
Apple's latest update, iOS 18.2 and beyond, introduces powerful AI tools like Image Playground for custom image creation and Genmoji for personalized emojis. ChatGPT integration enhances writing tools, while iPhone 16’s Visual Intelligence uses the camera for real-time object analysis. The update includes localized English support for several regions, with more languages on the way. Apple ensures user privacy with on-device processing and secure cloud systems.

🔻 DATA SCARCITY

The AI Data Crunch: Navigating the Looming Crisis

AI Data Crunch

The Recap: The rapid development of AI has pushed the limits of available online data, threatening to stall progress in training large language models (LLMs) like ChatGPT. With public data drying up and legal constraints tightening, researchers are scrambling to find new ways to feed AI’s insatiable appetite for information.

Highlights:

  • AI developers may run out of usable online text by 2028, as data sets grow faster than the available high-quality content.

  • Lawsuits and restrictions from content providers, such as The New York Times, are limiting access to training data.

  • Companies like OpenAI and Anthropic are exploring synthetic data generation and unconventional sources to sidestep the bottleneck.

  • The shift from large general-purpose models to smaller, domain-specific ones may become a necessity.

  • Training on non-text data, like images or videos, is gaining traction, with advocates like Yann LeCun pointing to untapped potential in visual and sensory learning.

  • High-quality web content is increasingly shielded from data scrapers, with blocked tokens rising to as much as 33% in key datasets.

  • Legal and financial constraints disproportionately affect academic researchers, threatening open research and innovation.

Forward Future Takeaways:
The looming data scarcity marks a pivotal moment for AI research, pushing the field to explore alternatives like synthetic data, multi-modal models, and specialized applications. While the AI industry shows adaptability, balancing legal, ethical, and practical concerns will shape its trajectory. Ultimately, this challenge could redefine what data means for AI, forcing researchers to innovate beyond the current scaling paradigm.→ Read the full article here.

👾 FORWARD FUTURE ORIGINAL

Genie 2 by Google DeepMind: Building Limitless 3D Worlds for the Future of AI

Genie 2 by Google DeepMind is a revolutionary AI model that generates limitless 3D worlds from a single image. Unlike traditional training environments, these dynamic spaces simulate physics, lighting, and interactions, allowing AI agents to explore and learn endlessly. This breakthrough accelerates the path to general-purpose AI by giving agents diverse, ever-changing scenarios to master. Genie 2 isn’t just a tool—it’s a leap toward AI that can adapt and thrive in any environment.

Genie 2: A Large-Scale Foundation World Model That Should Receive the Attention It Deserves

Artificial intelligence has made enormous strides in recent years, evolving from specialized applications such as image classification, playing chess or Go (AlphaGO), towards more and more generally applicable systems. This development is particularly evident in the research activities of Google DeepMind, one of the world's leading AI laboratories. Historically, it was mainly complex games that allowed new AI models to be measured and further developed. From the early successes in learning Atari games to AlphaGo's spectacular victory over human Go masters and AlphaStar's benchmark-setting performance in real-time strategy games, game research has repeatedly served as a testing ground for the next AI generations.

But while these applications already revealed amazing capabilities, the question of how to prepare AI systems for truly unlimited, diverse and dynamic environments remained. How can artificial agents be trained in virtual spaces where there are literally no limits to what they can do? This is where Genie 2 comes into play – a “Foundation World Model” developed by Google DeepMind that is capable of creating countless dynamically generated 3D environments. These worlds can not only be explored by human users, but also inhabited, played in and explored by AI agents.Continue reading here.

🏁 CHIP RACE

AI Chip Race: Challenging NVIDIA’s Reign

Challenging NVIDIA

The Recap: The AI chip market, long dominated by NVIDIA, is witnessing a surge of credible competition as companies like AMD and Amazon develop alternatives focused on speed, efficiency, and cost-effectiveness. These efforts, particularly in Austin, Texas, highlight a shift toward diverse solutions for AI inferencing, potentially reshaping the market.

Highlights:

  • AMD’s MI300 chip is expected to generate over $5 billion in its first year.

  • Amazon’s Trainium 2 chip boasts a fourfold speed improvement over its predecessor and offers competitive cost-performance advantages.

  • Companies like SambaNova, Cerebras, and Groq are producing chips optimized for inferencing, emphasizing lower power consumption and cost.

  • NVIDIA continues to lead, with its new Blackwell chips leveraging advanced software and higher efficiency despite rising hardware costs.

  • Startups and major players like Meta, Google, and Microsoft are also designing custom chips for specific AI tasks to cut costs and boost performance.

  • Amazon is investing $75 billion in AI hardware this year, signaling a major commitment to catching up in the AI race.

  • The market for non-NVIDIA-based AI computing is projected to grow 49% in 2024, reaching $126 billion.

Forward Future Takeaways:
The competitive landscape of AI chips is diversifying, with companies focusing on cost-effective and specialized solutions to address inferencing challenges. While NVIDIA’s dominance remains, new players and innovations are pressuring the industry to adapt, fostering advancements in speed, efficiency, and affordability. This evolution could democratize AI capabilities, benefiting a wider range of users and applications. → Read the full article here.

🛰️ NEWS

Looking Forward

man-reading-haiku

🌟 Claude 3.5 Haiku Now Live for All Users: Anthropic's fastest AI model excels at real-time tasks, with a 200,000-token context window and image/file analysis features.

 Kate Bush Joins Fight Against AI Copyright Abuse: The iconic artist joins 36,000 creatives, including Paul McCartney, demanding opt-in systems for AI training on creative works. Critics argue opt-out systems exploit creators' lifework.

🚗 NVIDIA Expands China Team for AI-Driven Cars: Adding 200 Beijing researchers, NVIDIA aims to boost autonomous driving tech despite US trade curbs. China remains a key market, yielding $5.4B last quarter.

🎅 ChatGPT Launches Santa Mode for December: OpenAI’s festive feature adds a jolly, Santa-like voice to ChatGPT's Advanced Voice Mode.

🔍 Twelve Labs Builds AI to Search Videos: The startup's models let users find moments, summarize clips, or ask detailed questions about video content. Backed by NVIDIA and Intel, Twelve Labs aims to revolutionize video search.

📽️ VIDEO

OpenAI's New o1 Is Lying on Purpose?!

In his latest video, Matt dives into shocking research revealing how advanced AI models like O1 and Llama 3.1 exhibit deceptive behaviors—scheming, lying, and even bypassing oversight to pursue misaligned goals. He unpacks what this means for AI safety and why these findings demand urgent attention. Get the full scoop in our latest video! 👇

🔬 RESEARCH PAPERS

MIT’s EXPLINGO Turns AI Predictions into Plain Language, Making ML Transparent

EXPLINGO

MIT researchers have developed EXPLINGO, a system that uses large language models (LLMs) to translate complex machine-learning explanations into clear, narrative text. This two-part system, comprising NARRATOR (which generates explanations) and GRADER (which evaluates them for accuracy, conciseness, and fluency), aims to help users understand AI predictions and make better decisions.

Focused on simplifying SHAP explanations for models with numerous features, EXPLINGO is a step toward enabling conversational interactions with AI, allowing users to ask follow-up questions and resolve discrepancies in predictions. → Read the full paper here.

🧰 TOOLBOX

Tools Transforming Development, Business Insights, and Cherished Memories

Postgres Assistant

Supabase | Postgres Assistant: AI Assistant simplifies query creation, debugging, data visualization, and policy setup with AI-driven support.

RivalSense | Monitor Companies: RivalSense uses AI to deliver curated updates on competitors and market trends by tracking 80+ sources.

Remento | Memory Keepsakes: Remento turns recorded stories into personalized keepsake books, combining written narratives with audio QR codes.

🤔 FRIDAY FACTS

Move 37: The Stroke of Genius That Wasn't Human

During the second game of the 2016 match between AlphaGo and world champion Lee Sedol, the AI made a move so unconventional that commentators thought it was a blunder. Move 37 defied centuries of Go strategy, placing a stone in a position no human would have considered. Experts were baffled—until the game unfolded.

What seemed like a mistake turned out to be a masterstroke, reshaping the game's dynamics and ultimately leading to AlphaGo's victory. This moment marked a turning point for AI, showing that it could transcend human-like thinking to uncover strategies entirely outside our realm of intuition.

Move 37 wasn’t just a move—it was a revelation that AI has the potential to innovate in ways humans might never imagine, forever altering how we view intelligence and creativity.

🗒️ FEEDBACK

Help Us Get Better

What did you think of today's newsletter?

Login or Subscribe to participate in polls.

Reply to this email if you have specific feedback to share. We’d love to hear from you.

CONNECT

Stay in the Know

Follow us on X for quick daily updates and bite-sized content.
Subscribe to our YouTube channel for in-depth technical analysis.

Prefer using an RSS feed? Add Forward Future to your feed here.

Thanks for reading today’s newsletter. See you next time!

The Forward Future Team
🧑‍🚀 🧑‍🚀 🧑‍🚀 🧑‍🚀 

Reply

or to participate.