Good morning, it’s Friday. Today, we’re exploring DeepMind’s new tool that lets us peek into AI’s “mind.” From catching “cringe” to preventing models from thinking they're the Golden Gate Bridge, researchers are finally cracking open the black box of AI—and what they’re finding is wild.
In other news: Small Language Models provide efficient alternatives, Google launches Gemini app for iPhone, and AI data centers may soon face power shortages. Let's dive in!
Inside Today’s Edition:
Top Stories 🗞️
DeepMind’s Gemma Scope Reveals AI’s Inner Workings 🔍
Small Language Models in the LLM Era 🤏
[FF Original] The Future of Work: AI-Driven Automation Is Inevitable 👾
[New Video] AI Progress Limit, Chat.com for $15m, Qwen Coder 📽️
Tools for Kids, Code, and Marketing 🧰
🤔 FRIDAY FACTS
The term "bug" is one of tech's most famous terms, with origins dating back to 1947. But where did it come from, and how did it get its name?
Stick around for the answer! 👇️
☝️ POWERED BY VULTR
🔍 DECODING AI
Google DeepMind’s Gemma Scope Offers a Window Into AI’s “Mind”
The Recap: Google DeepMind has introduced Gemma Scope, a tool powered by autoencoders that lets researchers look inside AI models to better understand their inner workings. This innovation could pave the way for more interpretable and safer AI applications by shedding light on how neural networks make decisions.
Highlights:
DeepMind's focus on understanding AI decision-making aims to "read" an AI model's processes to detect potential risks like deception.
By applying sparse autoencoders to AI layers, researchers can examine detailed “features” in each layer, like how the model understands a concept, such as "dogs" or "cringe."
Gemma Scope’s code is available to encourage collaboration, allowing researchers to analyze the interpretability of AI systems and build on DeepMind's findings.
In partnership with Neuronpedia, a demo lets users experiment with AI responses, showing how specific features are activated, like the "cringe" feature when negative criticism is detected.
Research has used similar techniques to adjust AI associations, like reducing gender bias by deactivating “gendered” features.
Mechanistic interpretability helps identify and correct errors—such as mistaking numbers for dates, as seen with models erroneously ranking 9.11 over 9.8.
Future applications might involve disabling specific knowledge nodes (like bomb-making information) to create more ethically aligned AI models.
Forward Future Takeaways: As AI advances into sensitive domains like medicine and security, tools like Gemma Scope may be essential in making neural networks more transparent and controllable. By illuminating AI’s decision pathways, mechanistic interpretability could help ensure AI models behave safely and ethically, though achieving such fine-grained control remains complex. As research progresses, improved interpretability may become the backbone of reliable and trustworthy AI systems. → Read the full article here.
👾 FORWARD FUTURE ORIGINAL
Why AI-Driven Automation Is Inevitable
In a previous article, we asked the question: Will AI take my job? I did not give a conclusive answer, however. This is largely because, well, nobody knows the future. However, we can look at how things might change based on how things have changed in the past. Let’s set the stage to explore this further and take the discussion beyond where we left off last time.
One point I emphasized is that a ‘job’ is in fact a part of the wider system (and hence question) of the overall economy, and the ideas of innovation, production and efficiency. In other words, we cannot look at the question of jobs in isolation, without considering such systemic factors.
The other point I’ll also emphasize is that human progress - technological, and in tandem, institutional - is like riding a bicycle uphill. You stop and you fall. Even if one society or nation refuses to innovate and make progress, another will - so ultimately, progress is a matter of staying competitive, even of survival!
Human progress is like riding a bicycle: you stop you fall
So, let’s assume we do proceed along this path of technological progress and see where this might lead us, taking this wider economic context.
In recent decades, there has been rapid progress in information technology - in fact we’re practically walking computers, given the computing power and versatility in that flat and shiny object in your pocket. It may even be worth mentioning that a typical mobile phone today has more computing power than all of NASA did when they landed a man on the moon in the 60s. → Continue reading here.
📜 PAPERS
A Comprehensive Guide to Small Language Models in the Age of LLMs
The Recap: With large language models (LLMs) like LaPM 540B and Llama-3.1 pushing the limits of scale and computation, small language models (SLMs) are emerging as a viable, efficient alternative for specialized tasks in resource-limited environments. This survey explores the techniques, enhancements, applications, collaboration strategies with LLMs, and reliability factors that make SLMs essential for privacy, latency, and cost-sensitive applications.
Highlights:
SLMs offer low latency, reduced cost, and customizable configurations, making them ideal for localized, privacy-sensitive tasks on edge devices.
LLMs struggle with domain specificity (e.g., healthcare, law), are resource-intensive, and often raise privacy issues when using cloud-based APIs.
The authors propose defining SLMs based on their capacity for specialized tasks and their sustainability in resource-constrained settings, with a framework for emerging capabilities without overextending computational demands.
SLMs are optimized to handle specialized knowledge through lightweight fine-tuning, making them better suited for narrow, domain-specific applications.
By working in tandem with LLMs, SLMs can support a hybrid approach, leveraging LLMs for general tasks and SLMs for specialized, cost-effective applications.
The paper discusses techniques for improving SLM trustworthiness, ensuring consistency, accuracy, and reliability, especially in high-stakes domains.
Forward Future Takeaways: As LLMs continue to grow in scale, SLMs will play an increasingly important role in balancing efficiency and specialization in AI applications. Their adaptability to specific tasks and environments makes SLMs pivotal for organizations aiming to deploy AI with minimal resources and enhanced privacy. Moving forward, advancements in SLM reliability and integration with LLMs could lead to hybrid models that optimize both general and specialized language processing tasks. → Read the full paper here.
🛰️ NEWS
Looking Forward: More Headlines
Rise in AI-Driven Scams: Google reports a surge in cloaking scams and AI-powered fraud, enhancing protections to counter evolving threats.
François Chollet Leaves Google: AI pioneer François Chollet leaves Google to start a new venture, focusing on advancing human-like reasoning in AI.
Final Cut Pro 11 Debuts AI Tools: Apple’s Final Cut Pro 11 introduces AI-driven tools and spatial video editing for VR/AR, enhancing workflows for professionals.
Tessl Raises $125M for AI Coding: Tessl secures $125M to build an AI-driven platform for autonomous code generation and maintenance.
AI-Powered Infrastructure Mapping: Mach9's Digital Surveyor uses AI to convert mobile lidar scans into models, accelerating infrastructure monitoring.
ChatGPT vs. Copilot vs. Gemini: Explore the unique benefits of each chatbot to understand which is the right fit for your needs.
📽️ VIDEO
AI Progress Limit, Chat.com for $15m, Qwen Coder, Ollama Vision, Ex-OpenAI CTO Plans
We cover recent AI news, including slowed innovation at OpenAI and Google, updates on ChatGPT’s domain, Meta’s AI Ray-Ban glasses, and Amazon’s new AI chips. Get the full scoop in our latest video! 👇
🧰 TOOLBOX
AI Tools for Children's Content Creation, Code, and Marketing
ReadKidz | Kid's AI Content: ReadKidz is an AI platform for creating children’s multimedia stories, offering templates, customization, and one-click publishing.
Trag | Superlinter for Code: Trag streamlines code reviews by enforcing custom patterns and rules, integrates with GitHub, and is free for open-source projects.
Averi | Marketing Manager: Averi simplifies marketing operations with AI-driven tools and optional human support, freeing users to focus on creativity.
🗒️ FEEDBACK
Stickers, Hoodies, Bobbleheads—oh my!
Forward Future swag: Coming soon!What should we release first? |
🤔 FRIDAY FACTS
The First Computer “Bug” Was Actually a Moth…
On September 9, 1947, while working on the Harvard Mark II computer, a team of engineers made a surprising discovery: a moth had found its way into the machine’s relay, causing a series of baffling errors. Grace Hopper and her team humorously documented this in their logbook as the "first actual case of a computer bug." Though "bug" had already been used in engineering contexts—Thomas Edison famously referred to mechanical issues as "bugs"—this incident gave the term a whole new level of visibility. This moment didn’t just popularize "bug" and "debugging" but also highlighted the unpredictability of early computing, reminding us that even the smallest issues can impact technology in significant ways.
CONNECT
Stay in the Know
Follow us on X for quick daily updates and bite-sized content.
Subscribe to our YouTube channel for in-depth technical analysis.
Prefer using an RSS feed? Add Forward Future to your feed here.
Thanks for reading today’s newsletter. See you next time!
The Forward Future Team
🧑🚀 🧑🚀 🧑🚀 🧑🚀
