- Forward Future AI
- Posts
- AI Coding Assistants Are Changing Programming
Top Story: AI Coding Assistants Have a BIG Impact
Measuring GitHub Copilot’s Impact on Productivity: This study analyzes how developers perceive their productivity while using GitHub Copilot, an AI pair-programming tool, and how these perceptions align with objective data collected from their interactions with the tool.
Developers of all skill levels, especially juniors, report productivity gains in various aspects such as task completion time and cognitive load. The evaluation shows that the completion suggestion acceptance rate is a robust indicator of perceived productivity rather than the correctness of suggestions.
Acceptance rate varies among developers and evolves over time, highlighting the complex nature of measuring productivity in software development. This work emphasizes the challenge of offline code completion evaluation due to the subjective nature and multiplicity of "correct" code completions.
Instead, real-world usage data and developers' self-reported productivity assessments offer a more nuanced understanding of productivity with AI programming tools.
AI Energy Drain
Alex de Vries, a Dutch economist, explored the alarming energy consumption of both cryptocurrencies and artificial intelligence (AI). His Bitcoin Energy Consumption Index highlights that bitcoin mining surpasses the Netherlands' energy use and results in significant CO2 emissions and electronic waste.
Now, de Vries focuses on the energy demands of AI, noting that systems like ChatGPT consume massive amounts of electricity—akin to half a million kilowatt-hours daily. He urges greater awareness of AI's environmental impact, advocating for mandatory energy disclosure as a policy measure.
Despite AI's potential to optimize energy efficiency, de Vries and AI proponents like Sam Altman acknowledge a pressing need for groundbreaking energy solutions to sustain these rapidly advancing technologies.
How Selective Forgetting Can Help AI Learn Better - A team of computer scientists has developed a machine learning model that improves its language processing abilities by periodically resetting its learned knowledge, specifically the token embeddings. This method mimics human memory, which doesn't retain every detail but rather grasps the essence of experiences. In experiments, periodically forgetting information during initial training made it easier for the model, a modified version of Roberta, to adapt to new languages later, especially with limited data and computational resources. This technique could enhance understanding of language beyond mere word meanings and extend AI capabilities to less commonly used languages with fewer resources, like Basque, enriching the diversity of AI applications.
Pentagon seeks low-cost AI drones - The Pentagon aims to progress artificial intelligence (AI) in aviation with the Collaborative Combat Aircraft (CCA) project, a $6 billion initiative to add over 1,000 drones to the Air Force. Major defense companies like Boeing, Lockheed Martin, and General Atomics are contenders, each showcasing various AI advancements such as Boeing's Ghost Bat and Anduril's Roadrunner. These new drones' potential includes acting as escorts for piloted jets, conducting surveillance, and providing communication support.
OpenAI announces new members to board of directors - OpenAI has expanded its Board of Directors with three new members committed to guiding the organization’s growth and fulfilling its mission to ensure artificial general intelligence benefits all of humanity. The new appointees are Dr. Sue Desmond-Hellmann, a seasoned non-profit leader and physician with prior tenure as CEO of the Bill & Melinda Gates Foundation; Nicole Seligman, a respected lawyer with extensive corporate leadership experience, including roles at Sony and current board positions; and Fidji Simo, the current CEO of Instacart with a background in leading major consumer technology initiatives, including at Facebook. They join existing board members such as Adam D'Angelo and Larry Summers, and Sam Altman is also returning to the board.
Italy opens probe into OpenAI's new video tool Sora - Italy's data protection authority has initiated an investigation into OpenAI's new AI tool, Sora, due to potential personal data processing concerns for EU users, especially in Italy. Sora, still in testing, can create realistic short videos based on user prompts. The watchdog demands clarification from OpenAI on the nature of data used for training Sora, particularly whether it involves personal or sensitive categories, and how it will adhere to EU data protection regulations. This inquiry follows a temporary block on OpenAI's ChatGPT by Italian authorities over regulatory compliance and age verification issues.
OpenAI's video AI Sora will not be released "anytime soon" - The Sora research project by OpenAI, as discussed in an interview with Marques Brownlee, is not slated for release anytime soon. Tim Brooks of OpenAI emphasized that the project is currently in a phase of collecting feedback and contemplating necessary improvements, without a fixed timeline for product conversion. Sora’s development encounters specific challenges, particularly in simulating hands and complex physical motions. Users have indicated a desire for greater control beyond textual prompts in video generation, an aspect the team is considering.
Command-R is a new large language model (LLM) introduced by Cohere, optimized for enterprise-level production with a balance of efficiency and accuracy. It specializes in retrieval augmented generation (RAG) tasks and integrating external tools, featuring a long context capability of 128k tokens and lower pricing than its predecessor, Command. Command-R demonstrates strong performance in multiple languages and offers improved accuracy and latency for enterprise applications. Integrated with Cohere's Embed and Rerank models, it significantly advances RAG functionality. Cohere has also made Command-R's model weights available for research on HuggingFace and plans for widespread availability across major cloud providers.
Groq Supports Gemma - Groq has added Gemma 7b, Google’s open-source small model. I haven’t seen great results from Gemma yet, but it’s improving quickly due to people finding bugs and fixing them.
Gemma Bug Fixes - As mentioned, Gemma has a lot of bugs. But, since it’s open-source, a kind engineer named Daniel Han has been fixing them left and right and improving Gemma’s performance quickly.
Claude Drops a Prompt Library - Claude released their very own prompt library, which includes dozens of pre-written prompts for various use cases.
Awesome Research Papers
The Yi model family comprises advanced language and multimodal models, including 6B and 34B parameter variants. The models showcase versatile proficiency, excelling in benchmarks like MMLU and gaining user preference in platforms such as AlpacaEval and Chatbot Arena. Central to their success is the quality of data, achieved through meticulous engineering, which incorporates 3.1 trillion tokens across English and Chinese for pretraining, and a rigorously refined instruction dataset for finetuning. Yi models integrate language capabilities with a vision transformer for image-language tasks and showcase improvements in contextual understanding up to 200K tokens.
Cool Projects
BaoGPT - BobGPT is an AI utility designed for querying content within YouTube videos through a structured process. When presented with a question, the system strategically routes it as either a greeting or a query that requires a detailed response. For queries, it performs an analysis to extract key attributes, optimizing the question for enhanced retrieval from a vector database. Candidates for answers are then retrieved, ranked using Cohere's API, and the highest-ranking results are summarized by the AI. This comprehensive approach ensures that users receive a succinct summary with relevant source attribution. Users interacting with the system must configure settings in a YAML file and have Python 3.11.4 with a virtual environment set up to run the commands, which include crawling for YouTube transcripts and ingesting data for processing.
Reply