🏫 Perplexity Enters the Deep Research Arena

Last week, I wrote a piece titled "The Research Revolution: How AI is Rewriting the Rules of Knowledge Work," which focused on AI as a research partner. As this was being published, Perplexity launched a new Deep Research capability that has received high praise, so I wanted to take a look and share my thoughts.

If you haven't tried Perplexity yet, you should. It's excellent at what it does and has become one of my go-to tools for daily use. What is Perplexity? Perplexity is a conversational search engine launched in 2022 that employs artificial intelligence to provide users with direct answers to their questions, complete with citations and sources. Unlike traditional search engines that merely present a list of links, Perplexity AI summarizes information from across the web in natural language, saving users time and effort. Perplexity's value lies in its ability to deliver accurate, well-sourced information, streamline research, and personalize the search experience, making it a user-centric alternative to conventional search engines. It aims to revolutionize how people engage with information by offering a seamless and intuitive conversational search experience.

Perplexity Deep Research lets you generate in-depth research reports on any topic.  It's available to everyone for free – up to 5 queries per day for non-subscribers and 500 queries per day for Pro users.  It combines advanced web search, reasoning, and coding to iteratively search and refine analysis into clear, comprehensive reports.  It also completes most reports in under 3 minutes (OpenAI and Google can take significantly longer).

The remarkable thing to me was their post on how it scored on Humanity's Last Exam (21.1%) – coming in second to OpenAI's Deep Research capability.  I have not been able to find a score for Google's Deep Research capability, and if I find out, I'll tweet out an update.

Let's Run Some Experiments

I decided to run three tests to compare the output from OpenAI, Google, and Perplexity. The first test involved deep research into the locum tenens market, which involves temporary staffing of healthcare professionals in hospitals and other healthcare facilities. I would typically conduct this research in a strategy consulting engagement or the type a private equity firm would conduct when evaluating new markets. There should be solid data sources to search for to build a fact-base from across the web.

The second test involved gathering research on the Apple II piracy market, which I discussed last week. This topic contains significantly less current data and depends on older information from more random web sources. I knew many of the people involved in this in the past, so I wanted to see how well these tools could create a solid narrative.

The third test examined the UCaaS market (unified communications as a service), an industry I have consulted in and worked directly in on and off over the last 25 years. I keep up-to-date on the industry through contacts working in various companies and track their financials regularly. 

Experiment #1: Locum Tenens Market

This is the prompt I used with each of them:

Let me share the executive summary from each section:

OPENAI DEEP RESEARCH

GEMINI DEEP RESEARCH

PERPLEXITY DEEP RESEARCH

There's a massive difference in the output. OpenAI had the most detailed and robust results (and the rest of the report was similar). Gemini was a solid second overall, and Perplexity was a distant third. Not only was the resulting report far smaller, but opening with "The $XX.X billion..." immediately raised a red flag and made me question the rest. 

Experiment #2: Software Piracy in the Apple II Era

This is the prompt I used with each of them:

Here’s how each of the tools opened the response:

OPENAI DEEP RESEARCH

GEMINI DEEP RESEARCH

PERPLEXITY DEEP RESEARCH

So, I knew many of the people mentioned in these research reports and have firsthand knowledge of many of the groups from the 1980s. The summary from Perplexity contains a lot of hallucinations.  Specifically, "Apple Mafia" wasn't a group and didn't exist as described here.  If you do follow-up questions on Perplexity, it will admit that it doesn't exist.  Also, Black Bag (which was US-based, not European-based) didn't rely primarily on in-person trading; most of the trading they did was via BBS.  So I'd rank this OpenAI, Gemini, and then Perplexity.

Experiment #3: UCaaS Market Analysis

This is the prompt I used with each of them:

I’m going to focus on one area of the results that highlighted the difference – the revenue projections out to 2030.  I’m going to focus on the company 8x8. 

OPENAI DEEP RESEARCH

GEMINI DEEP RESEARCH

PERPLEXITY DEEP RESEARCH

Ok – so let me first say that OpenAI's analysis was insightful and reasonably accurate across all companies.  Gemini was massively over-optimistic that a company that declined last year would double in the next 5 years (massively outpacing the growth of the industry).  But Perplexity – I can't even begin to explain the size of the miss.  Almost 8x growth with a 16% CAGR in 5 years?  Bad math.  The logic in the estimate was utterly off.  8x8 is not going from ~$800M to $6.2B in 5 years with a 16% CAGR.  Every part of that table is wrong (not just for them, but for all companies).  This analysis alone caused me to lose faith in Perplexity's ability to do future estimates like this.

Here's a link to a dropbox directory where I put in the History of Apple II and UCaaS research files: https://www.dropbox.com/scl/fo/3qrd319947jw1ymzkala8/AEj1-AfRJyrHHTpvuAsEQOg?rlkey=ej0i1uoddb4xic8limz4uzeyf&dl=0

Final Thoughts: The Future of Deep Research with AI

After putting Perplexity's Deep Research through its paces alongside OpenAI and Google, one thing is clear: AI-driven research is evolving at an impressive pace, but not all tools are created equal. OpenAI consistently delivered the most comprehensive and well-structured reports, making it the current leader in the space. Google's Gemini, while somewhat less precise in financial projections, still produced solid, data-rich outputs. Perplexity, on the other hand, showed promise but struggled with depth, accuracy, and forward-looking estimates—key factors that make or break serious research efforts.

That being said, Perplexity still has its strengths. Its speed and user-friendly conversational search make it an excellent tool for quick research and summarization. However, when it comes to high-stakes business decisions, market analyses, or historical accuracy, the results suggest that OpenAI remains the go-to solution for now.

These tools will only improve as the AI research landscape continues to evolve. Perplexity may have work to refine its accuracy and analytical rigor, but its entry into deep research signals a growing demand for AI-powered knowledge work. The real question isn't whether AI will replace traditional research—it's how quickly it will become an indispensable partner in the process.

For now, I'll keep experimenting and pushing these tools to their limits. And if Perplexity catches up, you'll hear about it here first.