Install Alpaca / LLaMA Locally (Video Companion)

Contents:

• Hello to everyone new!

• Let’s go deeper into Dalai

• Troubleshooting Dalai

• Optimal Prompt Formatting

• Dalai Now Supports Alpaca 13b

• Additional Context for the Video

• Fine-tuning LLaMA & Alpaca

Hello Everyone New! 👋

First, I wanted to say hello to all the new subscribers. I’m excited that you’ll be part of our journey of exploring the world of AI. I talk about anything interesting I come across, including AI tools, tutorials, news, demos, and much more. I’m always open to feedback and suggestions, so feel free to email me at mbermanyt@gmail.com.

Going Deeper into Dalai 🦙

My last video received a great response. For those who didn’t get a chance to watch it, here it is:

In the video, I give a walkthrough of how to install LLaMA and Alpaca locally using a new tool called Dalai (as in…Dalai Llama :P). There were a lot of questions in the comments and even more requests for more info, so I figured I’d send a companion Substack to this video.

First, a big thank you to 🎉CocktailPeanut🎉 for building such an awesome tool in Dalai.

Troubleshooting Dalai Installation 🔨

I know a lot of people had trouble installing Dalai, especially on Windows, but it looks like CocktailPeanut is working furiously to solve those issues. I HIGHLY RECOMMEND getting in the Dalai discord because there’s already a bunch of information on how to solve many of those issues.

Here are some of the common issues and a link to the discord discussion about this:

• No response from the model (part 2)

• Models not appearing in the UI

• Weird character issues (github issue)

• cmake issue

Optimal Prompt Formatting 🤖

Thanks to Jeremy Kothe in the video’s comments for pointing out the best format for prompts to get the best responses is:

Remember, Alpaca works as if it were completing your prompt.

Dalai Now Supports Alpaca 13b! 🦙

Fresh off releasing support for LLaMA 7b 13b and Alpaca 7b, Dalai now supports Alpaca 13b. To install Alpaca 13b, follow these commands:

Clear your cache:

npx clear-npx-cache

Update npm just in case:

npm install npm@latest -g

install 13b:

npx --loglevel verbose dalai@0.2.50 alpaca install 13b

I have a feeling more models are coming soon 😉

Some Additional Context

Dalai, LLaMA, and Alpaca are completely free. The “$600” I was referring to in the video was to fine-tune the Alpaca model on the LLaMA model.

Much Requested: Fine-tuning LLaMA & Alpaca

The most requested follow-up to that video is how to fine-tune the model for specific needs. I’m currently researching how to do that and have some good information so far. However, it does look like you’ll need a pretty beefy (albeit consumer) graphics cards to do the fine-tuning. If I can find a way to make it work without needing to drop $1,500 on a graphics card (which is unreasonable for most people), I’ll make a video about it.

Thanks, everyone!!