This is a Visual Studio solution for showcasing how you can integrate LLMs to add content to your video game. No GPU is required, as we will be using quantized models to 3 or 5 bits.
As you will need to install LlamaSharp from NuGet, you need NuGet for Unity to manage the downloading and installation.
The plugin is alredy installed by default, and you should see this:
But if not, just click on the .package you will see in AddToProjects
Check if LlamaSharp is installed.
If not, go to NuGet -> Manage Nu Get Packages -> Online, type LlamaSharp and install it.
UniTask is a library to get full multithread in Unity.
The library should have been preinstalled for you in the solution. Check that the compiler does not complain about lñacUniTask library.
If it complains about it being missing, then install it the same way as with NuGet, just click on the .package you will see in AddToProjects.
In the demo I use two quantized 7B versions. Let's download both of them.
Go to: https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF
In there, look for:
You will need to click in the model names and then, in the opening tab, download them clicking here:
Proceed as mentioned for the two models.
It should look like this:
It takes some seconds to load the model into memory. After, you can type concepts and click on Submit to see what the model generates.
Do you see the difference?
- Feel free to download other GGUF-based LLM models supported by LlamaSharp, including Llama2 and Mistral.
- Do you want to run it in GPU? Let me know!