boludo
boludo is a personal AI assistant that lives in your terminal. It respects
your privacy by running specialized Large language models
directly on your ordinary computer.
boludo
is a user-friendly interface for llama.cpp.
Usage
boludo
uses subcommands to call locally hosted models with specific parameters.
boludo.toml
is the config file with subcommands definitions (see:
examples/boludo.toml). For example, if you defined
someconfig
, you can do:
$ boludo someconfig "How are you?"
I am fine, thanks.
$ boludo someconfig <input.txt >output.txt
In the config file, you can change the default behaviour of the model by adjusting two
parameters:
-
creativity modifies probabilities of the next word. For example:
0.0 < creativity < 1.0
- increase predictability of result. The smaller the
value, the more similar the result to the dataset used to train the model (useful
for code generation or plagiarizing The New York Times)
creativity = 1.0
- preserves probabilities computed during model training
(best for validating claims of the model author)
1.0 < creativity
- increase probabilities of unlikely next
words (best for making the output more diverse, but higher values increase
inconsistency).
In popular LLM runners, this parameter is known as a temperature.
-
cutoff discards statistically insignificant words (long tail)
during the next word prediction. For example, cutoff = 0.03
means discarding
words with a probability smaller than 3% of the value of the most probable word.
In popular LLM runners, this parameter is known as a min-p.
Proofreading
Karen TheEditor
is one of the most practical models. It's specialized in correcting writing
(in American English). With a configuration similar to examples/boludo.toml,
you can do:
$ boludo proofreader "Trolling is a art"
Trolling is an art.
With longer texts, you may be interested to know which words were changed. By using the tee
and git
commands, you can monitor progress and track changes:
$ boludo proofreader <original.txt | tee corrected.txt; git -c color.diff.new="italic brightgreen reverse black bold" -c color.diff.old="strike brightred reverse bold" diff --no-index --color-words=. original.txt corrected.txt
Coding
CodeNinja is
a decent programmer assistant. With a configuration similar to examples/boludo.toml,
you can:
-
find solutions for complex tasks:
$ boludo coder "Show me the unix shell code without description for the task: Given a text file and an integer k, print the k most common words in the file (and the number of their occurrences) in decreasing frequency."
cat text_file.txt | tr -s ' ' '\n' > words
sort words | uniq -c > uniq_words
awk '{if (length($0)<=k) print}' uniq_words > output.txt
rm words uniq_words
(not quite the McIlroy's way)
-
perform code completion:
$ boludo coder "# returns anagram
def anagram(str):"
# sort the string
sorted_str = ''.join(sorted(str.lower()))
# return the result
return sorted_str
(it correctly recognized Python code)
-
mentoring:
$ boludo coder "Which 3 malpractices are most common among C# programmers?"
1. Ignoring exceptions: Many C# programmers tend to ignore exceptions, which can lead to unhandled errors and unpredictable behavior in the application.
2. Incorrect use of `string.Format`: Programmers often make mistakes when using `string.Format` for string manipulation, resulting in potential vulnerabilities or incorrect output.
3. Lack of unit testing: Many C# developers neglect to write unit tests, which can lead to undiscovered bugs and poor code quality.
To avoid these common mistakes, it's essential for C# programmers to follow best practices, utilize error-handling techniques, and practice thorough testing.
(that's interesting - format string attack
is mostly known in the C world, but it's still relevant in C#).
Installation
You can manually build boludo
with commands: make && make build
.
Development
Use make
(GNU or BSD):
make
- install dependencies
make test
- runs test
make check
- static code analysis
make build
- compile binaries from latest commit
make dist
- compile binaries from latest commit for supported OSes
make clean
- removes compilation artifacts
make cli-release
- tag latest commit as a new release of CLI
make info
- print system info (useful for debugging).
Versioning
The repo contains command-line utility which versions are tagged as cli/vYYYY.0M.MICRO
(calendar versioning).
TODO
Some ideas for further development:
- restrict allowed kernel calls (with seccomp and pledge)
- embed
main
command from llama.cpp using WASM (this will also make app more secure by sandboxing main
command)
- implement native Go infering engine instead of llama.cpp (the
gguf
format is well defined)
- show spinning wheel at stderr before generation.
License
MIT (in simple words)