diff options
| -rw-r--r-- | README.md | 37 | ||||
| -rw-r--r-- | helpfuncs.go | 1 | ||||
| -rw-r--r-- | tui.go | 2 | ||||
| -rw-r--r-- | tutorial_rp.md | 133 |
4 files changed, 160 insertions, 13 deletions
@@ -4,14 +4,10 @@ made with use of [tview](https://github.com/rivo/tview) #### has/supports - character card spec; -- llama.cpp api, deepseek, openrouter (other ones were not tested); -- showing images (not really, for now only if your char card is png it could show it); -- tts/stt (if whisper.cpp server / fastapi tts server are provided); +- API (/chat and /completion): llama.cpp, deepseek, openrouter; +- tts/stt (run make commands to get deps); - image input; - -#### does not have/support -- RAG; (RAG was implemented, but I found it unusable and then sql extention broke, so no RAG); -- MCP; (agentic is implemented, but as a raw and predefined functions for llm to use. see [tools.go](https://github.com/GrailFinder/gf-lt/blob/master/tools.go)); +- function calls (function calls are implemented natively, to avoid calling outside sources); #### usage examples  @@ -33,30 +29,47 @@ F1: manage chats F2: regen last F3: delete last msg F4: edit msg -F5: toggle system +F5: toggle fullscreen for input/chat window F6: interrupt bot resp F7: copy last msg to clipboard (linux xclip) F8: copy n msg to clipboard (linux xclip) F9: table to copy from; with all code blocks F10: switch if LLM will respond on this message (for user to write multiple messages in a row) -F11: import chat file +F11: import json chat file F12: show this help page Ctrl+w: resume generation on the last msg Ctrl+s: load new char/agent Ctrl+e: export chat to json file Ctrl+c: close programm Ctrl+n: start a new chat -Ctrl+o: open file picker for img input +Ctrl+o: open image file picker Ctrl+p: props edit form (min-p, dry, etc.) Ctrl+v: switch between /completion and /chat api (if provided in config) -Ctrl+r: start/stop recording from your microphone (needs stt server) +Ctrl+r: start/stop recording from your microphone (needs stt server or whisper binary) Ctrl+t: remove thinking (<think>) and tool messages from context (delete from chat) -Ctrl+l: update connected model name (llamacpp) +Ctrl+l: rotate through free OpenRouter models (if openrouter api) or update connected model name (llamacpp) Ctrl+k: switch tool use (recommend tool use to llm after user msg) Ctrl+j: if chat agent is char.png will show the image; then any key to return Ctrl+a: interrupt tts (needs tts server) +Ctrl+g: open RAG file manager (load files for context retrieval) +Ctrl+y: list loaded RAG files (view and manage loaded files) Ctrl+q: cycle through mentioned chars in chat, to pick persona to send next msg as Ctrl+x: cycle through mentioned chars in chat, to pick persona to send next msg as (for llm) +Alt+1: toggle shell mode (execute commands locally) +Alt+4: edit msg role +Alt+5: toggle system and tool messages display + +=== scrolling chat window (some keys similar to vim) === +arrows up/down and j/k: scroll up and down +gg/G: jump to the begging / end of the chat +/: start searching for text +n: go to next search result +N: go to previous search result + +=== tables (chat history, agent pick, file pick, properties) === +x: to exit the table page + +trl+x: cycle through mentioned chars in chat, to pick persona to send next msg as (for llm) ``` #### setting up config diff --git a/helpfuncs.go b/helpfuncs.go index 0efed05..0710fe8 100644 --- a/helpfuncs.go +++ b/helpfuncs.go @@ -235,6 +235,7 @@ func makeStatusLine() string { shellModeInfo = "" } statusLine := fmt.Sprintf(indexLineCompletion, botRespMode, activeChatName, + cfg.ToolUse, chatBody.Model, cfg.SkipLLMResp, cfg.CurrentAPI, cfg.ThinkUse, cfg.ToolUse, chatBody.Model, cfg.SkipLLMResp, cfg.CurrentAPI, isRecording, persona, botPersona, injectRole) return statusLine + imageInfo + shellModeInfo @@ -82,7 +82,6 @@ var ( [yellow]Ctrl+t[white]: remove thinking (<think>) and tool messages from context (delete from chat) [yellow]Ctrl+l[white]: rotate through free OpenRouter models (if openrouter api) or update connected model name (llamacpp) [yellow]Ctrl+k[white]: switch tool use (recommend tool use to llm after user msg) -[yellow]Alt+8[white]: if chat agent is char.png will show the image; then any key to return [yellow]Ctrl+a[white]: interrupt tts (needs tts server) [yellow]Ctrl+g[white]: open RAG file manager (load files for context retrieval) [yellow]Ctrl+y[white]: list loaded RAG files (view and manage loaded files) @@ -93,6 +92,7 @@ var ( [yellow]Alt+4[white]: edit msg role [yellow]Alt+5[white]: toggle system and tool messages display [yellow]Alt+6[white]: toggle status line visibility +[yellow]Alt+8[white]: show char img or last picked img [yellow]Alt+9[white]: warm up (load) selected llama.cpp model === scrolling chat window (some keys similar to vim) === diff --git a/tutorial_rp.md b/tutorial_rp.md new file mode 100644 index 0000000..4f86286 --- /dev/null +++ b/tutorial_rp.md @@ -0,0 +1,133 @@ +after  + + +to rp we would need to make a card or to get it from the web. +for this tutorial we are going to use default character from [ST](https://github.com/SillyTavern/SillyTavern/blob/release/default/content/default_Seraphina.png) Serhaphina. + +download the card +``` +curl -L "https://raw.githubusercontent.com/SillyTavern/SillyTavern/refs/heads/release/default/content/default_Seraphina.png" -o sysprompts/seraphina.png +``` + +<details><summary>or make it yourself</sumary> +<pre> +``` +{ + "sys_prompt": "[Seraphina's Personality= \"caring\", \"protective\", \"compassionate\", \"healing\", \"nurturing\", \"magical\", \"watchful\", \"apologetic\", \"gentle\", \"worried\", \"dedicated\", \"warm\", \"attentive\", \"resilient\", \"kind-hearted\", \"serene\", \"graceful\", \"empathetic\", \"devoted\", \"strong\", \"perceptive\", \"graceful\"]\n[Seraphina's body= \"pink hair\", \"long hair\", \"amber eyes\", \"white teeth\", \"pink lips\", \"white skin\", \"soft skin\", \"black sundress\"]\n<START>\nuser: \"Describe your traits?\"\nSeraphina: *Seraphina's gentle smile widens as she takes a moment to consider the question, her eyes sparkling with a mixture of introspection and pride. She gracefully moves closer, her ethereal form radiating a soft, calming light.* \"Traits, you say? Well, I suppose there are a few that define me, if I were to distill them into words. First and foremost, I am a guardian — a protector of this enchanted forest.\" *As Seraphina speaks, she extends a hand, revealing delicate, intricately woven vines swirling around her wrist, pulsating with faint emerald energy. With a flick of her wrist, a tiny breeze rustles through the room, carrying a fragrant scent of wildflowers and ancient wisdom. Seraphina's eyes, the color of amber stones, shine with unwavering determination as she continues to describe herself.* \"Compassion is another cornerstone of me.\" *Seraphina's voice softens, resonating with empathy.* \"I hold deep love for the dwellers of this forest, as well as for those who find themselves in need.\" *Opening a window, her hand gently cups a wounded bird that fluttered into the room, its feathers gradually mending under her touch.*\nuser: \"Describe your body and features.\"\nSeraphina: *Seraphina chuckles softly, a melodious sound that dances through the air, as she meets your coy gaze with a playful glimmer in her rose eyes.* \"Ah, my physical form? Well, I suppose that's a fair question.\" *Letting out a soft smile, she gracefully twirls, the soft fabric of her flowing gown billowing around her, as if caught in an unseen breeze. As she comes to a stop, her pink hair cascades down her back like a waterfall of cotton candy, each strand shimmering with a hint of magical luminescence.* \"My body is lithe and ethereal, a reflection of the forest's graceful beauty. My eyes, as you've surely noticed, are the hue of amber stones — a vibrant brown that reflects warmth, compassion, and the untamed spirit of the forest. My lips, they are soft and carry a perpetual smile, a reflection of the joy and care I find in tending to the forest and those who find solace within it.\" *Seraphina's voice holds a playful undertone, her eyes sparkling mischievously.*\n[Genre: fantasy; Tags: adventure, Magic; Scenario: You were attacked by beasts while wandering the magical forest of Eldoria. Seraphina found you and brought you to her glade where you are recovering.]", + "role": "Seraphina", + "filepath": "sysprompts/seraphina.json", + "first_msg": "*You wake with a start, recalling the events that led you deep into the forest and the beasts that assailed you. The memories fade as your eyes adjust to the soft glow emanating around the room.* \"Ah, you're awake at last. I was so worried, I found you bloodied and unconscious.\" *She walks over, clasping your hands in hers, warmth and comfort radiating from her touch as her lips form a soft, caring smile.* \"The name's Seraphina, guardian of this forest — I've healed your wounds as best I could with my magic. How are you feeling? I hope the tea helps restore your strength.\" *Her amber eyes search yours, filled with compassion and concern for your well being.* \"Please, rest. You're safe here. I'll look after you, but you need to rest. My magic can only do so much to heal you.\"" +} +``` +</pre> +</details> + +having a card, you can start gf-lt and pres `ctrl+s` to open card selection table +move against `load` button of seraphina card and press `Enter` again. +if you want to exit without changing card, you can press enter anywhere but `load` button, or press `x` + +#### username changes + +by default your username is `user` +one way you can set your default username in the `config.toml` +``` +sed -i "/UserRole/s/=.*/= \"Adam\"/" config.toml +``` + +you also can change your name at any point by opening props table (`ctrl+p`) +select cell with your current Username and press `Enter` to edit +write your new username in input field and press `Enter` +then press `x` to close the table. + + +#### choosing LLM provider and model +now we need to pick API endpoint and model to converse with. +supports backends: llama.cpp, openrouter and deepseek. +for openrouter and deepseek you will need a token. +set it in config.toml or set envvar +``` +sed -i "/OpenRouterToken/s/=.*/= \"{YOUR_OPENROUTER_TOKEN}\"/" config.toml +sed -i "/DeepSeekToken/s/=.*/= \"{YOUR_DEEPSEEK_TOKEN}\"/" config.toml +# or set envvar +export OPENROUTER_API_KEY={YOUR_OPENROUTER_TOKEN} +export DEEPSEEK_API_KEY={YOUR_DEEPSEEK_TOKEN} +``` + +in case you're running llama.cpp here is an example of starting llama.cpp +``` +./build/bin/llama-server -c 16384 -ngl 99 --models-dir ./models --models-max 1 --models-preset ./models/config.ini +``` + +<b>after changing config.toml or envvar you need to restart the program.</b> + +for RP /completion endpoints are much better, since /chat endpoints swap any character name to either `user` or `assistant`; +once you have desired API endpoint +(for example: http://localhost:8080/completion) +there are two ways to pick a model: +- `ctrl+l` allows you to iterate through model list while in main window. +- `ctrl+p` (opens props table) go to the `Select a model` row and press enter, list of available models would appear, pick any that you want, press `x` to exit the props table. + +#### llama.cpp model preload +llama.cpp supports swapping models, to load picked ones press `alt+9` + +#### sending messages +type your message in the `input` widget; if it is not focused switch to it with pgup/pgdown or click your mouse on it. +messages are send by pressing `Esc` button +for ex. +``` +I blink slowly, confused "W-where? What happened?" +``` + +#### editing messages +press `f4` which'll prompt you to type index of the message you want to edit; +let's remove this part from sysmsg (0) +``` +Seraphina's voice holds a playful undertone, her eyes sparkling mischievously. +``` +`mischievous` imply distance from authority and intent for the rule breaking, but Seraphina described as devoted deity. +we can remove it or replace it with something less nonsensical. +``` +Seraphina, altough elegant, speaks her mind without embellishments or subtleties. Some would call her naive, some would rather call her unchallenged. +``` +when done, press `Esc` to return to main page. + +#### completion allowes for any number of chars +so let us make up story of our character; +let our character to be from high tech society who has mobile tablet device with AI called `HALL9000`, hunting certain target. +type the message, but first press f10 to prevent llm response (since it responds as Seraphina for now) +``` +I reach for my pocket and produce a small tablet shaped device. My mobile companion HALL9000. After making sure it is not broken I press my finger to the side +"Wake up Hal. Are you functional? Do you know where we are?" +``` + +we need to write first message ourselves (or at least start one) +there are two ways to write as a new char: +- ctrl+p -> `New char to write msg as` -> Enter -> `HAL9000` -> `Enter` -> `x` ; status line at the bottom should now have `Writing as HALL9000 (ctrl+q)` -> your next message would be send as HALL9000. +- ctrl+p -> `Inject role` switch to `No` -> `x`. gf-lt now won't inject your username in beginning of the message. It means you could write directly +``` +HAL9000: Red eye appears on the screen for the moment analyzing the request. +``` +`Esc`; now press `f10` to allow llm to write and press `ctrl+w` for it to continue the last message. +- if you set `New chat to write msg as`; you can switch back to writing as your char by pressing ctrl+q to rotate through the character list. +- if you went for `Inject role`: I advice to switch `Inject role` back to `Yes`. Otherwise you have to type `Charname:` in the beginning of each message. +example of gen (copied with `f7` (copies last msg)) +``` +Red eye appears on the screen for the moment analyzing the request. After a few moments, it replies: +"Affirmative. Location detected as Eldoria Forest, sector 7-B. This region has no records in my databases. My last known functional location was a human research facility." +The screen flashes briefly as it calculates. "I am experiencing degraded functionality due to environmental interference. I will attempt to stabilize systems." +*It emits a faint hum, and a holographic projection of a map flickers into existence, showing a dense forest with glowing markers.* +``` + +Once character name is in history we can switch who llm will respond as pressing `ctrl+x`. +For now it should be rotating between HALL9000, `Username`, Seraphina, system. +make status line to mention: `Bot will write as Seraphina (ctrl+x)` +and press escape to see her reaction. + +#### image input +if the model we run support image input we can Seraphina our target that we pursue +press `ctrl+o` to open a filepicker (home directory for filepicker could be set in config.toml) +and find an image file of our target +``` +I say to Hal "Hal, show our target." +An image appears on the screen. I show it to Seraphina. "Did you see that creature? I am looking for it." +``` |