feat: chat session response prefix by giladgd · Pull Request #375 · withcatai/node-llama-cpp · GitHub
Skip to content

feat: chat session response prefix - #375

Merged
giladgd merged 18 commits into
masterfrom
gilad/chatResponsePrefix
Oct 29, 2024
Merged

feat: chat session response prefix#375
giladgd merged 18 commits into
masterfrom
gilad/chatResponsePrefix

Conversation

@giladgd

@giladgd giladgd commented Oct 27, 2024

Copy link
Copy Markdown
Member

Description of change

  • feat: chat session response prefix
  • feat: improve context shift strategy
  • feat: use RAM and swap sizes in memory usage estimations
  • feat(inspect gguf command): print a single key flag
  • feat: faster building from source
  • fix: Electron crash with some models on macOS when not using Metal (via fix: use vm_allocate to allocate CPU backend buffer on macOS ggml-org/llama.cpp#9875)
  • fix: adapt to llama.cpp breaking changes
  • fix: improve CPU compatibility score
  • docs: mention the need to treat node-llama-cpp as an external module in Electron
  • docs: sitemap fixes
  • docs(README): make the logo be a link
  • chore: update issue config

Fixes #361

Pull-Request Checklist

  • Code is up-to-date with the master branch
  • npm run format to apply eslint formatting
  • npm run test passes with this change
  • This pull request links relevant issues as Fixes #0000
  • There are new or updated unit tests validating the change
  • Documentation has been updated to reflect this change
  • The new commits and pull request title follow conventions explained in pull request guidelines (PRs that do not follow this convention will not be merged)

@giladgd
giladgd requested a review from ido-pluto October 27, 2024 00:04
@giladgd giladgd self-assigned this Oct 27, 2024

@ido-pluto ido-pluto left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@giladgd
giladgd merged commit ea12dc5 into master Oct 29, 2024
@giladgd
giladgd deleted the gilad/chatResponsePrefix branch October 29, 2024 23:54
@github-actions

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Electron sample app crash on Mac with speciffic model Meta-Llama-3.1-8B-Instruct.Q4_K_M.gguf

2 participants