InfiniLM-ModelHub/cpp_backends at master · InfiniTensor/InfiniLM-ModelHub · GitHub
Skip to content

Latest commit

 

History

History

Folders and files

C++ Backend Plugins

This directory contains out-of-tree C++ backend plugin examples.

  • modelhub_backend_adapters.cpp demonstrates registering new model_type values that delegate to existing InfiniLM backends.
  • New architectures that need their own decoder blocks, cache behavior, or model-specific operators should add a dedicated subdirectory here.

See docs/out_of_tree_backend.md for the complete backend/operator integration flow.