Native C++ tools, GPU & edge AI, and performance engineering.
I turn real-world problems into practical tools, and share the experiments, methods, and trade-offs behind them.
Technical writing on Medium · Projects & contributions · Research portfolio
How does a model reach different silicon, and what does that actually cost? My work connects model conversion, native runtimes, GPU memory, compiler-generated kernels, and reproducible measurement.
- Multi-backend edge inference: a C++20 framework with a stable execution-provider interface and separate model-family logic. I evaluate detection and RIFE frame interpolation across ten providers, spanning Windows, Jetson, and Android, and investigate IREE/RVV compiler paths.
- GPU-resident video: keep frames on the GPU across decode, interpolation, and encode. I study buffer interoperability, the CPU transfer envelope, and the effects on latency and energy—not just the neural-network kernel.
- LLM performance engineering: use a shared HTTP harness to compare vllm.cpp, llama.cpp, and vLLM; separate prefill from decode, control caching and concurrency, and investigate paged KV memory, dispatch overhead, GEMM/GEMV, and Vulkan/Triton CPU kernels.
Research portfolio & methodology · Published edge-inference deep dive
I contribute to vllm.cpp, a community C++ inference engine. My port/vulkan development branch brings together work on Vulkan compute kernels and subgroup portability, an optional Triton CPU provider, native CPU optimization, and Windows/CUDA build support.
- Vulkan: capability-gated NVIDIA cooperative-matrix paths, attention kernels that use the device's subgroup width, and decode GEMV tuning.
- CPU: an opt-in Triton CPU provider with native fallback, plus AVX-512 attention and matmul optimizations. The Triton CPU kernel loader currently serves operations on Linux.
- Windows & CUDA: MSVC portability repairs, consumer Blackwell Triton AOT assets, and fork-specific release work.
Explore my branch · Contribution record & selected commits
This is a contribution fork of mudler/vllm.cpp. The contribution record links my implementation commits and separates branch work from verified upstream acceptance.
I write in English and Traditional Chinese about native software, GPU pipelines, local AI, and the measurements behind engineering decisions.
- Edge inference across CPU, GPU, NPU, and RISC-V — execution-provider design, correctness checks, and GPU-resident video pipelines. Research write-up; a public standalone engine repository is not listed here.
- GPU thermal protection for sustained AI workloads — the investigation behind GpuThermalGuard.
- A modern C++ hooking runtime — the design behind Splice.
- Explore more articles on Medium →
GWT agent integrations · Midp2EXE · Windows SDK 7.1 / Visual Studio
A Windows utility for removing ASUS software, services, drivers, and leftover registry entries, with a dry-run preview.
Current repository & usage · Original gist / migration notice
The tool moved from the gist to the repository. Start with the current repository; the gist preserves the original entry point and directs existing users to the new home.
My repositories include tools I maintain, forks with substantial implementation work, and forks used for research. Start with projects & contributions to explore my work. A repository's fork label describes its lineage; the linked contributions show what I added.