Stars
DeepInfer: High-performance, large-scale inference framework for DeepSeek models
Code base for ICLR 2024 "Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature".
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.
a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features (GGUF, RadixAttention, Cache-aware scheduling, ...)
A fast Clojure Tensor & Deep Learning library
Fast Clojure Machine Learning Model Integration
Official inference framework for 1-bit LLMs
Measure object memory consumption in Clojure
An uber-fast parallelized Java classpath scanner and module scanner.
FlashSampling: Fast and Memory-Efficient Exact Sampling (https://huggingface.co/papers/2603.15854)
On-device AI across mobile, embedded and edge for PyTorch
Py4J enables Python programs to dynamically access arbitrary Java objects
Speech-to-text, text-to-speech, speaker diarization, speech enhancement, source separation, and VAD using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Andr…
Matplot++: A C++ Graphics Library for Data Visualization 📊🗾
⚡ Datoviz: high-performance GPU rendering for scientific data visualization
BLAS-like Extensions for Neanderthal, Fast Clojure Matrix Library
lightweight, standalone C++ inference engine for Google's Gemma models.
Helpful kernel tutorials, examples and SKILLs for tile-based GPU programming
Up to 100x faster strings for C, C++, CUDA, Python, Rust, Swift, JS, & Go, leveraging NEON, AVX2, AVX-512, SVE, GPGPU, & SWAR to accelerate search, hashing, sorting, edit distances, sketches, and m…
onnxruntime-extensions: A specialized pre- and post- processing library for ONNX Runtime
Awesome LLM Books: Curated list of books on Large Language Models