- Context and per-sequence state snapshots (
llama_state_get_data,llama_state_seq_get_data/_set_data,llama_state_seq_save_file/_load_file) — checkpoint a conversation, or cache a shared prefix instead of recomputing it. - New introspection and steering bindings: extra
llama_model_info()fields,llama_vocab_get_attr()and friends,llama_lora_meta(),llama_apply_control_vector(),llama_perf_sampler().
- Engine updated to upstream llama.cpp (expanded architectures, incl. Qwen3.5).
- Multimodal (vision/OCR) support via the mtmd subsystem (
llama_mtmd_load,llama_image_load,llama_image_eval,llama_gen_begin_at). - Tool calling via chat templates with structured output parsing (
llama_chat_build,llama_chat_parse). - Anthropic Messages API server (
llama_serve_anthropic) for local models — runs Claude Code against a local model. llama_serve_anthropic()optional vision:vision_model_path+mmproj_pathenable a caption-then-reason pipeline (a vision model describes images, the text model answers).enable_thinkingargument onllama_chat_build()/llama_serve_anthropic()to toggle hybrid thinking models' reasoning.
llama_gen_begin()/llama_gen_next()/llama_gen_end()— token-by-token generation matchingllama_generate()output, with valid-UTF-8 chunks.llama_serve_openai()— serve a local GGUF model over an OpenAI-compatible HTTP API (/v1/models,/v1/chat/completions, streaming and blocking) via the optionaldrogonRpackage.chat_ll 8000 amar()— returns anellmer::Chatbacked by a local model, connecting to a running server (base_url=) or spawning one (model_path=);chat_llamar_stop()stops a spawned server.
llama_n_ctx_seq()— per-sequence context window size.llama_n_batch()— logical batch size (max tokens perllama_decodecall).llama_n_ubatch()— physical micro-batch size.llama_n_seq_max()— maximum number of concurrent sequences.llama_n_threads()/llama_n_threads_batch()— read back thread counts set viallama_set_threads().llama_pooling_type()— pooling type of the context as a string ("none","mean","cls","last","rank").
llama_get_logits_ith()— logit vector for a specific token position in the last decoded batch. Supports negative indexing (-1= last token).
embed_llamar()— high-level embedding provider compatible withragnar_store_create(embed = ...). Supports partial application (lazy model loading), direct call returning a matrix, and data.frame input. L2 normalization on by default.
llama_embed_batch()— embed multiple texts in one call. Uses true pooled batch decode (llama_get_embeddings_seq) for embedding models, with automatic fallback to sequential last-token decode for generative models.llama_get_embeddings_ith()— get embedding vector for the i-th token (supports negative indexing).llama_get_embeddings_seq()— get pooled embedding for a sequence ID.
llama_new_context()gainsembeddingparameter. WhenTRUE, setscparams.embeddings = trueand disables causal attention at creation time.llama_embed_batch()uses this flag to choose the optimal code path.
llama_load_model()gainsdevicesparameter for explicit backend selection. Accepts device names fromllama_backend_devices(), type keywords ("cpu","gpu"), or numeric indices. Multiple devices enable multi-GPU split.llama_backend_devices()— list all available compute devices (CPU, GPU, iGPU, accelerator) as a data.frame.
llama_numa_init()— NUMA optimization with strategies: disabled, distribute, isolate, numactl, mirror.llama_time_us()— current time in microseconds.
- 40+ new test blocks covering all new functions.
- Total: 143 passing, 4 expected skips.
llama_token_to_piece()— convert a single token ID to its text piece.llama_encode()— run the encoder pass for encoder-decoder models (e.g. T5, BART).llama_batch_init()/llama_batch_free()— low-level batch allocation and release with automatic GC finalizer.
- Fixed compilation failure on macOS with Apple clang 17 / Xcode 16.4:
removed
extern "C"block wrapping#include <R.h>inr_llama_compat.h(C++ templates cannot appear insideextern "C"linkage). - Fixed macro conflict between
Rinternals.h#define length(x)andstd::codecvt::length()inr_llama_interface.cpp: C++ standard headers are now included before R headers, followed by#undef length.
- Added 9 new test blocks covering
llama_token_to_piece,llama_batch_init,llama_batch_free, andllama_encode, including GPU context variants. - Total: 103 passing, 4 expected skips.
llama_hf_list()— list GGUF files in a Hugging Face repository.llama_hf_download()— download a GGUF model with local caching. Supports exact filename, glob pattern, or Ollama-style tag selection.llama_load_model_hf()— download and load a model in one step.llama_hf_cache_dir()— get the cache directory path.llama_hf_cache_info()— inspect cached models.llama_hf_cache_clear()— clear the model cache.
- Added
jsonliteandutilstoImports.
- Added Vulkan linking support to
configure.winandMakevars.win.in. - Windows builds now link with Vulkan when
ggmlRis built with GPU support.
- Added
exit()/_Exit()overrides tor_llama_compat.hto prevent process termination (redirects toRf_error()).
- Requires
ggmlR>= 0.5.4. - Bumped minimum R version to 4.1.0 (matches
ggmlR).
- Updated description to mention Vulkan GPU support via
ggmlR.
- Expanded all acronyms in DESCRIPTION (LLMs, GPU).
- Added detailed
\valuetags to all exported functions describing return class, structure, and meaning. - Replaced
\dontrun{}with\donttest{}in all examples.
- Added Georgi Gerganov as copyright holder (
cph) for bundled 'llama.cpp' code.
- Included
NEWS.mdin the package tarball 547C (removed from.Rbuildignore). - Created
cran-comments.md. - Cleaned up duplicate entries in
.Rbuildignore.
Full LLM inference cycle is now available from R:
llama_load_model()/llama_free_model()— load and free GGUF modelsllama_new_context()/llama_free_context()— context managementllama_tokenize()/llama_detokenize()— tokenization and detokenizationllama_generate()— text generation with temperature, top_k, top_p, greedy supportllama_embeddings()— embedding extractionllama_model_info()— model metadata
Model and context are wrapped as ExternalPtr with automatic GC finalizers. The context holds a reference to the model ExternalPtr, preventing premature collection.
llama_generate() runs the full pipeline in a single C++ call: prompt
tokenization → encode → autoregressive decode loop with a sampler chain →
detokenization of generated tokens.
19 assertions across 7 test blocks, all passing.
- Basic package structure with llama.cpp integration
- Links against
libggml.afrom ggmlR package - Includes all llama.cpp model implementations (~100 architectures)
- Vulkan GPU support (optional)
- Requires ggmlR >= 0.5.1 for static library export
ggml_build_forward_selectreplaced with simplified branch selection