[GGUF FE] Simplify decoder API and refactor stateless/stateful KV cache

Refactor the frontend so conversion is device- and mode-agnostic, trim the
decoder interface to what the translators actually consume, drop dead code, and
make the frontend installable while hidden from FrontEndManager.

Decoder API:
- Surface GGUF weights as genuine GGML_OP_NONE leaf nodes (marked by a "data"
  attribute) instead of a synthetic op.
- Make GgufDecoder node-scoped: visit_subgraph binds the decoder to one node and
  the per-node accessors drop the node_idx parameter.
- Fold has_rope / use_per_op_rope / get_rope_config into a single
  get_attribute("rope_config"); drop is_swa_layer, is_static, is_stateful,
  get_model_output_shape and get_kv_param_res_names from the interface (they are
  backend-internal or unused).
- Drop unused NodeContext members (translate_session, get_output_name) and the
  dead, never-registered FuseToSDPA pass (the flash-attn translator emits
  ScaledDotProductAttention directly, so there is nothing to fuse).
- Use the real ov::pass::MarkCompressedFloatConstants transformation (link
  openvino::offline_transformations) instead of a hand-copied forward-declaration
  header.
- Remove dead code carried over from the initial copy: unused translator
  declarations (translate_add / translate_mul), unused utils helpers
  (getCurrentTime, dump_ov_model, non_cont_dim, argsort_descend, sorted_descend,
  is_permuted, permute), the unused make_fused_qkv_weights / slice_rows weight
  splitters, the dead stateful branch/parameter of make_sin_cos, and the
  never-defined get_shape(gguf_tensor) declaration (gguf_fill_mxfp4 now computes
  rows/cols from tensor.dim directly, fixing a Windows link error).

Stateless / stateful KV cache:
- The frontend always emits the stateless, llama.cpp-faithful graph. Every ggml
  SET_ROWS becomes an internal SetRows op, lowered in the normalization stage:
  the built-in LowerSetRowsStateless rewrites it to ScatterUpdate by default.
- A caller (the ggml-openvino backend) may register a stateful lowering via
  DecoderTransformationExtension to convert the attention KV writes into an
  OpenVINO stateful ReadValue/Concat/Assign subgraph -- selected structurally,
  with no decoder-side flag.
- The NPU-only SqueezeMatmul pass moves out of the frontend to the backend.

Install + discoverability:
- Install the frontend library and headers alongside the other frontends (drop
  SKIP_INSTALL) so callers can link openvino::frontend::gguf directly, and add
  the GGUF component to the Debian/RPM packaging (copyright, package name).
- Add FrontEndPluginInfo::m_hidden: a hidden frontend loads normally but is
  excluded from FrontEndManager::get_available_front_ends() and from
  load_by_model / load_by_framework. The GGUF frontend registers hidden (and
  supported_impl returns false), so it is not exposed via the generic model
  loading API while remaining usable by direct linkers.

Tests:
- expect_near uses a combined absolute + relative tolerance so the op tests pass
  on hardware that runs the graph in fp16 (e.g. ARM CPU by default), where a
  fixed fp32-tuned absolute tolerance is too tight.

Author: mvafin

cmake/packaging/debian.cmake

@@ -327,6 +327,19 @@ macro(ov_cpack_settings)
         set(tensorflow_lite_copyright "generic")
     endif()
 
+    if(ENABLE_OV_GGUF_FRONTEND)
+        set(CPACK_COMPONENT_GGUF_DESCRIPTION "OpenVINO GGUF Frontend")
+        set(CPACK_COMPONENT_GGUF_DEPENDS "${OV_CPACK_COMP_CORE}")
+        set(CPACK_DEBIAN_GGUF_PACKAGE_NAME "libopenvino-gguf-frontend-${cpack_name_ver}")
+        # since GGUF FE is a linkable target, we need to call ldconfig (i.e. `def_triggers`)
+        set(CPACK_DEBIAN_GGUF_PACKAGE_CONTROL_EXTRA "${def_postinst};${def_postrm};${def_triggers}")
+        ov_debian_add_lintian_suppression(gguf
+            # we have different package name strategy; it suggests libopenvino-gguf-frontend202230
+            "package-name-doesnt-match-sonames")
+        list(APPEND frontends gguf)
+        set(gguf_copyright "generic")
+    endif()
+
     #
     # core_dev: depends on core and frontends (since frontends don't want to provide its own dev packages)
     #

cmake/packaging/rpm.cmake

@@ -277,6 +277,15 @@ macro(ov_cpack_settings)
         set(tensorflow_lite_copyright "generic")
     endif()
 
+    if(ENABLE_OV_GGUF_FRONTEND)
+        set(CPACK_COMPONENT_GGUF_DESCRIPTION "OpenVINO GGUF Frontend")
+        set(CPACK_RPM_GGUF_PACKAGE_NAME "libopenvino-gguf-frontend-${cpack_name_ver}")
+        set(CPACK_RPM_GGUF_POST_INSTALL_SCRIPT_FILE "${def_triggers}")
+        set(CPACK_RPM_GGUF_POST_UNINSTALL_SCRIPT_FILE "${def_triggers}")
+        _ov_add_package(frontend_packages gguf)
+        set(gguf_copyright "generic")
+    endif()
+
     #
     # core_dev: depends on core and frontends (since frontends don't want to provide its own dev packages)
     #

src/frontends/common/include/openvino/frontend/manager.hpp

@@ -123,6 +123,12 @@ using FrontEndVersion = uint64_t;
 struct FrontEndPluginInfo {
     std::string m_name;
     FrontEndFactory m_creator;
+    /// When true, the frontend is hidden from the generic model-loading API: it is not returned
+    /// by FrontEndManager::get_available_front_ends() and is never auto-selected by
+    /// load_by_model / load_by_framework. Such a frontend is intended for direct linkage only
+    /// (the caller constructs it explicitly). It is still installed and loadable, so it must
+    /// export get_api_version / get_front_end_data like any other plugin.
+    bool m_hidden = false;
 };
 
 }  // namespace frontend

src/frontends/common/src/manager.cpp

@@ -58,6 +58,10 @@ class FrontEndManager::Impl {
         // Load plugins until we found the right one
         for (auto& plugin : m_plugins) {
             OPENVINO_ASSERT(plugin.load(), "Cannot load frontend ", plugin.get_name_from_file());
+            // Hidden frontends are not selectable by name through the generic API.
+            if (plugin.get_creator().m_hidden) {
+                continue;
+            }
             if (plugin.get_creator().m_name == framework) {
                 return make_frontend(plugin);
             }
@@ -74,6 +78,10 @@ class FrontEndManager::Impl {
                 OPENVINO_DEBUG("Frontend load failed: ", plugin_info.m_file_path, "\n");
                 continue;
             }
+            // Hidden frontends are for direct linkage only; do not advertise them.
+            if (plugin_info.get_creator().m_hidden) {
+                continue;
+            }
             names.push_back(plugin_info.get_creator().m_name);
         }
         return names;
@@ -91,6 +99,10 @@ class FrontEndManager::Impl {
             if (!plugin.load()) {
                 continue;
             }
+            // Hidden frontends are for direct linkage only; never auto-select them.
+            if (plugin.get_creator().m_hidden) {
+                continue;
+            }
             auto fe = plugin.get_creator().m_creator();
             OPENVINO_ASSERT(fe, "Frontend error: frontend '", plugin.get_creator().m_name, "' created null FrontEnd");
             if (fe->supported(variants)) {

src/frontends/gguf/include/openvino/frontend/gguf/decoder.h

@@ -20,103 +20,93 @@ namespace gguf {
 // gguf-file decoder by construction), so the op translators never need to know ggml's
 // memory layout.
 struct RopeConfig {
-    int n_dims = 0;
+    int n_dims = 0;  // 0 means the model uses no RoPE (replaces a separate has_rope() query)
     int n_ctx_orig = 0;
     float freq_base = 0.0f;
     float freq_scale = 0.0f;
     float ext_factor = 0.0f;
     float attn_factor = 0.0f;
     float beta_fast = 0.0f;
     float beta_slow = 0.0f;
+    // When true, each ROPE op builds its own sin/cos from its per-op config (e.g. gemma4, where
+    // SWA and global layers use different n_dims), so the shared rope_cos/rope_sin table that
+    // TranslateSession::preprocess would otherwise pre-build is skipped.
+    bool per_op = false;
 };
 
 // Decoder interface consumed by the gguf frontend translators.
 //
+// Following the established OpenVINO frontend pattern (cf. the PyTorch frontend), a GgufDecoder
+// is node-scoped: visit_subgraph hands the visitor a fresh decoder bound to a single node, and
+// every per-node accessor (get_attribute, get_input_*, get_output_*, get_op_*) refers to that
+// node -- no node index is threaded through. The same object type, queried at model scope (the
+// instance returned by InputModel::get_decoder and iterated by visit_subgraph), answers the
+// model-level questions (get_model_inputs, get_model_output_names, get_rope_config, ...).
+//
 // This is a typed, ggml-free interface: operation parameters are exposed through
-// get_attribute(node_idx, name) / get_input_view_offset / RopeConfig rather than as raw
-// ggml `op_params` int32 arrays. A concrete decoder (e.g. the llama.cpp cgraph decoder)
-// only has to translate ggml's layout into these typed accessors -- the op translators
-// here never touch ggml memory directly.
+// get_attribute(name) / get_input_view_offset / RopeConfig rather than as raw ggml `op_params`
+// int32 arrays. A concrete decoder (e.g. the llama.cpp cgraph decoder) only has to translate
+// ggml's layout into these typed accessors -- the op translators here never touch ggml memory.
 class GgufDecoder : public DecoderBase {
 public:
+    // Per-node typed attribute access (the bound node). The op translators use this to read
+    // scalar operation parameters (e.g. "eps", "scale", "bias", "swapped", "rope_config")
+    // without dereferencing ggml's raw op_params layout.
     ov::Any get_attribute(const std::string& name) const override = 0;
 
-    // Per-node typed attribute access. This is the mechanism the op translators use to
-    // read scalar operation parameters (e.g. "eps", "scale", "bias", "swapped",
-    // "softmax_axis", "rope_config") without dereferencing ggml's raw op_params layout.
-    virtual ov::Any get_attribute(int node_idx, const std::string& name) const = 0;
-
-    virtual PartialShape get_input_shape(int node_idx, const std::string& name) const = 0;
+    virtual PartialShape get_input_shape(const std::string& name) const = 0;
 
-    virtual std::vector<size_t> get_input_stride(int node_idx, const std::string& name) const = 0;
+    virtual std::vector<size_t> get_input_stride(const std::string& name) const = 0;
 
     // Byte offset of an input that is a gguf VIEW into a larger tensor (0 when the input
     // is not a view). Replaces the raw `get_input_op_params(...)[0]` read: a view's start
     // offset is a single semantic scalar the decoder knows (by parsing ggml, or by
     // construction), so translators never touch ggml's op_params layout. Translators
     // convert bytes -> elements using get_input_stride as needed.
-    virtual int64_t get_input_view_offset(int node_idx, const std::string& name) const = 0;
+    virtual int64_t get_input_view_offset(const std::string& name) const = 0;
 
-    virtual element::Type get_input_type(int node_idx, const std::string& name) const = 0;
+    virtual element::Type get_input_type(const std::string& name) const = 0;
 
     size_t get_input_size() const override = 0;
 
-    virtual size_t get_input_size(int node_idx) const = 0;
-
     void get_input_node(size_t input_port_idx,
                         std::string& producer_name,
                         std::string& producer_output_port_name,
                         size_t& producer_output_port_index) const override = 0;
 
-    virtual std::vector<std::string> get_input_names(int node_idx) const = 0;
+    virtual std::vector<std::string> get_input_names() const = 0;
 
-    virtual PartialShape get_output_shape(int node_idx) const = 0;
+    virtual PartialShape get_output_shape() const = 0;
 
-    virtual element::Type get_output_type(const int node_idx) const = 0;
+    virtual element::Type get_output_type() const = 0;
 
-    virtual std::vector<std::string> get_output_names(int node_idx) const = 0;
+    virtual std::vector<std::string> get_output_names() const = 0;
 
     const std::string& get_op_type() const override = 0;
 
-    virtual const std::string& get_op_type(int node_idx) const = 0;
-
     const std::string& get_op_name() const override = 0;
 
-    virtual const std::string& get_op_name(int node_idx) const = 0;
-
-    virtual void visit_subgraph(std::function<void(std::shared_ptr<GgufDecoder>, int node_idx)> node_visitor) const = 0;
+    virtual void visit_subgraph(std::function<void(std::shared_ptr<GgufDecoder>)> node_visitor) const = 0;
 
-    virtual int get_op_case(int node_idx) const = 0;
+    virtual int get_op_case() const = 0;
 
     virtual const std::map<std::string, std::shared_ptr<ov::Node>>& get_model_inputs() const = 0;
     virtual const std::map<std::string, std::shared_ptr<ov::Node>>& get_model_extra_inputs() const = 0;
     virtual std::vector<std::string> get_model_output_names() const = 0;
 
     // NOTE: there is no get_model_weights(). A GGUF weight is surfaced as a regular node in
-    // visit_subgraph with op type "GGML_OP_WEIGHT": the decoder exposes the raw weight bytes
-    // via get_attribute<ov::Tensor>(node_idx, "data"), the ggml quant type name via
-    // get_attribute<std::string>(node_idx, "quant_type") (e.g. "Q4_K", "F16") and the logical
-    // [rows, cols] shape via get_output_shape(node_idx). The frontend's translate_weight does
-    // the dequant / repacking / requantization, so the decoder never builds OV nodes itself.
-
-    // Model-level RoPE configuration, used by TranslateSession::preprocess to pre-build
-    // the shared rope sin/cos. has_rope() == false means the model uses no RoPE.
-    virtual bool has_rope() const = 0;
-
-    // When true, each ROPE op generates its own sin/cos from its per-op rope_config
-    // (e.g. gemma4 where SWA and global layers have different n_dims). The shared
-    // rope_cos/rope_sin tensor is NOT added to the tensor map.
-    virtual bool use_per_op_rope() const = 0;
-
-    virtual RopeConfig get_rope_config() const = 0;
-
-    virtual std::map<std::string, std::string> get_kv_param_res_names() const = 0;
-
-    virtual bool is_static() const = 0;
-
-    virtual bool is_stateful() const = 0;
-
-    virtual bool is_swa_layer(int layer) const = 0;
+    // visit_subgraph with the genuine ggml leaf op type "GGML_OP_NONE": the decoder marks it as a
+    // weight by exposing the raw weight bytes via get_attribute<ov::Tensor>("data"), the ggml
+    // quant type name via get_attribute<std::string>("quant_type") (e.g. "Q4_K", "F16") and the
+    // logical [rows, cols] shape via get_output_shape(). The frontend's translate_weight does the
+    // dequant / repacking / requantization, so the decoder never builds OV nodes itself. (Model
+    // inputs are also GGML_OP_NONE leaves, but they are returned via get_model_inputs() and
+    // resolved to Parameters before the walk, so they carry no "data".)
+
+    // RoPE configuration, exposed through get_attribute<RopeConfig>("rope_config"):
+    //   - at model scope, used by TranslateSession::preprocess to pre-build the shared rope
+    //     sin/cos table (skipped when RopeConfig::n_dims == 0, i.e. no RoPE, or per_op == true);
+    //   - at node scope, the ROPE translator reads the same key for the op's own config.
 };
 
 }  // namespace gguf

src/frontends/gguf/include/openvino/frontend/gguf/set_rows_op.hpp

@@ -0,0 +1,43 @@
+// Copyright (C) 2018-2026 Intel Corporation
+// SPDX-License-Identifier: Apache-2.0
+//
+
+#pragma once
+
+#include "openvino/frontend/gguf/visibility.hpp"
+#include "openvino/op/op.hpp"
+
+namespace ov {
+namespace frontend {
+namespace gguf {
+
+// Internal placeholder op for a ggml SET_ROWS (a scatter-write of rows into a destination
+// tensor). translate_set_rows always emits this op instead of a device-specific ScatterUpdate,
+// so conversion is identical regardless of execution mode. Inputs:
+//   input 0: data    (rows to write, reshaped to the destination layout [1, 1, seq, emb])
+//   input 1: indices (destination row indices, squeezed)
+//   input 2: dst     (the tensor written into -- a Parameter for a KV cache)
+// The output is the updated tensor; both the cache Result and the attention read path consume it.
+//
+// A normalization-stage lowering replaces every SetRows: the built-in LowerSetRows (default)
+// rebuilds the llama.cpp-faithful ScatterUpdate form. A caller may register (via
+// DecoderTransformationExtension) an alternative lowering that recognizes the SetRows feeding
+// attention as a KV-cache write and turns it into an OpenVINO stateful ReadValue/Concat/Assign
+// subgraph. The op never reaches a compiled model. It is public so a caller-side lowering pass
+// can match it by type.
+class GGUF_FRONTEND_API SetRows : public ov::op::Op {
+public:
+    OPENVINO_OP("SetRows", "gguf");
+
+    SetRows() = default;
+    SetRows(const ov::Output<ov::Node>& data,
+            const ov::Output<ov::Node>& indices,
+            const ov::Output<ov::Node>& dst);
+
+    void validate_and_infer_types() override;
+    std::shared_ptr<ov::Node> clone_with_new_inputs(const ov::OutputVector& new_args) const override;
+};
+
+}  // namespace gguf
+}  // namespace frontend
+}  // namespace ov

src/frontends/gguf/src/CMakeLists.txt

@@ -2,14 +2,21 @@
 # SPDX-License-Identifier: Apache-2.0
 #
 
-# The GGUF frontend is an MVP port of llama.cpp's ggml-openvino frontend.  It is built
-# only when ENABLE_OV_GGUF_FRONTEND is ON and is marked SKIP_INSTALL so it is never
-# shipped in the OpenVINO release package (it is for development / testing only).
+# The GGUF frontend is an MVP port of llama.cpp's ggml-openvino frontend, built only when
+# ENABLE_OV_GGUF_FRONTEND is ON. It installs the library + headers alongside the other
+# frontends (LINKABLE_FRONTEND, no SKIP_INSTALL) so callers -- the llama.cpp ggml-openvino
+# backend and OpenVINO GenAI -- can link openvino::frontend::gguf directly.
+#
+# It is intentionally NOT discoverable via FrontEndManager: it exports no plugin-registration
+# entry points (get_api_version / GetFrontEndData), so it never appears in available_front_ends()
+# and is never auto-selected by load_by_model / load_by_framework, even though it is installed in
+# the frontend directory. See the discoverability note in frontend.cpp. This keeps the
+# not-yet-production frontend hidden from the generic model-loading API while remaining usable by
+# direct linkers.
 ov_add_frontend(NAME gguf
                 LINKABLE_FRONTEND
-                SKIP_INSTALL
                 FILEDESCRIPTION "FrontEnd to convert GGUF models"
-                LINK_LIBRARIES openvino::util openvino::core::dev)
+                LINK_LIBRARIES openvino::util openvino::core::dev openvino::offline_transformations)
 
 ov_build_target_faster(openvino_gguf_frontend PCH)
 
@@ -27,8 +34,5 @@ elseif(MSVC)
     target_compile_options(openvino_gguf_frontend PRIVATE /wd4244 /wd4267)
 endif()
 
-# SKIP_INSTALL keeps the frontend out of every install component, so it never ships
-# in the release package and is never placed in the OpenVINO frontend search
-# directory (where FrontEndManager would try to load it).  The unit tests do not link
-# this library: they compile the frontend sources directly (see tests/CMakeLists.txt),
-# keeping the test binary self-contained.
+# The unit tests do not link this library: they compile the frontend sources directly
+# (see tests/CMakeLists.txt), keeping the test binary self-contained.