[GPU] Preserve weight sharing when cloning weights Convert in ConvertMatMulToFullyConnected

[alvoron]

Issue:

#35993 introduced a GPU memory regression that inflates on-device weight memory.

Root cause:

ConvertMatMulToFullyConnected normalizes weights by inserting a Transpose in front of the weights and re-wiring the decompression Convert. #35993 changed this to always convert->clone_with_new_inputs(...) - correct in intent (it prevents corrupting a Convert shared by sibling MatMuls), but clone_with_new_inputs() has two side effects that were not handled:

1. Lost runtime-info markers. The clone starts with empty rt_info, and both Decompression and DisableConstantFolding are intentionally non-copyable (is_copyable() == false), so they cannot be carried over. Without them, the subsequent ConstantFolding folds the cloned f16-const → Convert(f32) weights path into a materialized f32 weight constant, doubling that weight's device memory.
2. Broken weight sharing. When one decompression Convert is shared by several MatMuls, cloning it per consumer produces N independent weight subgraphs that ConstantFolding materializes into N separate weight constants instead of one.

Fix:

In convert_matmul_to_fc.cpp:

1. Re-apply the non-copyable markers on the cloned Convert - mark_as_decompression and disable_constant_folding - when the original Convert had them, so the compressed weights are not const-folded into a materialized f32 weight.
2. Deduplicate clones via a per-pass cache. Sibling MatMuls sharing the same Convert reuse a single cloned weights subgraph, preserving cross-layer weight sharing.

The cache is std::map<std::pair<std::shared_ptr<Node> /*original Convert*/, bool /*is_small_matmul*/>, ov::Output<Node>>:

• The shared_ptr keeps the key node alive so its address cannot be reused by an unrelated node.
• is_small_matmul is part of the key because it selects the weights layout (and hence the resulting FullyConnected's transpose_b). Sibling MatMuls sharing one Convert can differ in is_small_matmul (they may differ in M), so a clone built for one layout must not be reused for the other. The existing sibling guard already forces such siblings to share matmul->transpose_b, so (Convert, is_small_matmul) fully determines the normalized weights.

The cache lookup is performed before weights normalization: on a cache hit the already-normalized weights are reused and both the Transpose insertion and the Convert clone are skipped. Activation normalization (transpose_a) is hoisted above this block so it still runs for every MatMul, including cache hits - it is independent of the weights path. This keeps #35993's shared-Convert-corruption fix intact (the clone still isolates each transpose) while avoiding throwaway node creation on hits for models with many MatMuls sharing one weights Convert.

Tickets:

• CVS-187458

[Copilot]

Pull request overview

This PR fixes a GPU on-device weight memory regression introduced in ConvertMatMulToFullyConnected by preserving compressed-weight runtime markers on cloned Convert nodes and by deduplicating clones when the same weights-path Convert is shared across sibling MatMuls.

Changes:

• Add a per-pass cache to reuse a single cloned weights subgraph for shared Convert nodes (keyed by (Convert, is_small_matmul)), preserving weight sharing.
• Re-apply Decompression and DisableConstantFolding markers to cloned Convert nodes to prevent unintended constant folding into materialized f32 weights.
• Add unit tests covering (1) no duplicate converts for shared weights and (2) preservation of the no-fold marker on a cloned Convert.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
src/plugins/intel_gpu/src/plugin/transformations/convert_matmul_to_fc.cpp	Deduplicates shared weights-path Convert cloning and preserves key rt_info markers on the clone.
src/plugins/intel_gpu/tests/unit/transformations/convert_matmul_to_fc_test.cpp	Adds regression tests for shared-convert deduplication and no-fold marker preservation on the cloned convert.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

[e-ddykim]

It looks like we can reduce code duplication by moving this logic into a separate helper function.

[e-ddykim]

Suggested change

	if (is_small_matmul) {
	auto can_reuse_transpose = [&transpose_node](const ov::Output<ov::Node>& weights) {
	if (!transpose_node || transpose_node->get_input_size() != 2) {
	return false;
	}
	auto order_constant = ov::as_type_ptr<ov::op::v0::Constant>(transpose_node->get_input_node_shared_ptr(1));
	if (!order_constant) {
	return false;
	}
	std::vector<size_t> expected_order(weights.get_partial_shape().size());
	std::iota(expected_order.begin(), expected_order.end(), 0);
	std::swap(*(expected_order.end() - 1), *(expected_order.end() - 2));
	return order_constant->cast_vector<size_t>() == expected_order;
	};
	if (is_small_matmul) {

[e-ddykim]

Suggested change

	bool can_reuse_transpose = false;
	if (!matmul->get_transpose_b()) {
	if (transpose_node && transpose_node->get_input_size() == 2) {
	auto order_constant = ov::as_type_ptr<ov::op::v0::Constant>(transpose_node->get_input_node_shared_ptr(1));
	if (order_constant) {
	std::vector<size_t> order = order_constant->cast_vector<size_t>();
	std::vector<size_t> expected_order(fc_input_b.get_partial_shape().size());
	std::iota(expected_order.begin(), expected_order.end(), 0);
	std::swap(*(expected_order.end() - 1), *(expected_order.end() - 2));
	can_reuse_transpose = order == expected_order;
	}
	}
	fc_input_b = can_reuse_transpose ? transpose_node
	: create_transpose(fc_input_b, matmul->get_friendly_name() + "/transpose_b");
	if (!matmul->get_transpose_b()) {
	fc_input_b = can_reuse_transpose(fc_input_b)
	? transpose_node
	: create_transpose(fc_input_b, matmul->get_friendly_name() + "/transpose_b");

[e-ddykim]

Suggested change

	if (transpose_node && transpose_node->get_input_size() == 2) {
	auto order_constant = ov::as_type_ptr<ov::op::v0::Constant>(transpose_node->get_input_node_shared_ptr(1));
	if (order_constant) {
	std::vector<size_t> order = order_constant->cast_vector<size_t>();
	std::vector<size_t> expected_order(fc_input_b.get_partial_shape().size());
	std::iota(expected_order.begin(), expected_order.end(), 0);
	std::swap(*(expected_order.end() - 1), *(expected_order.end() - 2));
	if (order == expected_order)
	fc_input_b = transpose_node;
	}
	if (can_reuse_transpose(fc_input_b)) {
	fc_input_b = transpose_node;