[CPU][TRANSFORMATIONS][SNIPPETS] Keep Sin/Cos argument computation in f32 under bf16/f16 inference precision

[goyaladitya05]

Details:

Root cause: when RoPE tables are computed at runtime inside the model (e.g. LTX-Video's 3-axis rope, whose grid depends on width/height/num_frames inputs, so RoPEFusion cannot apply), bf16 enforcement lowers the angle computation feeding Sin/Cos to bf16. Angles reach theta·π/2 (~1.6e4 rad), where bf16's 2⁻⁸ relative precision means ~60 rad quantization steps. sin/cos of such angles is noise, scrambling the high-frequency positional components.
Confirmed by node-precision bisect on LTX-Video: keeping only the rope table in f32 restores the f32 latent trajectory.
PyTorch computes these tables in f32 unconditionally; MarkFloatingPointRange doesn't help because it stops propagating at the first arithmetic op after Range.

This PR introduces:

• New MarkSinCosInputToKeepInMixedPrecision pass (common transformations): marks the floating-point producer cone of each Sin/Cos (up to Constants/Parameters/MatMul/Convolution) with disable_conversion(f16) via visit_path. The cone is cheap once-per-inference elementwise math; MatMuls are never crossed.

• Snippets tokenization propagates DisablePrecisionConversion from the original node to the Subgraph, since copy_runtime_info drops the non-copyable attribute and the marking was otherwise lost when the angle op got fused.

• ConvertToPowerStatic re-applies the non-copyable DisablePrecisionConversion attribute after copy_runtime_info, so the marking survives lowering to PowerStatic.

Partial fix for: #36715

AI Assistance:

• AI assistance used: no / yes yes
• If yes, summarize how AI was used and what human validation was performed (build/tests/manual checks).
  used at several points. Verified by running inference using LTX-Video using original reproduction script (output attached).

[Copilot]

Pull request overview

This PR addresses a CPU bf16/f16 inference accuracy regression for models that compute RoPE sin/cos tables at runtime (where RoPEFusion cannot apply) by keeping the Sin/Cos argument computation in f32 and preventing Snippets precision enforcement from collapsing the marked subgraph.

Changes:

• Added a new CPU transformation pass (MarkSinCosInputsPrecision) to mark Sin/Cos producer cones with disable_conversion(f16) and exclude them from Snippets tokenization.
• Registered the new pass in the CPU post-LPT transformation pipeline for bf16/f16 inference precision.
• Updated ConvertToPowerStatic to re-apply the (non-copyable) DisablePrecisionConversion marker after copy_runtime_info, plus added unit/functional regression tests.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/plugins/intel_cpu/tests/unit/transformations/mark_sin_cos_inputs_precision_test.cpp	Unit tests validating traversal/stop conditions for the new marking pass.
src/plugins/intel_cpu/tests/functional/custom/subgraph_tests/src/x64/rope_table_precision.cpp	Functional bf16 regression test covering runtime-computed RoPE angle stability on CPU x64.
src/plugins/intel_cpu/src/transformations/transformation_pipeline.cpp	Registers MarkSinCosInputsPrecision in the bf16/f16 post-LPT pipeline.
src/plugins/intel_cpu/src/transformations/cpu_opset/common/pass/mark_sin_cos_inputs_precision.hpp	Declares the new model pass and documents intended behavior.
src/plugins/intel_cpu/src/transformations/cpu_opset/common/pass/mark_sin_cos_inputs_precision.cpp	Implements the marking + Snippets-skipping traversal from Sin/Cos backward through FP producers.
src/plugins/intel_cpu/src/transformations/cpu_opset/common/pass/convert_to_power_static.cpp	Re-applies DisablePrecisionConversion after copy_runtime_info during PowerStatic replacement.

[goyaladitya05]

#36718 and #36719 combined are able to fix the issue. I validated by building OpenVINO from these pull requests and running the reproduction script, obtained output is attached. Quality improved a lot, inference time was increased by about ~1 sec, but the speedup against fp32 is still almost 3.8x.

ltx_t2v_output.5.mp4

[maxnick]

@v-Golubev , could you please review?

[goyaladitya05]

@v-Golubev all checks passed!

[v-Golubev]

@goyaladitya05 do I understand correctly that sin/cos layers should be kept in f32 precision not only for f32->bf16, but also for f32->f16 conversion? If yes, than this transformation and the corresponding test should be shared between CPU and GPU

[v-Golubev]

We shouldn't disable Snippets tokenization for such nodes. The root cause is that we don't copy runtime info to a new Subgraph node during Snippets tokenization. It looks like we need to fix the Subgraph node creation in the same way as it is done in ConvertToPowerStatic, but take into account that it is a composite node

Let's check all the nodes in a body in Subgraph's constructor and if one of them has DisablePrecisionConversion attribute. copy it to the Subgraph's rt_info:

openvino/src/common/snippets/src/op/subgraph.cpp

Lines 219 to 233 in b0194f0

	Subgraph::Subgraph(const OutputVector& args, const std::shared_ptr<ov::Model>& body)
	: SubGraphOp(args),
	m_generator(nullptr) {
	SubGraphOp::set_function(body);
	init_config();
	constructor_validate_and_infer_types();
	for (size_t i = 0; i < body->get_parameters().size(); ++i) {
	m_input_descriptions[0].push_back(std::make_shared<InvariantInputDescription>(i, i));
	}
	for (size_t i = 0; i < body->get_output_size(); ++i) {
	m_output_descriptions[0].push_back(std::make_shared<BodyOutputDescription>(i, i));
	}
	m_transformations_allowed = false;
	m_shape_infer = std::make_shared<OVShapeInfer>(body);
	}

[goyaladitya05]

afaik there's a constraint here: snippets is built before transformations, so the Subgraph constructor can't link DisablePrecisionConversion/is_conversion_disabled. Also, this is a pre-existing general gap (MarkRopeInputsToKeepInMixedPrecision has the same exposure, and tokenization_utils already copy_runtime_infos to the Subgraph but skips the non-copyable attribute).

I'd prefer to handle it as a separate snippets PR, if that works.

[goyaladitya05]

This turned out to be an issue, with static shapes the angle op gets tokenized and the mark was lost. Note the Subgraph constructor can't fix it: tokenization clones nodes into the body (clone_with_new_inputs), and since DisablePrecisionConversion is non-copyable the body op no longer carries the mark. The originals do, so I propagate it from the source node(s) to the Subgraph at the creation chokepoints, build_subgraph (covers single-node wrap_node_as_subgraph) and both tokenization sites. It's copied by rt_info key because snippets can't link the transformations lib; if you'd prefer the typed API we'd need to move DisablePrecisionConversion to core.

[CuriousPanCake]

You're mentioning bf16 in the description, but only disabling for f16, why?

[goyaladitya05]

disable_conversion(f16) protects both:

f16: ConvertPrecision(f32->f16) honors it (filter_precisions_for_node).
bf16 (CPU): node.cpp:190 -> if (is_conversion_disabled(op, element::f16)) keepOriginalPrecision = true, EnforceInferencePrecision skips it.
also, MarkRopeInputsToKeepInMixedPrecision uses the same disable_conversion(element::f16).

[CuriousPanCake]

@v-Golubev , should we make this pass a part of MarkSugraphsToKeepInMixedPrecision? What do you think?

[goyaladitya05]

@v-Golubev @CuriousPanCake MarkSugraphsToKeepInMixedPrecision only runs inside ConvertPrecision when inferencePrecision == f16. This pass needs PostLpt, gated on bf16 or fp16, and just sets DisablePrecisionConversion consumed at CPU node-construction for runtime execution, the IR stays fp32. Folding it in would mean it never fires for bf16, which is what this fix also targets.

[mryzhov]

yes, and don't we need to extend it instead and support the both inference precisions bf16 and fp16, @v-Golubev what do you think?

[CuriousPanCake]

If it's already marked, do we need to enter the callback and do the marking? Or is this check omitted to traverse and mark the nodes above?

[CuriousPanCake]

For example, in a similar PR: #36504 there's not marking of the nodes above? I'm wondering what's the proper approach. @v-Golubev

[goyaladitya05]

@CuriousPanCake, MarkRMS only marks the matched node itself, so filtering already-marked nodes out of the pattern match should be sufficient. This pass also has to mark the whole upstream angle computation, so visited is "have I already walked this subgraph," not just "is this node marked", it's the same guard MarkRopeInputsToKeepInMixedPrecision uses above. It's a perf/redundancy guard, correctness doesn't depend on it, since visit_path de-dupes internally and marking is idempotent.

[CuriousPanCake]

build_jenkins

[goyaladitya05]

@goyaladitya05 do I understand correctly that sin/cos layers should be kept in f32 precision not only for f32->bf16, but also for f32->f16 conversion? If yes, than this transformation and the corresponding test should be shared between CPU and GPU

Yes @v-Golubev, it's an fp16 concern too, and disable_conversion(f16), is honoured by both f16 ConvertPrecision and the CPU bf16 path. I've moved the pass and its test into common transformations. I've registered it only in the CPU pipeline for now, because GPU fp16 already works correctly on these models, and has its own passes. Also, MarkSubgraphsToKeepInMixedPrecision has no sin/cos rule, so a general pass could replace or consolidate the GPU ones, but I would prefer to do that in a follow up PR.

[goyaladitya05]

The CI failure is unreleted to my changes, could we please re-run the tests?

[v-Golubev]

@goyaladitya05 As I understand it, these sin/cos operations should remain in FP32 even for FP32-to-FP16 conversion. On the other hand, we do not currently observe any accuracy issues on the GPU.

You mentioned that the GPU has its own passes. Are these passes part of MarkSubgraphsToKeepInMixedPrecision, or are there any dedicated GPU-specific passes which prohibit FP32->FP16 conversion for such subgraphs?

[goyaladitya05]

@goyaladitya05 As I understand it, these sin/cos operations should remain in FP32 even for FP32-to-FP16 conversion. On the other hand, we do not currently observe any accuracy issues on the GPU.

You mentioned that the GPU has its own passes. Are these passes part of MarkSubgraphsToKeepInMixedPrecision, or are there any dedicated GPU-specific passes which prohibit FP32->FP16 conversion for such subgraphs?

They're dedicated GPU-specific passes, not part of MarkSubgraphsToKeepInMixedPrecision (which has no Sin/Cos handling). In the GPU pipeline, just before ConvertPrecision(f32->f16), there's a set of narrow pattern passes that disable_conversion(f16) on specific Sin/rope structures, DisableFP16ComForGPTOSSROPEPattern, DisableFP16ComSinGenPatternForHiFiGAN, DisableFP16CompCumSumSinGen, DisableFP16CompForGemma3RMSPattern. So GPU keeps these in FP32 for F32->F16 too, per-pattern rather than generally, which is probably why there's no observed GPU issue. This pass is the general form of the same idea. @v-Golubev

[goyaladitya05]

@v-Golubev @CuriousPanCake could you please retrigger CI? it should pass now

[CuriousPanCake]

Does this mean we should make this attribute copyable? (Not a request for you)

[goyaladitya05]

That would work, and it'd let us drop both workarounds here (the ConvertToPowerStatic re-apply and the snippets propagation), but it's global. every copy_runtime_info from a marked node would propagate "keep in fp32", including decomposition/fusion passes, so there's an over-marking risk (e.g. a fusion pulling the mark onto a MatMul).
If that's acceptable, I'll set flipping is_copyable() to true and removing the manual re-applies to see what happens, but preferably as a separate PR.

[v-Golubev]

@goyaladitya05 My concern is that a general pass may end up skipping more layers/subgraphs than actually necessary, which could potentially lead to performance degradation.

Since GPU already handles this correctly with a set of targeted per-pattern passes, my suggestion would be to reuse those existing GPU passes instead of introducing a new generic pass for CPU. We could move them from the GPU-specific code into src/common/transformations/src/transformations/fp16_compression and make them available for both plugins.

Then, in the CPU pipeline, these passes could be registered in 2 places:

1. For fp16 conversion, right before ConverPrecision is called (as it is done in GPU);
2. For bf16 conversion, in postLPT stage.

This approach would preserve the current fine-grained behavior already proven on GPU, while avoiding the risk of disabling FP16 conversion for a broader set of subgraphs than required.

[CuriousPanCake]

@goyaladitya05 are you planning on making the changes?

[goyaladitya05]

@CuriousPanCake @v-Golubev yes, I'm planning to do the changes, please give me a day or two so I can scope and validate them properly.