为什么seq_cls任务中，在score之后再pool，这样不是浪费推理资源吗

[ckx506772099]

Checklist / 检查清单

• I have searched existing issues, and this is a new question or discussion topic. / 我已经搜索过现有的 issues，确认这是一个新的问题与讨论。

Question Description / 问题描述

为什么seq_cls任务中，在score之后再pool，这样不是浪费推理资源吗

[hiSandog]

我也觉得这里值得说明一下设计取舍。如果当前路径是 hidden_states -> score -> pool，那等于先对所有 token 做一次 num_labels 投影，再聚合；对常见 seq_cls 只取最后有效 token 或 mean pool 的场景，理论上先 pool 再 score 会更省。

可能现在的顺序是为了兼容 packing、token-level 输出或某些特殊 pooling 策略。建议维护者在文档里说明原因，或者给普通 seq_cls 增加一个先 pool 后 score 的 fast path，并用同一 batch size 对比一下显存和 latency。

[Jintao-Huang]

说的是 megatron 那里嘛

[ckx506772099]

是swift/model/patcher.py中的这里

`
def transformers_seq_cls_forward(self, *args, origin_forward, padding_side=None, **kwargs):
labels = kwargs.pop('labels', None)
return_dict = kwargs.pop('return_dict', None)
return_dict = return_dict if return_dict is not None else self.config.use_return_dict
input_ids = kwargs.get('input_ids')
inputs_embeds = kwargs.get('inputs_embeds')

output = origin_forward(*args, **kwargs)
if hasattr(output, 'logits'):
    output.logits = output.logits.to(self.score.weight.dtype)
elif 'last_hidden_state' in output:
    output.logits = output['last_hidden_state'].to(self.score.weight.dtype)
logits = self.score(output.logits)
if input_ids is not None:
    batch_size = input_ids.shape[0]
else:
    batch_size = inputs_embeds.shape[0]

if padding_side == 'left':
    pooled_logits = logits[:, -1]
else:
    pad_token_id = HfConfigFactory.get_config_attr(self.config, 'pad_token_id')
    if pad_token_id is None and batch_size != 1:
        raise ValueError('Cannot handle batch sizes > 1 if no padding token is defined.')
    if pad_token_id is None:
        sequence_lengths = -1
    else:
        if input_ids is not None:
            # if no pad token found, use modulo instead of reverse indexing for ONNX compatibility
            sequence_lengths = torch.eq(input_ids, pad_token_id).int().argmax(-1) - 1
            sequence_lengths = sequence_lengths % input_ids.shape[-1]
        elif kwargs.get('attention_mask') is not None:
            sequence_lengths = get_last_valid_indices(kwargs['attention_mask'])
        else:
            sequence_lengths = -1
    if isinstance(sequence_lengths, torch.Tensor):
        sequence_lengths = sequence_lengths.to(logits.device)

    pooled_logits = logits[torch.arange(batch_size, device=logits.device), sequence_lengths]

loss = None
if labels is not None:
    labels = labels.to(logits.device)
    if self.config.problem_type is None:
        if self.num_labels == 1:
            self.config.problem_type = 'regression'
        elif self.num_labels > 1 and (labels.dtype == torch.long or labels.dtype == torch.int):
            self.config.problem_type = 'single_label_classification'
        else:
            self.config.problem_type = 'multi_label_classification'

    if self.config.problem_type == 'regression':
        loss_fct = MSELoss()
        if self.num_labels == 1:
            loss = loss_fct(pooled_logits.squeeze(), labels.squeeze())
        else:
            loss = loss_fct(pooled_logits, labels)
    elif self.config.problem_type == 'single_label_classification':
        loss_fct = CrossEntropyLoss()
        loss = loss_fct(pooled_logits.view(-1, self.num_labels), labels.view(-1))
    elif self.config.problem_type == 'multi_label_classification':
        loss_fct = BCEWithLogitsLoss()
        loss = loss_fct(pooled_logits, labels)
if not return_dict:
    output = (pooled_logits, ) + output[1:]
    return ((loss, ) + output) if loss is not None else output

return SequenceClassifierOutputWithPast(
    loss=loss,
    logits=pooled_logits,
    past_key_values=output.past_key_values,
    hidden_states=output.hidden_states,
    attentions=output.attentions,
)

`