Work On CPU

[ZepinLi]

Hi Sequoia team,

Can this code framework fit in cpu devices? If so, how can we do it? Any insights?

Regards

[dreaming-panda]

I do not think this can be used on CPU. CPU does not have such high FLOPS for speculative decoding, so even if the code can run on CPU, no speed up will be found, unless you talk about some very advanced CPUs (I have no ideas about them).