Addressing Meltdown and Spectre in the kernel

Speaking as the mostly clueless, would there be any sense in halting speculation if it leads to a load that was not present in L1? In that case, at least latency of L2/L3 will always be involved, if not a bus transaction. This would seem to be efficient where it matters, e.g. tight loops where the instruction stream is already cached, and for other cases PREFETCHxx could be used to explicitly request population where it was known it could not create problems