8000 perf: Cortex-A76/A55 counters and ARM PMU fixes by JosVoigt · Pull Request #62 · miniosv/miniosv · GitHub
[go: up one dir, main page]

Skip to content

perf: Cortex-A76/A55 counters and ARM PMU fixes - #62

Closed
JosVoigt wants to merge 11 commits into
masterfrom
cortex-counters-v2
Closed

JosVoigt wants to merge 11 commits into
masterfrom
cortex-counters-v2

Conversation

@JosVoigt
@JosVoigt JosVoigt commented Oct 9, 2026
Copy link
Copy Markdown
Collaborator

Fixed in the Cortex commit (problems on master):

  • pmc_stop() wrote the counter number into PMCNTENCLR_EL0, a bit mask, and stopped no counter or the wrong one.
  • Events 0x4000-0x403F were checked against the PMCEID bits of events 0x00-0x3F.
  • An unsupported event was not programmed but still reported 0 as a valid count.
  • On 32-bit PMUs every counter wrap lost 2^32 counts.
  • std::hex stayed set on std::cout / std::cerr after a warning.

Fixed in follow-up commits (problems on master):

  • The STALL* events had empty names: no report header, not addressable via getCounter().
  • The Neoverse V1 + KVM workaround was skipped when counters had also been trimmed.
  • pmc_read() returned an uninitialised value for counters >= 6, and pmc_start_with_conf() enabled them without writing an event type.
  • The cycle counter filter (PMCCFILTR_EL0) was never written; its reset value is UNKNOWN, so the cycle counter may not count at EL1.
  • Detaching a sampler disabled the overflow interrupt of every counter.
  • enable_pmu() wrote PMCR_EL0.LP, which is RES0 before FEAT_PMUv3p5.
  • is_kvm_guest() issued HVC unconditionally (UNDEFINED without EL2), compared 64-bit instead of 32-bit results and did not declare the registers SMCCC v1.0 may corrupt.

Fixed in follow-up commits (problems introduced with the Cortex commit):

  • Shifting by 64 for 64-bit counters in Event (undefined behaviour).
  • The counter width probe overwrote event counter 0 and cached the result of the first cpu that ran it.
  • One counter table for the whole machine: concurrent PerfEvents on different cpus took each other's counters, and the wrap counting interrupt was only enabled on the first cpu that counted.

🤖 Generated with Claude Code

Meandres and others added 11 commits October 8, 2026 17:16
Also adds some event definitions that were missing in aarch64.
Also introduces a wrap-around feature since some ARM cores only count
with 32bits registers (overflows if you count cycles over more than a
few seconds).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The four STALL* events had empty names: their report columns had no header
and getCounter() could not address them. The comments of LL_CACHE,
LL_CACHE_MISS, L1D_CACHE_MISS and L2D_CACHE_LMISS_RD now say that they count
reads / long-latency reads only (Linux include/linux/perf/arm_pmuv3.h:
LL_CACHE_RD, LL_CACHE_MISS_RD, L1D_CACHE_LMISS_RD, L2D_CACHE_LMISS_RD).
Report names are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Event::report() computed wraps << width and Event::stop() 1ull << width also
for width == 64 (the cycle counter with PMCR_EL0.LC, every counter with
FEAT_PMUv3p5). Shifting a 64-bit value by 64 is undefined behaviour
([expr.shift]). A 64-bit counter does not wrap, so nothing is added.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…immed

PMCSelectCore dropped counter 0 on a Neoverse V1 under KVM only when no
counters had been trimmed (else if). Both conditions are independent: on a
VM with sliced counters (the AWS case named in the comment) running on a V1,
counter 0 stayed in the pool.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pmc_read() only handles event counters 0..5 and the cycle counter; for any
other id it returned an uninitialised variable (undefined behaviour). It
reads 0 now.

pmc_start_with_conf() had no case for those ids either: it wrote no event
type, still enabled the counter i
8000
n PMCNTENSET_EL0 and returned true, so the
counter counted whatever stale event type it held and the event was taken
as valid. It now returns false for them, and Event marks the event invalid.

PMUv3 has up to 31 event counters (PMCR_EL0.N); using counters 6..30 needs
the PMSELR_EL0 + PMXEVTYPER_EL0 / PMXEVCNTR_EL0 indirection (Arm ARM DDI
0487), left for a later change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Starting the cycle counter only set PMCNTENSET_EL0 bit 31 and left
PMCCFILTR_EL0 as it was. Its filter bits reset to an UNKNOWN value (Arm ARM
DDI 0487, PMCCFILTR_EL0); with P set the counter does not count at EL1, where
MiniOSv runs, and cpu-cycles / IPC are wrong. The filter bits of the event
bitmap are written now (none for PERF_COUNT_HW::CPU_CYCLES: count at EL0 and
EL1), like the event counters' PMEVTYPER<n>_EL0. Linux does the same
(drivers/perf/arm_pmuv3.c, write_pmccfiltr()).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pmc_detach_overflow_handler() on aarch64 wrote ~0 to PMINTENCLR_EL1 and so
disabled the overflow interrupt of every counter: other samplers and the
wrap counting of running PerfEvents went silent (and the RES0 upper half was
written with ones). The sampler now disables the interrupt of its own
counter with the new pmc_overflow_disable() (PMINTENCLR_EL1 / PMOVSCLR_EL0,
one bit per counter, Arm ARM DDI 0487), also when start() fails. On x64
pmc_stop() already clears the interrupt enable in the event select.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
enable_pmu() set PMCR_EL0.LP unconditionally, but LP is RES0 before
FEAT_PMUv3p5 (software must write 0), and decided on 64-bit counters by
reading it back. pmu_probe_event_counter_width() wrote 2^32 into event
counter 0 to find out, which overwrote a counter that may be in use (and
counter 0 is excluded on Neoverse V1 under KVM), and cached the result of
whichever processor ran it first.

PMUVer >= 6 (FEAT_PMUv3p5, Arm ARM DDI 0487, ID_AA64DFR0_EL1) is the
architected check, as in Linux (drivers/perf/arm_pmuv3.c,
armv8pmu_has_long_event()). LP is only written there;
pmu_event_counter_width() replaces the probe; pmu_dump_state() also prints
PMUVer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
is_kvm_guest() executed HVC unconditionally. HVC is UNDEFINED at EL1 without
EL2 or with SCR_EL3.HCE == 0 (Arm ARM DDI 0487, HVC), and where EL2 exists
without a hypervisor it lands in whatever EL2 code firmware left. It now
takes the conduit from the ACPI FADT ARM boot flags, like
arch/aarch64/psci.cc, and only calls over HVC (KVM always uses HVC; Linux
likewise only uses the firmware-described conduit, drivers/firmware/smccc).

Also, per the SMC Calling Convention (Arm DEN 0028): 0x8600FF01 is an
SMC32/HVC32 call, so only W0..W3 are compared, and X4..X17 are declared
clobbered (SMCCC v1.0 lets the callee corrupt them). The result is computed
once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
default_core_pmcs() was a single PMCSelectCore for the whole machine,
although every cpu has its own PMU. Concurrent PerfEvents in threads on
different cpus took counters from one table: on x64 under KVM with 4 pinned
threads and the default PerfEvent (4 events), 10 of the 16 events were
reported as occupied and measured 0; only the first thread got all of them.
The wrap counting handler was attached once, from the first cpu that
counted, and the PMU interrupt (a per-cpu PPI) was only enabled there.

default_core_pmcs() now returns a PerCpuPMCSelect: acquire() reserves from
the calling cpu's own table, created on that cpu on first use (it reads that
cpu's PMU, core types can differ), release() returns a counter to the table
it came from. Every table attaches its own wrap counting handler from its
cpu and ignores the other cpus' interrupts. PerfEvent, PerfEventBlock and
the sampler keep their interface; PMCSelect's reservation functions are
virtual now.


8000
Tested with 4 threads pinned to 4 vcpus (x64, KVM, AMD host): all 16 events
measured. aarch64 compile-checked only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@JosVoigt
JosVoigt commented Oct 9, 2026
Copy link
Copy Markdown
Collaborator Author

This also contains (and hopefully improves) commits by @Meandres from PR #48 . Those are therefore closely linked. Prob should approve #48 before this one.

@JosVoigt
Copy link
Copy Markdown
Collaborator Author

Superseded by Martin-Lndbl/miniosv#8 (merged with #52).

@JosVoigt JosVoigt closed this Oct 12, 2026
@Meandres
Meandres deleted the cortex-counters-v2 branch October 28, 2026 07:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

0