[go: up one dir, main page]

Skip to content

OmerRaviv/PdbBenchmarks

Repository files navigation

What is this?

This repo contains benchmarks for several popular managed PDB readers (for both Windows and Portable PDBs), with a specific interest in gauging which library is best for production diagnostic use cases, where the PDB reading will be performed in a production environment, and possibly from within the monitored application itself.

Why even bother doing a benchmark on PDB readers?

In most use cases, production diagnostics tools (such as debugger and profilers) only need to make a few (dozens or hundreds) reads from PDB at any given time. Even in the case of code coverage tools, which need to read sequence points for each and every executed method, the overall performance likely won't be hugely effected by PDB reading performance.

The motivation for creating these benchmarks is more around measuring the memory impact - as different libraries have wildly different characteristics both in terms of native and managed allocations. This is very significant, especially in memory constrained production environments (e.g. linux containers running in k8).

How do these benchmarks work?

This repo utilizes Benchmark.NET and xUnit side by side, so that you can easily benchmark or run/debug each variation. In the unit tests, we employ snapshot testing with Verify in order to ensure the code fragments being benchmarked for the different libraries all return the exact same result.

What libraries are included?

See PDB Library enum.

Why isn't library XYZ included in this benchmark?

  • PPDB is a very nice native implementation of a Portable PDB reader, that seems to be very much based on System.Reflection.Metadata's design. This library was not included because it seems it has not been thoroughly tested in production use-cases.
  • dbghelp is a Win32 API that can be easily used via PInvoke to read Windows PDBs with excellent performance. It was not included in this benchmark because, being a legacy API, it only has the notion of line numbers, and not column numbers, which makes it inadequate for handling modern C# code which may contain several lambda methods in the same line of code.

If you feel there is another library that is worth including in these benchmarks, please don't hesitate and create an issue/PR for it.

Results

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.22000
Intel Core i7-8750H CPU 2.20GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
  [Host]     : .NET Framework 4.8 (4.8.4470.0), X64 RyuJIT
  Job-OYCNNG : .NET Framework 4.8 (4.8.4470.0), X64 RyuJIT

IterationCount=100  LaunchCount=10  RunStrategy=Monitoring  
WarmupCount=0  
Method pdbType readerLibrary Mean Error StdDev Median Gen 0 Gen 1 Gen 2 Allocated Allocated native memory Native memory leak
GetLocalsAndSequencePoints_FromMethodToken WindowsPdb DiaNativeSymReader 2,155.9 μs 26.01 μs 249.22 μs 2,088.7 μs - - - 336.69 KB 1542.02 KB 2.33 KB
GetLocalsAndSequencePoints_FromMethodToken WindowsPdb MonoCecil 8,258.0 μs 52.65 μs 504.48 μs 8,119.9 μs - - - 4427.26 KB 2.52 KB 0.84 KB
GetLocalsAndSequencePoints_FromMethodToken WindowsPdb Dnlib_Managed 10,971.8 μs 113.83 μs 1,090.69 μs 10,713.0 μs 1000.0000 - - 7000.02 KB 1.94 KB 0.09 KB
GetLocalsAndSequencePoints_FromMethodToken WindowsPdb Dnlib_DiaSymReader 2,439.1 μs 40.23 μs 385.48 μs 2,324.0 μs - - - 352.69 KB 1574.61 KB 2.71 KB
GetLocalsAndSequencePoints_FromMethodToken PortablePdb DiaNativeSymReader 275.0 μs 6.68 μs 64.00 μs 282.2 μs - - - 136.05 KB 243.3 KB 0.13 KB
GetLocalsAndSequencePoints_FromMethodToken PortablePdb Dnlib_Managed 846.0 μs 16.94 μs 162.31 μs 801.9 μs - - - 969.44 KB 2.79 KB 0.94 KB
Method pdbType readerLibrary Mean Error StdDev Median Gen 0 Gen 1 Gen 2 Allocated Allocated native memory Native memory leak
GetILOffsetAndLocals_FromFileLine WindowsPdb DiaNativeSymReader 3.974 ms 0.0805 ms 0.7716 ms 3.688 ms - - - 251.66 KB 1575.74 KB 1.79 KB
GetILOffsetAndLocals_FromFileLine WindowsPdb Dnlib_Managed 12.050 ms 0.1385 ms 1.3274 ms 11.683 ms 1000.0000 - - 7257.67 KB 2.09 KB 0.25 KB
GetILOffsetAndLocals_FromFileLine WindowsPdb Dnlib_DiaSymReader 3.307 ms 0.0364 ms 0.3486 ms 3.212 ms - - - 259.66 KB 1607.28 KB 2.12 KB
GetILOffsetAndLocals_FromFileLine PortablePdb DiaNativeSymReader 2.327 ms 0.0329 ms 0.3151 ms 2.233 ms - - - 1264.05 KB 243.62 KB 0.44 KB
GetILOffsetAndLocals_FromFileLine PortablePdb Dnlib_Managed 1.076 ms 0.0206 ms 0.1970 ms 1.002 ms - - - 1089.65 KB 2.13 KB 0.28 KB

About

Benchmark for the most popular managed PDB readers.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages