ncnn is a high-performance neural network inference framework.
See https://github.com/Tencent/ncnn for more info about ncnn.
- The benchmark score is the inference time, less is better.
- Runs on all cpu big cores or vulkan, and fp16 arithmetic is enabled if supported.
- Table contents are ordered by mobilenet.
- rpi5b data from Tencent/ncnn#5058
| small-board | image | spec | squeezenet | mobilenet | shufflenet |
|---|---|---|---|---|---|
| NVIDIA Jetson AGX Orin | Ampere 1.3GHz x 2048 Tensor Core x 64 |
2.13 | 2.03 | 2.57 | |
| NVIDIA Jetson AGX Orin | A78AE 2.2GHz x 12 | 3.50 | 3.49 | 5.08 | |
| Radxa Rock5B | RK3588 A76 2.4GHz x 4 A55 1.8GHz x 4 |
3.65 | 5.41 | 3.37 | |
| Orange Pi 5 | RK3588S A76 2.4GHz x 4 A55 1.8GHz x 4 |
3.83 | 5.75 | 3.57 | |
| ZYSJ RK3588 | ARM Mali-G610 | 7.09 | 9.16 | 5.88 | |
| ZYSJ RK3588 | RK3588 A76 2.4GHz x 4 A55 1.8GHz x 4 |
7.57 | 11.01 | 7.95 | |
| Raspberry Pi 5B | BCM2712 A76 2.4GHz x 4 |
8.56 | 11.32 | 4.56 | |
| M4N DOCK | AX650 A55 1.7Ghz x 8 |
10.61 | 11.86 | 10.64 | |
| NVIDIA Jetson Nano | Maxwell 921MHz x 128 | 9.03 | 15.39 | 10.61 | |
| Khadas VIM4 | ARM Mali-G52 800MHz x MP8(8EE) |
12.77 | 20.17 | 12.81 | |
| Unisoc Tiger T710 | Unisoc Tiger T710 A75 1.82GHz x 4 A55 1.82GHz x 4 |
16.25 | 20.31 | 10.59 | |
| Radxa Rock3A | RK3568 A55 2.0GHz x 4 |
23.20 | 30.78 | 19.49 | |
| Khadas VIM3 | ARM Mali-G52 800MHz x MP4(6EE) |
19.54 | 31.62 | 17.91 | |
| Panther X2 | RK3566 A55 1.8GHz x 4 |
25.91 | 33.17 | 21.81 | |
| NanoPi M4 | ARM Mali-T864 | 24.57 | 35.86 | 33.90 | |
| Radxa RockPi X | Intel HD Graphics (Cherry Trail) 500 MHz | 29.91 | 36.41 | 24.54 | |
| Khadas VIM4 | A311D2 A73 2.2GHz x 4 A53 2.0GHz x 2 |
24.27 | 37.65 | 19.79 | |
| NVIDIA Jetson Nano | A57 1.43GHz x 4 | 28.55 | 40.25 | 19.44 | |
| Khadas VIM3 | A311D A73 2.2GHz x 4 A53 1.8GHz x 2 |
30.98 | 42.57 | 21.62 | |
| Lichee Pi 4A | T-Head TH1520 RV64GCV 2.0GHz x 4 |
42.29 | 47.73 | 120.13 | |
| NanoPi Fire3 | S5P6818 A53 1.4GHz x 8 |
41.81 | 52.33 | 33.54 | |
| Debix Model A | i.MX8M Plus A53 1.6GHz x 4 |
40.15 | 54.01 | 29.33 | |
| Odroid XU4 | Exynos 5422 A15 2GHz x 4 A7 1.3GHz x 4 |
35.73 | 54.17 | 25.75 | |
| Tinker Board S R2.0 | ARM Mali-T764 | 41.48 | 60.28 | 55.87 | |
| Radxa Zero | S905Y2 A53 1.8GHz x 4 |
46.82 | 60.49 | 35.18 | |
| Raspberry Pi 4B | BCM2711B0 A72 1.5GHz x 4 |
46.28 | 60.74 | 32.91 | |
| NanoPi M4 | RK3399 A72 1.8GHz x 2 A53 1.5GHz x 4 |
43.73 | 64.28 | 27.43 | |
| BeagleBone AI 64 | TDA4VM A72 2.0GHz x 2 |
42.23 | 65.78 | 29.97 | |
| NanoPi K2 | S905 A53 1.5GHz x 4 |
56.96 | 75.25 | 38.70 | |
| Radxa RockPi X | Atom x5-Z8350 x86-64 1.92 GHz x 4 |
50.22 | 80.12 | 37.96 | |
| Tinker Board S R2.0 | RK3288CGW A17 1.8GHz x 4 |
52.86 | 82.19 | 38.17 | |
| NanoPi R2S | RK3328 A53 1.3GHz x 4 |
62.20 | 82.88 | 52.34 | |
| EAIDK 310 | RK3228H A53 1.3GHz x 4 |
61.87 | 83.88 | 45.83 | |
| Radxa Zero | ARM Mali-G31 850 MHz x 2 | 57.95 | 98.62 | 51.71 | |
| Raspberry Pi 3B+ | BCM2837B0 A53 1.4GHz x 4 |
84.74 | 107.84 | 58.38 | |
| Banana Pi M3 | AllWinner A83T A7 1.8GHz x 8 |
88.35 | 109.72 | 80.91 | |
| MangoPI MQ-Quad | AllWinner H616 A53 1.5GHz x 4 |
99.60 | 129.70 | 66.90 | |
| Raspberry Pi Zero 2W | BCM2710A1 A53 1.0GHz x 4 |
119.52 | 162.60 | 72.72 | |
| EASY EAI Nano | RV1126 A7 1.5GHz x 4 |
105.16 | 163.82 | 68.79 | |
| Radxa RockPi S | RK3308 A35 1.3GHz x 4 |
142.02 | 214.56 | 102.44 | |
| StarFive VisionFive V2 | SiFive’s U74 RV64GFC 1.5GHz x 4 |
150.07 | 256.21 | 91.14 | |
| RVB-ICE | T-Head C910 RV64GC 1.2GHz x 2 |
161.10 | 274.39 | 118.61 | |
| Loongson Pie Lite | Loongson 2K1000 GS264 1.0GHz x 2 |
185.28 | 276.03 | 124.39 | |
| Ingenic X2000 | Ingenic X2000 XBurst2 1.2GHz x 2 |
175.99 | 282.25 | 118.4 | |
| Raspberry Pi 2B | BCM2836 A7 900MHz x 4 |
179.50 | 296.97 | 132.49 | |
| Banana Pi M2 Zero 2 | AllWinner H2+ A7 1.2GHz x 4 |
218.73 | 303.92 | 138.43 | |
| Raspberry Pi 4B | Broadcom VideoCore VI | 286.71 | 365.04 | 162.14 | |
| PICO-PI-IMX7 | i.MX7D A7 1.0GHz x 2 |
220.10 | 366.92 | 137.02 | |
| Microphase Z7-Lite | XC7Z020CLG400 A9 766MHz x 2 |
389.18 | 623.71 | 212.20 | |
| MangoPI MQ-Pro | D1 C906 RV64GCV 1GHz x 1 |
385.21 | 691.47 | 487.67 | |
| NeZha D1 | D1 C906 RV64GCV 1GHz x 1 |
396.18 | 727.97 | 495.87 | |
| StarFive VisionFive V1 | SiFive’s U74 RV64GFC 1.5GHz x 2 |
609.41 | 993.87 | 358.98 | |
| Raspberry Pi B+ | ARM1176JZF-S 700MHz x 1 |
4285.24 | 7684.05 | 2102.22 |