CN116069453A - a simulation system - Google Patents

Abstract

The application discloses a simulation system relates to the simulation field, and the simulation system is used for NVME equipment simulation, includes: a virtual host; the controller simulation model comprises an executable unit and a plurality of functional units, wherein the executable unit is used for running firmware to be simulated and comprises an interrupt management module, a software engine module, a bus simulation module, a cache simulation module and a memory simulation module, and the bus simulation module is used for realizing interconnection between the executable unit and each functional unit; and the message forwarding module is used for connecting the virtual host and the controller simulation model. The controller simulation model is further used for processing the information sent by the virtual host based on the firmware to be simulated so as to simulate the firmware to be simulated. Aiming at the problem that simulation debugging and verification of controller firmware cannot be achieved in the prior art, the method and the device provide an executable unit capable of supporting operation of the firmware to be simulated, so that the firmware in the controller can be independently designed and developed.

Description

a simulation system

technical field

The present application relates to the field of simulation, and in particular to a simulation system.

Background technique

As shown in Figure 1, when performing NVMe device simulation, the existing solutions usually use KVM to simulate the CPU unit and build an executable environment (the executable environment generally includes CPU, memory, cache, and bus devices that can execute instructions). Qemu performs peripheral emulation, creates a virtualized environment on the basis of the Host OS, and runs the Guest OS. It needs to use the user program and NVMe driver of the simulated NVMe device to run in the Guest OS, and the NVMe controller emulator runs in Qemu of the Host OS. in progress.

But for the NVMe driver of the Guest OS, the NVMe controller emulator is a simulated device. The real NVMe device includes a PCIE-EP interface, an NVMe controller with a processor, memory, Cache, bus, storage media, and memory. The controller is used to provide an executable environment for executing the internal firmware program of the NVMe controller. . For the firmware program inside the NVMe controller chip, when performing simulation tests, since the NVMe controller simulator does not provide an independent software executable environment that can support the operation of the NVMe device controller firmware, it is impossible to simulate the controller firmware. Debug verification.

Contents of the invention

In view of this, in order to overcome at least one aspect of the above problems, an embodiment of the present application proposes a simulation system, which is used for NVME device simulation, including:

virtual host;

The controller simulation model includes an executable unit and several functional units for running the firmware to be simulated, and the executable unit includes an interrupt management module, a software engine module, a bus simulation module, a cache simulation module and a memory simulation module, wherein , the bus emulation module is used to realize the interconnection between the executable unit and each of the functional units;

A message forwarding module, configured to connect the virtual host and the controller simulation model.

Wherein, the controller simulation model is further used to process a message sent by the virtual host based on the firmware to be simulated to simulate the firmware to be simulated.

In some embodiments, the message forwarding module includes a proxy module and an access module;

The agent module is used for encapsulating the message sent by the virtual host and sending it to the access module, and receiving and parsing the message sent by the access module and sending it to the virtual host;

The access module is used for receiving and analyzing the message sent by the agent module and sending it to the controller simulation model, and encapsulating the message sent by the controller simulation model and sending it to the agent module.

In some embodiments, the proxy module and the virtual host run in a first process.

In some embodiments, the access module and the controller simulation model run in a second process.

In some embodiments, the simulation system also includes:

Storage medium simulation model, including attribute acquisition operation interface, setting operation interface, reading operation interface, and writing operation interface.

In some embodiments, the attribute acquisition operation interface and the setting operation interface are used by the firmware to be simulated, and the read operation interface and write operation interface are used by the controller simulation model for data reading and writing. .

In some embodiments, the functional unit includes a PCIE interface unit, an NVMe core unit, a storage medium unit, and a commissioning and tracking unit, the PCIE interface unit, the NVMe core unit, the storage medium unit, the commissioning interconnection between the tracking unit and the executable unit;

Wherein, the PCIe interface unit is connected to the message forwarding module for simulating the PCIe bus, simulating reading and writing of the PCIe configuration space, transmitting bus data, and processing message signal interruption or extended message signal interruption;

The NVMe core unit is used to process the reading and writing of configuration registers, forward management commands to the executable unit, process IO commands, process doorbells, and move DMA data;

The storage medium unit is also connected to the storage medium simulation model for providing a control management interface and an IO data interface to the core unit to access the storage medium simulation model;

The commissioning and tracking unit provides print and log control management and output functions for the PCIE interface unit, the NVMe core unit, the storage medium unit and the executable unit.

In some embodiments, the interrupt management module is used to provide an interrupt function;

The software engine module provides the operating environment and executes the firmware to be simulated;

The bus emulation module is used to realize the interconnection between the PCIE interface unit, the NVMe core unit, the storage medium unit, the commissioning and tracking unit and the executable unit;

The cache emulation module is used to provide a cache function to the software engine module;

The memory emulation module is used to provide memory functions to the software engine module.

In some embodiments, the software engine module is configured to receive the management command, save the management command to a cache location configured based on the firmware to be simulated, and trigger an interrupt to notify the firmware to be simulated to process.

In some embodiments, after the firmware to be emulated receives the interrupt, it reads and processes the management command in the cache location, wherein the management command includes creating or deleting a submission queue for data input and output commands, commands to create or delete completion queues for data input and output, commands to get namespace information, commands to get properties, and commands to set properties.

In some embodiments, the PCIE interface unit, the NVMe core unit, the storage medium unit, and the commissioning and tracking unit register the socket of the initiator interface and the socket of the target interface with the bus emulation module respectively. Sockets to realize the interconnection access between each unit.

In some embodiments, the PCIE interface unit further includes a PCIe interface module and a PCIe register module.

In some embodiments, the PCIe interface module is used to simulate the simulation of the PCIe bus and transmit bus data, so as to pass the message sent by the virtual host to the NVMe core unit and the message sent by the NVMe core unit to the virtual host.

In some embodiments, the PCIe register module is used for simulating reading and writing of the PCIe configuration space, and saving the configuration information of the virtual host for the message signal interrupt or the extended message signal interrupt.

In some embodiments, the NVMe core unit includes an NVMe register module, a submission queue module for management, a completion queue module for management, a submission queue module for data input and output, and a completion queue for data input and output module, storage module, doorbell trigger module, interrupt trigger module, DMA data transmission module.

In some embodiments, the NVMe register module is used to handle reading and writing of NVMe configuration registers.

In some embodiments, the doorbell trigger module is used to receive the doorbell operation from the host, and notify the delivery queue module for management or the delivery queue module for data input and output according to the queue corresponding to the doorbell operation to process.

In some embodiments, the management delivery queue module is configured to package the management command corresponding to the entry of the management delivery queue into a bus-level transaction request and send it to the executable unit for processing;

The completion queue module for management is used to put the execution unit's processing result of the management command corresponding to the entry of the delivery queue for management into the entry of the completion queue for management, and call the interrupt trigger module notifies the virtual host.

In some embodiments, the DMA data transmission module is configured to perform data transfer according to the doorbell trigger module or transfer the processing result of the entry of the completed queue for management to the virtual host.

In some embodiments, the submission queue module for data input and output is used to process the IO command corresponding to the entry of the submission queue for data input and output;

The completion queue module for data input and output is used to process the processing result of the IO command corresponding to the entry of the submission queue for data input and output, and put it into the entry of the completion queue for data input and output, and call the The interrupt trigger module notifies the virtual host;

The storage module is used for creating a storage space, and writing data into and/or reading data from the storage space based on the IO command.

In some embodiments, the storage medium unit includes a storage medium model framework module, a storage medium control module and a storage medium I/O module.

In some embodiments, the storage medium model framework module provides a control management interface and an IO data interface to access the storage medium simulation model;

The storage medium control module is used to provide storage medium control information;

The storage medium I/O module is used for providing storage medium data IO operations.

In some embodiments, the commissioning and tracking unit includes a register operation module, a printing and log control module, and a unit test management module.

In some embodiments, the register operation module initiates read and write access to registers in the executable unit, the PCIE interface unit, the NVMe core unit, the storage medium unit, and the commissioning and tracking unit;

The print and log control module is used to provide print control and log control;

The unit test module is used to provide test entries of the executable unit, the PCIE interface unit, the NVMe core unit, the storage medium unit and the commissioning and tracking unit, and is used to process unit test commands.

The present application has one of the following beneficial technical effects: the scheme proposed by the present application sets an executable unit for running the firmware to be simulated in the controller simulation model, and the executable unit includes an interrupt management module, a software engine module, a bus simulation module, A cache emulation module and a memory emulation module, wherein the bus emulation module is used to realize the interconnection between the executable unit and each functional unit. In this way, the executable unit provides an independent software executable environment that can support the running needs of the NVMe device controller firmware, so that the internal firmware of the commissioning controller can be independently designed and developed.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present application, and those skilled in the art can obtain other embodiments according to these drawings without creative efforts.

Fig. 1 is the block diagram of the simulation system in the prior art;

FIG. 2 is a block diagram of a simulation system provided by some embodiments of the present application;

Fig. 3 is a schematic diagram of the simulation system provided by some embodiments of the present application in the operating system;

4 is a schematic diagram of a controller simulation model provided by some embodiments of the present application;

Fig. 5 is a schematic diagram of executable units provided by some embodiments of the present application;

Fig. 6 is the schematic diagram of the PCIE interface unit that some embodiments of the present application provide;

Fig. 7 is a schematic diagram of the NVME core unit provided by some embodiments of the present application;

FIG. 8 is a schematic diagram of a storage medium unit provided by some embodiments of the present application;

FIG. 9 is a schematic diagram of a commissioning and tracking unit provided by some embodiments of the present application;

Fig. 10 is a schematic diagram of the operation of the simulation system provided by some embodiments of the present application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application clearer, some embodiments of the present application will be further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

It should be noted that in some embodiments of the present application, all the expressions of "first" and "second" are used to distinguish two entities with the same name but different parameters or different parameters. It can be seen that "first" and "second" "two" is only for the convenience of expression, and should not be understood as a limitation on the embodiment of the present application, and the subsequent embodiments will not explain it one by one.

English abbreviations appearing in some embodiments of the present application have the following definitions:

Admin CQ: The completion queue used for management, commonly expressed as ACQ;

Admin SQ: Submission queue for management, often expressed as ASQ;

BAR: Base Address Register base address register, the register that saves the window base address in the PCIe protocol;

Cache: cache;

CPU: Central processing unit, as the computing and control core of the computer system, is the final execution unit of information processing and program operation;

CQ: Completion Queue completion queue;

CQE: Completion Queue Entry completes the queue entry;

DB: Doorbell, doorbell;

DMA: Direct Memory Access direct memory access;

FIFO: First In/First Out queue;

FW: Firmware firmware, a program that works inside a chip, because it is solidified inside the chip, it is usually called firmware;

Get Feature command: A command defined by the NVMe protocol to obtain specific attribute information of the NVMe controller;

Guest OS: usually refers to the operating system of the virtual machine based on the host operating system;

Host: generally refers to the host or host;

Host OS: host operating system;

Identify command: A command defined by the NVMe protocol to obtain some information about the NVMe controller;

IO: Input /Output, usually refers to data input and output;

IO SQ: Submission queue for data input and output;

IO CQ: Completion queue for data input and output;

KVM: a virtualization CPU technology;

LBA: Logical Block Address Logical block address;

Memory: memory;

MSI: Message Signaled Interrupt message signal interrupt;

MSI-X: extended message signal interrupt;

NVMe: Non Volatile Memory Express, non-volatile memory interface specification, is a logical device interface specification, a bus transmission protocol specification based on the device logic interface (equivalent to the application layer in the communication protocol);

PCIe: Peripheral Component Interconnect Express is a high-speed serial computer expansion bus standard;

PCIe-EP: PCIe EndPoint, PCIe end node;

PCIe-RC: PCIe RootComplex, PCIe root node;

PRP: Physical Region Page, the memory page used to save data in the NVMe protocol;

Qemu: QuickEmulator is a pure software virtualization simulator that can simulate hardware devices sc_event: an event model defined in the SystemC library;

sc_fifo: A FIFO model defined in the SystemC library;

sc_thread: A threading model defined in the SystemC library;

Set Feature command: A command defined by the NVMe protocol, used to set specific attribute information of the NVMe controller;

SGL: Scatter/Gather List Scatter/gather list, in this article refers to the organizational format of memory pages used to save data in the NVMe protocol;

SQ: Submission Queue submission queue;

SQE: Submission Queue Entry Submission queue entry;

SQID: Submission Queue ID number submission queue identifier;

SystemC: A library written in C++ language, often used for simulation modeling;

TLM: Transaction-Level Modeling transaction-level modeling.

According to one aspect of the present application, an embodiment of the present application proposes a simulation system, the simulation system is used for NVME device simulation, as shown in Figure 2, the simulation system includes:

virtual host;

The controller simulation model includes an executable unit and several functional units for running the firmware to be simulated, and the executable unit includes an interrupt management module, a software engine module, a bus simulation module, a cache simulation module and a memory simulation module, wherein , the bus emulation module is used to realize the interconnection between the executable unit and each of the functional units;

A message forwarding module, configured to connect the virtual host and the controller simulation model.

Wherein, the controller simulation model is further used to process a message sent by the virtual host based on the firmware to be simulated to simulate the firmware to be simulated.

The scheme proposed in this application sets an executable unit for running the firmware to be simulated in the controller simulation model, and provides an independent software executable environment that can support the operation needs of the NVMe device controller firmware through the executable unit, so that it can be independently designed and developed Debug the internal firmware of the controller.

In some embodiments, the message forwarding module includes a proxy module and an access module;

The agent module is used for encapsulating the message sent by the virtual host and sending it to the access module, and receiving and parsing the message sent by the access module and sending it to the virtual host;

The access module is used for receiving and analyzing the message sent by the agent module and sending it to the controller simulation model, and encapsulating the message sent by the controller simulation model and sending it to the agent module.

In some embodiments, the proxy module and the virtual host run in a first process.

In some embodiments, the access module and the controller simulation model run in a second process.

Specifically, as shown in Figure 3, the virtual host runs on the Qemu process (the first process), and the operating system (Guest OS) of the virtual host has an NVME device interface and a KVM virtualized executable environment (including virtualized CPU, memory , cache, bus), and run user applications, communicate with the agent module through the Qemu virtualization interface, and the agent module is interconnected with the agent module running in the NVME controller emulator process (second process) through inter-process communication.

The proxy module is responsible for encapsulating the Guest OS's request for the controller simulation model module into a host OS inter-process communication message, and passing it to the NVMe controller simulator access module of the NVMe controller simulator process. In the other direction, it is responsible for receiving the message sent by the NVMe controller emulator access module of the NVMe controller emulator process, and processing it, and processing the response of the NVMe device connected to the Guest OS.

The access module is responsible for receiving the encapsulation message sent by the NVMe controller emulator agent module, and after parsing the message, pass it to the NVMe controller emulation model module of the NVMe controller emulator process. In the other direction, the request or response of the NVMe controller simulation model module is encapsulated into an interprocess communication message of HostOS, and passed to the proxy module of the Qemu process.

In some embodiments, the simulation system also includes:

Storage medium simulation model, including attribute acquisition operation interface, setting operation interface, reading operation interface, and writing operation interface.

In some embodiments, the attribute acquisition operation interface and the setting operation interface are used by the firmware to be simulated, and the read operation interface and write operation interface are used by the controller simulation model for data reading and writing. .

Specifically, the storage medium simulation model is responsible for simulating operation interfaces such as attribute acquisition, setting, reading, and writing of the storage medium. Among them, the attribute acquisition and setting interfaces are used by the firmware to be simulated. The read and write interfaces are used for data reading and writing by the NVMe controller simulation model.

In some embodiments, the functional unit includes a PCIE interface unit, an NVMe core unit, a storage medium unit, and a commissioning and tracking unit, the PCIE interface unit, the NVMe core unit, the storage medium unit, and the commissioning and tracking unit interconnected with the executable unit;

Wherein, the PCIe interface unit is connected to the message forwarding module for simulating the PCIe bus, simulating reading and writing of the PCIe configuration space, transmitting bus data, and processing message signal interruption or extended message signal interruption;

The NVMe core unit is used to process the reading and writing of configuration registers, forward management commands to the executable unit, process IO commands, process doorbells, and move DMA data;

The storage medium unit is also connected to the storage medium simulation model for providing a control management interface and an IO data interface to the core unit to access the storage medium simulation model;

The commissioning and tracking unit provides print and log control management and output functions for the PCIE interface unit, the NVMe core unit, the storage medium unit and the executable unit.

Specifically, as shown in Figure 4, the controller simulation model module is responsible for the simulation of the PCIe bus, the simulation of reading and writing of the PCIe configuration space, the transmission of bus data, and MSI/MSI-X interruption (message signal interruption, extended message signal interruption) Simulation processing, simulating the registers defined by the NVMe protocol, responsible for initializing the executable unit, providing the executable unit program loading function, forwarding NVMe Amdin commands (management commands) to the firmware to be simulated, and processing IO commands.

In some embodiments, the internal initialization of the NVMe controller simulation model is performed when the NVMe controller simulation model is running, and the necessary internal notification signal events (sc_event type), internal processing logic threads (sc_thread type), and FIFO (sc_fifo type) are initialized. )wait. The external initialization of the registers of the NVMe controller simulation model is initiated by the NVMe driver of the GuestOS in the external Qemu process.

The controller simulation model may include an executable unit, a PCIE interface unit, an NVMe core unit, a storage medium unit, and a commissioning and tracking unit, the PCIE interface unit, the NVMe core unit, the storage medium unit, the commissioning and tracking unit The unit is interconnected with the executable unit.

Among them, the PCIe interface unit completes the simulation of the PCIe bus, the simulation of reading and writing of the PCIe configuration space, the transmission of bus data, and the processing of MSI/MSI-X interrupts.

The NVMe core unit completes the simulation of the NVMe protocol and the simulation of the registers defined by the NVMe protocol. Including the processing of reading and writing of NVMe controller configuration registers, forwarding Admin commands to the firmware to be simulated running on the executable unit, processing IO commands, processing doorbells, and moving DMA data.

The executable unit is responsible for providing an executable environment, including simulating the CPU, memory, Cache, and bus devices, loading and running the firmware to be simulated, completing the execution of the firmware to be simulated, and completing the configuration of the bus model, address mapping, Cache and memory. Manage model initialization and configuration.

The storage medium unit is connected with the storage medium simulation model, and is used to provide a common storage medium operation interface according to the characteristics of the storage medium simulation model. It includes control management interface and IO data interface, and provides the interface to the NVMe core unit to simulate and complete the operation of the NVMe controller on the storage medium simulation model.

The commissioning and tracking unit provides printing and log control management and output functions for each unit and module constituting the controller simulation model, completes printing and log recording related functions, and is used for testing and problem location of each unit and module.

In some embodiments, the executable unit includes an interrupt management module, a software engine module, a bus emulation module, a cache emulation module, and a memory emulation module;

The interrupt management module is used to provide an interrupt function;

The software engine module provides the operating environment and executes the firmware to be simulated;

The bus emulation module is used to realize the interconnection between the PCIE interface unit, the NVMe core unit, the storage medium unit, the commissioning and tracking unit and the executable unit;

The cache emulation module is used to provide a cache function to the software engine module;

The memory emulation module is used to provide memory functions to the software engine module.

In some embodiments, the software engine module is configured to receive the management command, save the management command to a cache location configured based on the firmware to be simulated, and trigger an interrupt to notify the firmware to be simulated to process.

In some embodiments, after the firmware to be emulated receives the interrupt, it reads and processes the management command in the cache location, wherein the management command includes creating or deleting a submission queue for data input and output commands, commands to create or delete completion queues for data input and output, commands to get namespace information, commands to get properties, and commands to set properties.

Specifically, as shown in FIG. 5 , the executable unit may include an interrupt management module, a software engine module, a bus simulation module, a Cache simulation module, and a memory simulation module.

Among them, the interrupt management module is responsible for simulating the interrupt initialization of the executable unit, interrupt connection, interrupt trigger, interrupt signal clearing and other functions. The bus simulation module is used for simulating the connection between the software engine module and each module of other units. The Cache simulation module is used to provide a Cache simulation function for the software engine module. The memory emulation module is used to provide the memory emulation function for the software engine module.

The software engine module is responsible for simulating the instruction loading and execution of the processor, and executing the version file provided by the custom firmware module. After the software engine module loads the firmware program to be simulated into the executable environment of the simulated CPU, it starts to execute the firmware program to be simulated. When there is a transaction operation on the software engine sent by the bus transaction, after analyzing the transaction content, the content is saved according to the buffer position configured by the firmware program, and an interrupt is triggered to notify the firmware to be simulated to process.

After the emulated firmware receives the interrupt, it reads the contents of the buffer, identifies the NVMe command type, and processes it. The Admin commands processed can include the Identify command defined by the NVMe protocol (a command defined by the NVMe protocol to obtain some information about the NVMe controller), and the Get Feature command (a command defined by the NVMe protocol to obtain information about the NVMe controller) Specific attribute information), Set Feature command (a command defined by the NVMe protocol, used to set specific attribute information of the NVMe controller), create IO SQ (submission queue for data input and output) command, create IO CQ (use Completion queue for data input and output) command, delete IO SQ command, delete IO CQ command, configure doorbell buffer command.

For example, when receiving the command to create an IO SQ, read the base address of the IO SQ in the SQE, the queue depth, the queue ID, the queue ID, and the corresponding CQ queue ID, and store them in the local storage with the queue ID as the index, and set the queue establishment sign. When the create IO CQ command is received, read the base address of the IO CQ in the SQE, the queue depth, the queue ID, the queue ID, and the corresponding interrupt vector, store them in the local storage with the queue ID as the index, and set the queue establishment flag. When the delete IO SQ command is received, the queue ID is used as the index to clear the queue establishment flag. When the delete IO CQ command is received, the queue ID is used as the index to clear the queue creation flag. When the Get Feature command is received, the configuration information of the NVMe controller is obtained. When receiving the Set Feature command, set the configuration information of the NMVe controller. When the Identify command is received, process NVMe namespace related information and other information.

The firmware to be simulated is also responsible for the initialization of each unit in the NVMe controller simulation model, and completes background tasks such as garbage collection, bad block processing, and wear leveling processing based on the storage medium model, and completes exception handling. In addition, the firmware to be simulated can also initiate access to the bus transaction through the internal bus address of the executable unit, and complete the read and write access to the registers of the NVMe controller emulator. And the bus transaction access is initiated through the internal bus address of the executable unit, and the acquisition and setting access to the attributes of the storage medium are completed.

In some embodiments, the PCIE interface unit, the NVMe core unit, the storage medium unit, and the commissioning and tracking unit register the socket of the initiator interface and the socket of the target interface with the bus emulation module respectively. Sockets to realize the interconnection access between each unit.

Specifically, the units of the controller simulation model can be interconnected through the bus simulation module.

In some embodiments, the TLM model based on SystemC can be used to provide each unit with the socket of the initiator interface and the target interface. Through the TLM model library of SystemC, the socket is registered to the bus simulation module, and the bus The simulation module performs transaction-level distribution according to the addresses mapped by each unit, and completes the simulation of bus interconnection transactions, thereby realizing the interconnection access between various modules and completing the simulation. And when each unit is initialized, it can initialize the internal events required by the simulation unit itself, the internal FIFO, the processing thread (sc_thread type), initialize the triggering of waiting events, and process the transaction-level transmission processing program corresponding to the target interface socket of the simulation unit Register to the bus target side. When a transaction arrives at the simulation unit, the transaction-level transfer handler corresponding to the target socket of the simulation unit will analyze the transaction, save the analyzed content in the FIFO of the simulation unit, and then trigger an internal event. After the event is triggered, the SystemC system will schedule the processing thread, and the processing thread will read the content in the FIFO, complete the processing of the event, and continue to wait for the next event to be triggered.

In this way, the executable unit loads the firmware program to complete the execution of the management plane management command, and the NVMe core unit completes the execution of the data platform IO command.

It should be noted that the above-mentioned transaction-level model based on SystemC, the construction of the NVMe controller simulation system, and the bus-level transaction distribution based on the method of transaction simulation of the executable unit accessing peripherals are only for the convenience of understanding the implementation of this application, not for To limit this application, any modifications and changes made without departing from the design and scope of this application, especially the modifications and changes to related system diagrams, mapping relationships, and related software and firmware frameworks, are within the scope of protection of this application .

In some embodiments, the PCIE interface unit further includes a PCIe interface module and a PCIe register module.

In some embodiments, the PCIe interface module is used to simulate the simulation of the PCIe bus and transmit bus data, so as to pass the message sent by the virtual host to the NVMe core unit and the message sent by the NVMe core unit to the virtual host.

In some embodiments, the PCIe register module is used for simulating reading and writing of the PCIe configuration space, and saving the configuration information of the virtual host for the message signal interrupt or the extended message signal interrupt.

Specifically, as shown in FIG. 6 , the PCIE interface unit further includes a PCIe interface module and a PCIe register module. The PCIe interface module is responsible for completing the simulation of the PCIe bus and the transmission of bus data. Simulate the access of the PCIe EP controller to the internal configuration registers and the BAR space; convert the memory access of the virtual host to the BAR space of the PCIe device on the PCIe bus into the memory access to the NVMe device, and pass it to the NVMe core unit; in the opposite direction, Complete the simulation of the NVMe core unit accessing the memory of the virtual host through the PCIe bus.

The PCIe register module completes the virtual host's simulation of reading and writing to the PCIe configuration space, and the configuration and processing of MSI/MSI-X interrupts.

Wherein, the address of the PCIe interface module and the target interface can be registered in the bus simulation module of the executable unit based on the TLM model in the initialization phase. When the NVMe controller simulation model inside qume calls the PCIe interface to initiate PCIe bus access to the PCIe device where the NVMe device is located, the type of PCIe bus access is based on the device register read, register write, address read, address write and other bus data transmission types, Encapsulate it as an internal communication message, call the proxy module and send it to the access module through the inter-process communication interface; after receiving it, the access module parses the message and converts it into a request from the initiator of the TLM model, and maps the connections on the bus The register address or Memory data address of the PCIe interface module initiates access.

The PCIe interface module registers a memory window address space for accessing the host address and the target interface to the bus emulation module of the executable unit based on the TLM model during initialization. When the NVMe controller core unit initiates access to the virtual host address, it initiates transaction-level access to the memory window address space used to access the virtual host address, triggering the processing of the PCIe interface module. The PCIe interface module calls the NVMe controller emulator access module to send the message to the host for processing. The agent module starts the software process to receive the message sent from the access module through the inter-process communication interface, and converts the message into an operation on the host address running inside Qemu.

The PCIe register module is responsible for completing the virtual host's configuration of MSI/MSI-X interrupts. When the virtual host driver invokes the PCIe interface through the NVME driver inside qume to initiate MSI/MSI-X configured PCIe bus access to the PCIe device where the NVMe device is located, The PCIe register module is responsible for saving the configuration information of the corresponding MSI/MSI-X.

When the NVMe core unit needs to initiate an MSI/MSI-X interrupt to the virtual host, the NVMe core unit notifies the PCIe interface module to initiate a PCIe transaction access to the address configured by the MSI/MSI-X, and the PCIe interface module calls the NVMe controller to simulate The server access module sends the message to the virtual host for processing.

In some embodiments, the NVMe core unit includes an NVMe register module, a submission queue module for management, a completion queue module for management, a submission queue module for data input and output, and a completion queue for data input and output module, storage module, doorbell trigger module, interrupt trigger module, DMA data transmission module.

In some embodiments, the NVMe register module is used to handle reading and writing of NVMe configuration registers.

In some embodiments, the doorbell trigger module is used to receive the doorbell operation from the host, and notify the delivery queue module for management or the delivery queue module for data input and output according to the queue corresponding to the doorbell operation to process.

In some embodiments, the management delivery queue module is configured to package the management command corresponding to the entry of the management delivery queue into a bus-level transaction request and send it to the executable unit for processing;

The completion queue module for management is used to put the execution unit's processing result of the management command corresponding to the entry of the delivery queue for management into the entry of the completion queue for management, and call the interrupt trigger module notifies the virtual host.

In some embodiments, the DMA data transmission module is configured to perform data transfer according to the doorbell trigger module or transfer the processing result of the entry of the completed queue for management to the virtual host.

In some embodiments, the submission queue module for data input and output is used to process the IO command corresponding to the entry of the submission queue for data input and output;

The completion queue module for data input and output is used to process the processing result of the IO command corresponding to the entry of the submission queue for data input and output, and put it into the entry of the completion queue for data input and output, and call the The interrupt trigger module notifies the virtual host;

The storage module is used for creating a storage space, and writing data into and/or reading data from the storage space based on the IO command.

Specifically, as shown in Figure 7, the NVMe core unit mainly includes an NVMe register module, an Admin SQ module (a delivery queue module for management), an Admin CQ module (a completion queue module for management), and an IO SQ module (for Submission queue module for data input and output), IO CQ module (complete queue module for data input and output), storage module, doorbell trigger module, interrupt trigger module, DMA data transmission module.

Among them, the NVMe register module is responsible for providing the simulation of reading and writing of NVMe configuration registers. During initialization, a section of the bus internal register address space of the NVMe controller simulation model is registered to the bus simulation module of the executable unit, and the registers of the NVMe from the virtual host PCIe bus are processed. The address request and the internal bus of the NVMe controller simulation model encapsulate the transaction-level operation interface for reading and writing NVMe registers, and the registers are saved. Through the storage space of the same register, it serves the request transactions of two different buses, completes the functional support for the host side to access the registers of the NVMe controller simulation model, and also completes the firmware side of the executable unit running in the NVMe controller simulation model Functional support for register access in the NVMe controller simulation model through the internal bus; the NVMe register module is also responsible for functions such as the trigger event status of write operations and read operations to specific registers.

The Admin SQ module is responsible for simulating and processing the NVMe management commands corresponding to the SQE in the Admin SQ queue. Including create IO SQ command, create IO CQ command, delete IO SQ command, delete IO CQ command, Identify command, get feature (Get Feature) command, set feature (Set Feature) command and other NVMe protocol commands.

The Admin CQ module is responsible for simulating the processing results of the NVMe management commands corresponding to the SQE in the Admin SQ queue, putting them into the CQE entry of the Admin CQ queue, and calling the interrupt trigger module to notify the host.

The IO SQ module is responsible for simulating and processing the NVMe IO commands corresponding to the SQE in the IO SQ queue. Including read command and write command.

The IO CQ module is responsible for simulating the processing results of the NVMe IO commands corresponding to the SQE in the IO SQ queue, putting them into the CQE entry of the IO CQ queue, and calling the interrupt trigger module to notify the host.

The storage module is responsible for creating the simulated storage space, writing data to and reading data from the storage space, and managing the storage space.

The doorbell trigger module is responsible for simulating the doorbell operation received from the host, and notifies the Admin SQ module or IO SQ module to process according to the queue corresponding to the doorbell.

The interrupt trigger module is responsible for simulating the MSI/MSI-X interrupt of PCIe and notifying the host.

The DMA data transmission module is responsible for the transfer of SQE and CQE, the transfer of IO data, and the transfer of data from the host to the controller, which is triggered by the doorbell trigger module. After the transfer of data from the controller to the host is completed, an interrupt is triggered to the host through the interrupt trigger module.

When the NVMe register module, as the target of the TLM model, receives the transaction message sent by the bus, it stores the message content in FIFO and sends an internal notification signal event. After the internal processing logic thread captures the event, it reads the FIFO content for processing. For read-only registers, a fixed value simulated by the NVMe controller is written at initialization, and when a read register transaction occurs, the contents of the register are returned to the bus as a transaction. When the write register transaction occurs, save the value of the write register to the local, and determine whether to perform subsequent processing according to the address of the write register, if it is necessary to trigger the transition of the NVMe controller simulation state machine, the transition of the state machine According to the definition of the NVMe protocol specification, details are not described here. When the address written into the register is the doorbell register, call the doorbell trigger module for processing.

The doorbell trigger module calculates the location of the current doorbell register and searches for the location of the corresponding SQ queue. The address index of the doorbell corresponding to the Admin SQ queue is 0, the address index of the doorbell corresponding to IO SQ 1 is 1, and the address index of the doorbell corresponding to IO SQ 2 is 2, and so on. After finding the position of the SQ queue, find the position of the latest SQE, and the DMA data transmission module will move the SQE from the host to the local, and write it into the FIFO corresponding to the queue. If it is the doorbell of Admin SQ, it will send an event to notify the Admin SQ module to process.

If it is the doorbell of IO SQ, an event is sent to notify the IO SQ module to process it. After the IO SQ module receives the event, it reads the content of the SQE from the FIFO of the queue, encapsulates the content of the SQE as a request for the storage medium, and sends it to the storage module, and the storage module completes the processing of the IO command. If it is a write command, read the host memory address corresponding to the PRP1/PRP2 or S/G List information carried in the IO command of IO SQE, copy the data of the address to the local cache, and call the interface provided by the storage medium model to write the IO command Specify the storage space provided by the storage medium model corresponding to the LBA parameter and the data length NLB parameter; if it is a read command, read the storage space provided by the storage medium model corresponding to the LBA parameter and the data length NLB parameter carried in the IO command of IO SQE Content, write the content to the host memory address corresponding to the PRP1/PRP2 or S/GList information carried in the command.

When the IO SQ command processing is completed, the execution result and the corresponding IO SQE entry ID are sent to the IO CQ module FIFO, and the IO CQ module register is written to notify the IO CQ module to process.

After the IO CQ module receives the register event trigger, after receiving the command processing result sent by the IO SQ from the FIFO, the IOCQ module fills in the corresponding CQE content, and initiates a DMA transmission request to the DMA data transmission module, and the DMA data transmission module sends the CQE Move to the host memory, move the CQE pointer of the IO CQ, and pass the CQ identification to the interrupt trigger module for processing. The interrupt trigger module reads the host address corresponding to the interrupt vector corresponding to the CQ, and triggers the MSIX interrupt to the host by initiating a transaction operation on the PCIe bus.

When the storage module receives the IO command, it calls the DMA data transmission module to move the host address data in the PRP/SGL information corresponding to the IO command to the local storage area, and after repackaging, initiates a bus transaction request and forwards it to the storage model unit for processing .

After the Admin SQ module receives the event, it reads the content of the SQE from the FIFO of the queue, encapsulates the content of the SQE into a bus-level transaction request, and sends it to the executable unit, which is then received and processed by the custom firmware program of the executable unit. When the custom firmware program of the executable unit finishes processing the Admin SQ command, the execution result and the corresponding Admins SQE entry ID are sent to the Admin CQ module FIFO, and the Admin CQ module register is written to notify the Admin CQ module to process. After the Admin CQ module receives the register event trigger, after receiving the command processing result sent by the custom firmware program of the executable unit from the FIFO, it fills in the corresponding Admin CQE content, initiates a DMA transfer request to the DMA data transfer module, and DMA data transfer The module initiates a PCIe data transmission request to the bus, moves the CQE content to the host memory through the PCIe interface module, moves the CQE pointer of the Admin CQ, and passes the CQ identifier to the interrupt trigger module for processing.

The interrupt trigger module reads the host address corresponding to the interrupt vector corresponding to the CQ according to the received CQ identifier. This address is configured by the host to the PCIe interface unit of the NVMe controller emulator when the host NVMe driver is initialized. The interrupt trigger module will interrupt the vector The identification is sent to the PCIe interface unit. Initiate the transaction operation on the PCIe bus through the PCIe interface unit, trigger the MSIX interrupt to the host.

Through the above modules, the NVMe core unit can complete the simulation of the NVMe protocol, including the simulation of reading and writing of the NVMe controller configuration register, forwarding the Admin SQ command to the custom firmware program running on the executable unit for processing, and the processing of the IO command. Doorbell processing, DMA data transfer. And receive the Doorbell interrupt corresponding to Admin SQ, check the entry item Admin SQE of Admin SQ, and send the management command in the entry item Admin SQE of Admin SQ to the custom firmware program module running on the executable unit for processing. It can also receive the content of the CQE completed by the executable unit, write it into the Admin CQE of the Admin CQ, and trigger an interrupt to notify the host. The NVMe core unit is responsible for receiving the Doorbell interrupt corresponding to the IO SQ, and checking the entry item IO SQE of the IO SQ; after the IO processing is completed, the processing result is written into the entry item IOCQE of the IO CQ, and an interrupt is triggered to notify the host.

In some embodiments, the storage medium unit includes a storage medium model framework module, a storage medium control module and a storage medium I/O module.

In some embodiments, the storage medium model framework module provides a control management interface and an IO data interface to access the storage medium simulation model;

The storage medium control module is used to provide storage medium control information;

The storage medium I/O module is used for providing storage medium data IO operations.

Specifically, as shown in FIG. 8 , the storage medium unit mainly includes a storage medium model framework module, a storage medium control module, and a storage medium I/O module.

Among them, the storage medium model framework module is used to provide the control interface and data interface of the storage medium. Provides a storage medium model registration framework for compatible management of different types of storage media. The storage medium model framework module maps the association between the identification of the storage medium and the corresponding operation, provides the registration interaction interface of the control information, and the registration interface of the I/O information. It is called in the process of the control path of media interaction to set or obtain the corresponding storage medium control information data, and it is called in the process of the I/O data path of the interaction between the NVMe core module and the storage medium to perform the corresponding storage medium data. Read and write operations.

The storage medium control module is used to provide simulation of storage medium control information.

The storage medium I/O module is used to provide simulation of storage medium data I/O operations.

The storage medium model supports custom firmware to complete read and write access to storage medium attribute information and configuration information by initiating bus transaction access. The storage medium model supports the NVMe controller simulation module to complete the read and write access to the IO data of the storage medium by initiating bus transaction access. The storage medium model accesses the storage medium simulation model registered in the storage medium framework to complete the read and write access to data IO.

In some embodiments, the commissioning and tracking unit includes a register operation module, a printing and log control module, and a unit test management module.

In some embodiments, the register operation module initiates read and write access to registers in the executable unit, the PCIE interface unit, the NVMe core unit, the storage medium unit, and the commissioning and tracking unit;

The print and log control module is used to provide print control and log control;

The unit test module is used to provide test entries of the executable unit, the PCIE interface unit, the NVMe core unit, the storage medium unit and the commissioning and tracking unit, and is used to process unit test commands.

Specifically, as shown in FIG. 9 , the commissioning and tracking unit mainly includes a register operation module, a printing and log control module, and a unit test management module.

Among them, the register operation module is used to provide list display of register content and read and write register operation. The print and log control module is used to provide print control and log control for each module. The unit test module is used to provide the unit test entry of each module and the processing of unit test commands.

In this way, the register operation module initiates read and write access to the registers of each module by initiating a bus-level transaction request. The print and log control module controls the print and log output of each simulation unit by initiating a bus-level transaction request and initiating read and write access to the registers used for debugging of each module. The unit test module organizes piling configuration and unit tests according to the needs of unit tests. Support for individual testing of individual units.

In some embodiments, as shown in FIG. 10 , when the NVMe controller simulation model is implemented in a specific application, an external Qemu process, a custom firmware program, a storage medium simulation model, etc. are required to be used together. As shown in Figure 9, S001 corresponds to the user application, S002 corresponds to the NVMe device interface; S003 corresponds to the NVMe controller simulator agent module inside Qemu; S004 corresponds to the NVMe controller simulator access module of the NVMe controller simulator process; S005 corresponds to PCIe interface unit; S006 corresponds to the executable unit; S007 corresponds to the NVMe core unit; S008 corresponds to the custom firmware program; S009 corresponds to the storage medium model unit; S010 corresponds to the storage medium simulation model; S011 corresponds to the commissioning and tracking unit.

In this way, the main flow of a complete management command can be as follows: S001, S002, S003, S004, S005, S006, S007, S006, S008, S006, S009, S010.

The main process of a complete IO command can be as follows: S001, S002, S003, S004, S005, S006, S007, S006, S009, S010.

The technical solution of this application proposes a simulation system for NVME equipment, which provides an executable unit that can support the operation of the firmware to be simulated, and provides internal access to the registers of the NVMe controller simulation model on the internal interconnection bus of the executable unit, thereby Able to independently design, develop and debug the internal firmware of the controller.

Those of ordinary skill in the art should understand that: the discussion of any of the above embodiments is exemplary only, and is not intended to imply that the scope (including claims) disclosed by the embodiments of the present application is limited to these examples; under the idea of the embodiments of the present application , the technical features in the above embodiments or different embodiments can also be combined, and there are many other changes in different aspects of the above embodiments of the present application, which are not provided in details for the sake of brevity. Therefore, within the spirit and principle of the embodiments of the present application, any omissions, modifications, equivalent replacements, improvements, etc., shall be included in the protection scope of the embodiments of the present application.

Claims (24)

1. A simulation system for non-volatile memory interface specification device simulation, comprising:

a virtual host;

the controller simulation model comprises an executable unit and a plurality of functional units, wherein the executable unit is used for running firmware to be simulated and comprises an interrupt management module, a software engine module, a bus simulation module, a cache simulation module and a memory simulation module, and the bus simulation module is used for realizing interconnection between the executable unit and each functional unit;

the message forwarding module is used for connecting the virtual host and the controller simulation model;

the controller simulation model is further used for processing the message sent by the virtual host based on the firmware to be simulated so as to simulate the firmware to be simulated.

2. The simulation system of claim 1, wherein the message forwarding module comprises a proxy module and an access module;

the agent module is used for packaging the information sent by the virtual host and sending the information to the access module, and receiving and analyzing the information sent by the access module and then sending the information to the virtual host;

The access module is used for receiving and analyzing the message sent by the proxy module and then sending the message to the controller simulation model, and packaging the sent message of the controller simulation model and sending the message to the proxy module.

3. The emulation system of claim 2, wherein the proxy module and the virtual host run in a first process.

4. The simulation system of claim 2, wherein the access module and the controller simulation model run in a second process.

5. The simulation system of claim 1, wherein the simulation system further comprises:

the storage medium simulation model comprises an attribute acquisition operation interface, a setting operation interface, a reading operation interface and a writing operation interface.

6. The emulation system of claim 5, wherein the attribute acquisition operation interface and the setup operation interface are for invocation by the firmware to be emulated, and the read operation interface and the write operation interface are for data read and write use by the controller emulation model.

7. The simulation system of claim 5, wherein the functional units include a high-speed serial computer expansion bus standard interface unit, a non-volatile memory interface specification core unit, a storage medium unit, and a debug trace unit, the high-speed serial computer expansion bus standard interface unit, the non-volatile memory interface specification core unit, the storage medium unit, the debug trace unit, and the executable unit being interconnected;

The high-speed serial computer expansion bus standard interface unit is connected with the message forwarding module and is used for simulating a high-speed serial computer expansion bus standard bus, simulating a read-write high-speed serial computer expansion bus standard configuration space, transmitting bus data and processing message signal interruption or expanded message signal interruption;

the nonvolatile memory interface standard core unit is used for processing the read-write of the configuration register, forwarding the management command to the executable unit, processing the IO command, processing the doorbell, and moving the direct memory access data;

the storage medium unit is also connected with the storage medium simulation model and is used for providing a control management interface and an IO data interface for the core unit to access the storage medium simulation model;

the debug trace unit provides print and log control management and output functions for the high-speed serial computer expansion bus standard interface unit, the nonvolatile memory interface specification core unit, the storage medium unit and the executable unit.

8. The simulation system of claim 7, wherein,

the interrupt management module is used for providing an interrupt function;

The software engine module provides an operating environment and executes the firmware to be emulated;

the bus simulation module is used for realizing interconnection among the high-speed serial computer expansion bus standard interface unit, the nonvolatile memory interface standard core unit, the storage medium unit, the debugging tracking unit and the executable unit;

the cache simulation module is used for providing a cache function for the software engine module;

the memory simulation module is used for providing memory functions for the software engine module.

9. The emulation system of claim 8, wherein the software engine module is to receive the management command, save the management command to a cache location based on the firmware configuration to be emulated, and trigger an interrupt to notify the firmware processing to be emulated.

10. The emulation system of claim 9, wherein the firmware to be emulated, upon receipt of the interrupt, reads and processes management commands in the cache location, wherein the management commands include a command to create or delete a commit queue for data input and output, a command to create or delete a completion queue for data input and output, a command to acquire namespace information, a command to acquire a property, a command to set a property.

11. The emulation system of claim 8, wherein the high-speed serial computer expansion bus standard interface unit, the nonvolatile memory interface specification core unit, the storage medium unit, and the debug trace unit register a socket of an initiator interface and a socket of a target interface with the bus emulation module, respectively, to enable interconnection access between the units.

12. The emulation system of claim 7, wherein the high-speed serial computer expansion bus standard interface unit further comprises a high-speed serial computer expansion bus standard interface module and a high-speed serial computer expansion bus standard register module.

13. The emulation system of claim 12, wherein the high-speed serial computer expansion bus standard interface module is to emulate and transfer bus data of a high-speed serial computer expansion bus standard to pass messages sent by the virtual host to the nonvolatile memory interface specification core unit and to pass messages sent by the nonvolatile memory interface specification core unit to the virtual host.

14. The emulation system of claim 12, wherein the high speed serial computer expansion bus standard register module is configured to emulate read and write high speed serial computer expansion bus standard configuration space, save configuration information of message signal interrupts or extended message signal interrupts by the virtual host.

15. The simulation system of claim 7, wherein the nonvolatile memory interface specification core unit comprises a nonvolatile memory interface specification register module, a commit queue module for management, a completion queue module for management, a commit queue module for data input and output, a completion queue module for data input and output, a storage module, a doorbell trigger module, an interrupt trigger module, and a direct memory access data transfer module.

16. The emulation system of claim 15, wherein the non-volatile memory interface specification register module is to handle reading from and writing to non-volatile memory interface specification configuration registers.

17. The simulation system of claim 15, wherein the doorbell trigger module is configured to receive a doorbell operation from the host, and notify a submitted queue module for management or a submitted queue module for data input and output for processing according to a queue corresponding to the doorbell operation.

18. The simulation system of claim 15, wherein the management queue module is configured to encapsulate a management command corresponding to an entry of a management queue into a bus-level transaction request and send the bus-level transaction request to the executable unit for processing;

The completion queue module for management is used for placing the processing result of the management command corresponding to the entry of the management submitting queue by the executable unit into the entry of the completion queue for management, and calling the interrupt triggering module to inform the virtual host.

19. The emulation system of claim 18, wherein the direct memory access data transfer module is configured to either move data according to the doorbell trigger module or to move the processing results of the entry of the completion queue for management to the virtual host.

20. The simulation system of claim 15, wherein the submit queue module for data input and output is configured to process IO commands corresponding to entries of the submit queue for data input and output;

the completion queue module for data input and output is used for processing the processing result of the IO command corresponding to the inlet of the submitting queue for data input and output, placing the processing result into the inlet of the completion queue for data input and output, and calling the interrupt triggering module to inform the virtual host;

the storage module is used for creating a storage space, and writing data into the storage space and/or reading data from the storage space based on the IO command.

21. The simulation system of claim 7, wherein the storage medium unit comprises a storage medium model framework module, a storage medium control module, and a storage medium I/O module.

22. The simulation system of claim 21, wherein the storage medium model framework module provides a control management interface and an IO data interface to access the storage medium simulation model;

the storage medium control module is used for providing storage medium control information;

the storage medium I/O module is used for providing storage medium data IO operation.

23. The simulation system of claim 7, wherein the debug trace unit comprises a register operation module, a print and log control module, and a unit test management module.

24. The emulation system of claim 23, wherein the register operating module initiates a read-write access to registers in the executable unit, the high-speed serial computer expansion bus standard interface unit, the nonvolatile memory interface specification core unit, the storage medium unit, and the debug trace unit;

the printing and log control module is used for providing printing control and log control;

The unit test module is used for providing test inlets of the executable unit, the high-speed serial computer expansion bus standard interface unit, the nonvolatile memory interface standard core unit, the storage medium unit and the modulation and test tracking unit, and is used for processing unit test commands.

Images