WO2025035906A1 - Reversing image display method and apparatus, device, and computer storage medium - Google Patents

Abstract

Disclosed are a reversing image display method and apparatus, a device, and a computer storage medium, which are applied to an in-vehicle terminal for installing an Android system. The reversing image display method comprises: mounting a target partition in a kernel_init thread startup phase of an Android system; running a preset program in the target partition to create a file system directory of a reversing application, and mounting a file system in the target partition to the file system directory, the file system comprising files required for running of the reversing application; re-running the preset program to set a user role and a read-write permission of the reversing application in the file system directory; and starting the reversing application to control a camera connected to the in-vehicle terminal to acquire reversing image data, and displaying the reversing image data.

Description

Reversing image display method, device, equipment, and computer storage medium

Citation of Related Applications

This disclosure claims all rights and interests in the invention patent application with application number 202311016273.0 filed with the State Intellectual Property Office of the People's Republic of China on August 11, 2023, and with the invention name “A reversing image display method, device, equipment, and computer storage medium”, and incorporates its entire contents into this disclosure by reference.

field

The present disclosure generally relates to the field of reversing image processing technology, and more specifically, to a reversing image display method, device, equipment, and computer storage medium.

background

Currently, electronic devices installed with the Android system must wait for the Android system to start before they can perform corresponding operations. For example, a vehicle-mounted terminal can operate its own camera to collect reversing image data for display only after the Android system is started. As a result, the camera startup time is too long, affecting the display speed of the reversing image data, and then affecting the rapidity of the reversing operation.

Overview

In a first aspect, the present disclosure provides a method for displaying a reversing image, which is applied to a vehicle-mounted terminal installed with an Android system, and includes:

During the kernel_init thread startup phase of the Android system, mounting the target partition;

Running a preset program in the target partition to create a file system directory of the reversing application, and mounting the file system in the target partition to the file system directory, wherein the file system includes files required for running the reversing application;

Re-running the preset program to set the user role and read-write permissions of the reversing application program in the file system directory; and

The reversing application is started to control the camera connected to the vehicle-mounted terminal to collect reversing image data and display the reversing image data.

In some embodiments, the step of starting the reversing application to control the camera connected to the vehicle-mounted terminal to collect reversing image data includes:

The reversing application is started to control the camera to collect the reversing image data through the V4L2 framework.

In some embodiments, during the kernel_init thread startup phase of the Android system, after mounting the target partition, the method further includes:

After the vehicle-mounted terminal is started, the target plug-in is used to interact with the driver of the camera through the V4L2 framework to light up the camera.

In some embodiments, after mounting the file system in the target partition to the file system directory and before re-running the preset program to set the user role and read/write permissions of the reversing application program in the file system directory, the method further includes:

Load the SE security policy and set the SE security mode.

In some embodiments, the processing of the reversing image data includes:

Displaying the reversing image data via a first graphics card;

After the reversing image is processed, the method further includes:

The other image data besides the reversing image data are displayed by the second graphics card.

In a second aspect, the present disclosure provides a reversing image display device, which is applied to a vehicle terminal installed with an Android system, and includes:

A preset program is used to create a file system directory of the reversing application during the kernel_init thread startup phase of the Android system, and mount the file system in the target partition of the vehicle terminal to the file system directory, wherein the file system includes files required for the operation of the reversing application; and set the user role and read and write permissions of the reversing application in the file system directory;

The reversing application is used to control the camera connected to the vehicle-mounted terminal to collect reversing image data and process the reversing image data.

In some embodiments, the reversing image display device further includes:

The V4L2 framework is used to interact with the driver of the camera to light up the camera under the control of the target plug-in after the vehicle terminal is started; and is used to control the camera to collect the reversing image data under the instruction of the reversing application.

In some embodiments, the reversing image display device further includes:

A first graphics card configured to display the reversing image data;

The second graphics card is configured to display other image data besides the reversing image data.

In a third aspect, the present disclosure provides a computer device, comprising:

a memory configured to store a computer program;

The processor is configured to implement the reversing image display method described in the present disclosure when executing the computer program.

In a fourth aspect, the present disclosure provides a computer storage medium, in which a computer program is stored. When the computer program is executed by a processor, the reversing image display method described in the present disclosure is implemented.

In certain implementation schemes, during the kernel_init thread startup phase of the Android system, a vehicle terminal installed with an Android system can mount and run a preset program in a target partition to create a file system directory for a reversing application, mount the file system in the target partition to the file system directory, and rerun the preset program to set the user role and read-write permissions of the reversing application in the file system directory, thereby setting up a running environment for the reversing application in advance, and starting the reversing application to control the camera to collect reversing image data and process the reversing image data, thereby achieving the goal of starting the camera for reversing display through the reversing application during the kernel_init thread startup phase of the Android system. Compared with a solution in which the camera can be started for reversing display only after waiting for the Android system to start, the camera startup speed is accelerated, and then the display speed of the reversing image data is accelerated, and finally the reversing display speed of the vehicle terminal is accelerated.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solution of the embodiment of the present disclosure, the drawings required for use in the embodiment will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

FIG1 is a first flow chart of a method for displaying a reversing image provided by an embodiment of the present disclosure;

FIG2 is a second flow chart of a method for displaying a reversing image provided by an embodiment of the present disclosure;

FIG3 is a third flow chart of a method for displaying a reversing image according to an embodiment of the present disclosure;

Figure 4 is a normal boot flow chart of the Android system;

FIG5 is a flowchart of reverse start of the reverse imaging device;

FIG6 is a schematic flow chart of the structure of a reversing image display device provided by an embodiment of the present disclosure; and

FIG. 7 is a structural diagram of a computer device 20 according to an embodiment of the present disclosure.

Details

The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

Please refer to FIG. 1 , which is a first flow chart of a method for displaying a reversing image according to an embodiment of the present disclosure.

The reversing image display method provided by the embodiment of the present disclosure is applied to a vehicle terminal installed with an Android system, and may include the following steps:

Step S101: during the kernel_init thread startup phase of the Android system, read the mount target partition;

Step S102: running a preset program in the target partition to create a file system directory of the reversing application, and mounting the file system in the target partition to the file system directory, wherein the file system includes files required for running the reversing application;

Step S103: re-running the preset program to set the user role and read/write permissions of the reversing application program in the file system directory; and

Step S104: Start the reversing application to control the camera connected to the vehicle terminal to collect reversing image data and display the reversing image data.

In certain implementation schemes, a vehicle terminal installed with an Android system can read the mounted target partition during the startup phase of the kernel_init thread of the Android system, so as to subsequently build a running environment for a reversing application that controls the camera in advance based on the mounted target partition, so as to start the camera connected to the vehicle terminal during the startup phase of the kernel_init thread of the Android system.

In some implementation schemes, the vehicle terminal installed with the Android system is based on the mount In the process of building the operating environment of the reversing application in the target partition, you can first run the preset program in the target partition to create the file system directory of the reversing application, and mount the file system in the target partition to the file system directory. The file system includes the files required for the operation of the reversing application. It should be noted that the file system may include dynamic libraries, configuration files, security policies, executable files, etc., and may include sysfs, selinuxfs, devtmpfs, etc.; then re-run the preset program to set the user role and read and write permissions of the reversing application in the file system directory.

In certain implementation schemes, after the vehicle terminal installed with the Android system mounts the file system in the target partition to the file system directory, before re-running the preset program to set the user role and read and write permissions of the reversing application in the file system directory, in order to reduce the dangers brought by the reversing application, the SE security policy in the target partition can also be loaded, and the SE security mode can be set to provide security protection for the reversing application.

In certain implementation schemes, after a vehicle terminal installed with an Android system builds an operating environment for a reversing application based on a target partition, the reversing application can be started to control the camera connected to the vehicle terminal to collect reversing image data and display the reversing image data, so that the camera can be started for reversing display during the kernel_init thread startup phase of the Android system, thereby quickly providing reversing services to users.

In certain implementation schemes, during the kernel_init thread startup phase of the Android system, a vehicle terminal installed with an Android system can mount and run a preset program in a target partition to create a file system directory for a reversing application, mount the file system in the target partition to the file system directory, and rerun the preset program to set the user role and read-write permissions of the reversing application in the file system directory, thereby setting up a running environment for the reversing application in advance, and starting the reversing application to control the camera to collect reversing image data and process the reversing image data, thereby achieving the goal of starting the camera for reversing display through the reversing application during the kernel_init thread startup phase of the Android system. Compared with a solution in which the camera can be started for reversing display only after waiting for the Android system to start, the camera startup speed is accelerated, and then the display speed of the reversing image data is accelerated, and finally the reversing display speed of the vehicle terminal is accelerated.

Please refer to FIG. 2 , which is a second flow chart of a method for displaying a reversing image according to an embodiment of the present disclosure.

The reversing image display method provided by the embodiment of the present disclosure is applied to a vehicle terminal installed with an Android system, and may include the following steps:

Step S201: During the kernel_init thread startup phase of the Android system, the target partition is mounted;

Step S202: after the vehicle terminal is started, the target plug-in is used to interact with the camera driver through the V4L2 framework to light up the camera;

Step S203: running a preset program in the target partition to create a file system directory of the reversing application, and mounting the file system in the target partition to the file system directory, wherein the file system includes files required for running the reversing application;

Step S204: re-running the preset program to set the user role and read/write permissions of the reverse application program in the file system directory; and

Step S205: Start the reversing application to control the camera to collect reversing image data through the V4L2 framework.

In some embodiments, the vehicle terminal installed with the Android system can light up the camera based on the target plug-in after mounting the target partition during the kernel_init thread startup phase of the Android system, that is, the camera can be quickly lit up based on the target plug-in after the vehicle terminal is started. The target plug-in can be written based on C language or C++ language, etc., which is not specifically limited in the present disclosure.

In certain implementation schemes, when a vehicle-mounted terminal installed with an Android system is lighting up a camera based on a target plug-in, the target plug-in can be used to interact with the camera driver through the V4L2 (Video for Linux2) framework, without having to wait for the Android system to fully start up, thereby achieving the purpose of quickly lighting up the camera.

In some embodiments, the process of mounting the target partition and quickly starting the user space program in the kernel_init thread can be as follows: in kernel_init() of kernel/msm-4.14/init/main.c, first call the kernel_init_freeable() function, in the kernel_init_freeable() function, call the newly added launch_early_services() function to mount the target partition and start the user space early_init program (preset program) through the call_usermodehelper() function, then call the get_early_services_status() function to query the execution result of the early_init program, and wait for the security policy to be loaded. The "/dev/sedone" identifier is created by the early_init program, and its code can be as follows:

In certain implementation schemes, when a vehicle terminal installed with an Android system starts a reversing application to control a camera connected to the vehicle terminal to collect reversing image data, the reversing application can be started to control the camera to collect reversing image data through a V4L2 framework. That is, the V4L2 framework can be used to quickly interact with the camera, and then the reversing image data can be quickly obtained for display.

Please refer to FIG. 3 , which is a third flow chart of a reversing image display method provided in an embodiment of the present disclosure.

The reversing image display method provided by the embodiment of the present disclosure is applied to a vehicle terminal installed with an Android system, and may include the following steps:

Step S301: During the kernel_init thread startup phase of the Android system, the target partition is mounted;

Step S302: running a preset program in the target partition to create a file system directory of the reversing application, and mounting the file system in the target partition to the file system directory, wherein the file system includes files required for running the reversing application;

Step S303: re-running the preset program to set the user role and read-write permissions of the reverse application program in the file system directory;

Step S304: starting a reversing application to control the camera connected to the vehicle-mounted terminal to collect reversing image data, and displaying the reversing image data through the first graphics card; and

Step S305: displaying other image data except the reversing image data via the second graphics card.

In some implementations, considering that displaying the reversing image data collected by the camera will occupy the display interface of the vehicle terminal, and in the early stage of booting, the Android surfacefinger service is not running, and the surfacefinger cannot be used for display. To avoid this situation, the image information and other images collected by the camera can be flexibly displayed during the kernel_init thread startup phase. In the process of processing the image information, the vehicle terminal can display the reversing image data through the first graphics card; accordingly, after displaying the reversing image data, other image data other than the reversing image data can also be displayed based on the second graphics card of the vehicle terminal, such as displaying the Android interface, displaying the road image when the car is moving forward, etc. That is, the vehicle terminal can use a dual graphics card mechanism to display the reversing image data collected by the camera and the Android interface. In this way, the reversing image data collected by the camera and the Android interface can be displayed correspondingly with two different layers, so that the display of the reversing image data collected by the camera does not affect the display process of the Android interface at all. For example, during the boot animation playback phase, even if the camera is frequently used to collect reversing image data, the two graphics cards can be used to seamlessly switch to display the reversing image data collected by the camera or display the boot animation.

In order to facilitate understanding of the reversing image display solution provided by the present disclosure, the startup process of the camera connected to the vehicle terminal is described in combination with the vehicle terminal, and the vehicle terminal adopts a dual graphics card mechanism, that is, there are two layers, one layer is used for displaying the original Android interface, using the /dev/dri/card0 node, and the other layer is specifically used for displaying the reversing screen, using the /dev/dri/card2 node, and assuming that the target partition is the early_services partition, it can include the following steps:

Step 1: Write the camera's reversing application early_camera and its required so dynamic library, configuration file, security policy, and executable file required for system startup into the early_services partition, mount the early_services partition through the kernel_init thread of the vehicle terminal; and set the early_init process to start the camera;

Step 2: After the vehicle terminal is started, use the target plug-in to interact with the camera driver through the V4L2 framework to quickly light up the camera;

Step 3: During the startup phase of the kernel_init thread of the Android system, the vehicle terminal runs the early_init process based on the early_services partition corresponding to the kernel_init thread. Specifically, in the first phase of the early_init process: create a file system directory and mount related file systems, including sysfs, selinuxfs, devtmpfs, etc.; load SE security policies and set SE security mode, and restart to enter the second phase through the execv() function; the second phase: set the user role and read and write permissions of the files in the early_services directory, and parse the early_init.conf configuration file to start the application defined in the early_init.conf. And the early_camera application is defined in this configuration file;

Step 4: The vehicle terminal sends the reversing command to the early_camera application, so that the early_camera application starts the camera through the V4L2 framework to collect the reversing image. After that, the early_camera application writes data to the /dev/dri/card2 node, and the vehicle terminal displays the reversing image on its own display screen based on the first layer. At this time, the reversing image covers the original Android interface. In this process, the early_camera application can use the DRM (DirectRenderingManager) graphics display framework for rendering; and

Step 5: After reversing is completed, the early_camera application stops writing data to the /dev/dri/card2 node, and the vehicle terminal displays the Android interface based on the second layer on its own display.

It should be noted that the reason why the early_init program is started instead of the early_camera reversing application program is that the environment required for the reversing application to run needs to be configured first. In layman's terms, the relevant processes in the init process that ensure the normal operation of the reversing application are streamlined and implemented in early_init. In addition, the role of the devtmpfs involved is to establish a preliminary /dev in the early stage of Linux kernel startup, so that general applications can start execution as soon as possible without waiting for udev to complete; and in order to support devtmpfs, the following configuration needs to be added to the kernel's xxx_defconfig and xxx-perf_defconfig configuration files:

CONFIG_DEVTMPFS=y

CONFIG_DEVTMPFS_MOUNT=y.

It should also be noted that the implementation of launch_early_services() and get_early_services_status() functions can be as follows:

As can be seen from the code, saved_early_userspace contains the device name of the early_services partition, such as /dev/block/sda14, which is passed from the Bootloader (EDK2). In EDK2, the part_name of the early_services partition is obtained through the GetSystemPath() function and passed to the Kernel through the command line. Therefore, corresponding modifications must be made in EDK2. The specific changes will not be repeated here. Among them, EDK2 is also written as EDKII, which is a set of open source codes implemented by Intel for the UEFI specification. UEFI (Unified Extensible Firmware) is the unified extensible firmware interface, which defines the interface specification between the computer and platform firmware.

It should be noted that the vehicle terminal can achieve rapid reversing after starting the camera according to the disclosed solution. In order to facilitate this effect, the reversing startup process of the vehicle terminal is described in combination with Figures 4 and 5, wherein Figure 4 is a normal startup flowchart of the Android system, and Figure 5 is a reversing startup flowchart of the vehicle terminal. It can be seen from Figure 4 that after the existing Android system is started, the early_camera application is started in init.rc to perform reversing display. It can be seen from Figure 5 that the present disclosure performs reversing display in the kernel_init thread. After the kernel_init thread is started, a lot of work is done until the init process runs to the second stage to trigger the init.rc script. These works will delay the reversing display. If the present disclosure performs reversing display directly in the kernel_init thread, the reversing display process can be accelerated. After testing, the disclosed solution can realize reversing image preview within 4 seconds after the vehicle terminal is turned on, which can meet the user's needs for rapid reversing and rear view.

Please refer to FIG. 6 , which is a schematic structural flowchart of a reversing image display device provided in an embodiment of the present disclosure.

The reversing image display device provided by the embodiment of the present disclosure is applied to a vehicle terminal installed with an Android system, and may include:

The preset program 101 is used to create a file system directory of the reversing application program during the kernel_init thread startup phase of the Android system, and mount the file system in the target partition of the vehicle terminal to the file system directory, wherein the file system includes files required for the reversing application program to run; set the user role and read/write permissions of the reversing application program in the file system directory; and

The reversing application 102 is used to control the camera connected to the vehicle-mounted terminal to collect reversing image data and process the reversing image data.

The embodiment of the present disclosure provides a reversing image display device, which is applied to a vehicle-mounted terminal installed with an Android system, and may further include:

The V4L2 framework 103 is used to interact with the camera driver under the control of the target plug-in 104 after the vehicle terminal is started to light up the camera; it is used in the reversing application Under the instruction of the user, the camera is controlled to collect reversing image data.

The embodiment of the present disclosure provides a reversing image display device, which is applied to a vehicle-mounted terminal installed with an Android system, and may further include:

A first graphics card 105, configured to display reversing image data;

The second graphics card 106 is configured to display other image data besides the reversing image data.

The description of the corresponding components in the reversing image display device provided by the present disclosure can be referred to the above embodiments, and will not be repeated here.

The embodiment of the present disclosure also provides a computer device. Fig. 7 is a structural diagram of a computer device 20 according to an exemplary embodiment, and the content in the diagram cannot be regarded as any limitation on the scope of application of the present disclosure.

FIG7 is a schematic diagram of the structure of a computer device 20 provided in an embodiment of the present disclosure. The computer device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input/output interface 25, and a communication bus 26, wherein the memory 22 is configured to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the aforementioned reversing image display method of the present disclosure. In some embodiments, the computer device 20 in this embodiment may be a server.

In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the computer device 20; the communication interface 24 can create a data transmission channel between the computer device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present disclosure, and is not specifically limited here; the input and output interface 25 is configured to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

In some embodiments, the memory 22 serves as a carrier for storing resources, which may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222, and video data 223, etc. The storage method may be temporary storage or permanent storage.

In some embodiments, the operating system 221 is used to manage and control the hardware devices and computer programs 222 on the computer device 20 to enable the processor 21 to calculate and process the massive video data 223 in the memory 22. The operating system 221 can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer programs 222 that can be used to complete the present invention, In addition to the disclosed computer program of the reversing image display method executed by the computer device 20 , the computer program that can be used to complete other specific tasks can also be included. In some embodiments, the data 223 can include various data collected by the computer device 20 .

In certain embodiments, the computer device may be a module capable of realizing a positioning communication function or a terminal device comprising a module, etc. The terminal device may be a mobile terminal and/or a smart device, etc. The mobile terminal may be at least one of a mobile phone, a tablet computer, a laptop computer, etc. The smart device may be at least one of a smart watch, a smart refrigerator, a smart speaker, a smart washing machine, a smart TV, etc. The module may be any one of a 2G communication module, a 3G communication module, a 4G communication module, a 5G communication module, a NB-IOT communication module, etc.

The embodiment of the present disclosure further discloses a computer storage medium, in which a computer program is stored. When the computer program is loaded and executed by a processor, the reversing image display method of the present disclosure is implemented.

It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

The above description of the disclosed embodiments enables those skilled in the art to implement or use the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

The method for displaying a reversing image is applied to a vehicle terminal installed with an Android system, and comprises: During the kernel_init thread startup phase of the Android system, mounting the target partition; Running a preset program in the target partition to create a file system directory of the reversing application, and mounting the file system in the target partition to the file system directory, wherein the file system includes files required for running the reversing application; Re-running the preset program to set the user role and read-write permissions of the reversing application program in the file system directory; and The reversing application is started to control the camera connected to the vehicle-mounted terminal to collect reversing image data and display the reversing image data. The method according to claim 1, wherein the step of starting the reversing application to control the camera connected to the vehicle-mounted terminal to collect reversing image data comprises: The reversing application is started to control the camera to collect the reversing image data through the V4L2 framework. The method according to claim 1 or 2, wherein, in the kernel_init thread startup phase of the Android system, after mounting the target partition, it further comprises: After the vehicle-mounted terminal is started, the target plug-in is used to interact with the driver of the camera through the V4L2 framework to light up the camera. The method according to any one of claims 1 to 3, wherein after mounting the file system in the target partition to the file system directory and before re-running the preset program to set the user role and read/write permissions of the reversing application program in the file system directory, further comprising: Load the SE security policy and set the SE security mode. The method according to any one of claims 1 to 4, wherein the displaying of the reversing image data comprises: Displaying the reversing image data via a first graphics card; After displaying the reversing image, the method further includes: The other image data besides the reversing image data are displayed by the second graphics card. A reversing image display device is applied to a vehicle terminal installed with an Android system, and includes: A preset program is used to create a file system directory of the reversing application during the kernel_init thread startup phase of the Android system, and mount the file system in the target partition of the vehicle terminal to the file system directory, wherein the file system includes files required for the operation of the reversing application; and set the user role and read and write permissions of the reversing application in the file system directory; The reversing application is used to control the camera connected to the vehicle-mounted terminal to collect reversing image data so as to display the reversing image data. The device of claim 6, further comprising: The V4L2 framework is used to interact with the driver of the camera to light up the camera under the control of the target plug-in after the vehicle terminal is started; and is used to control the camera to collect the reversing image data under the instruction of the reversing application. The device according to claim 6 or 7, further comprising: A first graphics card configured to display the reversing image data; and The second graphics card is configured to display other image data besides the reversing image data. Computer equipment, comprising: a memory configured to store a computer program; A processor is configured to implement the reversing image display method described in any one of claims 1 to 5 when executing the computer program. A computer storage medium, wherein the computer storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements any one of claims 1 to 5. The invention claims a method for displaying a reversing image.