WO2025097771A1 - Human-machine collaborative management and control method, system and device involving logistics storage and transportation - Google Patents

Abstract

The present invention relates to the technical field of intelligent logistics management and control. Disclosed are a human-machine collaborative management and control method, system and device involving logistics storage and transportation. The method comprises: on the basis of a logistics storage and transportation task, determining a target operation process and a corresponding target operation item queue; executing said task on the basis of the queue; determining whether the queue belongs to a manual operation item set; if the queue belongs to the manual operation item set, generating and issuing a manual operation notification instruction; determining whether a confirmation signal fed back on the basis of the manual operation notification instruction is received; if yes, updating the state of operation items to 'in progress'; determining whether the queue belongs to a feedback operation item set; if the queue belongs to the feedback operation item set, determining whether a manual feedback completion signal is received; if yes, updating the state of the operation items to 'completed'; and when the label of one of the operation items is the maximum label in the queue, confirming that said task is completed. According to the present invention, an instruction is sent to an operator and a manual feedback signal is received in time, so as to collaboratively manage and control the logistics storage and transportation operation process requiring manual participation.

Description

Human-machine collaborative control method, system and equipment related to logistics storage and transportation Technical Field

The present invention relates to the technical field of intelligent logistics control, and in particular to a human-machine collaborative control method, system and equipment for logistics storage and transportation.

Background Art

The steel industry needs to use a large amount of raw materials in its daily work. The raw materials entering each steel plant are large in quantity and variety. In steel plants that mainly use automobiles and other means of transportation, each process in the logistics and transportation process needs to be completed by both manpower and equipment. However, the operation process involving manpower is difficult to coordinate in a timely and accurate manner, and the degree of intelligent automation is low.

The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art.

Summary of the invention

The main purpose of the present invention is to provide a human-machine collaborative control method, system and equipment involving logistics storage and transportation, aiming to solve the technical problem that the manual operation process in the existing steel industry logistics and transportation process is difficult to coordinate as a whole in a timely and accurate manner.

To achieve the above objectives, the present invention provides a human-machine collaborative management and control method for logistics storage and transportation, comprising:

When receiving a logistics storage and transportation task, determining a target operation process according to the logistics storage and transportation task;

Determine the target operation item queue corresponding to the target operation process according to the operation item queue;

Execute the logistics storage and transportation task according to the operation items and operation sequence of the target operation item queue;

Determine whether the target operation item queue belongs to a manual operation item set;

When the target operation item queue belongs to the manual operation item set, generating and issuing a manual operation notification instruction;

Determining whether a confirmation signal based on the manual operation notification instruction feedback is received;

When the confirmation signal is received, updating the status of the operation item in the target operation item queue to being executed;

Determine whether the target job item queue belongs to a feedback job item set;

When the target operation item queue belongs to the feedback operation item set, determining whether a manual feedback completion signal is received;

When the manual feedback completion signal is received, the status of the work item in the target work item queue is updated to be completed;

When the digital number of the operation item is the maximum digital number of the target operation item queue, confirm that the object The flow storage and transportation tasks are completed.

In some embodiments, after determining whether the target operation item queue belongs to a manual operation item set, the method further includes:

When the target operation item queue does not belong to the manual operation item set, the corresponding execution equipment is automatically notified to start the operation through the industrial Internet of Things;

or,

When the target operation item queue does not belong to the manual operation item set, the operation status corresponding to the operation item that automatically triggers the target operation item queue is in execution, completed or faulty.

In some embodiments, after determining whether the target job item queue belongs to the feedback job item set, the method further includes:

When the target operation item queue does not belong to the feedback operation item set, determining whether an automated feedback completion signal is received;

When the automation feedback completion signal is received, the status of the operation item in the target operation item queue is updated to be completed.

In some embodiments, when the target operation item queue belongs to the manual operation item set, generating and issuing a manual operation notification instruction includes:

When the target operation item queue belongs to the manual operation item set, determining a current operation item of the target operation item queue;

Obtain name audio data and status audio data corresponding to the current job item;

Obtaining a job instruction keyword corresponding to the current job item;

Automatically generate a manual operation notification instruction based on the name audio data and the status audio data combined with the operation instruction keyword;

The manual operation notification instruction is issued through a mobile terminal.

In some embodiments, the sending of the manual operation notification instruction through the mobile terminal includes:

Determine the operator corresponding to the current operation project based on the preset mapping relationship;

Acquire information data of the operator, and search for the mobile terminal number of the operator according to the information data;

Establishing a communication connection with the operator through a mobile terminal based on the mobile terminal number;

When the communication connection is established, the manual operation notification instruction is played by voice and displayed on a preset display terminal.

In some embodiments, the method further comprises:

Determine the operation items and the operation sequence of the operation items according to the preset production control items;

Classifying the work tasks based on the work items and the work sequence to generate a work process type set;

A job item queue is constructed according to the job flow type set.

In some embodiments, the step of constructing a job item queue according to the job flow type set includes:

Extracting a plurality of operation processes and the operation items, operation sequence and initial conditions of each operation process according to the operation process type set;

Constructing a work task according to the work items, work sequence and initial conditions of the work flow; wherein each of the work tasks includes a plurality of work items;

A job item queue is constructed according to the job tasks.

In some embodiments, the method further comprises:

Determine the operation items according to the preset production control items;

The work items are classified according to their task execution attributes to generate a manual work item set and a feedback work item set.

In addition, to achieve the above-mentioned purpose, the present invention also proposes a human-machine collaborative management and control system involving logistics storage and transportation, comprising:

The operation task process control module is used to determine the target operation process according to the logistics storage and transportation task when receiving the logistics storage and transportation task;

The operation task process control module is further used to determine the target operation project queue corresponding to the target operation process according to the operation project queue;

The operation task process control module is also used to execute the logistics storage and transportation tasks according to the operation items and operation sequence of the target operation item queue;

The job task process control module is also used to determine whether the target job item queue belongs to the manual job item set;

The operation task process control module is further used to generate and issue a manual operation notification instruction when the target operation item queue belongs to the manual operation item set;

The operation task process control module is also used to determine whether a confirmation signal based on the manual operation notification instruction feedback is received;

A task status determination module, configured to update the status of the job item in the target job item queue to being executed upon receiving the confirmation signal;

The job task process control module is also used to determine whether the target job item queue belongs to the feedback job item set;

The job task process control module is also used to: When the manual feedback completion signal is received, it is determined whether the manual feedback completion signal is received;

The task status determination module is further configured to update the status of the work item in the target work item queue to completed when receiving the manual feedback completion signal;

The operation task process control module is also used to confirm that the logistics storage and transportation task is completed when the digital label of the operation project is the maximum digital label of the target operation project queue.

In addition, to achieve the above-mentioned objectives, the present invention also proposes a human-machine collaborative control device involving logistics storage and transportation, the human-machine collaborative control device involving logistics storage and transportation comprising: a memory, a processor, and a human-machine collaborative control program involving logistics storage and transportation stored on the memory and executable on the processor, the human-machine collaborative control program involving logistics storage and transportation being configured to implement the human-machine collaborative control method involving logistics storage and transportation as described above.

The present invention provides a human-machine collaborative management and control method involving logistics storage and transportation, comprising: upon receiving a logistics storage and transportation task, determining a target operation process according to the logistics storage and transportation task; determining a target operation project queue corresponding to the target operation process according to an operation project queue; executing the logistics storage and transportation task according to the operation items and operation sequence of the target operation project queue; judging whether the target operation project queue belongs to a manual operation project set; when the target operation project queue belongs to the manual operation project set, generating and issuing a manual operation notification instruction; judging whether a confirmation signal based on feedback of the manual operation notification instruction is received; upon receiving the confirmation signal, updating the status of the operation items in the target operation project queue to being executed; judging whether the target operation project queue belongs to a feedback operation project set; when the target operation project queue belongs to the feedback operation project set, judging whether a manual feedback completion signal is received; upon receiving the manual feedback completion signal, updating the status of the operation items in the target operation project queue to being completed; and when the digital label of the operation item is the maximum digital label of the target operation project queue, confirming that the logistics storage and transportation task is completed. In the present invention, the overall execution process of logistics storage and transportation tasks is intelligently controlled, operation instructions are sent to operators in a timely manner and manual feedback on the execution status is received, so as to coordinate the logistics storage and transportation operation process requiring manual participation, solving the technical problem that the operation process requiring manual participation in the existing logistics and transportation process in the steel industry is difficult to coordinate as a whole in a timely and accurate manner.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of the structure of a human-machine collaborative control device for logistics storage and transportation in a hardware operating environment according to an embodiment of the present invention;

FIG2 is a flow chart of a first embodiment of a human-machine collaborative control method for logistics storage and transportation according to the present invention;

FIG4 is a diagram of a human-machine collaborative control method for logistics storage and transportation according to the present invention;

FIG4 is a flow chart of a second embodiment of a human-machine collaborative control method for logistics storage and transportation according to the present invention;

FIG5 is a structural block diagram of the first embodiment of the human-machine collaborative control device for logistics storage and transportation according to the present invention.

The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings.

DETAILED DESCRIPTION

The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

In addition, the descriptions of "first", "second", etc. in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

Refer to Figure 1, which is a schematic diagram of the structure of a human-machine collaborative management and control device involving logistics storage and transportation in the hardware operating environment involved in an embodiment of the present invention.

As shown in FIG1 , the human-machine collaborative control device for logistics storage and transportation may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM memory) or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

Those skilled in the art will understand that the structure shown in FIG. 1 does not constitute a limitation on human-machine collaborative control equipment involved in logistics storage and transportation, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

As shown in FIG. 1 , the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a human-machine collaborative management and control program related to logistics storage and transportation.

In the human-machine collaborative control device involving logistics storage and transportation shown in Figure 1, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the human-machine collaborative control device involving logistics storage and transportation of the present invention can be set in the human-machine collaborative control device involving logistics storage and transportation, and the human-machine collaborative control device involving logistics storage and transportation calls the human-machine collaborative control program involving logistics storage and transportation stored in the memory 1005 through the processor 1001, and executes the human-machine collaborative control method involving logistics storage and transportation provided by an embodiment of the present invention.

An embodiment of the present invention provides a method for human-machine collaborative control of logistics storage and transportation. Referring to FIG. 2 , FIG. 2 is a flow chart of a first embodiment of a method for human-machine collaborative control of logistics storage and transportation according to the present invention.

As shown in FIG2 , the human-machine collaborative control method for logistics storage and transportation includes:

Step S10: upon receiving a logistics storage and transportation task, determining a target operation process according to the logistics storage and transportation task;

Step S20: determining the target operation item queue corresponding to the target operation process according to the operation item queue;

Step S30: executing the logistics storage and transportation task according to the operation items and operation sequence of the target operation item queue;

Step S40: determining whether the target operation item queue belongs to a manual operation item set;

Step S50: when the target operation item queue belongs to the manual operation item set, generating and issuing a manual operation notification instruction;

Step S60: determining whether a confirmation signal based on the manual operation notification instruction feedback is received;

Step S70: upon receiving the confirmation signal, updating the status of the operation item in the target operation item queue to being executed;

Step S80: determining whether the target operation item queue belongs to a feedback operation item set;

Step S90: when the target operation item queue belongs to the feedback operation item set, determining whether a manual feedback completion signal is received;

Step S100: when the manual feedback completion signal is received, updating the status of the work item in the target work item queue to completed;

Step S110: When the digital number of the operation item is the maximum digital number of the target operation item queue, confirm that the logistics storage and transportation task is completed.

It should be noted that the execution subject in this embodiment may be a human-machine collaborative control device involved in logistics storage and transportation. The human-machine collaborative control device involved in logistics storage and transportation may be a computer device with data processing functions, or other devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment, a computer device is used as an example for explanation.

By way of example, this embodiment takes the in-plant storage and transportation operation process of raw material procurement logistics of a steel enterprise as an example for explanation, and the mode of transportation is exemplarily automobile transportation.

In one embodiment, work items and the work order of the work items are determined according to preset production control items; work tasks are classified based on the work items and the work order to generate a set of work process types; and a work item queue is constructed according to the set of work process types.

Specifically, a set of operation flow types is established: each operation task is classified according to the operation items and their operation sequence included in the production control process, and an operation flow type set C _j is established; wherein 1≤j≤m, and m is the total number of types of all operation flows in the production control process.

In one embodiment, a job item queue is constructed according to the job process type set, including: extracting a number of job processes and the job items, job sequence and initial conditions of each of the job processes according to the job process type set; constructing job tasks according to the job items, job sequence and initial conditions of the job processes; wherein each of the job tasks includes a number of job items; and constructing a job item queue according to the job tasks.

Exemplarily, a task queue corresponding to each task flow type is established: a unique description standard is established for each type of task flow C _j (1≤j≤m), including all task items contained in the process, the order of execution of each task, and the initial conditions. For example, if task C _j contains n task items, a task queue C _ji is formed; where 1≤j≤m, 1≤i≤n.

In one embodiment, work items are determined according to preset production control items; and the work items are classified according to task execution attributes to generate a manual work item set and a feedback work item set.

Exemplarily, each work project is classified to establish a set of manual work projects T that need to wait for system notification before starting, and a set of feedback work projects Y that need to wait for manual feedback before confirmation of completion, and U is a set of all work projects.

Specifically, a set T of manual work items that need to wait for system notification before starting is established, T∈U. For the work items _Ti in the set T; where 1≤i≤k, it is assumed that the set T contains a total of k work items. When the work item _Ti in the set T is triggered to the "executable" state, the interactive application module can automatically issue a "start work" instruction to the on-site workers, notify the corresponding personnel to start the work, and receive feedback. Issue work instructions (such as voice, character or graphic type work instructions) through the mobile terminal, accept confirmation replies, and automatically update the work status. After confirming that the instruction is successfully issued, according to the state machine of the current task, the status of _Ti is updated to "executing".

Specifically, a set of work items that need to wait for manual feedback to confirm completion is established, namely, a feedback work item set Y, Y∈U. For work item _Yi in set Y; where 1≤i≤j, it is assumed that set Y contains a total of j jobs. When work item _Yi in set Y has been completed, the on-site worker wakes up the mobile terminal through voice commands or buttons, inputs feedback information, and can receive feedback information through the interactive application module. For example, the mobile terminal receives feedback information (such as voice and characters) from the on-site worker, automatically updates the work status, and confirms that the feedback information is successfully recognized. According to the state machine of the current task, the status of _Yi is updated to "Completed".

It should be noted that the state of a specific task can be adjusted through the job task state machine: for a specific job task W, according to its corresponding job flow type C _j (1≤j≤m), the corresponding job item queue C _ji (1≤j≤m, 1≤i≤n) can be determined. For each item in the job item queue C _ji , five states are set: "to be executed, executable, executing, completed, abnormal".

In actual applications, when the process starts, that is, when C _j1 meets the execution conditions, C _j1 is triggered to the "executable" state, and the C _ji state (1<i≤n) is set to "pending execution". For C _ji , (1<i≤n): when Cji is sensed to start execution, Cji is triggered to the "executing"state; when C _ji is sensed to be "completed", C _ji is triggered to the "completed"state; when C _ji (i<n) is in the "completed" state, C _j(i+1) is triggered to the "executable"state; when C _ji (i＝n) is in the "completed" state, the project status is set to "completed". When C _ji meets the triggering conditions of the above-mentioned "pending execution, executable, executing, completed" states, but does not enter the corresponding state within the set time, or the time in the "executable" or "executing" state exceeds the preset time, C _ji is triggered to the "abnormal" state. For C _ji , when i=n, when C _ji is in the "completed" state, it triggers the current project to be in the completed state.

It is understandable that establishing a multi-modal interactive information support dataset such as voice, pictures, and text for the information that humans need to interact with during the execution of tasks, that is, establishing a multi-modal human-computer interaction support dataset, is convenient for effectively improving the recognition accuracy and interaction efficiency of manual feedback information.

In practical applications, a multimodal human-computer interaction support dataset is established: for a job item C _ji (1≤i≤n), a unique job name instruction audio dataset N is established; a state instruction audio dataset S corresponding to the five job states of "to be executed, executable, executing, completed, and abnormal" is established; a feedback dataset is established, including a voice feedback keyword audio training dataset Ra, a picture feedback training dataset Rp, and a gesture feedback instruction dataset Rg; a voice query keyword audio dataset F is established; a job instruction keyword audio dataset O is established; an executor information set P (including the dialing number of a mobile terminal device) corresponding to each job is established; a mapping relationship dataset E between the operator and each specific job item is established to achieve the correspondence between each job C _i and its operator P _i (P _i ∈P) information. It should be noted that the specific content of the multimodal human-computer interaction support dataset includes but is not limited to the above-mentioned set, which can be set according to actual conditions.

It is understandable that an interactive information conversion module is established: for the data contained in the above sets N, S, Ra, Rp, Rg, F, O, P and E, efficient and reliable conversion between text and voice can be achieved. Character information can be converted into audio data in the above sets to form voice instructions, which can be played to on-site workers; audio information can also be converted into character information for operation process control through automatic recognition of voice instructions. Feedback pictures and gesture information can also be converted into character information for operation process control.

In one embodiment, when a logistics storage and transportation task is received, a target operation process is determined according to the logistics storage and transportation task, a target operation item queue corresponding to the target operation process is determined according to the operation item queue, and a target operation item queue corresponding to the target operation process is determined according to the target operation item queue. The logistics storage and transportation task is executed according to the operation items and operation sequence of the target queue: for a specific operation task, such as a vehicle driver receiving a certain operation and maintenance task in a waiting queue, the operation flow C _j corresponding to the operation and maintenance task is determined. For a specific operation task, its unique corresponding operation item queue C _ji is determined, _{and the variable i is initialized to 1. The i-th operation item in the execution queue C ji is} updated to "executable".

In one embodiment, it is determined whether the target operation project queue belongs to the manual operation project set; when the target operation project queue belongs to the manual operation project set, a manual operation notification instruction is generated and issued; it is determined whether a confirmation signal based on the manual operation notification instruction feedback is received; when the confirmation signal is received, the state of the operation project in the target operation project queue is updated to be executed: it is determined whether C _ji belongs to the set T (a manual operation project set that needs to wait for system notification to start), if C _ji belongs to the set T, the operation instruction is issued to the corresponding operator through the interactive control information conversion module and the multimodal interactive application module. If the corresponding operator feedback confirmation information is not received, the steps are repeated to issue the operation instruction to the corresponding operator through the interactive control information conversion module and the multimodal interactive application module; if the operator feedback confirmation information is still not received after the specified time, an abnormal prompt is issued; if the operator feedback confirmation information is received, the C _ji status is updated to "execution".

In one embodiment, when the target job item queue belongs to the manual job item set, a manual job notification instruction is generated and issued, including: when the target job item queue belongs to the manual job item set, determining the current job item of the target job item queue; obtaining the name audio data and status audio data corresponding to the current job item; obtaining the job instruction keyword corresponding to the current job item; automatically generating a manual job notification instruction based on the name audio data and status audio data combined with the job instruction keyword; and issuing the manual job notification instruction through a mobile terminal.

Specifically, C _ji belongs to the set T, and the i-th job item in the execution queue C _ji , when the job item _Ti in the set T is triggered to the "executable" state, the interactive control information conversion module and the multimodal interactive application module can automatically issue a "start job" instruction to the on-site workers, notify the corresponding personnel to start the job, and receive feedback. Issue job instructions (such as voice, character or graphic type job instructions) through the mobile terminal, accept confirmation replies, and automatically update the job status. After confirming that the instruction is successfully issued, according to the state machine of the current task, the status of _Ti is updated to "executing".

In another embodiment, after determining whether the target operation project queue belongs to the manual operation project set, it also includes: when the target operation project queue does not belong to the manual operation project set, automatically notifying the corresponding execution equipment through the industrial Internet of Things to start the operation; or, when the target operation project queue does not belong to the manual operation project set, automatically triggering the operation status corresponding to the operation project of the target operation project queue to be in execution, completed or faulty.

It can be understood that a set C of all job items other than the set T and the set Y is established, that is, when C does not belong to (T∪Y), for the job _Ci in the set C; where 1≤i≤n, it is assumed that the set C contains n jobs in total. According to the state machine of the current task, when _Ci is triggered to the "executable" state, the corresponding execution equipment can be automatically notified through the industrial Internet of Things to start the operation; _Ci can also be automatically triggered to the "executing", "completed" or "fault" state through the automatically collected basic automation system signals.

Specifically, if C _ji does not belong to the set T, an instruction is sent to the corresponding execution equipment to start automatic operation, and whether the operation has started is determined based on the automatically collected basic automation information. If the operation has started, the C _ji status is updated to "executing"; if the operation start signal is still not received within the specified time, an abnormal prompt is issued.

In one embodiment, it is determined whether the target job item queue belongs to a feedback job item set; when the target job item queue belongs to the feedback job item set, it is determined whether a manual feedback completion signal is received; when the manual feedback completion signal is received, the status of the job items in the target job item queue is updated to completed.

In one embodiment, after determining whether the target job item queue belongs to the feedback job item set, it also includes: when the target job item queue does not belong to the feedback job item set, determining whether an automated feedback completion signal is received; when the automated feedback completion signal is received, updating the status of the job items in the target job item queue to completed.

Specifically, determine whether C _ji belongs to set Y (the set Y of feedback operation items that need to wait for manual feedback to confirm completion). In one example, if C _ji belongs to set Y, determine whether the operation completion information of manual feedback is received. If received, the C _ji status is updated to "completed". If not received, continue to wait. If the operation completion information of the operator feedback is still not received after the specified time, an abnormal prompt is issued. In another example, if C _ji does not belong to set Y, determine whether the operation completion information of the automation system feedback is received. If received, the C _ji status is updated to "completed". If not received, continue to wait. If the operation completion information of the automation system feedback is still not received after the specified time, an abnormal prompt is issued.

It should be noted that when the digital label of the work item is the maximum digital label of the target work item queue, the logistics storage and transportation task is confirmed to be completed: if i<n (n is the total number of items in the queue C _ji ), then i=i+1, jump to the step execution queue C _ji , and update the C _ji status to "executable"; if i=n (n is the total number of items in the queue C _ji ), the task ends.

This embodiment determines the target operation process according to the logistics storage and transportation task when receiving the logistics storage and transportation task; determines the target operation project queue corresponding to the target operation process according to the operation project queue; executes the logistics storage and transportation task according to the operation project and operation sequence of the target operation project queue; determines whether the target operation project queue belongs to the manual operation project set; when the target operation project queue belongs to the manual operation project set, generates and issues a manual operation notification instruction; determines whether a confirmation signal based on the feedback of the manual operation notification instruction is received; when the confirmation signal is received, updates the status of the operation project in the target operation project queue to being executed; determines whether the target operation project queue belongs to the feedback operation project set; when the target operation project queue belongs to the feedback operation project set, determines whether a manual feedback completion signal is received; and when receiving the confirmation signal, When the completion signal is manually fed back, the status of the operation item in the target operation item queue is updated to completed; when the digital number of the operation item is the maximum digital number of the target operation item queue, the logistics storage and transportation task is confirmed to be completed. In this embodiment, the overall execution process of the logistics storage and transportation task is intelligently controlled, and operation instructions are sent to the operators in a timely manner and the execution status is received through manual feedback, so as to coordinate and control the logistics storage and transportation operation process that requires manual participation, solving the technical problem that the manual participation operation process in the existing steel industry logistics transportation process is difficult to coordinate in an overall manner in a timely and accurate manner.

In some embodiments, as shown in FIG3 , a second embodiment of the human-machine collaborative management and control method for logistics storage and transportation according to the present invention is proposed based on the first embodiment, and the step S50 includes:

Step S51: when the target operation item queue belongs to the manual operation item set, determining the current operation item of the target operation item queue;

Step S52: Acquire the name audio data and status audio data corresponding to the current operation item;

Step S53: obtaining the operation instruction keyword corresponding to the current operation item;

Step S54: Automatically generate a manual operation notification instruction based on the name audio data and the status audio data combined with the operation instruction keyword;

Step S55: Sending the manual operation notification instruction via the mobile terminal.

In one embodiment, the manual operation notification instruction is issued through a mobile terminal, including: determining the operator corresponding to the current operation project based on a preset mapping relationship; obtaining the information data of the operator, and searching for the mobile terminal number of the operator according to the information data; establishing a communication connection with the operator based on the mobile terminal number through the mobile terminal; when the communication connection is established, playing the manual operation notification instruction by voice and displaying the manual operation notification instruction on a preset display terminal.

Specifically, C _ji belongs to the set T, and the i-th job item in the execution queue C _ji , when the job item _Ti in the set T is triggered to the "executable" state, the interactive control information conversion module and the multimodal interactive application module can automatically issue a "start job" instruction to the on-site workers, notify the corresponding personnel to start the job, and receive feedback. Issue job instructions (such as voice, character or graphic type job instructions) through the mobile terminal, accept confirmation replies, and automatically update the job status. After confirming that the instruction is successfully issued, according to the state machine of the current task, the status of _Ti is updated to "executing".

It should be noted that a multimodal interactive application module is established to enable interaction between people and the management and control system through various forms such as voice commands, pictures, and gestures.

In one example, a mobile terminal sends a work instruction (including voice, characters and graphics), receives a confirmation reply, and automatically updates the work status: corresponding to the current work project C _ji , according to the project's corresponding name audio data N _i and status audio data S _i , combined with the work instruction keyword O _i , automatically generates a voice instruction "O _i +N _i +S _i " and a corresponding character/graphic instruction. The interactive module automatically maps the work project C _ji to the corresponding on-site operator P _i according to the mapping relationship E _i ; searches for the configured mobile terminal number according to P _i 's information data and automatically dials; after the call is connected, plays the voice instruction "O _i +N _i +S _i ", At the same time, the corresponding character/graphic instructions are displayed on the mobile terminal screen; finally, the character or voice confirmation information replied by the on-site operator is accepted. Identify character or voice reply information: For the voice reply information R _i , the voice reply content is automatically recognized by comparing with the voice feedback keyword audio data set R. If the character or voice reply information is recognized as confirmed successfully, the C _ji project status is automatically updated, and the updated status S _i is displayed through the screen interface of the mobile terminal, and the audio information S _i of the updated C _ji status is played at the same time. If the recognized character or voice reply information is not confirmed successfully, the operation instruction is automatically issued again, and the interactive module is repeatedly executed according to the mapping relationship E _i , and the operation project C _ji is automatically mapped to the corresponding on-site operator P _i ; according to the information data of P _{i ,} the configured mobile terminal number is searched and dialed automatically; after the connection is connected, the voice instruction "O _i +N _i +S _i " is played, and the corresponding character/graphic instruction is displayed on the mobile terminal screen; finally, the character or voice confirmation information replied by the on-site operator and the character or voice reply information recognition steps are accepted. If the recognition is still not successful after the set time and number of times, an abnormal state alarm is triggered.

For example, in actual applications: Scenario 1: In a logistics management system, vehicle drivers can obtain information about the operation of unloading materials immediately after entering the factory through voice commands played by the vehicle-mounted mobile terminal while waiting in line, and interact with the logistics management system through feedback confirmation information. Scenario 2: In an equipment operation and maintenance system, operation and maintenance personnel can obtain information about the operation of a certain operation and maintenance task immediately after starting the operation and maintenance task through voice commands played by mobile terminals such as head-mounted displays and PADs, or text commands displayed, and interact with the equipment operation and maintenance system through feedback confirmation information.

In another example, feedback information (including voice and characters) from on-site workers is received through a mobile terminal, and the operation status is automatically updated: Step 1. The on-site worker wakes up the mobile terminal through voice commands or buttons, and the mobile terminal sends a request to the interactive application module to obtain the operation project information corresponding to the worker. Step 2. After receiving the request from the mobile terminal, the interactive application module first verifies the identity of the worker P _i through the mobile terminal number and login information. The interactive module obtains the current operation project C _ji and the current status of C _ji corresponding to P _i according to the mapping relationship E _i , and sends the relevant information to the corresponding mobile terminal. Step 3. The on-site worker can view the current corresponding operation project and status through the mobile terminal, and input the feedback voice information according to the feedback instruction keyword; or input feedback information through the corresponding interface of the mobile terminal; or feedback the captured video or picture; or use gesture feedback. Then the feedback information is sent back to the interactive application module. Step 4. Identify and confirm the feedback information. According to the feedback data sets Ra, Rp, Rg, voice commands, image pictures, and gesture commands are recognized. If the recognition is successful, the C _ji operating status is updated according to the feedback content in combination with the current status of C _ji . The updated status is displayed through the screen interface of the mobile terminal, and the audio information of the updated C _ji status is played at the same time. Step 5. If the recognition in step 3 fails, and the on-site operator finds that the status is not updated normally, feedback can be provided again. After receiving the feedback information, the system repeats "steps 2" and "steps 3". Step 6. If the recognition is still not successful after exceeding the set time and number of times, an abnormal status alarm is triggered.

For example, in actual applications: Scenario 1: In a logistics management system, when a vehicle arrives at a manual operation location such as unloading or vehicle inspection, on-site workers can wake up terminals such as head-mounted displays and PADs through voice commands or buttons. The corresponding work items and status of the personnel are displayed on the terminal. After the operator confirms that everything is correct, the operator guides the camera device on the head display device to take photos of the specified parts such as the license plate and the carriage through gestures or voice, confirms the photo through voice or gestures, and sends the photo back to the interactive application module; or inputs feedback information such as "unloading completed" and "vehicle inspection results" through voice, and sends the feedback information back to the interactive application module after confirmation. Scenario 2: In the equipment operation and maintenance system, when the operation and maintenance personnel perform operations such as part replacement on site, they can wake up the head display, PAD and other terminals through voice commands or buttons. The corresponding work items and status of the personnel are displayed on the terminal. After the operator confirms that everything is correct, the operator guides the camera device on the head display device to take photos of the maintained equipment, spare parts to be replaced, key steps for replacing equipment, and damaged parts replaced through gestures or voice; confirms the photo through voice or gestures, and sends the photo back to the interactive application module. Or wake up the head display through voice commands, input feedback information such as "replacement completed" through voice, and send the feedback information back to the interactive application module after confirmation.

In another example, a mobile terminal receives a query (including voice and characters) from a field operator and responds: Step 1: The field operator wakes up the mobile terminal through voice commands or buttons, and inputs the query voice information or key characters with reference to the query keyword. Step 2: After receiving the query information, the interactive application module verifies the operator identity P _i through the mobile terminal number and login information, and obtains the information corresponding to P _i according to the mapping relationship E _i according to the interactive module. Step 3: Identify the query information. For the voice query information, the voice query content is automatically identified by comparing the voice query keyword audio data set Pa. Step 4: If the recognition is successful, the system retrieves the corresponding information in the database according to the query keyword, and converts the corresponding information into voice information and plays it through the mobile terminal, while providing relevant text and chart display. Step 5: If the recognition fails, the audio is played or a prompt message is displayed asking the operator to re-enter the query information. After receiving the query information, "step 2", "step 3" and "step 4" are repeated. Step 6: If the recognition is still unsuccessful after the set time and number of times, an abnormal state alarm is triggered.

For example, in actual applications: Scenario 1: In a logistics management system, warehouse managers can wake up mobile terminals such as head-mounted displays and PADs through voice commands or buttons, and then enter query instructions for the storage location of a certain type of goods through voice or text. Interactive application and other modules obtain query results through identity authentication, voice recognition, data retrieval and other processes, and then transmit the query results to the mobile terminal for playback or display. Scenario 2: In an equipment operation and maintenance system, operation and maintenance personnel can wake up mobile terminals such as head-mounted displays and PADs through voice commands or buttons, and then enter query instructions for the current operation project through voice or text. Interactive application and other modules obtain query results through identity authentication, voice recognition, data retrieval and other processes, and then transmit the query results to the mobile terminal for playback or display.

This embodiment determines the current operation project of the target operation project queue when the target operation project queue belongs to the manual operation project set; obtains the name audio data and status audio data corresponding to the current operation project; obtains the operation instruction keyword corresponding to the current operation project; automatically generates a manual operation notification instruction based on the name audio data and status audio data combined with the operation instruction keyword; and sends the manual operation notification instruction through a mobile terminal. In this embodiment, a multimodal interactive application module is established to interact with the control system through voice commands, pictures, gestures and other forms. The overall execution process of logistics storage and transportation tasks is intelligently controlled, and operation instructions are sent to operators in a timely manner and the execution status is received through manual feedback, so as to coordinate the logistics storage and transportation operation process that requires manual participation, and solve the technical problem that it is difficult to coordinate the operation process that requires manual participation in the existing steel industry logistics and transportation process in a timely and accurate manner.

4 , which is a structural block diagram of a first embodiment of a human-machine collaborative management and control system for logistics storage and transportation according to the present invention.

As shown in FIG4 , the human-machine collaborative management and control system involving logistics storage and transportation includes:

The operation task process control module 10 is used to determine the target operation process according to the logistics storage and transportation task when receiving the logistics storage and transportation task;

The operation task process control module 10 is further used to determine the target operation project queue corresponding to the target operation process according to the operation project queue;

The operation task process control module 10 is also used to execute the logistics storage and transportation task according to the operation items and operation sequence of the target operation item queue;

The operation task process control module 10 is also used to determine whether the target operation project queue belongs to the manual operation project set;

The operation task process control module 10 is further used to generate and issue a manual operation notification instruction when the target operation item queue belongs to the manual operation item set;

The operation task process control module 10 is also used to determine whether a confirmation signal based on the manual operation notification instruction feedback is received;

The task status determination module 20 is used to update the status of the operation item in the target operation item queue to being executed when receiving the confirmation signal;

The job task process control module 10 is also used to determine whether the target job item queue belongs to the feedback job item set;

The operation task process control module 10 is further used to determine whether a manual feedback completion signal is received when the target operation item queue belongs to the feedback operation item set;

The task status determination module 20 is further configured to update the status of the work item in the target work item queue to completed when receiving the manual feedback completion signal;

The operation task process control module 10 is also used to confirm that the logistics storage and transportation task is completed when the digital label of the operation project is the maximum digital label of the target operation project queue.

In this embodiment, a human-machine collaborative control system involving logistics storage and transportation includes an operation task process control module and a task status determination module. (Logistics storage and transportation) operation process is collaboratively controlled. The human-machine collaborative control system involving logistics storage and transportation can send operation instructions to operators in a timely manner through voice or mobile terminal display and receive manual feedback on the execution status, thereby solving the problem that the operation process involving manual participation is difficult to coordinate as a whole in a timely and accurate manner. It is understandable that the human-machine collaborative control system involving logistics storage and transportation can also include: an operation process control system (including a scheduling model for all waiting personnel in each step), a manual operation status collection system, a mobile application system, automatic voice content generation, a feedback recognition system, and a voice dialing system.

Exemplarily, referring to FIG5 , a human-machine collaborative management and control system involving logistics storage and transportation may include an operation task process management and control system, an operation task state machine, a multimodal human-machine interaction data set, an interactive control information conversion module, and a multimodal interactive application module.

Specifically, as shown in FIG5 , the task process control system: controls the execution process and status of the task according to the task items and sequence contained in the task, mainly including real-time automatic collection of system main information and issuing of task instructions. Task state machine: automatically determines and updates the current status of each task in real time according to the real-time information of the system, and automatically determines the task items that need to be executed in real time. Multimodal human-computer interaction data set: establishes a multimodal interactive information support data set such as voice, picture, and text for the information that humans need to interact with during the execution of the task, thereby effectively improving the recognition accuracy and interaction efficiency of human feedback information. Interactive control information conversion module: can convert system instructions into information such as images and voices that are easy for the task executors to recognize; can also recognize human feedback information such as voice, gestures, and pictures, and convert the recognition results into information that can be used by the system. Multimodal interactive application module: realizes real-time interaction between the task executors, production managers, and the control system through voice, graphics, gestures, photography, and video, mainly including the issuance of instructions, the reception of feedback information, and the query and issuance of task status.

This embodiment determines the target operation process according to the logistics storage and transportation task when a logistics storage and transportation task is received; determines the target operation project queue corresponding to the target operation process according to the operation project queue; executes the logistics storage and transportation task according to the operation items and operation sequence of the target operation project queue; determines whether the target operation project queue belongs to the manual operation project set; generates and issues a manual operation notification instruction when the target operation project queue belongs to the manual operation project set; determines whether a confirmation signal based on the feedback of the manual operation notification instruction is received; when the confirmation signal is received, updates the status of the operation items in the target operation project queue to being executed; determines whether the target operation project queue belongs to the feedback operation project set; when the target operation project queue belongs to the feedback operation project set, determines whether a manual feedback completion signal is received; when the manual feedback completion signal is received, updates the status of the operation items in the target operation project queue to completed; and confirms that the logistics storage and transportation task is completed when the digital label of the operation item is the maximum digital label of the target operation item queue. In this embodiment, the overall execution process of logistics storage and transportation tasks is intelligently controlled, work instructions are sent to operators in a timely manner, and manual feedback on the execution status is received, so as to coordinate the logistics storage and transportation operations that require manual participation, solving the technical problem that the manual participation operations in the existing steel industry logistics and transportation processes are difficult to coordinate in an accurate and timely manner.

In addition, for technical details that are not described in detail in the embodiment of the human-machine collaborative control system involving logistics storage and transportation, please refer to the human-machine collaborative control method involving logistics storage and transportation as described above provided in any embodiment of the present invention, and will not be repeated here.

It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present invention. In specific applications, technicians in this field can make settings as needed, and the present invention does not limit this.

It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as a read-only memory (ROM)/RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.

The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims (10)

A human-machine collaborative control method for logistics storage and transportation, characterized in that the human-machine collaborative control method for logistics storage and transportation comprises: When receiving a logistics storage and transportation task, determining a target operation process according to the logistics storage and transportation task; Determine the target operation item queue corresponding to the target operation process according to the operation item queue; Execute the logistics storage and transportation task according to the operation items and operation sequence of the target operation item queue; Determine whether the target operation item queue belongs to a manual operation item set; When the target operation item queue belongs to the manual operation item set, generating and issuing a manual operation notification instruction; Determining whether a confirmation signal based on the manual operation notification instruction feedback is received; When the confirmation signal is received, updating the status of the operation item in the target operation item queue to being executed; Determine whether the target job item queue belongs to a feedback job item set; When the target operation item queue belongs to the feedback operation item set, determining whether a manual feedback completion signal is received; When the manual feedback completion signal is received, the status of the work item in the target work item queue is updated to be completed; When the digital number of the operation item is the maximum digital number of the target operation item queue, it is confirmed that the logistics storage and transportation task is completed. The human-machine collaborative management and control method for logistics storage and transportation according to claim 1 is characterized in that after determining whether the target operation item queue belongs to the manual operation item set, it also includes: When the target operation item queue does not belong to the manual operation item set, the corresponding execution equipment is automatically notified to start the operation through the industrial Internet of Things; or, When the target operation item queue does not belong to the manual operation item set, the operation status corresponding to the operation item that automatically triggers the target operation item queue is in execution, completed or faulty. The human-machine collaborative management and control method involving logistics storage and transportation according to claim 1 is characterized in that after determining whether the target operation item queue belongs to the feedback operation item set, it also includes: When the target operation item queue does not belong to the feedback operation item set, determining whether an automated feedback completion signal is received; When the automation feedback completion signal is received, the status of the operation item in the target operation item queue is updated to be completed. The human-machine collaborative control method involving logistics storage and transportation according to claim 1 is characterized in that When the target operation item queue belongs to the manual operation item set, a manual operation notification instruction is generated and issued, including: When the target operation item queue belongs to the manual operation item set, determining a current operation item of the target operation item queue; Obtain name audio data and status audio data corresponding to the current job item; Obtaining a job instruction keyword corresponding to the current job item; Automatically generate a manual operation notification instruction based on the name audio data and the status audio data combined with the operation instruction keyword; The manual operation notification instruction is issued through a mobile terminal. The human-machine collaborative control method involving logistics storage and transportation according to claim 4 is characterized in that the manual operation notification instruction is issued through the mobile terminal, including: Determine the operator corresponding to the current operation project based on the preset mapping relationship; Acquire information data of the operator, and search for the mobile terminal number of the operator according to the information data; Establishing a communication connection with the operator through a mobile terminal based on the mobile terminal number; When the communication connection is established, the manual operation notification instruction is played by voice and displayed on a preset display terminal. The human-machine collaborative control method involving logistics storage and transportation according to claim 1, characterized in that the method further comprises: Determine the operation items and the operation sequence of the operation items according to the preset production control items; Classifying the work tasks based on the work items and the work sequence to generate a work process type set; A job item queue is constructed according to the job flow type set. The human-machine collaborative management and control method for logistics storage and transportation according to claim 6 is characterized in that the step of constructing a job item queue according to the job process type set comprises: Extracting a plurality of operation processes and the operation items, operation sequence and initial conditions of each operation process according to the operation process type set; Constructing a work task according to the work items, work sequence and initial conditions of the work flow; wherein each of the work tasks includes a plurality of work items; A job item queue is constructed according to the job tasks. The human-machine collaborative control method involving logistics storage and transportation according to claim 1, characterized in that the method further comprises: Determine the operation items according to the preset production control items; The work items are classified according to their task execution attributes to generate a manual work item set and a feedback work item set. A human-machine collaborative control system for logistics storage and transportation, characterized in that the human-machine collaborative control system for logistics storage and transportation comprises: The operation task process control module is used to determine the target operation process according to the logistics storage and transportation task when receiving the logistics storage and transportation task; The operation task process control module is further used to determine the target operation project queue corresponding to the target operation process according to the operation project queue; The operation task process control module is also used to execute the logistics storage and transportation tasks according to the operation items and operation sequence of the target operation item queue; The job task process control module is also used to determine whether the target job item queue belongs to the manual job item set; The operation task process control module is further used to generate and issue a manual operation notification instruction when the target operation item queue belongs to the manual operation item set; The operation task process control module is also used to determine whether a confirmation signal based on the manual operation notification instruction feedback is received; A task status determination module, configured to update the status of the job item in the target job item queue to being executed upon receiving the confirmation signal; The job task process control module is also used to determine whether the target job item queue belongs to the feedback job item set; The operation task process control module is further used to determine whether a manual feedback completion signal is received when the target operation item queue belongs to the feedback operation item set; The task status determination module is further configured to update the status of the work item in the target work item queue to completed when receiving the manual feedback completion signal; The operation task process control module is also used to confirm that the logistics storage and transportation task is completed when the digital label of the operation project is the maximum digital label of the target operation project queue. A human-machine collaborative control device involving logistics storage and transportation, characterized in that the human-machine collaborative control device involving logistics storage and transportation comprises: a memory, a processor, and a human-machine collaborative control program involving logistics storage and transportation stored in the memory and executable on the processor, the human-machine collaborative control program involving logistics storage and transportation being configured to implement the human-machine collaborative control method involving logistics storage and transportation as described in any one of claims 1 to 8.