[go: up one dir, main page]

WO2019187719A1 - Dispositif de traitement d'informations, procédé de traitement d'informations et programme - Google Patents

Dispositif de traitement d'informations, procédé de traitement d'informations et programme Download PDF

Info

Publication number
WO2019187719A1
WO2019187719A1 PCT/JP2019/005042 JP2019005042W WO2019187719A1 WO 2019187719 A1 WO2019187719 A1 WO 2019187719A1 JP 2019005042 W JP2019005042 W JP 2019005042W WO 2019187719 A1 WO2019187719 A1 WO 2019187719A1
Authority
WO
WIPO (PCT)
Prior art keywords
real
time
core
task
processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2019/005042
Other languages
English (en)
Japanese (ja)
Inventor
元 戸村
修祐 佐伯
琢麿 馬渕
健太 多田
佐藤　和美
寛 飯渕
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Corp
Original Assignee
Sony Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Corp filed Critical Sony Corp
Publication of WO2019187719A1 publication Critical patent/WO2019187719A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]

Definitions

  • the present disclosure relates to an information processing apparatus, an information processing method, and a program. More specifically, in an apparatus that can execute a plurality of processes in parallel using a plurality of cores (processors) such as a multi-core system, an information processing apparatus that realizes real-time performance of a real-time task, and information processing
  • the present invention relates to a method and a program.
  • each task is executed in parallel, so information with multiple processors, such as a multicore system capable of parallel processing by multiple cores (processors)
  • processors such as a multicore system capable of parallel processing by multiple cores (processors)
  • Tasks executed in such an information processing apparatus include tasks that require completion of processing up to a specified time, that is, real-time tasks that require real-time performance (real-time processing), and other tasks, that is, identification Non-real-time tasks (non-real-time processing) that do not require completion of processing until the specified time are mixed.
  • This shielding technique is set so that a dedicated core (real-time core) does not execute non-real-time processing, so that a dedicated core can reliably execute a real-time task when a real-time task occurs.
  • this configuration has a problem that it is necessary to set a dedicated core that executes only a real-time task, and is difficult to realize when the number of cores is small.
  • Patent Document 1 Japanese Patent Laid-Open No. 2005-157955 discloses a technique for disclosing a specific real-time task by a specified time without providing a dedicated core for executing a real-time task.
  • This Patent Document 1 calculates the processing time of a real-time task required by a processor, and if there is a surplus time before the specified completion time of the real-time task, causes the processor to execute another process and there is no room Is configured to execute only real-time tasks.
  • This configuration realizes a configuration that guarantees the real-time performance of the real-time task even in a configuration where the number of cores (processors) is not sufficient and a real-time dedicated core cannot be set.
  • this configuration has a problem that a lot of additional processing occurs, for example, it is necessary to calculate the processing time of the real-time task, and it is also necessary to set an interrupt for calculating the processing time. .
  • Another problem is that it is difficult to deal with real-time tasks that occur at unpredictable timing.
  • the present disclosure has been made in view of the above-described problems, for example, in a multi-core system or the like that uses a plurality of cores (processors) to execute a plurality of processes in parallel, and some cores are processed in real time.
  • Information processing apparatus, information processing method, and program capable of ensuring real-time performance of a real-time task without performing processing such as pre-calculating the processing time of each task The purpose is to provide.
  • the first aspect of the present disclosure is: Multiple cores that perform data processing; A task scheduler for scheduling tasks executed in the plurality of cores;
  • the plurality of cores includes a real-time core that executes a real-time task and a non-real-time core that executes a non-real-time task,
  • the task scheduler A task allocation process for causing the real-time core to execute not only a real-time task but also a non-real-time task;
  • the information processing apparatus executes migration for moving a non-real-time task assigned to the real-time core to another core in response to occurrence of a predetermined event.
  • the second aspect of the present disclosure is: An information processing method executed in an information processing apparatus,
  • the information processing apparatus includes: Multiple cores that perform data processing;
  • the plurality of cores includes a real-time core that executes a real-time task and a non-real-time core that executes a non-real-time task,
  • the task scheduler is A task allocation process for causing the real-time core to execute not only a real-time task but also a non-real-time task;
  • migration is performed in which a non-real-time task assigned to the real-time core is moved to another core in response to occurrence of a predetermined event.
  • the third aspect of the present disclosure is: A program for executing information processing in an information processing apparatus;
  • the information processing apparatus includes: Multiple cores that perform data processing;
  • the plurality of cores includes a real-time core that executes a real-time task and a non-real-time core that executes a non-real-time task,
  • the program is stored in the task scheduler. Causing the real-time core to execute a task assignment process for executing not only a real-time task but also a non-real-time task;
  • there is a program for executing a migration that moves a non-real-time task assigned to the real-time core to another core.
  • the program of the present disclosure is a program that can be provided by, for example, a storage medium or a communication medium provided in a computer-readable format to an information processing apparatus or a computer system that can execute various program codes.
  • a program in a computer-readable format, processing corresponding to the program is realized on the information processing apparatus or the computer system.
  • system is a logical set configuration of a plurality of devices, and is not limited to one in which the devices of each configuration are in the same casing.
  • a configuration in which non-real-time tasks are executed in the real-time core to improve processing efficiency and the real-time property of the real-time task is ensured is realized.
  • it has a real-time core that executes a real-time task, a plurality of cores configured by a non-real-time core that executes a non-real-time task, and a task scheduler that schedules the tasks of the plurality of cores.
  • the task scheduler executes task assignment that allows the real-time core to execute not only real-time tasks but also non-real-time tasks, and migrates non-real-time tasks assigned to real-time cores to other cores in response to the occurrence of a predefined event.
  • FIG. 25 is a diagram for describing an example hardware configuration of an information processing device.
  • a real-time task (real-time processing) will be described with reference to FIG.
  • an information processing apparatus having a plurality of cores (processors) such as a multi-core system in which a plurality of cores (processors) can be operated in parallel in order to execute a plurality of different tasks (processes) in parallel is used. .
  • A Processing that requires completion of processing up to a specified time, that is, a real-time task that requires real-time performance (real-time processing),
  • B Non-real-time task (non-real-time processing) that does not require completion of processing up to a specified time,
  • the time from the camera photographed image to the monitor (display unit) output is determined. If the delay occurs, the work may be delayed or the display may be distorted. At worst, it is impossible to work while looking at the front image of the monitor.
  • Image processing in this case basically needs to be executed as real-time processing.
  • An information processing apparatus that executes such image processing may perform, for example, a monitor output of a captured image and a process of recording the captured image in a storage unit such as a hard disk in parallel.
  • a monitor output of a captured image and a process of recording the captured image in a storage unit such as a hard disk in parallel.
  • Such processing is allowed to be executed as a non-real-time task (non-real-time processing).
  • the real-time task further includes the following two types of processing.
  • A1 Periodic processing a2) Aperiodic processing
  • Periodic processing is processing that occurs periodically in advance.
  • the above-described image processing for outputting the captured image of the camera to the monitor (display unit) is also periodic processing.
  • an image captured at a predetermined frame rate is cycled. It is necessary to perform processing periodically, which is a periodic process.
  • the camera captures images at various frame rates, such as 60 f / s for capturing an image of 60 frames per second and 120 f / s for capturing an image of 120 frames per second.
  • the core needs to perform periodic processing on each of the continuous images at the prescribed frame intervals.
  • non-periodic processing is to execute event-corresponding processing within a predetermined response time when a sudden event occurs.
  • focus adjustment is required when the focal length changes due to movement of a camera or subject during moving image shooting. If the focus adjustment cannot be completed within a predetermined time, only a blurred image can be obtained and a clear moving image cannot be taken.
  • camera shake correction process or the like if the correction process cannot be completed within a predetermined time after the camera shake is detected, the correction cannot be made in time and the purpose cannot be achieved.
  • Such processing is an aperiodic real-time task.
  • a periodic real-time task is required to execute one cycle of processing at a predetermined time, and an interrupt response-type real-time task is executed within a predetermined response time from the occurrence of an event such as the above-mentioned doctor operation. To complete the process.
  • FIG. 1 shows three examples of real-time task execution sequences by one core (processor) in an information processing apparatus having a plurality of cores (processors).
  • Te Processing completion specified time
  • Processing example 1 is an example in which processing within a specified time for a real-time task is successfully completed.
  • the processing by the core is started at time T1 after a predetermined delay time (latency), and the processing is ended at time T2.
  • the processing end time T2 is a time before the processing completion specified time Te, and this processing example 1 is a successful example of real-time processing.
  • Processing example 2 is a failure example of processing completion within a specified time of a real-time task.
  • the processing end time T2 is a time after the processing completion specified time Te, and this processing example 2 is an example of failure of real-time processing.
  • This processing example 2 is an example in which the processing time of the real-time task has become longer. This is because, for example, resources and data required for real-time task processing compete with resources and data used by other tasks (processing) executed by other cores, and there is a waiting time for acquiring resources and data. This is an example that can occur.
  • Processing example 3 is also an example of a process completion failure within a specified time of the real-time task.
  • the time from the process start time T1 to the process end time T2 is shorter than the process example 1, as a result, the process end time T2 is a time after the process completion specified time Te, and this process example 3 is also real time. This is an example of task processing failure.
  • the tasks executed in each core are various different tasks, and the required processing time is also different. Therefore, for example, in a configuration in which various tasks are executed using two cores, the balance of processing assigned to each core may deteriorate. Specifically, there is a case where the processing amount of the task assigned to one core is large, the processing amount of the task assigned to the other core is small, and an idle time occurs in the other core.
  • the task assigned to one core may be moved to the other core for processing.
  • Such task movement processing between cores is called migration.
  • FIG. 2 shows an example of migration in a configuration in which tasks (processes) are executed using two cores (CPUs). Time elapses in the order of time zones T10 to T20, T20 to T30, and T30 to T40 from the left to the right shown in the figure. The figure shows the task execution status of the two cores 1 (CPU 1) and 2 (CPU 2) in each time zone.
  • CPU 1 CPU 1
  • CPU 2 CPU 2
  • Core 1 executes task A, task B, and task C.
  • the core 1 (CPU 1) executes these three tasks as, for example, time division processing.
  • the size of the task A to C area of the core 1 (CPU 1) shown at (time T10 to T20) corresponds to the distribution ratio of the processing time of each task in the core 1 (CPU 1).
  • the core 2 executes only the task D and there is a free time during which no processing is performed.
  • the total processing time for example, the time until all tasks of tasks A to D are completed. Will become longer.
  • Core 1 (CPU 1) executes only task A and task C.
  • the core 2 (CPU 2) executes task D and task B. Note that arrows in the figure at times T30 to T40 mean that the distribution of processing time of each task in each core can be expanded.
  • the core 1 (CPU 1) can use the processing time of the task B, which has been executed during the time T10 to T20, as the processing time of the task A and the task C.
  • the core 2 (CPU 2) can use all of the free time in the times T10 to T20 as the processing time of the task B that has moved by migration.
  • FIG. 3 shows an example of shield processing for a multi-core system having four cores, that is, four cores, core 1 (CPU 1) to core 4 (CPU 4).
  • the second core 2 (CPU 2) from the left is set as a real-time core dedicated to real-time processing. That is, the core 2 (CPU 2) is shielded (isolated) and set as a real-time processing-dedicated core.
  • Other cores other than the core 2 (CPU 2) are non-real-time cores that execute non-real-time processing.
  • CPU 2 Only a real-time task is assigned to the core 2 (CPU 2) which is a real-time core.
  • Other cores (core 1 (CPU 1), core 3 (CPU 3), and core 4 (CPU 4)) other than core 2 (CPU 2) execute various applications (App) that are non-real-time tasks.
  • Each core also executes a kernel thread.
  • the kernel thread corresponds to a part of kernel processing that executes processing such as management of resources and memory necessary for task execution in each core and task switching.
  • the kernel is software that is the core of the OS (operating system).
  • Core 2 which is a real-time core, receives only an interrupt from a device necessary for processing a real-time task, device B shown in the figure. Interrupts to the real-time core from other devices are prohibited.
  • other cores core 1 (CPU 1), core 3 (CPU 3), core 4 (CPU 4)
  • core 2 CPU 2
  • the device is a device that requires a processing result by the core, and is configured by various devices such as a GPU (graphic processing unit), a hard disk (HD), and a USB.
  • a GPU graphics processing unit
  • HD hard disk
  • USB USB
  • the device B is the interrupt-permitting device for the core 2 (CPU 2), which is a real-time core, but there may be a plurality of devices.
  • the real-time task executed in the core 2 (CPU 2) is processing for monitor output of a captured image, as in the above-described example.
  • the device B is, for example, a GPU (graphic processing unit).
  • the core 2 which is a real-time core, executes generation of image processing parameters necessary for an image processing process in a GPU, for example.
  • Core 2 (CPU2).
  • the GPU uses the generation parameters of the core 2 (CPU 2) to generate an image for monitor output. These series of processes need to be executed without time delay in order to immediately output the camera-captured image to the monitor. That is, the image parameter generation process executed in the core 2 (CPU 2) is a real-time task.
  • the shield process with the two processes (1) and (2) above allows the core 2 (CPU 2), which is a real-time core, to execute a real-time task without being interrupted by other tasks or interrupts. Can be completed in time. That is, processing that guarantees real-time performance is possible.
  • CPU 2 CPU 2
  • the configuration of the present disclosure is a configuration that executes a process to which a new shielding technique is applied.
  • the shielded core that executes real-time processing is also configured to execute a non-real-time task as long as the real-time task executed in the core can be completed within a specified time.
  • the shield process of the core that executes the real-time task is a gentle shield process. That is, the real-time task execution core is also configured to execute a non-real-time task depending on the situation. As described above, by executing the non-real-time task even in the execution core of the real-time task, it becomes possible to improve the processing efficiency of the entire apparatus even in a configuration without a sufficient number of cores.
  • FIG. 4 is a diagram illustrating a conventional strict shield configuration
  • FIG. 5 is a diagram illustrating a gradual shield configuration of the present disclosure.
  • FIG. 4 is the same as the configuration described above with reference to FIG. 3, and is a multi-core system configuration having four cores, that is, four cores, core 1 (CPU 1) to core 4 (CPU 4). is there.
  • RT real-time task
  • CPU 2 CPU 2
  • CPU 3 core 3
  • CPU 4 core 4
  • non-RT non-real-time task
  • each core also executes a kernel thread, as described above with reference to FIG.
  • the kernel thread is a process corresponding to the kernel, which is the core software of the OS (operating system), management of resources and memory necessary for task execution in each core, and processing such as task switching Is included.
  • the four cores shown in FIG. 4, that is, core 1 (CPU 1) to core 4 (CPU 4), are processes corresponding to various devices such as a GPU (graphic processing unit), a hard disk (HD), and a USB. Execute.
  • the core 2 (CPU 2), which is a real-time core isolated by strict shielding processing, receives only an interrupt from a device necessary for processing a real-time task, and executes only processing corresponding to the device. Interrupts from other devices are prohibited for the core 2 (CPU 2), which is a real-time core isolated by strict shielding processing.
  • Interrupts from various devices are allowed for cores other than core 2 (CPU 2) (core 1 (CPU 1), core 3 (CPU 3), core 4 (CPU 4)).
  • the core 2 (CPU 2), which is a real-time core isolated by the strict shielding process shown in FIG. 4, has the following strict shielding in order to guarantee the real-time property of the real-time task, that is, the task completion up to a specified time. Processing is performed. (1) Only real-time tasks are assigned to real-time cores. (2) The real-time core allows only interrupts from devices necessary for processing real-time tasks, and prohibits interrupts from other devices.
  • the core 2 which is a real-time core, can execute a real-time task without being interrupted by other tasks or interrupts. Can be completed within a specified time. That is, processing that ensures real-time performance is possible.
  • the real-time core 2 (CPU 2) can execute only the real-time task (RT). Unassigned and cannot be executed. As a result, when many non-real-time tasks occur, a small number of non-real-time cores must execute all these non-real-time tasks, which causes a problem that the processing delay of the non-real-time tasks increases.
  • FIG. 5 A configuration of the present disclosure that solves such a problem, that is, a configuration of the present disclosure that performs gentle shielding processing will be described with reference to FIG. 5.
  • the configuration shown in FIG. 5 is the same as the configuration described with reference to FIG. 4, and is a multi-core system configuration having four cores, that is, four cores of core 1 (CPU 1) to core 4 (CPU 4).
  • the gentle shield (isolation) processing of the present disclosure is executed for the second core 2 (CPU 2) from the left, and the core 2 (CPU 2) is changed to the real-time task (RT).
  • RT real-time task
  • the core 2 (CPU 2) which is a real-time core, executes not only a real-time task (RT) but also a non-real-time task (non-RT).
  • non-real-time tasks that are running in the real-time core may not be processed (real-time inhibition processing) until the specified time (real-time performance) of the real-time task (RT) running in the real-time core is hindered. If it is set as an execution scheduled process of (non-RT), the non-real-time task (non-RT) is moved (migrated) to another core.
  • Non-RT non-real time cores that execute non-real time tasks
  • This pre-defined condition means that processing (real-time inhibition processing) that may prevent processing completion (real-time performance) up to the specified time of the real-time task (RT) being executed in the real-time core is This is the detection that the non-real-time task (non-RT) being executed is set as the scheduled execution process. When this condition occurs, the non-real-time task (non-RT) is moved (migrated) to another core.
  • each core also executes a kernel thread as described above with reference to FIG. 3.
  • the kernel thread is a process corresponding to the kernel, which is the core software of the OS (operating system), and manages resources and memory necessary for task execution in each core, and also performs processing such as task switching. is there.
  • the four cores shown in FIG. 5, that is, core 1 (CPU 1) to core 4 (CPU 4), are processes corresponding to various devices such as GPU (graphic processing unit), hard disk (HD), USB, and the like. Execute.
  • Core 2 (CPU 2), which is a real-time core isolated by gentle shield processing, receives not only interrupts from devices necessary for real-time task processing but also non-real-time processing interrupts without being prohibited.
  • the core 2 (CPU 2) which is a real-time core isolated by the strict shielding process shown in FIG. 4 described above, is prohibited from interrupting from other devices, but is isolated by the gentle shielding process shown in FIG.
  • the core 2 (CPU 2) which is a real-time core, is allowed to interrupt from other devices.
  • Other cores (core 1 (CPU 1), core 3 (CPU 3), core 4 (CPU 4)) other than core 2 (CPU 2) are allowed to interrupt from various devices.
  • the core 2 (CPU 2), which is a real-time core isolated by the gentle shield process shown in FIG. 5, performs a shield process with the following settings in order to guarantee the real-time property of the real-time process, that is, the completion of the process up to a specified time. .
  • a real-time task is assigned to a real-time core.
  • a non-real time task is also assigned to the real time core.
  • Non-real-time tasks (non-real-time processing) that are running in the real-time core non-real-time inhibition processing) that may interfere with processing completion (real-time performance) of the real-time task that is running in the real-time core. If it is set to the execution schedule process of (RT), the non-real-time task (non-RT) is moved (migrated) to another core.
  • the core 2 (CPU 2), which is a real-time core, can complete the real-time task within a specified time by the gentle shielding process involving the three processes (1), (2), and (3). That is, it is possible to process a real-time task that ensures real-time performance.
  • Non-real-time processing can be executed, the time until completion of non-real-time processing can be shortened, and the processing efficiency of the entire apparatus can be improved.
  • a gentle shield process is executed for a real-time core, and a real-time task and a non-real-time task are also assigned to the real-time core.
  • non-real-time tasks (non-RT) running in the real-time core may not be processed (real-time inhibition processing) that may prevent processing completion (real-time performance) of the real-time task running in the real-time core.
  • the non-real-time task (non-RT) is moved (migrated) to another core.
  • Task assignment to a plurality of cores and task migration (migration) are executed in a kernel which is a partial function of the OS.
  • a kernel which is a partial function of the OS.
  • FIG. 6 illustrates a multi-core 20 configured by a plurality of cores (CPUs) configuring an information processing apparatus of the present disclosure and a device group 10 configured by a plurality of devices.
  • the device group 10 includes various devices. Specifically, it is a device that requires a processing result by any one of the cores in the multi-core 20, and is configured by various devices such as a GPU (graphic processing unit), a hard disk (HD), and a USB.
  • GPU graphics processing unit
  • HD hard disk
  • USB USB
  • the multi-core 20 includes two or more cores (CPUs) as hardware.
  • FIG. 6 shows a software configuration (software stack) executed by a core (CPU) that is hardware in the multi-core 20.
  • a kernel (OS) 30 as a base stack and a plurality of tasks as its upper stack are shown.
  • the plurality of tasks include a number of tasks such as a real-time task (RT) 52 in which a process completion deadline and a processing start time are defined, and other non-real-time tasks (non-RT) 51, 53, and 54. These are executed in each of a plurality of cores constituting the multi-core 20.
  • RT real-time task
  • non-RT non-real-time tasks
  • a kernel thread is executed in each core constituting the multi-core 20. This is the kernel thread described above with reference to FIG.
  • the kernel thread is a process corresponding to the kernel that is the core software of the OS (operating system), and includes processing such as management of resources and memory necessary for task execution in each core, and task switching.
  • the kernel also has a function of a task scheduler 31.
  • the task scheduler 31 executes task scheduling processing such as task assignment to each core constituting the multi-core 20 and task movement processing between each core. These task scheduling processes are also part of the kernel thread executed in each core.
  • the task scheduler 31 of the kernel (OS) 30 shown in FIG. 6 executes processing such as task assignment to each core constituting the multi-core 20 and task movement between the cores.
  • the task scheduler 31 also executes the task movement process (migration) described above with reference to FIG.
  • processing real-time inhibition processing
  • processing completion real-time performance
  • non-RT real-time performance
  • the non-real-time task is moved (migrated) to another core.
  • Step S101 The task scheduler 31 first executes a normal task scheduling process in step S101. Specifically, task scheduling is executed in which real-time tasks are assigned to real-time cores, and non-real-time tasks are assigned according to free times of all cores including real-time cores and non-real-time cores.
  • step S102 the task scheduler 31 performs processing (real-time inhibition processing) that affects the real-time property of the real-time task being executed in the real-time core (the processing completion certainty up to the specified time) by the real-time core. It is determined whether or not the non-real-time task (non-RT) being executed is set as a scheduled execution process.
  • processing real-time inhibition processing
  • the task scheduler 31 monitors the processes being executed in all the cores, and further monitors and manages the schedules of all processes such as process requests from all devices and interrupt process requests.
  • the task scheduler 31 includes a process newly scheduled as an execution scheduled process of a non-real-time task (non-RT) being executed in the real-time core. “Processing that affects the real-time performance of the real-time processing being executed in the real-time core (process completion certainty up to the specified time) (real-time inhibition processing)” The process which determines whether it is is performed is performed.
  • processing that affects the real-time performance of the real-time processing being executed in the real-time core processes completion certainty up to the specified time
  • real-time inhibition processing For example, a process that causes an interruption of a real-time process being executed in the real-time core or the like.
  • the system call is an output process of instructions and functions issued by the OS (kernel) (for example, instructions and functions for providing and using functions for tasks).
  • the interrupt prohibition process is an interrupt prohibition process forcibly executed by interrupting a process being executed.
  • the preemption prohibition process is a preemption prohibition process for temporarily interrupting a task being executed.
  • step S102 the task scheduler 31 executes a non-real-time task (non-RT) in which a real-time inhibition process such as the processes (1) to (3) shown in FIG. It is determined whether or not the processing is set.
  • non-RT non-real-time task
  • step S102 determines whether the real-time inhibition process is set as the execution scheduled process for the non-real-time task (non-RT) being executed in the real-time core. If the determination in step S102 is Yes, that is, if it is determined that the real-time inhibition process is set as the execution scheduled process for the non-real-time task (non-RT) being executed in the real-time core, the process proceeds to step S103. Otherwise, the process returns to step S101, and normal task scheduling processing is continued.
  • Step S103 If the determination in step S102 is Yes, that is, if it is determined that the real-time inhibition process is set as the execution scheduled process for the non-real-time task (non-RT) being executed in the real-time core, the process proceeds to step S103.
  • the task scheduler 31 performs processing for moving (migrating) the non-real-time task (non-RT) set as the execution scheduled processing to the real-time inhibition processing to another core.
  • the real-time core that is executing the real-time task does not need to execute the non-real-time task that is scheduled to execute the real-time inhibition process, without causing an unexpected interruption of the real-time task, Real-time tasks can be completed by the scheduled time. That is, a real-time task that secures real-time performance can be executed.
  • the embodiment described below is an endoscope system that captures 4K images.
  • the 4K image is a high-quality image having high density pixels of 3840 ⁇ 2160 pixels, for example.
  • the endoscope system captures 120 frames per second of a 4K image and performs a process of executing a process of displaying the frame on a monitor.
  • an image processing apparatus that processes an image photographed by an endoscope must continuously input a 120 f / s image and continuously generate an image for output to a monitor. Don't be.
  • FIG. 9 shows a configuration example of the endoscope system 100.
  • the endoscope system 100 includes a camera device 110, an image processing device 120, and a monitor 140.
  • the camera 110 captures a 4K image at a frame rate of 120 f / s and inputs the captured image to the image processing apparatus 120.
  • the image processing apparatus 120 performs image processing on a 120 f / s image input from the camera 110, that is, processing on each of 120 captured images per second, and generates an output image to be output to the monitor 140. .
  • the output image generated by the image processing apparatus 120 is output to the monitor 140. For example, a doctor performs an appropriate treatment while viewing the output image of the monitor 140.
  • the image output to the monitor 140 needs to be a real-time image, and a large time delay is not allowed. Therefore, the image processing apparatus 120 needs to input an image with a frame rate of 120 f / s from the camera 110, perform image processing for each frame without time delay, and generate an output image of 120 frames per second. is there. This image generation process is a real-time task.
  • the image processing apparatus 120 includes a multi-core 130 having a plurality of cores (CPUs) and a device group 140 composed of a plurality of devices.
  • CPUs central processing units
  • device group 140 composed of a plurality of devices.
  • the multi-core 130 has two cores, a core 1 (CPU 1) 131 and a core 2 (CPU 2) 132.
  • the device group 140 includes a plurality of devices A (GPU) 141, device B (camera) 142, device C (USB) 143,. These devices are devices that require a processing result by the core, and are constituted by various devices such as a GPU (graphic processing unit), a camera, and a USB.
  • the core 1 (CPU 1) 131 that is a real-time core executes generation of image processing parameters necessary for an image processing process in the device A (GPU) 141, for example.
  • the core 1 (CPU 1) 131 is.
  • the generated parameter is provided to the device A (GPU) 141.
  • the device A (GPU) 141 generates an image for monitor output using the generation parameters of the core 1 (CPU 1).
  • the core 1 (CPU 1) 131 and the device A (GPU) 141 which are real-time cores, need to repeatedly perform image processing in units of one frame at intervals of 8.3 ms.
  • the sequence diagram shown in FIG. 10 shows the processing sequence of one frame image.
  • an image input interrupt corresponding to an image processing request is input from the device B (camera) 142 to the core 1 (CPU 1) 131 which is a real-time core.
  • control for inputting an image input interrupt to the core 1 (CPU 1) 131, which is a real-time core, is executed by a kernel task scheduler.
  • the core 1 (CPU 1) 131 which is a real-time core, generates image processing parameters used for actual processing, specifically, image processing in the GPU, from time t12 after a predetermined delay time. Start processing. This process is a real-time task, and the time until the process is completed is defined.
  • the core 1 (CPU 1) 131 which is a real-time core, completes the image processing parameter generation process at time t13 and outputs the generated image processing parameter to the device A (GPU) 141. Thereafter, the device A (GPU) 141 generates an output image and outputs it to the monitor 140 until time T2.
  • the core 1 (CPU 1) 131 which is a real-time core, is set to be allowed to execute only real-time tasks, and the task is executed for a time from time t13 to time T2. Therefore, it was necessary to wait until the next frame was processed.
  • the core 1 (CPU 1) 131 that is a real-time core is allowed to execute not only a real-time task but also a non-real-time task. Therefore, after the time t13, it is possible to execute the non-real-time task processing for the time up to the time T2.
  • the configuration having two cores enables parallel processing of non-real-time tasks using two cores, and the processing efficiency jumps compared to a configuration in which only one core executes non-real-time tasks. Can be increased.
  • the core 1 (CPU 1) 131 which is a real-time core, can also execute a non-real-time task in a time-sharing manner in addition to the real-time task that is the image processing parameter generation process even during the period of time t12 to t13. .
  • the multi-core 301 includes a plurality of cores (CPU: Central Processing Unit). As shown in FIG. 11, the multi-core 301 has at least two cores such as a core 1 (CPU 1) 351, a core 2 (CPU 2) 352, a core 3 (CPU 3) 353, and the like.
  • CPU Central Processing Unit
  • At least one of the cores in the multi-core 301 is set as a real-time core whose processing is limited by the gentle shield processing described in the above-described embodiment.
  • the shield process of the core that executes the real-time task is a gentle shield process, and the execution of the non-real-time task is allowed even in the real-time core depending on the situation.
  • non-real-time tasks (non-RT) running in the real-time core may not be processed (real-time inhibition processing) that may prevent processing completion (real-time performance) of the real-time task running in the real-time core.
  • real-time inhibition processing may prevent processing completion (real-time performance) of the real-time task running in the real-time core.
  • the real-time core can complete the real-time task within the specified time by performing a gentle shield process that is set to perform task migration (migration) under the above conditions for the real-time core. In other words, processing that ensures real-time performance of the real-time task is possible.
  • a GPU (Graphic Processing Unit) 302 is an image-dedicated processor that executes image processing. As described above, image processing using a parameter generated by the CPU is performed.
  • a ROM (Read Only Memory) 303 is used as a storage area for programs and parameters executed by the multi-core 301 and the GPU 302.
  • a RAM (Random Access Memory) 304 is used as a work area for processing executed by the multi-core 301 and the GPU 302, a parameter storage area, a recording area for other data, and the like.
  • These multi-core 301, GPU 302, ROM 303, and RAM 304 are mutually connected by a bus 305.
  • the multi-core 301 and the GPU 302 are connected to an input / output interface 306 via a bus 305.
  • the input / output interface 306 includes various switches, a keyboard, a touch panel, a mouse, a microphone, and a data acquisition unit such as a sensor and a camera.
  • An input unit 307, a display such as a monitor, and an output unit 309 including a speaker are connected.
  • the multi-core 301 receives commands, status data, and the like input from the input unit 307, executes various processes, and outputs the processing results to the output unit 308, for example.
  • the storage unit 309 connected to the input / output interface 306 includes, for example, a hard disk and stores programs executed by the multi-core 301 and various data.
  • the communication unit 310 functions as a data transmission / reception unit via a network such as the Internet or a local area network, and communicates with an external device.
  • the drive 311 connected to the input / output interface 306 drives a removable medium 312 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory such as a memory card, and executes data recording or reading.
  • a removable medium 312 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory such as a memory card
  • the technology disclosed in this specification can take the following configurations.
  • the plurality of cores includes a real-time core that executes a real-time task and a non-real-time core that executes a non-real-time task,
  • the task scheduler A task allocation process for causing the real-time core to execute not only a real-time task but also a non-real-time task;
  • An information processing apparatus that executes migration for moving a non-real-time task assigned to the real-time core to another core in response to occurrence of a predetermined event.
  • the task scheduler Real-time inhibition processing which is a process that may prevent processing completion (real-time performance) of a real-time task being executed in the real-time core from being executed, is scheduled to be executed for a non-real-time task being executed in the real-time core.
  • the real-time inhibition process is: The information processing apparatus according to (2), wherein the information processing apparatus is one of a system call, an interrupt prohibition process, a preemption prohibition process, a CPU exception process, or a cache lock.
  • the task scheduler Assign a real-time task only to the real-time core and execute it in the real-time core,
  • the information processing apparatus according to any one of (1) to (3), wherein a non-real-time task is assigned to the non-real-time core and the real-time core, and task assignment processing is executed by a plurality of cores.
  • each of the plurality of cores is a processor.
  • the information processing apparatus It is a configuration that executes image processing for inputting an image captured by a camera and generating an output image to be displayed on the display unit,
  • the real-time core is The image processing device according to any one of (1) to (5), wherein generation of a parameter provided to a GPU (Graphic Processing Unit) that executes the image processing is executed as the real-time task.
  • a GPU Graphic Processing Unit
  • the real-time core is The information processing apparatus according to (6), wherein the parameter generation processing corresponding to one image frame is executed as a real-time task, and the non-real-time task is executed in a free time thereafter.
  • An information processing method executed in the information processing apparatus includes: Multiple cores that perform data processing; A task scheduler for scheduling tasks executed in the plurality of cores; The plurality of cores includes a real-time core that executes a real-time task and a non-real-time core that executes a non-real-time task, The task scheduler is A task allocation process for causing the real-time core to execute not only a real-time task but also a non-real-time task; An information processing method for executing migration for moving a non-real-time task assigned to the real-time core to another core in response to occurrence of a predetermined event.
  • a program for executing information processing in an information processing device includes: Multiple cores that perform data processing; A task scheduler for scheduling tasks executed in the plurality of cores; The plurality of cores includes a real-time core that executes a real-time task and a non-real-time core that executes a non-real-time task, The program is stored in the task scheduler. Causing the real-time core to execute a task assignment process for executing not only a real-time task but also a non-real-time task; A program for executing migration to move a non-real-time task assigned to the real-time core to another core in response to occurrence of a predetermined event.
  • the series of processes described in the specification can be executed by hardware, software, or a combined configuration of both.
  • the program recording the processing sequence is installed in a memory in a computer incorporated in dedicated hardware and executed, or the program is executed on a general-purpose computer capable of executing various processing. It can be installed and run.
  • the program can be recorded in advance on a recording medium.
  • the program can be received via a network such as a LAN (Local Area Network) or the Internet and installed on a recording medium such as a built-in hard disk.
  • the various processes described in the specification are not only executed in time series according to the description, but may be executed in parallel or individually according to the processing capability of the apparatus that executes the processes or as necessary.
  • the system is a logical set configuration of a plurality of devices, and the devices of each configuration are not limited to being in the same casing.
  • a configuration in which a non-real-time task is executed in a real-time core to improve the processing efficiency and the real-time property of the real-time task is ensured is realized.
  • it has a real-time core that executes a real-time task, a plurality of cores configured by a non-real-time core that executes a non-real-time task, and a task scheduler that schedules the tasks of the plurality of cores.
  • the task scheduler executes task assignment that allows the real-time core to execute not only real-time tasks but also non-real-time tasks, and migrates non-real-time tasks assigned to real-time cores to other cores in response to the occurrence of a predefined event. Do. With this configuration, a non-real-time task is executed in the real-time core to improve processing efficiency, and a configuration that ensures the real-time property of the real-time task is realized.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)

Abstract

La présente invention permet d'obtenir une structure dans laquelle des tâches en différé sont effectuées dans un cœur en temps réel pour augmenter l'efficacité de traitement, et la propriété en temps réel de tâches en temps réel est garantie. La présente invention comprend : une pluralité de cœurs qui sont structurés à partir d'un cœur en temps réel effectuant des tâches en temps réel et un cœur en différé effectuant des tâches en différé ; et un planificateur de tâches qui programme des tâches pour la pluralité de cœurs. Le planificateur de tâches réalise une attribution de tâche pour amener le cœur en temps réel à effectuer non seulement des tâches en temps réel mais également des tâches en différé, et en réponse à l'apparition d'un événement préalablement prescrit, effectue une migration pour transférer une tâche en différé attribuée au cœur en temps réel à un noyau différent.
PCT/JP2019/005042 2018-03-28 2019-02-13 Dispositif de traitement d'informations, procédé de traitement d'informations et programme Ceased WO2019187719A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2018061340 2018-03-28
JP2018-061340 2018-03-28

Publications (1)

Publication Number Publication Date
WO2019187719A1 true WO2019187719A1 (fr) 2019-10-03

Family

ID=68058992

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/005042 Ceased WO2019187719A1 (fr) 2018-03-28 2019-02-13 Dispositif de traitement d'informations, procédé de traitement d'informations et programme

Country Status (1)

Country Link
WO (1) WO2019187719A1 (fr)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115080280A (zh) * 2022-08-16 2022-09-20 南方电网数字电网研究院有限公司 一种多时间尺度数据融合的边缘计算平台控制系统和方法
WO2023071643A1 (fr) * 2021-10-29 2023-05-04 华为技术有限公司 Procédé et appareil pour traiter une tâche, dispositif électronique et support
JP2024502329A (ja) * 2021-01-07 2024-01-18 インターナショナル・ビジネス・マシーンズ・コーポレーション 動的タスク移動を介したクラスタにまたがるエッジ・タイム・シェアリング
WO2024164701A1 (fr) * 2023-02-10 2024-08-15 中兴通讯股份有限公司 Procédé de planification de tâches, dispositif à plusieurs cœurs et support lisible par ordinateur
CN120540824A (zh) * 2025-07-29 2025-08-26 重庆长安汽车股份有限公司 一种调度系统、一种任务的调度方法、电子设备和介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005157955A (ja) * 2003-11-28 2005-06-16 Internatl Business Mach Corp <Ibm> 資源予約システムおよび資源予約方法および該方法を実行するためのプログラムが記録された記録媒体
JP2005190238A (ja) * 2003-12-26 2005-07-14 Matsushita Electric Ind Co Ltd リアルタイム制御方式
JP2013533524A (ja) * 2010-08-11 2013-08-22 日本電気株式会社 マルチプロセッサ・システムのためのプライマリ−バックアップに基づくフォールト・トレラント方法
US20130342713A1 (en) * 2012-03-20 2013-12-26 Xi'an Lianke Information Technology Co., Ltd. Cloud service based intelligent photographic method, device and mobile terminal
JP2014164635A (ja) * 2013-02-27 2014-09-08 Nec Computertechno Ltd リソース管理装置、リソースの管理方法、及びプログラム
JP2018045638A (ja) * 2016-09-16 2018-03-22 株式会社東芝 情報処理装置、情報処理方法及びプログラム

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005157955A (ja) * 2003-11-28 2005-06-16 Internatl Business Mach Corp <Ibm> 資源予約システムおよび資源予約方法および該方法を実行するためのプログラムが記録された記録媒体
JP2005190238A (ja) * 2003-12-26 2005-07-14 Matsushita Electric Ind Co Ltd リアルタイム制御方式
JP2013533524A (ja) * 2010-08-11 2013-08-22 日本電気株式会社 マルチプロセッサ・システムのためのプライマリ−バックアップに基づくフォールト・トレラント方法
US20130342713A1 (en) * 2012-03-20 2013-12-26 Xi'an Lianke Information Technology Co., Ltd. Cloud service based intelligent photographic method, device and mobile terminal
JP2014164635A (ja) * 2013-02-27 2014-09-08 Nec Computertechno Ltd リソース管理装置、リソースの管理方法、及びプログラム
JP2018045638A (ja) * 2016-09-16 2018-03-22 株式会社東芝 情報処理装置、情報処理方法及びプログラム

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2024502329A (ja) * 2021-01-07 2024-01-18 インターナショナル・ビジネス・マシーンズ・コーポレーション 動的タスク移動を介したクラスタにまたがるエッジ・タイム・シェアリング
WO2023071643A1 (fr) * 2021-10-29 2023-05-04 华为技术有限公司 Procédé et appareil pour traiter une tâche, dispositif électronique et support
CN115080280A (zh) * 2022-08-16 2022-09-20 南方电网数字电网研究院有限公司 一种多时间尺度数据融合的边缘计算平台控制系统和方法
CN115080280B (zh) * 2022-08-16 2022-12-27 南方电网数字电网研究院有限公司 一种多时间尺度数据融合的边缘计算平台控制系统和方法
WO2024164701A1 (fr) * 2023-02-10 2024-08-15 中兴通讯股份有限公司 Procédé de planification de tâches, dispositif à plusieurs cœurs et support lisible par ordinateur
CN120540824A (zh) * 2025-07-29 2025-08-26 重庆长安汽车股份有限公司 一种调度系统、一种任务的调度方法、电子设备和介质

Similar Documents

Publication Publication Date Title
WO2019187719A1 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations et programme
US11507420B2 (en) Systems and methods for scheduling tasks using sliding time windows
US8161491B2 (en) Soft real-time load balancer
EP2885707B1 (fr) Interruption logicielle sensible à la latence et planification de fil
EP3425502A1 (fr) Procédé et dispositif de planification de tâche
CN101501645A (zh) 计算机微作业
JP2021526257A (ja) マルチカーネル波面スケジューラ
WO2010082244A1 (fr) Dispositif et procédé de traitement d&#39;information
Elliott et al. Supporting real-time computer vision workloads using OpenVX on multicore+ GPU platforms
KR101392584B1 (ko) 리소스 모니터링을 이용한 동적 데이터 처리 장치 및 그 방법
US20050132038A1 (en) Resource reservation system and resource reservation method and recording medium storing program for executing the method
JP6372262B2 (ja) 印刷装置、およびプログラム
JP7671742B2 (ja) マルチスレッドマイクロプロセッサにおける共有リソース割り当て
CN109766168B (zh) 任务调度方法和装置、存储介质以及计算设备
JP2010287046A (ja) リソース配分システム、リソース配分方法及びリソース配分プログラム
US20050066093A1 (en) Real-time processor system and control method
JP5726006B2 (ja) タスクおよびリソースのスケジューリング装置及びその方法並びに制御装置
JP5299869B2 (ja) コンピュータマイクロジョブ
JP2011203428A (ja) 画像形成装置およびそのプログラム
EP2905703A1 (fr) Système informatique parallèle, procédé de commande d&#39;un système informatique parallèle et support de stockage lisible par ordinateur
JP4048638B2 (ja) スケジューリング装置及び方法並びに記録媒体
CN117806789B (zh) 多核操作系统的任务处理方法、装置及计算设备
CN114035926B (zh) 应用线程调度方法、装置、存储介质及电子设备
Souto et al. Overhead-aware schedulability evaluation of semi-partitioned real-time schedulers
JP2004234546A (ja) プログラム実行方法および装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19777107

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19777107

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP