WO2022003729A1

WO2022003729A1 - System and method for telestrating the operative procedures

Info

Publication number: WO2022003729A1
Application number: PCT/IN2021/050645
Authority: WO
Inventors: Charudutt PARANJAPE; Rahul DHARMADHIKARI
Original assignee: Cnr Medimentor Private Ltd
Current assignee: Cnr Medimentor Private Ltd
Priority date: 2020-07-02
Filing date: 2021-07-02
Publication date: 2022-01-06
Anticipated expiration: 2023-01-02

Abstract

The present invention system and method for telestrating the operative procedures provide an online platform and interface for telestrating the operative surgery or a diagnostic procedure to a larger audience. The system comprises of an interface for audio visual feedback, real time streaming, playing the existing streaming, playing the archived video, recording the real time streaming. The method of telestration comprises of multiple digital markers such as colors, shapes, text and critical situation markers which could be added as a layer on the real time streamed audio-visual content while the real time video is paused and the real time video stream is played away from the paused frame as a subset. The present invention has industrial application but not limited to telemedicine, real time mentoring and education of the pupils all over the globe.

Description

Field of Invention:

The present invention relates to remote real time operative procedures video collaboration for the telestration purposes. More particularly, the present invention relates to real time telestration on operative procedures video for effective mentoring of group of mentees by group of mentors by bringing the experts and the learners together on a virtual platform to provide real time, precise and focused audiovisual feedback complemented with digital markers which can be added as a layer over the real time streaming of operative procedures.

Background of Invention

Procedural Learning in medicine and in operative procedures has been following the traditional ways of since physicians and surgeons still practice some variation of “see one - do one - teach one” methodology. These currently practiced methods have many inherent flaws and most of the complications of a learner happens during and under the learning curve. All seasoned and trainee proceduralists have to learn new procedures while patients undergo the challenges such as slower pace of operative procedures, since the learning curves of the learner is slow and possibly unsafe ways to teach the learners. Larger health systems face the challenges of standardizing techniques and maintaining high quality and safety across the system. Having access to experts enhances the access to these procedures in many areas of the globe in emergency situations as well as elective procedures. Today’s learners are technologically savvy but the traditional method of learning does not allow them to innovate their learning process.

A paper published in Journal of Neurosurgery (JNS) titled “Virtual interactive presence for real time, long-distance surgical collaboration during complex microsurgical procedure” on August 2014 highlights the development of novel technologies that geographically extend the capabilities of individual surgeons and enhance surgical skills. The authors have developed “Virtual Interactive Presence” (VIP), a platform that allows remote participants to simultaneously view each other's visual field, creating a shared field of view for real-time surgical telecollaboration and concludes that participants were able to visually and verbally interact while identifying anatomical structures, guiding surgical maneuvers, and discussing overall surgical strategy. The paper recites the importance of telecollaboration while performing an operative surgery in the situations when there is shortage of surgeons performing the actual surgery and finds its application in surgical training programs, remote proctoring for proficiency and expert support for rural settings.

“5G-assisted telementored surgery” is review literature published on 4 September 2019 in Wiley Online Library recites both the safety and effectiveness of this form of surgical training and supervision, as well as the technical aspects, applicability and cost/benefit ratio of this technological solution.

The same concepts of telementored surgery may be applied to all types of procedures be it operative or diagnostic in nature.

Different stakeholders have been trying to find out a cost-effective solution to the existing problem of imparting the knowledge by proceduralists as well as incorporate the relevant experience in the operative procedures, new as well as prevailing, while the patient is being operated remotely. However, the drawbacks in these solutions are that the accurately pin pointing the area in a given field of view with the help of digital markers that could be actually added while the real time streaming of the operative surgery is happening such that the learners receive the real-time audio visual feedback complemented with digital markers which can be added as a layer over the real time streaming of operative procedures. The digital markers can be selected from text markers, shape markers, color markers and other critical situation markers.

Hence there is an unmet need for a platform to make these learning curves shorter, faster and safer by bringing the experts and the learners together on a virtual platform to provide real time, precise and focused audiovisual feedback complemented with digital markers which can be added as a layer over the real time streaming of operative procedures and also create a community of experts and learners to make the procedures safer, quicker and accessible.

The present invention overcomes the drawbacks of the existing practices and allows seamless standardization to maintain high standards of safety and quality by providing instant access to experts and providing key performance data points to improve the deficiencies.

Objects of the Invention

The primary object of the present invention is to provide a system and method for telestrating the operative procedures. Another object of the present invention is to provide a faster and efficient system and method for bringing experts and the learners together on a virtual platform to provide real time, precise and focused audio-visual feedback on operative procedures.

Yet another object of the present invention is to provide a system and method for real-time two- way communication on the virtual platform with the help of audio-visual feedback, digital markers and critical situation markers.

Still another object of the present invention is to provide a system and method for real-time streaming of the operative procedures for learning purposes.

Summary of the Invention

The system and method of the present invention provides safe, easy and technology assisted method to benefit the patients, learners, experts, health systems and companies. The current platform allows learners to access mentors/ proceduralists across the globe, obtain instant, precise and focused audiovisual feedback complemented with the digital markers which can be added as a layer over the real time streaming of operative procedures in real time to expedite their learning curve and archive their procedures for future reviews and finish the given operative procedures in stipulated time.

In one of the embodiments of the present invention provides a system by which any of such operative procedures can be attended by multiple of learners and proceduralists at the same time over LAN of the host or WAN or on separate servers for providing the critical improvisations, interventions and suggestions through the complementary digital markers which can be added as a layer over the real time streaming of operative procedures. The present invention allows the all the participants to draw, enter text and incorporate other critical situation digital markers in the real time streaming wherein the that video frame gets paused and is shown as frame in frame or as subset picture while the main streaming keeps on playing in the main interface ready to receive new inputs such as digital markers, audio visual feedback and other feedbacks enables by the user.

In one of the embodiment of the present invention provides a method by which any of the such operative procedures which are streamed in real time with critical suggestions and audio visual feedback complemented with the said digital markers can be recorded and archived and referred to in future operative procedures to perform the surgeries accurately and efficiently so that the learners learning curve is minimized and the patient also doesn’t suffer or feel overwhelmed with the time taken for the surgeries.

Detailed description of the invention

In the following description, a preferred embodiment of the invention is described with regard to system and method elements. However, those skilled in the art would recognize, after reading this application, that alternate embodiments of the invention may be implemented with regard to hardware or software without requiring undue invention.

Accordingly, the present invention provides a system and a method for telestrating the operative surgery to a larger audience. The system comprises of an interface for audio visual feedback, real time streaming, playing the existing streaming, playing the archived video, recording the real time streaming. The user interface also provides the options of creating plurality of sessions and inviting the other users to join the sessions. The system also has an option to have real time video feed from multiple of sources and playing the real time feed of choice and toggle between the sources.

The interface of the system of telestrating the operative procedures further comprise of multiple digital markers such as colors, shapes, text and critical situation markers which could be added as a layer on the real time streamed audio-visual content while the real time video is paused and the real time video stream is played away from the paused frame as a subset. The system and interface of the present invention is connected with servers and network connections in order to provide desired output for the intended beneficiaries i.e. proceduralist carrying out actual operational procedure, an expert who is examining the entire procedure and an expert who is instructing the entire procedure remotely.

The system of the present invention for telestrating the operative procedure is enabled with a method of using the system for monitoring the operating room activity. The system may consist of one or more mentors and mentees and a platform enabled by the system.

The method of enabling the above-mentioned system may consists of an administrator or a user, plurality of mentors, plurality of mentees, single or plurality of cameras at the operation room and plurality of servers connected over LAN or WAN. The method further consists of setting up system the which involves one of the administrators/ users:

Creating users or mentors/ mentees or single or plurality of users OR User registering themselves on the system

Defining the room with name, description of event/ surgery, one or more cameras

Defining the session with session name, description of procedure / surgery, mentor room name, mentor/ user, mentee/ user, mentee, room, date and time, steps of procedure.

Completing the setup

Joining the session through a web base application where plurality of mentors and mentees can join.

For Drawing on the real time video

Using the system interface, mentor to draw on the real time video output

Client sends pause command to server; server sends pause command to all the other clients / participants (mentees or mentors)

Each client pauses the real time video, take a snapshot and shows the snapshot overlaying the video.

Client send snapshot to server, server sends snapshot to all other clients, all other clients replaces the snapshot.

User adds another set of digital markers selected from text markers, shape markers, colour markers and other critical situation markers.

Client sends command(s) to server, server sends command(s) to other mentees, other clients replicate the action.

For resuming the real time video

User presses resume button. Client hides the images and shifts to real time video and sends command to play/resume, server sends commands to all other mentees, mentees hides the images and shows video. User clicks on video

Client brings the video to centre of interface and sends switch command to server, server sends command to all other clients, client switches the video.

For opening the audio channel

Client opens audio channel to audio server and sends “start audio” command to server, server sends “start audio” to all other mentees, mentees connects to audio server User clicks on broadcast button, client opens connect to broadcast server and starts transmitting video.

For recording the real time video

User presses record button Prompts for actions

Client informs server with recording, server stores recording parameters in database

Client starts stream to server, server stores video

Client stops recording video and sends stop command to server.

Server stops recording and marks recording completed.

For recording the real time video on client

User presses record button

Client starts recording in memory and browser storage User presses stop button

Client stops recording video and shows save to file dialog

User selects folder

Client saves file in selected folder Brief description of the drawing

Figure 1 shows logical flowchart for the administrator session setup

Figure 2 shows the logical flowchart for two-way communication between the mentor and mentee with the help of server

Figure 3 shows the real-time audio-visual playback on the application.

Figure 4 represents the logical flowchart for the real-time recording of the operative procedures on the server.

Figure 5 represents the interface layout for telestrating the operative procedures.

Figure 6 represents interface layout of the real time paused video and addition of digital marker arrow as shown in the figure.

Figure 7 represents the interface layout of the paused image in on main layout while the real time video is played in the subset of the main layout.

Figure 8 represents plurality audio visual feeds teletstrated in the interface for effective monitoring or mentoring of operative procedures.

Advantages:

• Proceduralists gain significant benefits from the current platform as the current platform allows them to guide the learners remotely, saving on time and energy to travel.

• Platform gives experts the ability to make available their expertise to a much broader audience across the globe.

• Proceduralists can showcase their procedures in a easy to leam, teletstrated, stepwise process.

• Proceduralists can mentor junior doctors / residents without the need to be physically present.

• As the procedures can be monitored and recorded, it helps to improve the quality of the procedures. Learners can access expertise at short notice

Claims

We claim

1. A system and method for telestrating the operative procedures, wherein the system comprising: an input device means for capturing the operative procedure in visual format, means for communication for two-way remote communication coupled with audio visual feedback, a central processing unit for processing the audio-visual input, a computing device selected from a laptop, mobile phone or a tablet for providing the audio-visual feedback remotely, an interface for facilitating the recording, retrieval, playback, pause, draw annotations, adding plurality of video feeds in real time in the telestration, a means for authentication, coordination and storage

A networking means for connecting plurality of said computing devices for enabling remote communication; and

Plurality of users including mentors and mentees participating in the telestration via the said interface.

2. A system and method for telestrating the operative procedures as claimed in claim 1, wherein, the input device means for capturing the operative procedure in visual format is selected from video output from medical devices, video recorder, close circuit camera, a smart camera, machine vision camera, a DSLR camera coupled with the existing operative setup.

3. A system and method for telestrating the operative procedures as claimed in claim 1, wherein the said means for communication is two-way remote communication such that said plurality of users can communicate over the distance with the help of said networking means selected from local, LAN, wired internet or mobile internet.

4. A system and method for telestrating the operative procedures as claimed in claim 1, wherein, said interface for facilitating the recording, retrieval, playback, pause, draw annotations, adding plurality of video feeds in real time can be accessed with the help of said computing device selected from a laptop, desktop, all-in-one computer, mobile phone, a tablet or custom-built computing device for providing the audio-visual feedback in real time.

5. A system and method for telestrating the operative procedures as claimed in claim 1, wherein the said interface for facilitating the recording, retrieval, playback, pause, draw annotations, adding plurality of video feeds in real time comprises of plurality of digital markers such as colors, shapes, images, text and critical situation markers to be added as a layer on the real time streamed audio visual content while the real time video may optionally be paused and the real time video telestration is continued in a subset in the same interface away from the paused frame of said telestration.

6. A system and method for telestrating the operative procedures as claimed in claim 1, wherein said means for authentication and coordination is selected from web server, real time communication server, streaming server and means for storage is selected from database, cloud storage and file storage and combination of either of the means for authentication, coordination and storage.

7. A system and method for telestrating the operative procedures as claimed in claim 1, wherein the said plurality of users are connected with the help of said networking and said interface for telestrating an operative surgery in presence of the plurality of mentors and mentees in such a manner that the plurality of mentors can utilize the said interface by recording, retrieval, playback, pause, draw annotations for guiding the mentees with the help of said plurality digital markers such as colors, shapes, text and critical situation markers.

8. A system and method for telestrating the operative procedures as claimed in claim 1, wherein, the said interface comprises of main video playback interface, plurality of video playback, plurality of controls for the said plurality of input devices, a provision for pausing the telestration for drawing annotation with help of said digital markers selected from colors, shapes, text and critical situation markers to guide the said plurality of mentees by said plurality of mentors.

9. A system and method for telestrating the operative procedures as claimed in claim 1, wherein the said interface allows to pause the said telestration and add said digital marker and to play the real time video telestration in a subset in the same interface away from the paused frame of said telestration.

10. A system and method for telestrating the operative procedures as claimed in claim 1, wherein the method for telestrating the operative procedures comprise the steps of: a. Creating users as mentors / mentees - single or plurality of users b. Defining the session with operative procedure name, description of operative procedure, said mentor, said mentee, steps of operative procedure. c. Logging in the session through web-based application having access through said computing devices. d. Starting the telestration session with the help of said input device means for capturing the operative and diagnostic procedure in visual format. e. Using the said interface to annotate on the real time video output with the help of said digital markers including but not limited to colors, shapes, text and critical situation markers. f. Sending pause command to server for communicating with said plurality of mentors and mentees g. Pausing the video and telestrating the operative procedure in a subset in the same interface. h. Using the said interface to annotate on the real time video output with the help of said another set of digital markers including but not limited to colors, shapes, text and critical situation markers. i. Sending the same command to server and to other plurality of mentees and replicating the action. j. Resuming the button through said interface’s resume button k. Hiding all the images of snapshot (frozen screens) and shifting to real time telestration. l. Clicking on another telestration being made available in the said interface for real time playback and repeating the steps of (a) to (1) above for continuous telestration of plurality of video playbacks, annotation with said digital markers and pausing the real time telestration. m. Recording and storing the recording on the plurality of computing devices and alternatively on said means for authentication, coordination and storage.