WO2025235385A1

WO2025235385A1 - Retractor-array objects and uses thereof

Info

Publication number: WO2025235385A1
Application number: PCT/US2025/027762
Authority: WO
Inventors: Timothy Alton
Original assignee: Individual
Current assignee: Individual
Priority date: 2024-05-06
Filing date: 2025-05-05
Publication date: 2025-11-13
Anticipated expiration: 2026-11-06

Abstract

A method for configuring a boundary includes imaging locational markers in a surgical environment. The markers are of a retractor-array object and are physically connected to a retractor portion to retract an anatomical structure. The method determines locations of the markers in 3D space of the surgical environment, and, based on the locations and a known size, shape, and position of the retractor portion relative to the locations, determines a location of the retractor portion in the 3D space. The method relates, in a surgical navigation system, the determined location of the retractor portion in the 3D space to a location of an anatomical structure of the patient in the 3D space, and, based on the relating, configures a boundary for the surgical navigation system. The boundary is in locational proximity to the anatomical structure for automatically triggering an action based on an object reaching or exceeding the boundary.

Description

RETRACTOR-ARRAY OBJECTS AND USES THEREOF

BACKGROUND

[0001] Aspects described herein relate to surgical applications, and more specifically to surgical instruments and tracking thereof during surgical procedures.

SUMMARY

[0002] Shortcomings of the prior art are overcome and additional advantages are provided through the provision of a computer-implemented method. The method images, by at least one imaging device, locational markers in a surgical environment. The locational markers are of a retractor-array object and are physically connected to a retractor portion configured for retracting an anatomical structure of a patient of the surgical environment. The method also determines locations of the locational markers in three-dimensional space of the surgical environment based on the imaging. Then based on the determined locations of the locational markers, a known size and shape of the retractor portion, and a known position of the retractor portion relative to the detected locations of the locational markers, the method determines a location of the retractor portion in the three-dimensional space. The method also relates, in a surgical navigation system, the determined location of the retractor portion in the three- dimensional space to a location of an anatomical structure of the patient in the three- dimensional space, and based on this relating, configures a boundary for the surgical navigation system. The boundary is in locational proximity to the anatomical structure for automatically triggering an action based on an object reaching or exceeding the boundary.

[0003] In one or more embodiments, the boundary is configured at a perimeter or periphery of the retractor portion such that the action is triggered based on the object reaching the perimeter or periphery of the retractor portion.

[0004] In one or more embodiments, the action includes control of a robotic surgical system to control a surgical action taken by the robotic surgical system. In one or more embodiments, the surgical action includes movement and/or operation of a surgical tool controlled by the robotic surgical system. In one or more embodiments, the surgical tool is a surgical saw. In one or more embodiments, the surgical action includes cutting by the surgical saw. In one or more embodiments, the control of the surgical action includes (i) slowing or stopping cutting by the surgical saw and/or (ii) movement of the surgical saw away from the boundary.

[0005] In one or more embodiments, the action includes a providing an audio and/or visual alert. In one or more embodiments, the action includes providing haptic feedback to a user. In one or more embodiments, the action includes triggering a mechanical interlock configured to prevent injury to the anatomical structure.

[0006] In one or more embodiments, the method further includes building and presenting a graphical interface for display. The graphical interface includes a virtual representation of patient anatomy including the anatomical structure and a graphical element representing the retractor-array object in the graphical interface. The building positionally relates the virtual representation of the patient anatomy and the graphical element in the graphical interface based on, and corresponding to, the related location of the retractor portion in the three-dimensional space to the location of the anatomical structure of the patient in the three-dimensional space to visually reflect an accurate positioning of the retractor portion related to the anatomical structure in the three dimensional space.

[0007] In one or more embodiments, configuration of the retractor portion for retracting the anatomical structure includes curvature of the retractor portion for curvature at least partially around other patient anatomy based on insertion of the retractor portion into the patient in an anterior-to-posterior trajectory, wherein the anatomical structure is located posterior to the other patient anatomy. In one or more embodiments, the curvature provides a tip of the retractor portion in proximity to the anatomical structure posterior to the other patient anatomy based on operative position of the retractor portion in the patient. In one or more embodiments, the anatomical structure is a neurovascular bundle of the patient posterior to the patient tibia or femur.

[0008] In one or more embodiments, the object is the retractor-array object and wherein the boundary is set to trigger an alert based on the retractor-array object reaching the boundary. [0009] In accordance with one or more aspects, each of the embodiments noted above and herein are separable and optional from one another. Further, embodiments may be combined with one another.

[0010] In a further embodiment, a computer system is provided that includes a memory and a processing circuit in communication with the memory, where the computer system is configured to perform any method/embodiment above or herein.

[0011] In yet a further embodiment, a computer program product is provided that includes a computer readable storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing any method/embodiment above or herein.

[0012] The present summary is not intended to illustrate each aspect of, every implementation of, and/or every embodiment of the present disclosure. Additional features and advantages are realized through the concepts described herein.

BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Aspects described herein are particularly pointed out and distinctly claimed as examples in the claims at the conclusion of the specification. The foregoing and other objects, features, and advantages of the disclosure are apparent from the following detailed description taken in conjunction with the accompanying drawings in which:

[0014] FIG. 1 depicts an example showing retractor-array objects in accordance with aspects described herein operatively positioned for use during a surgery;

[0015] FIG. 2 depicts another example showing a retractor-array object in accordance with aspects described herein operatively positioned for use during a surgery;

[0016] FIG. 3 depicts an example system for imaging a retractor-array object and providing a graphical depiction of the retractor-array object relative to a patient anatomical model, in accordance with aspects described herein;

[0017] FIGS. 4A-4B depict another example of a retractor-array object in accordance with aspects described herein; [0018] FIG. 5 depicts an example retractor-array object in accordance with aspects described herein to facilitate rod placement for a spinal fusion procedure;

[0019] FIG. 6 depicts an example of navigated rod placement in a spinal fusion procedure, in accordance with aspects described herein;

[0020] FIG. 7 depicts an example process for configuring a boundary, in accordance with aspects described herein; and

[0021] FIG. 8 depicts an example computer system to perform aspects described herein.

DETAILED DESCRIPTION

[0022] A retractor is a surgical instrument (also referred to herein as a device or object) used in surgery to move, position, align, or protect tissue during a surgical procedure, such as an operation. For instance, the retractor might separate the edges of a surgical incision/wound or hold certain organs and tissues so that anatomy underneath may be accessed during the surgical procedure. A retractor can be any shape, material, or size and can be either hand-held or connected to another object, for instance a patient, surgical table, other instrument, tool, or navigation system. A navigation system can be one that performs surgery on a body part in any specialty of medicine.

[0023] An array can be any form of device that communicates with and/or provides locational markers for a navigation system. An array may include a set of fiducials and/or ‘markers’ that are imaged or otherwise detected so that their locations in three-dimensional space (physical, ‘real-world’ space) can be determined. Such objects are often metal ball-like structures connected by metal arms and placed in a field of view of an imaging system to provide reference points. Locations of other objects in the images may be determined relative to these markers and used for various tasks, for instance robotic navigation, among others.

[0024] Aspects described herein provide retractor-array devices/objects, in which, in embodiments, an array is connected to (coupled to, affixed to, or the like) a retractor device. In other examples, the retractor and array may be a single, unitary object and/or integrally manufactured and sold as a single unit. In any case, the array interacts with a navigation system and provides the navigation system with spatial information serving as a basis for relating to the retractor to determine the location of the retractor in physical space and based on the known shape of the retractor.

[0025] As an example, a visual array is connected to a retractor, such as one commonly used in total knee replacement. By way of specific example, the retractor may be “Z” shaped and used to protect anatomy, such as soft tissue like the medial collateral ligament (MCL) as one example. In this instance, the retractor may be positioned around the posterior tibia, protecting the neurovascular structures behind the knee. The navigation system could then determine exactly where the whole retractor is located by way of registering the retractor’s shape/position based on the known shape of the retractor and on determining the retractor’s position relative to the detected position of the array to which the retractor is affixed. Possible benefits would enable the navigation system or other system (such as a robotic control system responsible for controlling robotic operation, if different than the navigation system) to determine anatomic safe areas and control operation of tools, such as a surgical saw, based on that knowledge of exactly where the retractor is. For example, the system could stop the saw from cutting before reaching/passing the location of the retractor. This would increase safety and decrease iatrogenic injuries, as examples. The system could also determine whether a retractor itself is coming too close to protected anatomy and take action accordingly.

[0026] As another example, a robotically-assisted knee replacement system may utilize aspects discussed herein. During a surgical procedure, an array may be connected to a retractor of which the exact size and shape is known. A robotic system can determine and track where the retractor is located in three-dimensional space at all times based on tracking the affixed array and knowledge of the retractor’s position and pose relative to that array. One benefit of knowing the location of the retractor is that it would enable the system to create boundaries and/or haptics, for instance to trigger various actions, alerts, mechanical interlocks, or the like, for instance to prevent retracted structures from being injured by the retractor itself and/or other objects such as other surgical tools. Multiple different known retractor sizes could be set up, and corresponding different haptics or boundaries could be defined based on those in order to protect vital structures. [0027] Arrays can be camera based (as is commonly used), though in other examples they can be based on other forms of a computer system and/or navigation system that communicates with an object/array such that the location of both may be known by that system in real time and space, and that can be learned by the system so that interactions can be had between the system, the array, and the operator, in real time.

[0028] This technology has broad applications, including applications of orthopedic surgery such as robotic knee replacements, shoulder surgery, partial knee replacement surgery, and hip replacement surgery. Other applications include, but are not limited to, abdominal surgery, spinal surgery, general surgery, vascular surgery, neurological surgery, and others. It can be used with current robotic platforms, navigation platforms, and current optical arrays, as examples. Any array or other tracking modality (electromagnetic, ultrasound, etc.) and associated computer system that interacts with the array to facilitate surgery can be used and connected to any shape, size, or material retractor for which characteristics of the retractor (e.g., size, shape, etc.) are known or can be determined. The computer/navigation system can know where that retractor is located relative to the patient anatomy, for instance bone and soft tissues, and interact with the flow of the operation and the procedure.

[0029] As noted, the location of a retractor can be used to establish barrier(s) at the perimeter/periphery of the retractor to create no-go areas, haptic barriers (ones that will trigger haptic feedback to an operator or other user), or the like to trigger actions or responses if another object, such as a surgical saw, reaches or traverses such barrier(s). This can ultimately help in protecting patients. This could also decrease registration times of patient native anatomy and improve intra-operative safety for patients.

[0030] Referring to FIG. 1, shown is a superior (top) view of a patient tibia 102 with two retractor-array objects 120, 130 in accordance with aspects described herein operatively positioned for use during a surgery. At a medial side 104 is the MCL 106 of the patient. At a lateral side 108 is the lateral collateral ligament (LCL) 110 of the patient. At an anterior side 112 is an extensor mechanism (EXT) 114 used in a surgical procedure. At a posterior side 116 is a neurovascular bundle (NVB) 118 of the patient. [0031] Retractor-array object 120 at the medial side 104 is shown in operative position, in which it has been inserted between the MCL 106 and the medial side of tibia 102. Specifically, a curved retractor end portion 121 was inserted from the front (anterior) direction between the MCL 106 and medial portion of the tibia 102 and slid around the medial portion of tibia 102 to a posterior portion thereof in the direction of the NVB 118. The retractor-array object 120 includes an array portion 122 with fiducials and a retractor portion 124. The array portion 122 is, in this example, attached to the retractor portion 124 with clip 126. Other forms of solid connection, such as screws, clamps, or the like, may be used. Any clip or similar object used could be reusable or disposable, and could attach at any place on the retractor-array object 120 desired.

[0032] Similarly, retractor-array object 130 at the lateral side 106 is shown in operative position, in which it has been inserted between the LCL 110 and the lateral side of tibia 102. Specifically, a curved retractor end portion 131 was inserted from the front (anterior) direction initially between the anterior tibia 120 and the EXT 114, then around to the lateral portion of tibia 120, between the LCL 110 and the lateral portion of the tibia 102, and slid around the lateral portion of tibia 102 to a posterior portion thereof in the direction of the NVB 118. The retractor-array object 130 includes an array portion 132 with fiducials and a retractor portion 134. The array portion 132 is, in this example, attached to the retractor portion 134 with clip 136. Other forms of solid connection, such as screws, clamps, or the like, may be used. Any clip or similar object used could be reusable or disposable, and could attach at any place on the retractor-array object 130 desired.

[0033] In this manner, two retractor-array objects 120, 130 of differing shapes are shown with clips 126, 136 that connect arrays 122, 132 to retractors 124, 134. Retractor portions (e.g., 121, 132) can be placed between bony structures and soft tissue structures between various tissue planes to move tissues out of the way, protect them, and interact with the robotic/navigation system in real time.

[0034] FIG. 2 depicts another example of a retractor-array object in accordance with aspects described herein operatively positioned for use during a surgery. Shown in FIG. 2 is a lateral (side) view of a patient knee anatomy, including tibia 202 and femur 240, in a position of approximately 90 degrees flexion with a retractor-array object 220 in accordance with aspects described herein. Retractor-array object 220 is shown in operative position in which it has been inserted at a posterior side (216; in contrast to anterior side 212) of tibia 202 between tibia 202 and femur 240. Specifically, a curved retractor end portion 221 was inserted at the posterior tibial plateau and slid down (distally). The retractor-array object 220 includes an array portion 222 with fiducials and a retractor portion 224. The array portion 222 is, in this example, attached to the retractor portion 224 with clip 226. Other forms of solid connection, such as screws, clamps, or the like, may be used. Any clip or similar object used could be reusable or disposable, and could attach at any place on the retractor-array object 220 desired.

[0035] FIG. 3 depicts an example system for imaging a retractor-array object and providing a graphical depiction of the retractor-array object relative to a patient anatomical model, in accordance with aspects described herein. Shown in FIG. 3 is an example retractor-array object 320 having array portion 322 and retractor portion 324 affixed using a clip 326. Also shown is an imaging device 350 (such as a camera) that images the retractor-array object 320 and that is in wired or wireless communication over communication link 352 with a system 360, for instance a robotic control system or navigation system implemented as a computer system, as an example. System 360 can generate and output images, user interfaces, and/or other graphical elements. These can include depictions of patient anatomy and other imaged objects, including retractor-array object 320. Image 372 is a generated image showing a model 374 of patient anatomy and the positioning of retractor-array object 320 relative to the patient anatomy by way of a model graphical element 376 representing the retractor-array object 320. The image may be presented on a display device of, or in communication with, system 360, for instance.

[0036] In addition, the system 360 could be used for safe and effective robotic navigation and other functions. System 360 can determine exactly where array 322 is located in three-dimensional space, and could use the location information provided by imaging the array 322 to determine locations where the retractor portion 324 and/or portions thereof (for instance an end 321 of the retractor portion 324) are relative to that array in three-dimensional space or a two-dimensional space relative to that three-dimensional space. If the retractor comes too close to protected anatomy, an alert could be raised. Additionally or alternatively, the system could make decisions regarding navigation, cutting, and/or other robotic functions based on the determined locations. In some embodiments, system 360 interacts with another system, for instance a separate navigation system, to provide information for taking surgical actions based on this information determined by system 360.

[0037] Thus, in accordance with some aspects, a navigated ‘Z’ -style retractorarray object is provided as a Z retractor with an array of fiducials affixed to the Z retractor. The retractor-array object can then become a structure that can inform boundaries, tool control, alerts, or the like. In some examples, a proximity alert is delivered when the system detects that a surgical tool, such as a saw, comes in sufficient proximity of the retractor-array object, for instance to within a threshold distance from a portion of the retractor-array object. The portion of the retractor can serve as a reference for the distance between the surgical tool and selected and/or identified anatomy, for instance the medial collateral ligament or neurovascular structures (e.g., posteriorly in the case of a knee surgery). In this manner, proximitybased alerts can be raised when the surgical tool comes too close (e.g. within a threshold distance that may be configurable) to selected anatomy as indicated by the position of the retractor that is detected on account of the navigation achieved based on the attached array. In some examples, a proximity alert could be raised then the retractor itself comes too close to protected anatomy.

[0038] By way of example with respect to a knee surgery, the Z-style retractorarray object may be placed at a level of the tibia bone and inside of the medial collateral ligament. Because of the curvature of the retractor portion, it is intimately opposed to the back of (posterior of ) the tibia bone. Since the system can have knowledge of the exact size and shape of the retractor portion, for instance by way of pre-configuration and/or imaging the retractor prior to use, the system can project the location of any portion of the retractor-array object. The retractor, for instance an end portion extending to the posterior of the patient, can protect the neurologic structures such as those that are part of the neurovascular bundle. If a saw used for resecting the tibia, for instance, comes too close to the critical anatomy as determined based on the position of the retractor portion, then the system can raise an alert, deactivate the saw, provide haptic feedback, and/or constrain saw movement, as examples. It is noted that conventional navigation systems that do not have preoperative imaging, such as preoperative CT scanning, are not aware of where the back of the patient tibia is because this part of the anatomy was not registered, as it is not generally accessible for registration through incisions on the anterior knee, as is common.

[0039] Aspects presented in examples above are discussed in the context of a knee surgery, but can be useful in various other types of surgeries, for instance others in which navigation can be very useful. One such type is spine surgery. Navigation in spine surgery has produced good results in terms of accuracy of hardware placement (e.g., pedicle screws), reduced incision size, accuracy of implant device placement, decreases in blood loss and radiation exposure, and improved surgeon experience, for instance increased confidence in object placement and elimination of the need to wear lead protection, as examples.

[0040] Additionally, retractor portions of example retractor-array objects disclosed herein can be other than ‘Z’-style retractors. For instance, another example of a retractor-array object in accordance with aspects described herein is presented in FIGS. 4A-4B, which depicts a side view and rear view, respectively, of an example retractor-array object 400. The retractor-array object 400 includes a handle portion 402, an array portion 404 with fiducials 406a, 406b, 406c, and a retractor portion 408. The retractor portion 408 extends from handle portion 402 to a retractor end portion 410 at the opposite end of the retractor portion 408. The array portion 404 is attached to the handle portion 402 by way of an attachment portion 412 using any suitable approach for attachment, for instance clip(s), screw(s), clamp(s), or the like.

[0041] One example clinical scenario in which the example retractor-array object 400 may be used is an anterior lumbar interbody fusion (ALIF) procedure. A clinical advantage for an ALIF operation is that it places relatively large grafts in a biomechanically favorable position, for instance in front of part of the patient’s spine to correct alignment deformities, and accomplishes an indirect neurologic decompression to take pressure off of the nerves as they’re being pressed in the back by opening up the front, all with relatively less blood loss than alterative procedures. One of the challenges, however, is that this is done vie a relatively large anterior incision. The neurologic surgeon is usually therefore assisted by an approach or vascular surgeon to help escort the neurologic surgeon down to the spine. This assistance can be critical because the aorta and other vital cardiovascular anatomy run right in the front of the spine where the procedure takes place. At level of L4 and L5 vertebrae, the aorta and common iliac arteries get retracted. At the L5-S1 spinal motion segment (the lumbosacral joint) the left common iliac vein and artery get retracted laterally between the bifurcation where they split. One particular vein - the iliolumbar vein - is critical and commonly ligated for exposure and is a critical aspect of the ALIF procedure. Anterior lumbar surgery can therefore be of great importance but vascular injury presents a real threat.

[0042] The provision of a retractor-array object such as that of FIG. 4 can be used in ALIF procedures to provide improvements, as it introduces the possibility of navigation-assisted surgery. Currently, navigation is not used in ALIF procedures beyond perhaps planning incisions, even though preoperative imaging (e.g., CT/MRI) is common before lumbar spine surgery.

[0043] In an example process for a navigated anterior approach for an ALIF surgery, preoperative advanced imaging (CT/MRI) is performed. This imaging shows veins and arteries, among other anatomy which may be indicated as being protected anatomy. These tethered in location to the spine, so where they appear in the preoperative imaging is accurately reflective of their location relative to the spine during the surgical approach. That preoperative imaging study can then be linked to intra-operative navigation. Retractors are rigid structures with known three- dimensional shape and size that does not change. The process can ‘teach’ the computer the size and shape of the retractor(s) used, through imaging or other input of the characteristics and specification of the retractor-array object(s), and then this information can be integrated into the intra-operative navigation to show the retractor(s) on the navigation screen/monitor/display in real time. With the vascular anatomy and the retractor(s) projected in the navigation view on the display, this combines retractor location(s) with that of the vulnerable anatomical structures to understand the spatial relation between them. As discussed above, alerts such as those based on proximity to a retractor can be raised to alert surgeons as to proximity to vascular structures based on the proximity to the retractor. Ultimately, this will allow for smaller incisions and safer approaches, and will limit the need for approach surgeons, while increasing the safety and prevalence of ALIF procedures with navigation. [0044] The above process adds navigation to an ALIF procedure for which it is already common to acquire preoperative advanced imaging. This imaging may be linked to intraoperative navigation via the retractor-array object(s) as described above to relate the retractor object(s) with the vascular anatomy and provide alerts, boundaries, or the like based on that relation. Examples of proximity alerts when another object, such as a surgical tool reaches within a threshold distance of protected anatomy, such as vascular structures, include visual and/or audio alerts.

[0045] Another example clinical scenario in which the example retractor-array object 400 may be used is a Lateral lumbar interbody fusion (LLIF) procedure. LLIF advantageously requires no vascular exposure, provides indirect spinal decompression, and yields shorter hospital stays, less blood loss, shorter surgical times, and improved deformity correction. It also enables relatively large implants with high fusion rates. The problem is that LLIF relies on a largely blind approach since it is retroperitoneal, meaning behind the abdominal muscles and through the psoas muscle. Often it is performed as tube-based procedure with neuromonitoring / fluoroscopic imaging. Neurologic injury can result if retractors are placed in wrong positions and too much force is put on them.

[0046] Thus, in accordance with aspects described herein, navigated retractor(s) are used in the LLIF procedure, which can help decrease nerve injury and render a minimally invasive surgical option for LLIF. A process for navigated LLIF can be similar to that of the process described above for ALIF, with preoperative advanced imaging (CT/MRI), navigation of the retractor(s), and integration of the preoperative advanced imaging showing the neurological structures - nerve roots, etc. - with projected onto the navigation. The preoperative imaging is linked to the intraoperative navigation, showing the retractor(s) on the monitors in real time again knowing that retractor(s) are known size and shape. Integration with the LLIF procedure relates retractor location to the location and neurological anatomy (as opposed to vascular anatomy in the case of the ALIF) and enables creation of, e.g., proximity based alerts, such as visual and/or audio alerts, when other objects such as a surgical tool comes sufficiently close to a protected neurostructure.

[0047] Another example scenario in which an example retractor-array object as described herein may be used is in navigating a rod placement process as part of a spinal fusion operation. As is known, pedicle screws are placed in pedicles and then connected via rods threaded through slots. Rod placement is not performed using navigation, and is instead performed based on fluoroscopy or surgeon ‘feel’ through a relatively large incision and often going through muscle. This can lead to injury and/or poor surgical outcomes. Use of a retractor-array object described herein can make this a safer, minimally invasive procedure with improved patient outcomes.

[0048] Referring to FIG. 5, which depicts an example retractor-array object in accordance with aspects described herein to facilitate rod placement for a spinal fusion procedure of a patient spine 510, shown is retractor-array object 500 having a handle portion 503, array portion 504 with three fiducials in this example, and retractor portion 506. The retractor portion 506 holds, temporarily, a rod 508 for delivery through slots (not shown) in sleeves 514, the rod to eventually bond the placed pedicle screws 512 once secured. In this setup, the retractor-array 500 is trackable and so too is the rod of known size and shape. Therefore, the position of the rod and any portion thereof, including the tip 509, is known and tracked by the navigation in real time and shown on the display relative to patient anatomy. FIG. 6 depicts an example of this for rod placement.

[0049] Referring to FIG. 6, a display device 602 (may also be referred to as a monitor, display, screen, or the like) is shown and presents a built graphical interface 604 depicting elements of the navigated surgery, include a model of the rod 608 and models 620 of the slots through which the rod 608 is threaded for rod placement. The locations of the slots 620 may be known because the pedicle screws can be navigated and connected to arrays; their 3D size, shape, and position may be known, allowing the rod to be placed into the slots. The positioning of the rod 608 and the tip 609 thereof is known by way of tracking the retractor-array object 500 that delivers the rod. The graphical interface 604 includes four views of rod 608 and its approach to the slots 620 into which it is to be inserted. The four views are a three-dimensional isometric view (top left), a side view (top right), an end view (bottom left) and a top view (bottom right), all showing the approach of the rod tip 609 to opening of the first slot 620 through which the rod is to pass. Using this interface, the surgeon knows the precise positioning of the rod and portions thereof, including the tip, to facilitate a proper and efficient approach of the rod into and through the slots of the pedicle screws/sleeves. The interface could include additional elements, including anatomical elements if desired. This might be useful in situations where the surgeon wants to visualize the approach of the rod tip as it travels from the incision toward the site of the pedicle screws and slots.

[0050] Accordingly, navigated retractors have several applications in the spine space in which vascular and/or neurological anatomy (sometimes already captured by preoperative imaging) is provided in the navigation screen along with retractor position. Proximity -based audio and/or visual alerts and/or other actions, such as color changes to graphical elements or vibrations to surgical equipment, actions can be performed to inform the surgeon when the retractor and/or other surgical instrumentation is too close to one of these protected structures. Through combining knowledge of retractor positioning and positioning of vulnerable structures, and raising alert(s), aspects help to improve patient outcomes through safer, less invasive, and more efficient surgical procedures.

[0051] Other surgical applications are possible, including unicompartmental knee replacement, total knee replacement, and hip replacement operations, though more generally any surgical procedure involving vulnerable vascular, neurological, or other vulnerable anatomic structures may benefit from aspects described herein.

[0052] Software implementing features described herein can be integrated into robotic surgery applications for performing robotic surgeries in hospitals and ambulatory surgery center operating rooms, as examples.

[0053] One or more embodiments described herein may be incorporated in, performed by, and/or used by one or more computer systems, such as one or more computer systems that are incorporated into and/or in communication with an orthopedic surgical robot system. Processes described herein, and/or aspects thereof, may be performed singly or collectively by one or more such computer systems. A computer system may also be referred to herein as a data processing device/system, computing device/system/node, or simply a computer. The computer system may be based on one or more of various system architectures and/or instruction set architectures.

[0054] FIG. 7 depicts an example process/method for configuring a boundary, for instance a boundary for a system such as a surgical navigation system and/or robotic surgical system, as examples. The process can be performed in whole or part by system(s), including that system and/or another system, either of which may be regarded as a computer or computer system as described herein. Referring to FIG. 7, the process images (702), by at least one imaging device, locational markers in a surgical environment. The locational markers are of a retractor-array object and are physically connected to a retractor portion configured for retracting an anatomical structure of a patient of the surgical environment. The process also determines (704) locations of the locational markers in three-dimensional space of the surgical environment based on the imaging. Then, based on the determined locations of the locational markers, a known size and shape of the retractor portion, and a known position of the retractor portion relative to the detected locations of the locational markers (by way of knowing the spatial relation of the retractor portion to the connected markers), the process determines (706) a location of the retractor portion in the three-dimensional object in space. By this is meant that the location of any one or more points of the retractor portion is determined. The process relates (708), in a surgical navigation system, the determined location of the retractor portion in the three-dimensional space to a location of an anatomical structure of the patient in the three-dimensional space, and, based on this relating, configures (710) a boundary for the surgical navigation system. The boundary is set to be in locational proximity (i.e., near, next to, within a configurable threshold, etc.) to the anatomical structure and if for automatically triggering an action based on an object reaching or exceeding the boundary.

[0055] In examples, the boundary is configured at a perimeter or periphery of the retractor portion such that the action is triggered based on the object reaching the perimeter or periphery of the retractor portion. The action can include any one or more of a variety of action(s). For instance, the action can include, but is not limited to, providing an audio and/or visual alert, providing haptic feedback to a user, and/or triggering a mechanical interlock configured to prevent injury to the anatomical structure, as examples.

[0056] In some embodiments, the action includes control of a robotic surgical system to control a surgical action taken by the robotic surgical system. In examples, the surgical action includes movement and/or operation of a surgical tool controlled by the robotic surgical system. The surgical tool could be a surgical saw, for instance. In embodiments, the surgical action includes cutting by the surgical saw, and the control of the surgical action includes (i) slowing or stopping cutting by the surgical saw and/or (ii) movement/moving of the surgical saw away from the boundary.

[0057] Continuing with FIG. 7, the process further builds and presents (712) a graphical interface for display. The graphical interface includes a virtual representation of patient anatomy including the anatomical structure and a graphical element representing the retractor-array object in the graphical interface. The building can positionally relate the virtual representation of the patient anatomy and the graphical element in the graphical interface based on, and corresponding to, the related location of the retractor portion in the three-dimensional space to the location of the anatomical structure of the patient in the three-dimensional space, in order to to visually reflect an accurate positioning of the retractor portion related to the anatomical structure in the three dimensional space.

[0058] In embodiments, the configuration of the retractor portion for retracting the anatomical structure includes a curvature of the retractor portion for curvature at least partially around other patient anatomy based on insertion of the retractor portion into the patient in an anterior-to-posterior trajectory. This may be particularly helpful to protect anatomical structure(s) located posterior to the other patient anatomy around which the curved retractor portion curves. For instance, in some examples the anatomical structure is a neurovascular bundle of the patient posterior to the patient tibia or femur. In some embodiments, the curvature provides enough proximity to the anatomical structure that the boundary established is sufficient to keep a surgical tool (such as a saw) from contacting the anatomical structure, even if the curved retractor portion does not extend to such an extent that it physically blocks the tool from reaching the anatomical structure. For instance, the curvature could be sufficient to provide a tip of the retractor portion in proximity to the anatomical structure posterior to the other patient anatomy based on operative position of the retractor portion in the patient, such that an effective boundary is established that would prevent the tool from reaching the anatomical structure.

[0059] In embodiments, the object that reaches or exceeds the boundary is the retractor-array object itself, where the boundary is set to trigger an alert based on the retractor-array object reaching the boundary. In this embodiment, retractor-array object is tracked and locationally related to a protected anatomical structure. A boundary can be set at a point where an alert is to be raised if the retractor portion (i.e., one or more points or portions thereof) come too close to the anatomical structure such that it reaches that set boundary.

[0060] FIG. 8 shows a computer system 800 in communication with external device(s) 812. Computer system 800 includes one or more processor(s) 802, for instance central processing unit(s) (CPUs). A processor can include functional components used in the execution of instructions, such as functional components to fetch program instructions from locations such as cache or main memory, decode program instructions, and execute program instructions, access memory for instruction execution, and write results of the executed instructions. A processor 802 can also include register(s) to be used by one or more of the functional components. Computer system 800 also includes memory 804, input/output (I/O) devices 808, and I/O interfaces 810, which may be coupled to processor(s) 802 and each other via one or more buses and/or other connections. Bus connections represent one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include the Industry Standard Architecture (ISA), the Micro Channel Architecture (MCA), the Enhanced ISA (EISA), the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI).

[0061] Memory 804 can be or include main or system memory (e.g., Random Access Memory) used in the execution of program instructions, storage device(s) such as hard drive(s), flash media, or optical media as examples, and/or cache memory, as examples. Memory 804 can include, for instance, a cache, such as a shared cache, which may be coupled to local caches (examples include LI cache, L2 cache, etc.) of processor(s) 802. Additionally, memory 804 may be or include at least one computer program product having a set (e.g., at least one) of program modules, instructions, code, or the like that is/are configured to carry out functions of embodiments described herein when executed by one or more processors.

[0062] Memory 804 can store an operating system 805 and other computer programs 806, such as one or more computer programs/applications that execute to perform aspects described herein. Specifically, programs/applications can include computer readable program instructions that may be configured to carry out functions of embodiments of aspects described herein.

[0063] Examples of VO devices 808 include but are not limited to microphones, speakers, Global Positioning System (GPS) devices, RGB, IR, spectral, and/or other forms of cameras, lights, accelerometers, gyroscopes, magnetometers, sensor devices configured to sense light, proximity, heart rate, body and/or ambient temperature, blood pressure, and/or skin resistance, registration probes, robotic tools, and activity monitors. An VO device may be incorporated into the computer system as shown, though in some embodiments an VO device may be regarded as an external device (812) coupled to the computer system through one or more VO interfaces 810.

[0064] Computer system 800 may communicate with one or more external devices 812 via one or more VO interfaces 810. Example external devices include a keyboard, a pointing device, a display, and/or any other devices that enable a user to interact with computer system 800. Other example external devices include any device that enables computer system 800 to communicate with one or more other computing systems or peripheral devices such as a printer. A network interface/ adapter is an example VO interface that enables computer system 800 to communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), and/or a public network (e.g., the Internet), providing communication with other computing devices or systems, storage devices, or the like. Ethernet-based (such as Wi-Fi) interfaces and Bluetooth® adapters are just examples of the currently available types of network adapters used in computer systems (BLUETOOTH is a registered trademark of Bluetooth SIG, Inc., Kirkland, Washington, U.S.A.).

[0065] The communication between VO interfaces 810 and external devices 812 can occur across wired and/or wireless communications link(s) 811, such as Ethernetbased wired or wireless connections. Example wireless connections include cellular, Wi-Fi, Bluetooth®, proximity -based, near-field, or other types of wireless connections. More generally, communications link(s) 811 may be any appropriate wireless and/or wired communication link(s) for communicating data. [0066] Particular external device(s) 812 may include one or more data storage devices, which may store one or more programs, one or more computer readable program instructions, and/or data, etc. Computer system 800 may include and/or be coupled to and in communication with (e.g., as an external device of the computer system) removable/non-removable, volatile/non-volatile computer system storage media. For example, it may include and/or be coupled to a non-removable, nonvolatile magnetic media (typically called a “hard drive”), a magnetic disk drive for reading from and writing to a removable, non-volatile magnetic disk (e.g., a “floppy disk”), and/or an optical disk drive for reading from or writing to a removable, nonvolatile optical disk, such as a CD-ROM, DVD-ROM or other optical media.

[0067] Computer system 800 may be operational with numerous other general purpose or special purpose computing system environments or configurations. Computer system 800 may take any of various forms, well-known examples of which include, but are not limited to, personal computer (PC) system(s), server computer system(s), such as messaging server(s), thin client(s), thick client(s), workstation(s), laptop(s), handheld device(s), mobile device(s)/computer(s) such as smartphone(s), tablet(s), and wearable device(s), multiprocessor system(s), microprocessor-based system(s), telephony device(s), network appliance(s) (such as edge appliance(s)), virtualization device(s), storage controller(s), set top box(es), programmable consumer electronic(s), network PC(s), minicomputer system(s), mainframe computer system(s), and distributed cloud computing environment(s) that include any of the above systems or devices, and the like.

[0068] Aspects of the present invention may be a system, a method, and/or a computer program product, any of which may be configured to perform or facilitate aspects described herein.

[0069] In some embodiments, aspects of the present invention may take the form of a computer program product, which may be embodied as computer readable medium(s). A computer readable medium may be a tangible storage device/medium having computer readable program code/instructions stored thereon. Example computer readable medium(s) include, but are not limited to, electronic, magnetic, optical, or semiconductor storage devices or systems, or any combination of the foregoing. Example embodiments of a computer readable medium include a hard drive or other mass-storage device, an electrical connection having wires, random access memory (RAM), read-only memory (ROM), erasable-programmable read-only memory such as EPROM or flash memory, an optical fiber, a portable computer disk/diskette, such as a compact disc read-only memory (CD-ROM) or Digital Versatile Disc (DVD), an optical storage device, a magnetic storage device, or any combination of the foregoing. The computer readable medium may be readable by a processor, processing unit, or the like, to obtain data (e.g., instructions) from the medium for execution. In a particular example, a computer program product is or includes one or more computer readable media that includes/stores computer readable program code to provide and facilitate one or more aspects described herein.

[0070] As noted, program instruction contained or stored in/on a computer readable medium can be obtained and executed by any of various suitable components such as a processor of a computer system to cause the computer system to behave and function in a particular manner. Such program instructions for carrying out operations to perform, achieve, or facilitate aspects described herein may be written in, or compiled from code written in, any desired programming language. In some embodiments, such programming language includes object-oriented and/or procedural programming languages such as C, C++, C#, Java, etc.

[0071] Program code can include one or more program instructions obtained for execution by one or more processors. Computer program instructions may be provided to one or more processors of, e.g., one or more computer systems, to produce a machine, such that the program instructions, when executed by the one or more processors, perform, achieve, or facilitate aspects of the present invention, such as actions or functions described in flowcharts and/or block diagrams described herein. Thus, each block, or combinations of blocks, of the flowchart illustrations and/or block diagrams depicted and described herein can be implemented, in some embodiments, by computer program instructions.

[0072] Although various embodiments are described above, these are only examples.

[0073] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising”, when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and/or groups thereof.

[0074] The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below, if any, are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of one or more embodiments has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiment was chosen and described in order to best explain various aspects and the practical application, and to enable others of ordinary skill in the art to understand various embodiments with various modifications as are suited to the particular use contemplated.

Claims

1. A computer-implemented method including: imaging, by at least one imaging device, locational markers in a surgical environment, the locational markers being of a retractor-array object and being physically connected to a retractor portion configured for retracting an anatomical structure of a patient of the surgical environment; determining locations of the locational markers in three-dimensional space of the surgical environment based on the imaging; based on the determined locations of the locational markers, a known size and shape of the retractor portion, and a known position of the retractor portion relative to the detected locations of the locational markers, determining a location of the retractor portion in the three-dimensional space; relating, in a surgical navigation system, the determined location of the retractor portion in the three-dimensional space to a location of an anatomical structure of the patient in the three-dimensional space; based on the relating, configuring a boundary for the surgical navigation system, the boundary being in locational proximity to the anatomical structure for automatically triggering an action based on an object reaching or exceeding the boundary.

2. The method of claim 1, wherein the boundary is configured at a perimeter or periphery of the retractor portion such that the action is triggered based on the object reaching the perimeter or periphery of the retractor portion.

3. The method of claim 1, wherein the action includes control of a robotic surgical system to control a surgical action taken by the robotic surgical system.

4. The method of claim 3, wherein the surgical action includes movement and/or operation of a surgical tool controlled by the robotic surgical system.

5. The method of claim 4, wherein the surgical tool is a surgical saw, wherein the surgical action includes cutting by the surgical saw, and wherein the control of the surgical action includes (i) slowing or stopping cutting by the surgical saw and/or (ii) movement of the surgical saw away from the boundary.

6. The method of claim 1, wherein the action includes a providing an audio and/or visual alert.

7. The method of claim 1, wherein the action includes providing haptic feedback to a user.

8. The method of claim 1, wherein the action includes triggering a mechanical interlock configured to prevent injury to the anatomical structure.

9. The method of claim 1, further including building and presenting a graphical interface for display, the graphical interface including a virtual representation of patient anatomy including the anatomical structure and a graphical element representing the retractor-array object in the graphical interface, wherein the building positionally relates the virtual representation of the patient anatomy and the graphical element in the graphical interface based on, and corresponding to, the related location of the retractor portion in the three-dimensional space to the location of the anatomical structure of the patient in the three-dimensional space to visually reflect an accurate positioning of the retractor portion related to the anatomical structure in the three dimensional space.

10. The method of any of claim 1 to 9, wherein configuration of the retractor portion for retracting the anatomical structure includes curvature of the retractor portion for curvature at least partially around other patient anatomy based on insertion of the retractor portion into the patient in an anterior-to-posterior trajectory, wherein the anatomical structure is located posterior to the other patient anatomy.

11. The method of claim 10, wherein the curvature provides a tip of the retractor portion in proximity to the anatomical structure posterior to the other patient anatomy based on operative position of the retractor portion in the patient.

12. The method of claim 10, wherein the anatomical structure is a neurovascular bundle of the patient posterior to the patient tibia or femur.

13. The method of claim 1, wherein the object is the retractor-array object and wherein the boundary is set to trigger an alert based on the retractor-array object reaching the boundary.

14. A computer system comprising: a memory; and a processing circuit in communication with the memory, wherein the computer system is configured to perform a method of any of claims 1 to 13.

15. A computer program product compri sing : a computer readable storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method of any of claims 1 to 13.