WO2025235518A1

WO2025235518A1 - Navigation controller for image capture and diagnostics in medical procedures

Info

Publication number: WO2025235518A1
Application number: PCT/US2025/027999
Authority: WO
Inventors: Moshe Bouhnik; Roee Shibolet
Original assignee: Intuitive Surgical Operations Inc
Current assignee: Intuitive Surgical Operations Inc
Priority date: 2024-05-07
Filing date: 2025-05-06
Publication date: 2025-11-13
Anticipated expiration: 2026-11-07

Abstract

Aspects of this technical solution can receive, via a robotic medical system, a data set corresponding to a three-dimensional (3D) model of an anatomical structure, identify, by a machine learning model that receives as input the 3D model, a level of completeness of a portion of the 3D model, and generate, by the machine learning model and responsive to the level of completeness of the portion being less than or equal to a threshold, an action to control at least one of a position or an orientation of a sensor of the robotic medical system to increase the level of completeness of the portion of the 3D model of the anatomical structure.

Description

NAVIGATION CONTROLLER FOR IMAGE CAPTURE AND DIAGNOSTICS IN MEDICAL PROCEDURES

CROSS-REFERENCES TO RELATED APPLICATIONS

[0001] This application claims the benefit of priority under 35 U.S.C. § 119 to U.S. Provisional Patent Application No. 63/643,807, filed May 7, 2024, which is hereby incorporated by reference herein in its entirety.

TECHNICAL FIELD

[0002] This disclosure relates generally to medical devices, including, but not limited to, a navigation controller for image capture and diagnostics in medical procedures.

INTRODUCTION

[0003] Screening procedures, such as colonoscopy, upper gastrointestinal (GI) inspection, stomach inspection before endoscopic sleeve gastroplasty (ESG) procedure, enteroscopy, and more, entail inspecting some or all parts of an organ (e.g., colon, small intestine, stomach). Such screening is demanding on an operating physician, and is susceptible to error through incomplete viewing of the inspected organ. However, it can be challenging to efficiently, reliably, and completely inspect an organ, without consuming excessive computing resources or introducing delays, which can result in heightened risk of a medical diagnosis that does not identify a health hazard present in the organ.

SUMMARY

[0004] Systems, methods, apparatuses, and non-transitory computer-readable media of the technical solutions disclosed herein can identify missing portions (e.g., “holes”) of a model of a body part or anatomical structure, and determine a navigation path for a sensing device (e.g., endoscope) through the body part to detect the missing portions of the model. A system according to this disclosure can, for example identify holes in a 3D model of a body part, identify a navigation path through the body part to capture 3D data to “fill” areas having holes, advance a sensing device (e.g., endoscope) through the body part along the navigation path, or automatically orient the sensing device toward areas having the holes as the device advances. Thus, a technical solution for a navigation controller for image capture and diagnostics in medical procedures is provided. [0005] At least one aspect is directed to a system. The system can include one or more processors, coupled with memory. The system can receive, via a robotic medical system, a data set corresponding to a three-dimensional (3D) model of an anatomical structure. The system can identify, by a machine learning model that receives as input the 3D model, a level of completeness of a portion of the 3D model. The system can generate, by the machine learning model and responsive to the level of completeness of the portion being less than or equal to a threshold, an action to control at least one of a position or an orientation of a sensor of the robotic medical system to increase the level of completeness of the portion of the 3D model of the anatomical structure.

[0006] At least one aspect is directed to a method. The method can include receiving, via a robotic medical system, a data set corresponding to a three-dimensional (3D) model of an anatomical structure. The method can include identifying, by a machine learning model that receives as input the 3D model, a level of completeness of a portion of the 3D model. The method can include generating, by the machine learning model and responsive to the level of completeness of the portion being less than or equal to a threshold, an action to control at least one of a position or an orientation of a sensor of the robotic medical system to increase the level of completeness of the portion of the 3D model of the anatomical structure.

[0007] At least one aspect is directed to a non-transitory computer readable medium can include one or more instructions stored thereon and executable by a processor. The processor can receive, via a robotic medical system, a data set corresponding to a three-dimensional (3D) model of an anatomical structure. The processor can identify, via a machine learning model that receives as input the 3D model, a level of completeness of a portion of the 3D model. The processor can generate, via the machine learning model and responsive to the level of completeness of the portion being less than or equal to a threshold, an action to control at least one of a position or an orientation of a sensor of the robotic medical system to increase the level of completeness of the portion of the 3D model of the anatomical structure.

BRIEF DESCRIPTION OF THE FIGURES

[0008] These and other aspects and features of the present implementations are depicted by way of example in the figures discussed herein. Present implementations can be directed to, but are not limited to, examples depicted in the figures discussed herein. Thus, this disclosure is not limited to any figure or portion thereof depicted or referenced herein, or any aspect described herein with respect to any figures depicted or referenced herein. [0009] FIG. 1 depicts an example system, according to this disclosure.

[0010] FIG. 2 depicts an example computer architecture, according to this disclosure.

[0011] FIG. 3A depicts an example first navigation position of a robotic medical system, according to this disclosure.

[0012] FIG. 3B depicts an example second navigation position of a robotic medical system, according to this disclosure.

[0013] FIG. 3C depicts an example third navigation position of a robotic medical system, according to this disclosure.

[0014] FIG. 4 depicts an example capture for a 3D model of an anatomical structure, according to this disclosure.

[0015] FIG. 5 depicts an example modification of a 3D model of an anatomical structure, according to this disclosure.

[0016] FIG. 6 depicts an example method of navigation control for image capture and diagnostics in medical procedures, according to this disclosure.

DETAILED DESCRIPTION

[0017] Aspects of the technical solutions are described herein with reference to the figures, which are illustrative examples of this technical solution. The figures and examples below are not meant to limit the scope of this technical solution to the present implementations or to a single implementation, and other implementations in accordance with present implementations are possible, for example, by way of interchange of some or all of the described or illustrated elements. Where certain elements of the present implementations can be partially or fully implemented using known components, only those portions of such known components that are necessary for an understanding of the present implementations are described, and detailed descriptions of other portions of such known components are omitted to not obscure the present implementations. Terms in the specification and claims are to be ascribed no uncommon or special meaning unless explicitly set forth herein. Further, this technical solution and the present implementations encompass present and future known equivalents to the known components referred to herein by way of description, illustration, or example. [0018] The technical solutions described herein can determine one or more steps of a navigation path through a body part using a machine learning model. The machine learning model can, for example, correspond to a neural network, or a deep neural network, but is not limited thereto. The machine learning model can receive as input a three-dimensional (3D) model of a body part. For example, the 3D model of the body part can be a point cloud model of an internal surface of a colon as captured by a sensor of an endoscope inserted into the colon. The 3D model of the body part can be represented as a point cloud model of the internal surface of the anatomical structure (e.g., colon). The point cloud model can be generated or obtained through from data captured by the sensor of the endoscope, which can be inserted into the anatomical structure. The point cloud model can refer to or include a collection of discrete data points in a three-dimensional coordinate system, which each point can represent a specific location on an interior or exterior surface of the anatomical structure. These points can collectively form a 3D digital representation of the shape and surface characteristics of the anatomical structure. The points can represent spatial measurements and characteristics of the anatomical structure, such as internal contours of the colon.

[0019] The point cloud can include one or more holes corresponding to regions having no points or a density of points below a given threshold. The holes can refer to gaps, absences, nulls, or voids in a portion of the anatomical structure. The holes can result from the lack of data or points corresponding to the portion of the anatomical structure, which can be due to an incomplete scan by the sensor of the anatomical structure, missing data points, the portion of the anatomical structure being obfuscated from the field of view of the sensor, or erroneous, corrupt or otherwise unusable data points.

[0020] The machine learning model can generate an output for each of one or more steps through the body part, where each step corresponds to a movement either into (forward) or out of (backward) the colon. The output can include an orientation of a sensor for each of the steps. The machine learning model can identify an orientation at a given step (e.g., position along the colon) that captures the greatest number of points from the holes present at that position along the colon. For example, the system can either advance through each step in response to user input (e.g., a semi-automatic system with fully supervised physician control), or can advance through each step actively, and pause at a given step or orientation to alert a physician of presence of an abnormality at that step (e.g., a fully automatic system with semi-supervised physician control). Abnormalities can include, for example, a polyp, diverticulitis, or an unusual dirty bowel section. In addition to or instead of pausing at the given step in response to detecting an abnormality, the system of this technical solution can suggest or automatically perform other actions, such as reaching the cecum, reaching the transverse colon, etc. The automatic halting criteria can be based on or dependent on a type of procedure, such as upper gastrointestinal or colonoscopy, etc. In some cases, the technical solution can invoke or utilize a model trained with machine learning that is configured to detect or classify the type of abnormality. Thus, the machine learning model can generate as output a navigation path through the body part to traverse through the body part to capture data to efficiently and effectively complete or substantially improve a 3D model with holes, while minimizing movement within the body part to minimize displacement of the body part and impact on the patient. With the capture of additional points, a system can more accurately perform polyp detection and polyp classification.

[0021] FIG. 1 depicts an example system, according to this disclosure. As illustrated by way of example in FIG. 1, a system 100 can include one or more of a network 101, a data processing system 102, a client system 103, a robotic system 104, or instruments 170.

[0022] The network 101 can include any type or form of network. The geographical scope of the network 101 can vary widely and the network 101 can include a body area network (BAN), a personal area network (PAN), a local-area network (LAN), e.g., Intranet, a metropolitan area network (MAN), a wide area network (WAN), or the Internet. The topology of the network 101 can be of any form and can include, e.g., any of the following: point-to-point, bus, star, ring, mesh, or tree. The network 101 can include an overlay network which is virtual and sits on top of one or more layers of other networks 101. The network 101 can be of any such network topology as known to those ordinarily skilled in the art capable of supporting the operations described herein. The network 101 can utilize different techniques and layers or stacks of protocols, including, e.g., the Ethernet protocol, the Internet protocol suite (TCP/IP), the ATM (Asynchronous Transfer Mode) technique, the SONET (Synchronous Optical Networking) protocol, or the SD (Synchronous Digital Hierarchy) protocol. The 'TCP/IP Internet protocol suite can include application layer, transport layer, Internet layer (including, e.g., IPv6), or the link layer. The network 101 can include a type of a broadcast network, a telecommunications network, a data communication network, or a computer network.

[0023] The data processing system 102 can include a physical computer system operatively coupled or coupleable with one or more components of the system 100, either directly or directly through an intermediate computing device or system. The data processing system 102 can include one or more of a virtual computing system, an operating system, and a communication bus to effect communication and processing. The data processing system 102 can include one or more of a system processor 110, an interface controller 112, a 3D model processor 120, a navigation processor 130, a robotic device controller 140, and a system memory 150.

[0024] The system processor 110 can execute one or more instructions associated with the system 100. The system processor 110 can include an electronic processor, an integrated circuit, or the like including one or more of digital logic, analog logic, digital sensors, analog sensors, communication buses, volatile memory, nonvolatile memory, and the like. The system processor 110 can include, but is not limited to, at least one microcontroller unit (MCU), microprocessor unit (MPU), central processing unit (CPU), graphics processing unit (GPU), physics processing unit (PPU), embedded controller (EC), or the like. The system processor 110 can include a memory operable to store or storing one or more instructions for operating components of the system processor 110 and operating components operably coupled to the system processor 110. For example, the one or more instructions can include one or more of firmware, software, hardware, operating systems, embedded operating systems. The system processor 110 or the system 100 generally can include one or more communication bus controller to effect communication between the system processor 110 and the other elements of the system 100.

[0025] The interface controller 112 can link the data processing system 102 with one or more of the network 101 and the client system 103, by one or more communication interfaces. A communication interface can include, for example, an application programming interface (“API”) compatible with a particular component of the data processing system 102, or the client system 103. The communication interface can provide a particular communication protocol compatible with a particular component of the data processing system 102 and a particular component of the client system 103. The interface controller 112 can be compatible with particular content objects and can be compatible with particular content delivery systems corresponding to particular content objects, structures of data, types of data, or any combination thereof. For example, the interface controller 112 can be compatible with transmission of text data or binary data structured according to one or more metrics or data of the system memory [0026] The 3D model processor 120 can generate one or more 3D models corresponding to an interior surface of a structure. For example, the 3D model processor 120 can obtain surface data corresponding to the interior surface of the structure, from one or more sensors positioned at least partially in an interior of the structure. For example, the surface data can include at least one of one or more points, point cloud data, time of flight information, sensor device orientations, or any combination thereof, but is not limited thereto. For example, the 3D model processor 120 can generate a surface of an interior of a structure based on the surface data. To do so, the 3D model processor 120 can utilize one or more components depicted in FIG. 2, including, for example, an image capture processor 212, point cloud processor 214, or a 3D model generator 216. For example, the surface can correspond to a point cloud having a plurality of points. For example, the structure can correspond to an interior surface of at least a portion of a gastrointestinal tract, but is not limited thereto. The 3D model processor 120 can generate the 3D model having one or more 3D surface features that correspond to biological structures associated with the gastrointestinal tract. For example, the one or more of the 3D surface features can correspond to polyps. Thus, the 3D model processor 120 can provide a technical solution to generate a 3D model of an interior of a structure at a granularity sufficient to support medical diagnostics.

[0027] The navigation processor 130 can determine a path of traversal of the robotic system 104 or one or more of the instruments 170 through an organ. To do so, the navigation processor 130 can include or use one or more components depicted in FIG. 2, including, for example, a model completeness processor 222, sensor telemetry processor 224, or a machine learning model processor 226. The navigation processor 130 determine the path based at least partially on the 3D model corresponding to the structure or an organ as discussed herein, but is not limited thereto. For example, the navigation processor 130 can determine one or more portions of the 3D model that are incomplete, as having a surface portion absent in the 3D model, for example. The navigation processor 130 can apply a machine learning model processor 226 to generate a sequence of positions of an instrument 170 of the robotic system 104 to capture the absent surface portion of an interior surface of the structure or organ. The navigation processor 130 can generate a path according to the machine learning model that maximizes a percentage or amount of the absent surface portion that is captured, and minimized a number of movements within the structure or organ. The navigation processor 130 can determine the path, including one or more orientations of the instrument 170 or the robotic system 104 within the structure or organ, based on one or more properties of the sensors of the robotic system 104. For example, the navigation processor 130 can determine a navigation path through a gastrointestinal tract with fewer movements and with higher percentage of capture of the absent surface portion, based on input to the machine learning model identifying a camera sensor of the robotic system 104 having a higher resolution or wider viewing angle. For example, the navigation path can include one or more positions of the sensors of the robotic system 104 within the organ, and one or more orientations of the sensors at the one or more positions. For example, as discussed herein, positions can correspond to a coordinate in a Euclidean coordinate system (e.g., x, y and z coordinates) that identifies a location of the sensor with respect to the organ or a portion thereof (e.g., an opening of the organ into a body cavity). For example, as discussed herein, orientations can correspond to one or more angular positions (e.g., pitch, yaw, roll) at a coordinate in the Euclidean coordinate system, that identifies a field of view of the sensor.

[0028] The robotic device controller 140 can instruct or cause the robotic system 104 or one or more of the instruments 170 to a position or an orientation corresponding to the navigation path. To do so, the robotic device controller 140 can include or utilize one or more components depicted in FIG. 2, such as, for example, a position input processor 232, anatomy property processor 234, or navigation path processor 236. For example, the robotic device controller 140 can obtain or identify one or more steps of a navigation path, where each step includes a position of the robotic system 104 or the instrument 170 within the structure or organ (e.g., a linear position in a gastrointestinal tract with respect to an opening, a body cavity, or any combination thereof). For example, the robotic system 104 can determine position according to a coordinate system having an origin aligned with an opening of a body cavity. The robotic system 104 can determine distance from the origin and position with respect to the origin according to sensor telemetry or robotic device telemetry. The telemetry can be determined according to relative positioning of the robotic system 104 via an internal coordinate system of the robotic system (e.g., position of an endoscope with respect to a base of the endoscope outside the patient). The robotic device controller 140 can cause the robotic system 104 to move within the structure or organ to the position associated with the navigation path, or a plurality of positions associated with the navigation path in sequential order according to the navigation path. For example, the robotic device controller 140 can obtain or identify one or more steps of a navigation path, where each step includes an orientation of the robotic system 104 or the instrument 170 within the structure or organ (e.g., an angular position in a gastrointestinal tract with respect to an interior surface or an absent surface portion of the structure or organ, or any combination thereof). The robotic device controller 140 can cause the robotic system 104 or the instruments 170 (e.g., a camera or depth sensor) to move or rotate within the structure or organ to the orientation associated with the navigation path, or a plurality of orientations associated with the navigation path in sequential order according to the navigation path. In an aspect, the orientation corresponds to a relative position of the sensor of the robotic medical system with respect to the portion of the 3D model or the anatomical structure.

[0029] The system memory 150 can store data associated with the system 100. The system memory 150 can include one or more hardware memory devices to store binary data, digital data, or the like. The system memory 150 can include one or more electrical components, electronic components, programmable electronic components, reprogrammable electronic components, integrated circuits, semiconductor devices, flip flops, arithmetic units, or the like. The system memory 150 can include at least one of a non-volatile memory device, a solid-state memory device, a flash memory device, or aNAND memory device. The system memory 150 can include one or more addressable memory regions disposed on one or more physical memory arrays. A physical memory array can include a NAND gate array disposed on, for example, at least one of a particular semiconductor device, integrated circuit device, and printed circuit board device. The system memory 150 can include a video data 152, robot metrics 154, performance metrics 156, and an anatomy metrics 158. The system memory 150 can correspond to a non-transitory computer readable medium. In an aspect, the non-transitory computer readable medium can include one or more instructions executable by the system processor 110. The system processor 110 can generate, by the machine learning model, a second action to modify the position of the sensor of the robotic medical system with respect to a direction of traversal through the anatomical structure.

[0030] The video data 152 can depict one or more medical procedures from one or more viewpoints associated with corresponding medical procedures. For example, the video data 152 can correspond to still images or frames of video images that depict at least a portion of a medical procedure, medical environment, or patient site from a given viewpoint. For example, the image capture processor 212 can identify one or more depictions in an image or across a plurality of images. Each time can, for example, be associated with a given task or phase of a workflow as occurring during that task or phase.

[0031] The robot metrics 154 can be indicative of one or more states of one or more components of the robotic system 104. Components of the robotic system 104 can include, but are not limited to, actuators of the robotic system 104 as discussed herein. For example, the robot metrics 154 can include one or more data points indicative of one or more of an activation state (e.g., activated or deactivated), a position, or orientation of a component of the robotic system 104. For example, the robot metrics 154 can be linked with or correlated with one or more medical procedures, one or more phases of a given medical procedure, or one or more tasks of a given phase of a given medical procedure. For example, a robot metric among the robot metrics 154 can correspond to corresponding positions of one or more actuators of a given arm of the robotic system 104 at a given time or over a given time period. Each time can, for example, be associated with a given task or phase of a workflow as occurring during that task or phase.

[0032] The performance metrics 156 can be indicative of one or more actions during one or more medical procedures. For example, the performance metrics 156 can correspond to OPIs as discussed herein. The patient metrics 158 can be indicative of one or more characteristics of a patient during one or more medical procedures. For example, the patient metrics 158 can indicate various conditions (e.g., diabetes, low blood pressure, blood clotting) or various traits (e.g., age, weight) corresponding to the patient in each medical procedure. The surgical data processor 130 or the model processor 120 can obtain the patient metrics 158, and can filter or modify any segments of video in response to the obtained patient metrics 158. For example, the surgical data processor 130 or the model processor 120 can include video segments restricted to patients with patient metrics 158 indicative of diabetes.

[0033] The anatomy metrics 158 can be indicative of one or more anatomical features associated with the structures. For example, the anatomy metrics 158 can correspond to one or more features indicative of a polyp in a gastrointestinal tract. Thus, the anatomy metrics 158 can correspond to data indicative of a given anatomical feature with respect to a given structure or organ. The anatomy metrics 158 can be provided as input to a machine learning model configured to detect anatomical features, as machine learning features. The anatomy metrics 158 can be generated as output from a machine learning model configured to detect anatomical features, as trained machine learning features associated with a given anatomical feature (e.g., a polyp), a given structure (e.g., a gastrointestinal tract) or any combination thereof.

[0034] In an aspect, at least one of the anatomy metrics 158 corresponds to a tissue pressure metric indicative of a force on the anatomical structure caused the traversal through the anatomical structure, and the second threshold is indicative of a maximum force on the anatomical structure. For example, the navigation processor 130 can execute a machine learning model with input including the tissue pressure metric, to identify or generate a path through an organ that is restricted to applying pressure on the organ at or below the second threshold according to the tissue pressure metrics generated by the machine learning model for the given navigation path. In an aspect, at least one of the anatomy metrics 158 corresponds to an inflation metric indicative of an amount of inflation to inspect at least a portion of the anatomical structure corresponding to the traversal through the anatomical structure, and the second threshold is indicative of a maximum inflation of the portion of the anatomical structure. For example, the navigation processor 130 can execute a machine learning model with input including the inflation metric, to identify or generate a path through an organ that is restricted to applying pressure on the organ at or below the second threshold according to the inflation metrics generated by the machine learning model for the given navigation path.

[0035] The client system 103 can include a computing system associated with a database system. For example, the client system 103 can correspond to a cloud system, a server, a distributed remote system, or any combination thereof. For example, the client system 103 can include an operating system to execute a virtual environment. The operating system can include hardware control instructions and program execution instructions. The operating system can include a high-level operating system, a server operating system, an embedded operating system, or a boot loader. The client system 103 can include a user interface 160.

[0036] The user interface 160 can include one or more devices to receive input from a user or to provide output to a user. For example, the user interface 160 can correspond to a display device to provide visual output to a user and one or more or user input devices to receive input from a user. For example, the input devices can include a keyboard, mouse or touch-sensitive panel of the display device, but are not limited thereto. The display device can display at least one or more presentations as discussed herein, and can include an electronic display. An electronic display can include, for example, a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, or the like. The display device can receive, for example, capacitive or resistive touch input. The display device can be housed at least partially within the client system 103.

[0037] The robotic system 104 can include one or more robotic devices configured to perform one or more actions of a medical procedure (e.g., a surgical procedure). For example, a robotic device can include, but is not limited to a surgical device that can be manipulated by robotic device. For example, a surgical device can include, but is not limited to, a scalpel or a cauterizing tool. The robotic system 104 can include various motors, actuators, or electronic devices whose position or configuration can be modified according to input at one or more robotic interfaces. For example, a robotic interface can include a manipulator with one or more levers, buttons, or grasping controls that can be manipulated by pressure or gestures from one or more hands, arms, fingers, or feet. The robotic system 104 can include a surgeon console in which the surgeon can be positioned (e.g., standing or seated) to operate the robotic system 104. However, the robotic system 104 is not limited to a surgeon console co-located or on-site with the robotic system 104. The robotic system 104 can include an instrument s) 170. The instrument s) 170 can include components of the robotic system 104 that can be moved in response to input by a surgeon at the surgeon console of the robotic system 104 (which can also be referred to as or include a robotic device). The components can correspond to or include one or more actuators that can each move or otherwise change state to operate one or more of the instruments 170 of the robotic system 104. For example, one or more of the instruments 170 can include one or more sensors or be associated with one or more sensors to capture image data or depth data for a point cloud. For example, the instrument 10 is a camera that is affixed to an end portion or along a tube that can be actuated by the robotic system 104 or comprises a portion of the robotic system 104.

[0038] FIG. 2 depicts an example computer architecture, according to this disclosure. As illustrated by way of example in FIG. 2, a computer architecture 200 can include at least the 3D model processor 120, the navigation processor 130, and the robotic device controller 140. The image capture processor 120 can produce one or more representations of a structure or organ as discussed herein. For example, the image capture processor 120 can obtain one or more images, and can generate a 3D model based on one or more of the images. For example, the image capture processor 120 can obtain one or more depth maps including depth data, and can generate a 3D model based on the depth data. The 3D model processor 120 can include an image capture processor 212, a point cloud processor 214, and a 3D model generator 216.

[0039] The point cloud processor 212 can generate a point cloud based on one or more of image data or depth data. For example, the point cloud processor 212 can generate a point cloud including a plurality of points in a Cartesian coordinate system that each indicate that a surface exists at a position indicated by the respective coordinates of the points. For example, the point cloud processor 212 can generate the point cloud by identifying one or more image features in image data that correspond to a surface of a structure, according to a machine learning model configured to detect image features. The machine learning model configured to detect image features can correspond to a neural network or a deep learning system, for example. The neural network can be configured to receive input data including point cloud data, image data, or both. The neural network can include a feedback architecture to identify portions of the 3D model identified as incomplete, to focus the neural network on completing the incomplete portions of the 3D model. Thus, the neural network can provide technical improvement to improve completeness and accuracy of 3D models captured according to a robotic device as discussed herein, via a feedback architecture to focus the neural network on incomplete regions. For example, the point cloud processor 212 can generate the point cloud by identifying coordinates associated with the one or more image features in the image data that correspond to the surface of the structure, according to a machine learning model configured to detect image features. For example, the point cloud processor 212 can generate the point cloud by identifying coordinates associated with the one or more points of depth data along a vector in a predetermined direction to the surface of the structure.

[0040] The 3D model generator 216 can generate or modify a 3D model of a structure or an organ (e.g., a gastrointestinal tract) based on the point cloud, or a partial point cloud. For example, the 3D model generator 216 can obtain a first point cloud including a plurality of points in a Cartesian coordinate system corresponding to a surface of the structure and bounded by one or more holes corresponding to absent surface portions. For example, the 3D model generator 216 can obtain a partial point cloud including a plurality of points in a Cartesian coordinate system corresponding to the absent surface portion in the point cloud. For example, the 3D model generator 216 can modify the first point cloud to include the partial point cloud, resulting in a second point cloud or a modified first point cloud that reduces or eliminates the hole by filling the absent surface portion with the point cloud data of the partial point cloud.

[0041] For example, the point cloud processor 212 can generate the point cloud by identifying one or more image features in image data that correspond to a surface of a structure, according to a machine learning model configured to detect image features. In an aspect, the 3D model corresponds to a point cloud indicative of the anatomical structure. For example, the point cloud processor 212 can generate the point cloud by identifying coordinates associated with the one or more image features in the image data that correspond to the surface of the structure, according to a machine learning model configured to detect image features. For example, the point cloud processor 212 can generate the point cloud by identifying coordinates associated with the one or more points of depth data along a vector in a predetermined direction to the surface of the structure.

[0042] The navigation processor 130 can generate, according to one or more machine learning models, one or more positions of the robotic system 104 or a component thereof, to traverse a structure as discussed herein. For example, the navigation processor 130 can generate a path of traversal through an organ in a forward direction (entering the organ) or a backward direction (exiting the organ). The navigation processor 130 can generate the path to orient a sensor of the robotic system 104 to detect a surface associated with one or more holes or absent surface portions. The navigation processor 130 can use, access or include a machine learning model trained to generate a path that maximizes detection of a surface and minimizes movement, contact, force, or any combination thereof within the structure or organ (e.g., to minimize or prevent tissue damage). In some cases, the navigation processor 130 can use a deep neural network, reinforcement machine learning model, or execute an optimization technique to generate the path. The navigation processor 130 can include a model completeness processor 222, a sensor telemetry processor 224, and a machine learning model processor 226.

[0043] The model completeness processor 222 can determine whether a 3D model is complete, and can determine a quantitative level of completeness of a 3D model or a portion thereof. For example, the model completeness processor 222 can determine whether a point cloud associated with a 3D model satisfies a threshold indicative of completeness or level of completeness of the 3D model. For example, the model completeness processor 222 can determine whether a portion of a point cloud associated with a corresponding portion of a 3D model satisfies a threshold indicative of completeness or level of completeness of the corresponding portion of the 3D model. The model completeness processor 222 can identify a portion of the 3D model, for a corresponding portion of the point cloud not satisfying the threshold, as incomplete or a hole in the 3D model. The model completeness processor 222 can identify a portion of the 3D model, for a corresponding portion of the point cloud satisfying the threshold, as complete or a surface portion of the 3D model. In an aspect, the threshold is indicative of a portion of the point cloud having a quantity of points below a predetermined quantity. For example, a portion of the point cloud having a number of points below 100 can be identified by the model completeness processor 222 as incomplete or as a hole. In an aspect, the threshold is indicative of a portion of the point cloud having a density of points below a predetermined density of points. For example, a portion of the point cloud having a number of points below 10 per mm2 can be identified by the model completeness processor 222 as incomplete or as a hole. The amount of completeness that is desired may vary depending on the portion of the organ, type of organ, type of procedure, or any combination thereof, but is not limited thereto. For example, a region of the 3D model may be 50% complete, but the region may be of no consequence to the type of medical procedure. Here, the data processing system 102 may ignore the portion of the 3D model according to the determination that the portion is not relevant to the medical procedure. The navigation processor 130 can determine that a region where a polyp is identified historically with high frequency, and can increase the threshold of completeness for that portion of the 3D model. For example, the 3D model maybe 90% complete, but because it is an area of interest, that may not be sufficient. The navigation processor 130 can increase the level of completeness threshold to 99% for that region.

[0044] The sensor telemetry processor 224 can determine one or more states associated with one or more instruments 170 of the robotic system 104. The sensor telemetry processor 224 can identify a state corresponding to a position of one or of the instruments 170. For example, the sensor telemetry processor 224 can identify a position corresponding to an instrument of the robotic system 104 that corresponding to an articulable tube of a medical device to traverse a gastrointestinal tract or other organ along a direction of traverse along the organ. For example, the sensor telemetry processor 224 can determine a position of one or more portions of the articulable tube or a sensor of the articulable tube with respect to a coordinate system, with respect to the surface of the 3D model, with respect to the surface of the organ, or any combination thereof. For example, the sensor telemetry processor 224 can determine a position of an end of an articulable tube with respect to one or more surfaces of the organ and with respect to a distance from a point of entry to or exit from the organ (e.g., a rectal cavity opening). Thus, the sensor telemetry processor 224 can determine a location of a sensor within the structure or organ with respect to a given portion of a 3D model. For example, the sensor telemetry processor 224 can determine an orientation of one or more portions of the articulable tube or a sensor of the articulable tube with respect to a coordinate system, with respect to the surface of the 3D model, with respect to the surface of the organ, or any combination thereof. For example, the sensor telemetry processor 224 can determine an angle of an end of an articulable tube with respect to one or more surfaces of the organ and with respect to a distance from a point of entry to or exit from the organ (e.g., a rectal cavity opening). Thus, the sensor telemetry processor 224 can determine a direction of a field of view of a sensor with respect to a given portion of a 3D model.

[0045] The machine learning model processor 226 can execute or train one or more machine learning models as discussed herein. For example, the machine learning model processor 226 can obtain one or more features, one or more metrics, or any combination thereof, as inputs, but is not limited thereto. For example, the machine learning model processor 226 can execute a machine learning model configured to receive as input images features or image data corresponding to a given structure, given organ, or given point cloud. For example, the machine learning model processor 226 can train a machine learning model to identify a navigation path based on points clouds for a plurality instances of a structures (e.g., gastrointestinal tract data from a number of specimens), and a number of steps to complete a maximal amount of a point cloud with a minimal amount of movement, contact, force, or other interaction with an organ as discussed herein. Thus, the machine learning model processor 226 as discussed herein can provide a technical improvement to increase accuracy of identification of a surface of a structure, including an interior surface of an organ, at a level of granularity sufficient for medical diagnostics and an amount of motion and displacement to minimize tissue damage in vivo.

[0046] In an aspect, the machine learning model processor 226 can generate, by the machine learning model, a second action to modify the position of the sensor of the robotic medical system with respect to a direction of traversal through the anatomical structure. In an aspect, the machine learning model processor 226 can receive, by the machine learning model and responsive to the prompt, an instruction to modify the position of the sensor. The system can generate, by the machine learning model, the second action in response to the instruction. In an aspect, the machine learning model processor 226 can modify, by the machine learning model and responsive to the second action, the 3D model to include a second data set indicative of a portion of the 3D model of the anatomical structure corresponding to the modified position of the sensor.

[0047] The machine learning model to action to modify the position of the sensor of the robotic medical system, can correspond to a neural network or a deep learning system, for example. The neural network can be configured to receive input data including point cloud data, image data, or both. The neural network can include a feedback architecture to identify portions of the 3D model identified as incomplete, to focus the neural network on maximizing orientation of a field of view to incomplete portions with a minimum of movements. Thus, the neural network can provide a technical improvement to improve completeness and accuracy of 3D models captured according to a robotic device as discussed herein, via a feedback architecture to focus the neural network on incomplete regions.

[0048] The image capture processor 212 can identify one or more features in depictions in video data as discussed herein. For example, the depictions can include portions of a patient site, one or more medical instruments, or any combination thereof, but are not limited thereto. The image capture processor 212 can identify one or more edges, regions, or a structure within an image and associated with the depictions. For example, an edge can correspond to a line in an image that separates two depicted objects (e.g., a delineation between an instrument and a patient site). For example, a region can correspond to an area in an image that at least partially corresponds to a depicted object (e.g., an instrument tip). For example, a structure can correspond to an area in an image that at least partially corresponds to a portion of a depicted object or a predetermined type of an object (e.g., a scalpel edge). For example, the image capture processor 212 can include or correspond to a first machine learning model configured to identify the one or more features in the one or more images. For example, the system can generate, with the first machine learning model and based on the first feature, a second feature indicating an economy of motion for each of the plurality of videos.

[0049] The robotic device controller 140 can cause the robotic system 104 to one or more positions and orientations as discussed herein. For example, the articulable tube and sensor thereof can correspond to an endoscope. The robotic device controller 140 can effect robotic control of endoscope movement in a semi-automatic or an automatic mode. For example, in a semiautomatic mode of robotic control of navigation, the robotic device controller 140 can prompt the user via the user interface 160 to advance the endoscope instrument step-by-step along the path of traversal. In the semiautomatic mode, the robotic system 104 can pause advancing through the organ until an input is received at the user interface 160 that indicates an instruction to advance to a step in traversal through the organ. The user interface can include one or more user interface elements to instruct the endoscope instrument to advance to a next step, and to present one or more capture metrics indicative of one or more threshold, one or more levels of completeness, or presence of one or more features (e.g., polyps). For example, in an automatic robotic control of navigation, the robotic device controller 140 can prompt the user via the user interface 160 to authorize automatic navigation through multiple steps of the navigation path. For example, a navigation path can correspond to multiple changes of positions along a direction of traversal in increments ranging from 1 mm to 1 cm as determined by the machine learning model for the given structure.

[0050] The robotic device controller 140 can cause the user interface 160 to present one or more alerts to a physician user in response to detection of one or more image features by the machine learning model trained to detect image features (e.g., pause auto-navigation if new polyp detected). For example, a pause in auto-navigation can correspond to a temporary stoppage in advancement of the robotic system though the organ. The temporary stoppage can occur in accordance with a predetermined time period (e.g., 30 seconds) or can be ended in accordance with a conditional event (e.g., approval of advancement by an input at the user interface 160). For example, an alert can correspond to a visual indication at a display device of the user interface 160 that presents text, image, or media content or any sequence or combination thereof, but is not limited thereto. The visual indication can be indicative of a request to review a machine-identified polyp for confirmation by a medical professional. For example, the visual indication can include text stating, “POSSIBLE POLYP FOUND: REVIEW REQUESTED”). For example, an alert can correspond to an audio indication including a voice, a chime, a beep, a tone, or any sequence or combination thereof, but is not limited thereto. The audio indication can be indicative of a request to review a machine- identified polyp for confirmation by a medical professional. For example, the audio indication can include a voice stating, “Possible polyp found. Please review ”). For example, an alert can correspond to a haptic indication including force feedback having a given magnitude or pattern, or any sequence or combination thereof, but is not limited thereto. The haptic indication can be indicative of a request to review a machine-identified polyp for confirmation by a medical professional. For example, the haptic indication can include a vibration of a manipulator or seat in a predetermined pattern. Thus, the robotic device controller 140 can provide a technical improvement to achieve real-time analysis of new capture information by a vision model (e.g., to detect polyps and request physician user intervention), by a technical solution to automatically navigate to portions of interest of anatomical structures. For example, real-time can correspond to generation of a 3D model within 1 second of capture of sensor data from a robotic system 104 at least partially inserted in a cavity (e.g., an organ). Thus, the real-time analysis as discussed herein can provide a technical improvement at least to transform sensor data into usable 3D models for medical diagnostics, during execution of a medical procedure involving advancement of a sensor within an organ (e.g., endoscopy). [0051] The robotic device controller 140 can include a position input processor 232, an anatomy property processor 234, and a navigation path processor 236. The position input processor 232 can determine one or more positions of the instrument. For example, the position input processor 232 can determine one or more positions of one or more portions of the instrument at a current step. For example, the position input processor 232 can specify an example 3D model as point cloud, can specify position as along forward/b ackward traversal direction of navigation, or can specify orientation as angular vector or spherical vector within a 3D model coordinate space as discussed herein.

[0052] The anatomy property processor 234 can determine one or more properties of one or more portions of the organ according to one or more images or points of a point cloud corresponding to the organ. For example, the anatomy property processor 234 can determine one or more states of one or more portions of the organ according to a machine learning model configured to receive as input one or more both the image data, image features, and point clouds corresponding to the organ. For example, the anatomy property processor 234 can perform tissue pressure estimation to take into account the amount of pressure the endoscope puts on the tissue when planning the navigation plan. For example, the anatomy property processor 234 can perform inflation estimation to indicate an amount of inflation to inspect the tissue. The navigation path processor 236 can execute modification of the position and orientation of the robotic system 104 according to the navigation path. For example, the navigation path processor 236 can select the semi-automatic mode or the automatic mode in response to user input to select the corresponding mode at the user interface. For example, the navigation path processor 236 can re-generate a navigation path at each step, according to an updated model including an augmented portion of the 3D model as discussed herein. For example, the navigation path processor 236 can re-generate a navigation path in response to determining that a scan captures a significant amount of holes for the next step, allowing the next step to be recalculated for greater efficiency. An example scan that can be provided as input to the navigation path processor 236 for the re-generating, including the augmented portion, is illustrated in FIG. 5. In an aspect, the navigation path processor 236 can output, via a user interface, a prompt to modify the position of the sensor of the robotic medical system with respect to the direction of traversal. For example, the surgeon can retract the endoscope, while the endoscope automatically articulates in one or more angular directions to capture one or more holes in the 3D model during extraction. [0053] FIG. 3A depicts an example first navigation position of a robotic medical system, according to this disclosure. As illustrated by way of example in FIG. 3A, a first navigation position of a robotic medical system 300 A can include at least an organ 302, and an endoscopic sensor device 310A. The organ 302 can correspond to a gastrointestinal tract as discussed herein, but is not limited thereto. A forward direction with respect to the organ 302 can correspond to movement of the endoscopic sensor device 310 into the gastrointestinal tract from the point of entry into the body cavity. A backward direction with respect to the organ 302 can correspond to movement of the endoscopic sensor device 310 out of the gastrointestinal tract toward the point of entry into the body cavity. The endoscopic sensor device 310A can be in a first position within the organ 302. For example, the endoscopic sensor device 310A is in a first position corresponding to entry into the gastrointestinal tract from the rectal cavity. For example, the first position can correspond to a first step of a navigation path through the organ 302 in a forward direction.

[0054] FIG. 3B depicts an example second navigation position of a robotic medical system, according to this disclosure. As illustrated by way of example in FIG. 3B, a second navigation position of a robotic medical system 300B can include at least an endoscopic sensor device 310B. The endoscopic sensor device 310B can be in a second position within the organ 302. For example, the endoscopic sensor device 310A is in a second position corresponding to traversal into the gastrointestinal tract from the first position. For example, the second position can correspond to a second step of a navigation path through the organ 302 in a forward direction.

[0055] FIG. 3C depicts an example third navigation position of a robotic medical system, according to this disclosure. As illustrated by way of example in FIG. 3C, a third navigation position of a robotic medical system 300C can include at least an endoscopic sensor device 310C. The endoscopic sensor device 310C can be in a third position within the organ 302. For example, the endoscopic sensor device 310A is in a third position corresponding to traversal into the gastrointestinal tract from the second position. For example, the third position can correspond to a third step of a navigation path through the organ 302 in a forward direction.

[0056] FIG. 4 depicts an example capture for a 3D model of an anatomical structure, according to this disclosure. As illustrated by way of example in FIG. 4, a capture state 400 for a 3D model of an anatomical structure can include at least a 3D model 410 of an anatomical structure, a sensor of the robotic system 104, and an orientation of the sensor at a field of view 422. For example, the anatomical structure can refer to, include, or otherwise correspond to the organ 302 as discussed herein. For example, the capture state 400 can correspond to a first step of a navigation path traversing the anatomical structure corresponding to the 3D model 410. The 3D model 410 can be generated by the 3D model generator 216 as discussed herein. The 3D model 410 of an anatomical structure can include incomplete portions 412, 414 and 416. The incomplete portions 412, 414 and 416 can each correspond to distinct holes or distinct absent surface portions of the 3D model 410. The sensor 420 can correspond to a camera of an endoscope or a depth sensor of the endoscope. The sensor 420 can have an orientation toward the incomplete portion 412. The orientation of the sensor 420 can correspond to a field of view 422 oriented at a given angle with respect to the path of traversal of the endoscope through the anatomical structure.

[0057] FIG. 5 depicts an example modification of a 3D model of an anatomical structure, according to this disclosure. As illustrated by way of example in FIG. 5, a modification state 500 of a 3D model of an anatomical structure can include at least an augmented portion of the 3D model 510. For example, the modification state 500 can correspond to a second step of a navigation path traversing the anatomical structure corresponding to the 3D model 410, subsequent to the first step of FIG. 4. The sensor 420 can capture one or more of a partial point cloud or one or more images to complete a portion of the 3D model corresponding to the incomplete portion 412. The augmented portion of the 3D model 510 can be modified as discussed herein to include a partial point cloud generated based on data captured by the sensor 420, and can be integrated with the 3D model 410 to arrive at the 3D model 510.

[0058] In an aspect, the system can include modifying, by the machine learning model and responsive to the second action, the 3D model to include a second data set indicative of a portion of the 3D model of the anatomical structure corresponding to the modified position of the sensor. In an aspect, the system can receive, via the robotic medical system and responsive to the second action, the second data set. For example, the second data set is indicative of a portion of the 3D model of the anatomical structure corresponding to the modified position of the sensor according to the augmented portion of the 3D model 510.

[0059] FIG. 6 depicts an example method of navigation control for image capture and diagnostics in medical procedures, according to this disclosure. At least one of the system 100, the data processing system 102, or any component thereof, can perform method 600. For example, the data processing system 12 can perform at least a portion of the method 600 in an automatic mode, or in a semi-automatic mode, according to a configuration of the robotic system 104. For example, the robotic system 104 can be configurable to operate in a semiautomatic mode or an automatic mode, and the data processing system 102 can perform at least a portion of the method 600 according to a selection of the semi-automatic mode or the automatic mode for the robotic system 104. For example, the robotic system 104 can be configurable to operate in the semi-automatic mode, and the data processing system 102 can perform at least a portion of the method 600 according to a determination that the robotic system 104 supports the semi-automatic mode. For example, the robotic system 104 can be configurable to operate in the automatic mode, and the data processing system 102 can perform at least a portion of the method 600 according to a determination that the robotic system 104 supports the automatic mode.

[0060] At 610, the method 600 can receive a data set for a 3D model of an anatomical structure. For example, the 3D model processor 120 can receive a data set for a 3D model of an anatomical structure. For example, the image capture processor can receive a data set including video data or image data that corresponds to a 3D model of an anatomical structure. For example, the point cloud processor 214 can receive a data set including point cloud data that corresponds to a 3D model of an anatomical structure. For example, the data set can include at least one of the video data 152, the robot metrics 154, or any combination thereof, corresponding to a given medical procedure or type of medical procedure. At 612, the method 600 can receive the data set via a robotic medical system. For example, the 3D model processor 120 can receive the data set from the robotic system 104, via the network 101. For example, the 3D model processor 120 can receive the data set from a robotic endoscope system during an endoscopy procedure.

[0061] At 620, the method 600 can identify a level of completeness of the 3D model. For example, the navigation processor 130 can identify a level of completeness of the 3D model. For example, the model completeness processor 222 can identify a level of completeness of the 3D model. For example, the completeness processor 222 can identify a level of completeness of the 3D model based on the 3D model output by the 3D model generator 216 to the navigation processor 130 or the model completeness processor 222. At 622, the method 600 can identify the level of completeness of a portion of the 3D model. For example, the navigation processor 130 can identify the level of completeness of the portion of the 3D model based on a field of view corresponding to one or more sensors of the robotic system 104. For example, the sensors can correspond to one or more cameras of a robotic endoscopy device. The navigation processor 130 can obtain data for the sensors from the sensor telemetry processor 224. At 624, the method 600 can identify the level of completeness by a machine learning model that receives as input the 3D model. For example, the machine learning model can be configured to determine completeness according to a density or presence of one or more data points within the field of view as discussed herein, but is not limited thereto.

[0062] At 630, the method 600 can generate an action to increase the level of completeness of the 3D model. At 632, the method 600 can generate the action of the portion of the 3D model of the anatomical structure. At 634, the method 600 can generate the action to control at least one of a position and an orientation of a sensor of the robotic medical system. At 636, the method 600 can generate the action responsive to the level of completeness of the portion being less than or equal to a threshold. At 638, the method 600 can generate an action by the machine learning model. In an aspect, the method can include receiving, by the machine learning model and responsive to the prompt, an instruction to modify the position of the sensor. The method can include generating, by the machine learning model, the second action in response to the instruction. In an aspect, the method can include generating, by the machine learning model, a second action to modify the position of the sensor of the robotic medical system with respect to a direction of traversal through the anatomical structure. In an aspect, the system can determine a metric indicative of the traversal through the anatomical structure. The system can generate the second action in response to a determination that the metric satisfies a second threshold. In an aspect, the method can include causing a user interface to present a prompt to modify the position of the sensor of the robotic medical system with respect to the direction of traversal.

[0063] Having now described some illustrative implementations, the foregoing is illustrative and not limiting, having been presented by way of example. In particular, although many of the examples presented herein involve specific combinations of method acts or system elements, those acts and those elements may be combined in other ways to accomplish the same objectives. Acts, elements and features discussed in connection with one implementation are not intended to be excluded from a similar role in other implementations.

[0064] The phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of "including," "comprising," "having," "containing," "involving," "characterized by," "characterized in that," and variations thereof herein, is meant to encompass the items listed thereafter, equivalents thereof, and additional items, as well as alternate implementations consisting of the items listed thereafter exclusively. In one implementation, the systems and methods described herein consist of one, each combination of more than one, or all of the described elements, acts, or components.

[0065] References to "or" may be construed as inclusive so that any terms described using "or" may indicate any of a single, more than one, and all of the described terms. References to at least one of a conjunctive list of terms may be construed as an inclusive OR to indicate any of a single, more than one, and all of the described terms. For example, a reference to "at least one of 'A' and 'B'" can include only 'A', only 'B', as well as both "A¹ and 'B'. Such references used in conjunction with "comprising" or other open terminology can include additional items. References to "is" or "are" may be construed as nonlimiting to the implementation or action referenced in connection with that term. The terms "is" or "are" or any tense or derivative thereof, are interchangeable and synonymous with "can be" as used herein, unless stated otherwise herein.

[0066] Directional indicators depicted herein are example directions to facilitate understanding of the examples discussed herein, and are not limited to the directional indicators depicted herein. Any directional indicator depicted herein can be modified to the reverse direction, or can be modified to include both the depicted direction and a direction reverse to the depicted direction, unless stated otherwise herein. While operations are depicted in the drawings in a particular order, such operations are not required to be performed in the particular order shown or in sequential order, and all illustrated operations are not required to be performed. Actions described herein can be performed in a different order. Where technical features in the drawings, detailed description or any claim are followed by reference signs, the reference signs have been included to increase the intelligibility of the drawings, detailed description, and claims. Accordingly, neither the reference signs nor their absence have any limiting effect on the scope of any clam elements.

[0067] Scope of the systems and methods described herein is thus indicated by the appended claims, rather than the foregoing description. The scope of the claims includes equivalents to the meaning and scope of the appended claims.

Claims

WHAT IS CLAIMED IS:

1. A system, comprising: one or more processors, coupled with memory, to: receive, via a robotic medical system, a data set corresponding to a three-dimensional (3D) model of an anatomical structure; identify, by a machine learning model that receives as input the 3D model, a level of completeness of a portion of the 3D model; and generate, by the machine learning model and responsive to the level of completeness of the portion being less than or equal to a threshold, an action to control at least one of a position or an orientation of a sensor of the robotic medical system to increase the level of completeness of the portion of the 3D model of the anatomical structure.

2. The system of claim 1, comprising the one or more processors to: generate, by the machine learning model, a second action to modify the position of the sensor of the robotic medical system with respect to a direction of traversal through the anatomical structure.

3. The system of claim 2, comprising the one or more processors to: output, via a user interface, a prompt to modify the position of the sensor of the robotic medical system with respect to the direction of traversal.

4. The system of claim 3, comprising the one or more processors to: receive, by the machine learning model and responsive to the prompt, an instruction to modify the position of the sensor; and generate, by the machine learning model, the second action in response to the instruction.

5. The system of claim 2, comprising the one or more processors to: modify, by the machine learning model and responsive to the second action, the 3D model to include a second data set indicative of a portion of the 3D model of the anatomical structure corresponding to the modified position of the sensor.

6. The system of claim 5, comprising the one or more processors to: receive, via the robotic medical system and responsive to the second action, the second data set.

7. The system of claim 2, comprising the one or more processors to: determine a metric indicative of the traversal through the anatomical structure; and generate the second action in response to a determination that the metric satisfies a second threshold.

8. The system of claim 7, wherein the metric corresponds to a tissue pressure metric indicative of a force on the anatomical structure caused the traversal through the anatomical structure, and the second threshold is indicative of a maximum force on the anatomical structure.

9. The system of claim 7, wherein the metric corresponds to an inflation metric indicative of an amount of inflation to inspect at least a portion of the anatomical structure corresponding to the traversal through the anatomical structure, and the second threshold is indicative of a maximum inflation of the portion of the anatomical structure.

10. The system of claim 1, wherein the 3D model corresponds to a point cloud indicative of the anatomical structure.

11. The system of claim 10, wherein the threshold is indicative of a portion of the point cloud having a quantity of points below a predetermined quantity.

12. The system of claim 10, wherein the threshold is indicative of a portion of the point cloud having a density of points below a predetermined density of points.

13. The system of claim 1, wherein the orientation corresponds to a relative position of the sensor of the robotic medical system with respect to the portion of the 3D model or the anatomical structure.

14. A method, comprising: receiving, by one or more processors coupled with memory, via a robotic medical system, a data set corresponding to a three-dimensional (3D) model of an anatomical structure; identifying, by the one or more processors, using a machine learning model that receives as input the 3D model, a level of completeness of a portion of the 3D model; and generating, by the one or more processors, using the machine learning model and responsive to the level of completeness of the portion being less than or equal to a threshold, an action to control at least one of a position or an orientation of a sensor of the robotic medical system to increase the level of completeness of the portion of the 3D model of the anatomical structure.

15. The method of claim 14, further comprising: generating, by the one or more processors using the machine learning model, a second action to modify the position of the sensor of the robotic medical system with respect to a direction of traversal through the anatomical structure.

16. The method of claim 15, further comprising: output, by the one or more processors via a user interface, a prompt to modify the position of the sensor of the robotic medical system with respect to the direction of traversal.

17. The method of claim 16, further comprising: receiving, by the one or more processors via the machine learning model and responsive to the prompt, an instruction to modify the position of the sensor; and generating, by the machine learning model, the second action in response to the instruction.

18. The method of claim 15, further comprising: modifying, by the one or more processors via the machine learning model and responsive to the second action, the 3D model to include a second data set indicative of a portion of the 3D model of the anatomical structure corresponding to the modified position of the sensor.

19. A non-transitory computer readable medium including one or more instructions stored thereon and executable by a processor to: receive, by the processor via a robotic medical system, a data set corresponding to a three-dimensional (3D) model of an anatomical structure; identify, by the processor via a machine learning model that receives as input the 3D model, a level of completeness of a portion of the 3D model; and generate, by the processor via the machine learning model and responsive to the level of completeness of the portion being less than or equal to a threshold, an action to control at least one of a position or an orientation of a sensor of the robotic medical system to increase the level of completeness of the portion of the 3D model of the anatomical structure.

20. The non-transitory computer readable medium of claim 19, the non-transitory computer readable medium further including one or more instructions executable by the processor to: generate, by the processor via the machine learning model, a second action to modify the position of the sensor of the robotic medical system with respect to a direction of traversal through the anatomical structure.