WO2019179023A1 - Indoor image processing method, server and computer readable storage medium - Google Patents
Indoor image processing method, server and computer readable storage medium Download PDFInfo
- Publication number
- WO2019179023A1 WO2019179023A1 PCT/CN2018/102079 CN2018102079W WO2019179023A1 WO 2019179023 A1 WO2019179023 A1 WO 2019179023A1 CN 2018102079 W CN2018102079 W CN 2018102079W WO 2019179023 A1 WO2019179023 A1 WO 2019179023A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- picture
- degree
- mobile terminal
- stitched
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/16—Real estate
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/63—Control of cameras or camera modules by using electronic viewfinders
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
Definitions
- the present application relates to the field of computer technologies, and in particular, to an indoor image processing method, a server, and a computer readable storage medium.
- real estate plays an important role in the e-commerce business.
- vision will be the most intuitive experience for customers.
- customers often choose to conduct a basic screening of the property on the e-commerce platform, and then examine the filtered property online. In this case, if the indoor image details of the property cannot be better displayed to the customer on the e-commerce platform, the customer may not further investigate the corresponding property, thereby losing the possible transaction and affecting the corresponding performance. which performed.
- the present application provides an indoor image processing method, a server, and a computer readable storage medium, which can acquire a real-time indoor scene image according to the mobile terminal, and quickly acquire an indoor 360-degree or 720-degree panoramic image and display it to the end user.
- the indoor situation of the target property can be more stereoscopically displayed on the online, so that the consumer obtains more first sensory information.
- the present application provides a server, which includes a memory, a processor, and an image processing program executable on the processor, where the image processing program is The processor performs the following steps: receiving a picture uploaded by the mobile terminal; determining, according to the picture uploaded by the mobile terminal, the collection angle of the picture by the mobile terminal; and selecting a corresponding picture splicing algorithm according to the collection angle, The picture uploaded by the mobile terminal is spliced by using the picture splicing algorithm; and the spliced picture is displayed.
- the present application further provides an indoor image processing method, which is applied to a server, and the server is connected to a mobile terminal, and the method includes: receiving a picture uploaded by the mobile terminal; and uploading according to the mobile terminal. Determining, by the picture, the collection angle of the picture by the mobile terminal, selecting a corresponding picture splicing algorithm according to the collection angle, splicing the picture uploaded by the mobile terminal by using the picture splicing algorithm, and performing splicing the picture Show.
- the present application further provides a computer readable storage medium storing an indoor image processing program, the indoor image processing program being executable by at least one processor, so that The at least one processor performs the steps of the indoor image processing method as described above.
- the indoor image processing method, the server, and the computer readable storage medium proposed by the present application firstly receive a picture uploaded by the mobile terminal; secondly, determine the mobile according to the picture uploaded by the mobile terminal.
- the terminal collects the collection angle of the picture; again, selects a corresponding picture splicing algorithm according to the collection angle, and uses the picture splicing algorithm to splicing the picture uploaded by the mobile terminal; finally, displaying the spliced picture .
- the real-time scene image can be obtained according to the mobile terminal, and the 360-degree or 720-degree panoramic image of the target indoor space can be quickly obtained and displayed to the end user, which can make the display on the target line more stereoscopic, so that the consumer obtains more A sensory data.
- 1 is a schematic diagram of an optional hardware architecture of the server of the present application.
- FIG. 2 is a block diagram showing the program of the first embodiment of the image processing program of the present application.
- FIG. 3 is a flowchart of a first embodiment of a method for processing indoor images of the present application
- FIG. 4 is a flow chart of a second embodiment of the indoor image processing method of the present application.
- FIG. 1 it is a schematic diagram of an optional hardware architecture of the server 1.
- the server 1 may be a computing device such as a rack server, a blade server, a tower server, or a rack server.
- the server 1 may be a standalone server or a server cluster composed of multiple servers.
- the server 1 may include, but is not limited to, a memory 11, a processor 12, and a network interface 13 that can communicate with each other through a system bus.
- the server 1 connects to the network through the network interface 13 to obtain information.
- the network may be an intranet, an Internet, a Global System of Mobile communication (GSM), a Wideband Code Division Multiple Access (WCDMA), a 4G network, or a 5G network.
- Wireless or wired networks such as networks, Bluetooth, Wi-Fi, and call networks.
- Figure 1 only shows the server 1 with the components 11-13, but it should be understood that not all illustrated components are required to be implemented, and more or fewer components may be implemented instead.
- the memory 11 includes at least one type of storage medium (computer readable storage medium) including a flash memory, a hard disk, a multimedia card, a card type memory (eg, SD or DX memory, etc.), a random access memory. (RAM), static random access memory (SRAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), programmable read only memory (PROM), magnetic memory, magnetic disk, optical disk, and the like.
- the memory 11 may be an internal storage unit of the server 1, such as a hard disk or memory of the server 1.
- the memory 11 may also be an external storage device of the server 1, such as a plug-in hard disk equipped with the server 1, a smart memory card (SMC), and a secure digital (Secure Digital). , SD) card, flash card (Flash Card), etc.
- the memory 11 can also include both the internal storage unit of the server 1 and its external storage device.
- the memory 11 is generally used to store an operating system installed in the server 1 and various types of application software, such as program code of the image processing program 200. Further, the memory 11 can also be used to temporarily store various types of data that have been output or are to be output.
- the processor 12 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments.
- the processor 12 is typically used to control the overall operation of the server 1, such as performing data interaction or communication related control and processing, and the like.
- the processor 12 is configured to run program code or process data stored in the memory 11, such as running the image processing program 200 and the like.
- the network interface 13 may comprise a wireless network interface or a wired network interface, which is typically used to establish a communication connection between the server 1 and other electronic devices.
- an image processing program 200 is installed and run in the server 1.
- the server 1 first receives a picture uploaded by the mobile terminal; secondly, according to the movement The picture uploaded by the terminal determines that the mobile terminal collects the collection angle of the picture; again, selects a corresponding picture splicing algorithm according to the collection angle; and then splices the picture uploaded by the mobile terminal by using the picture splicing algorithm; Finally, the stitched pictures are displayed.
- the real-time scene image can be obtained according to the mobile terminal, and the 360-degree or 720-degree panoramic image of the target space can be quickly obtained and displayed to the end user, which can make the display on the target line more stereoscopic, and use the consumer to obtain more first.
- Sensory information can be obtained according to the mobile terminal, and the 360-degree or 720-degree panoramic image of the target space can be quickly obtained and displayed to the end user, which can make the display on the target line more stereoscopic, and use the consumer to obtain more first.
- Sensory information can be obtained according to the mobile terminal, and the 360-degree or 720-degree panoramic image of the target space can be quickly obtained and displayed to the end user, which can make the display on the target line more stereoscopic, and use the consumer to obtain more first.
- the present application proposes an indoor image processing program 200.
- FIG. 2 it is a program block diagram of the first embodiment of the indoor image processing program 200 of the present application.
- the server 1 includes a series of computer program instructions stored in the memory 11, that is, the indoor image processing program 200.
- the indoor image processing program 200 can be divided into one or more modules based on the particular operations implemented by the various portions of the computer program instructions.
- the indoor image processing program 200 can be divided into a receiving module 201, a determining module 202, an algorithm selecting module 203, a splicing module 204, and a display module 205. among them:
- the receiving module 201 is configured to receive a picture uploaded by the mobile terminal.
- the mobile terminal (not shown) is a portable terminal having an image acquisition function and a data transmission function, such as a smart phone, a personal digital assistant, a tablet computer, and the like.
- the picture uploaded by the mobile terminal is obtained by taking a picture of the scene according to the requirement of the user. Specifically, during the photo taking process, if the effect is not satisfactory after a single photo, you can choose to cancel the photo result. The photo will not be uploaded to the server for splicing. If the effect is satisfactory, you need to save the photo.
- the steps of receiving the picture uploaded by the mobile terminal are described in detail in FIG. 4.
- the determining module 202 is configured to determine, according to the picture uploaded by the mobile terminal, the collection angle of the picture collected by the mobile terminal.
- an acquisition angle digital identifier will be embedded in the acquired image and hidden in the image pixel set.
- the determining module 203 parses the pixel set of the image, and then can determine the collection angle of the currently analyzed image.
- the algorithm selection module 203 is configured to select a corresponding image splicing algorithm according to the collection angle.
- the picture stitching algorithm includes an opencv picture 360 degree panoramic stitching algorithm and an opencv picture 720 degree panoramic stitching algorithm.
- the 360-degree panoramic splicing algorithm of the opencv picture includes the following steps: 1. performing feature point extraction on each picture; 2. matching feature points, where only feature extraction or matching is needed, only need to extract or match The spliced picture is symmetrical on both sides of the image acquisition direction, such as a 360-degree picture acquisition relative to the left and right direction of the photographer, only need to extract and match the feature points of the left and right symmetrical edges; 3. Perform image registration; 4.
- the image is copied to a specific position of another image, and the specific position is located within a certain distance from the edge of the other image, and the setting of the distance may be set according to user needs, which is not limited herein; 5.
- Special treatment of overlapping boundaries such as removing pixels with the same brightness and hue, and removing image noise.
- the 720 degree panoramic splicing algorithm of the opencv picture includes the following steps: 1. performing feature point extraction on each picture; 2. matching feature points, when performing feature point extraction or matching, all edges of the picture to be spliced are needed The area performs feature point extraction and matching; 3. performs image registration; 4. copies the image to a specific position of another image, the specific position is within a certain distance from the edge of the other image, The setting of the distance can be set according to the needs of the user, and is not limited herein; 5. Special processing is performed on the overlapping boundary, such as removing pixels with the same brightness and hue, and removing image noise.
- the splicing module 204 is configured to splicing a picture uploaded by the mobile terminal by using the picture splicing algorithm.
- the feature points are collected and matched on the symmetrical two sides of the acquired image in the acquisition direction, and then the image splicing is performed.
- the image splicing algorithm corresponding to 720 degrees is selected, the feature points are extracted and matched to the portions within the predetermined range around the acquired image, thereby performing the above-mentioned picture splicing.
- the display module 205 is configured to display the stitched picture.
- the determining module 202 further analyzes the spliced picture to determine whether the spliced picture conforms to the 360-degree spliced picture or the 720-degree picture. Specifically, the judging module 202 analyzes the spliced picture to obtain a picture overlapping area. If the picture overlapping area only belongs to the left and right or the upper and lower symmetrical distribution, the splicing picture conforming to 360 degrees is determined, and if the overlapping area of the picture is closed, The area is judged to match the 720-degree stitched picture.
- the display module 205 processes and processes the 360-degree spliced picture by using a krpano engine, and generates an html page, where the html page is used for access by the mobile terminal.
- the krpano engine is the Krpano panoramic display engine, but it adopts xml data structure and flash-based 3D panoramic display technology, and is a powerful 3D panoramic display engine.
- the presentation module 205 can cause a 360 degree stitched picture to be quickly rendered to produce a panoramic image.
- an html page is generated, which can enable the mobile terminal to quickly access and display the panoramic image to the user.
- the display module 205 returns a 720-degree spliced picture to the mobile terminal, and the mobile terminal renders the 720-degree spliced picture by using an open source opengl technology.
- Perform a 720 degree display For the 720-degree stitched picture, the corresponding rendering process is not performed on the server side, but the stitched picture is directly transmitted back to each mobile terminal that needs to browse the service, and the mobile terminal directly spliced the picture through the opengl technology. Rendering shows 720 degrees.
- the amount of 720-degree stitched picture data is larger than the data amount of the general 360-degree stitched picture, by transferring the post-render processing to the mobile terminal, the running pressure of the server 1 end can be effectively reduced, and the picture display efficiency is improved.
- the server of the present application receives the picture uploaded by the mobile terminal, and determines, according to the picture uploaded by the mobile terminal, the collection angle of the picture by the mobile terminal, according to the program module 201-205; Selecting a corresponding image splicing algorithm, and splicing the image uploaded by the mobile terminal by using the image splicing algorithm; and displaying the spliced image.
- the real-time scene image can be obtained according to the mobile terminal, and the 360-degree or 720-degree panoramic image of the target indoor space can be quickly obtained and displayed to the end user, which can make the indoor space of the target appear more stereoscopic on the line, so that the consumer can obtain more. More first sensory information.
- the present application also proposes an indoor image processing method.
- FIG. 3 it is a schematic flowchart of the implementation of the first embodiment of the indoor image processing method of the present application.
- the order of execution of the steps in the flowchart shown in FIG. 3 may be changed according to different requirements, and some steps may be omitted.
- Step S301 Receive a picture uploaded by the mobile terminal.
- the mobile terminal is a portable terminal having an image acquisition function and a data transmission function, such as a smart phone, a personal digital assistant, a tablet computer, and the like.
- the picture uploaded by the mobile terminal is obtained by taking a picture of the scene according to the requirement of the user. Specifically, during the photo taking process, if the effect is not satisfactory after a single photo taking, you can choose to cancel the photo result. The photo will not be uploaded to the server 1 for splicing. If the effect is satisfactory, you need to save the photo.
- the step of receiving the picture uploaded by the mobile terminal is specifically described in the second embodiment of the indoor image processing method of the present application (as shown in FIG. 4).
- Step S302 Determine, according to the picture uploaded by the mobile terminal, the collection angle of the picture collected by the mobile terminal.
- an acquisition angle digital identifier will be embedded in the acquired image and hidden in the image pixel set.
- the determining module 203 parses the pixel set of the image, and then can determine the collection angle of the currently analyzed image.
- Step S303 selecting a corresponding picture stitching algorithm according to the collection angle.
- the picture stitching algorithm includes an opencv picture 360 degree panoramic stitching algorithm and an opencv picture 720 degree panoramic stitching algorithm.
- the 360-degree panoramic splicing algorithm of the opencv picture includes the following steps: 1. performing feature point extraction on each picture; 2. matching feature points, where only feature extraction or matching is needed, only need to extract or match The spliced picture is symmetrical on both sides of the image acquisition direction, such as a 360-degree picture acquisition relative to the left and right direction of the photographer, only need to extract and match the feature points of the left and right symmetrical edges; 3. Perform image registration; 4. Put The image is copied to a specific position of another image, and the specific position is located within a certain distance from the edge of the other image, and the setting of the distance may be set according to user needs, which is not limited herein; 5. Special treatment of overlapping boundaries, such as removing pixels with the same brightness and hue, and removing image noise.
- the 720 degree panoramic splicing algorithm of the opencv picture includes the following steps: 1. performing feature point extraction on each picture; 2. matching feature points, when performing feature point extraction or matching, all edges of the picture to be spliced are needed The area performs feature point extraction and matching; 3. performs image registration; 4. copies the image to a specific position of another image, the specific position is within a certain distance from the edge of the other image, The setting of the distance can be set according to the needs of the user, and is not limited herein; 5. Special processing is performed on the overlapping boundary, such as removing pixels with the same brightness and hue, and removing image noise.
- step S304 the picture uploaded by the mobile terminal is spliced by using the picture splicing algorithm.
- the feature points are collected and matched on the symmetrical two sides of the acquired image in the acquisition direction, and then the image splicing is performed.
- the image splicing algorithm corresponding to 720 degrees is selected, the feature points are extracted and matched to the portions within the predetermined range around the acquired image, thereby performing image splicing.
- step S305 the stitched picture is displayed.
- the spliced picture is further analyzed to determine whether the spliced picture conforms to a 360-degree spliced picture or a 720-degree picture.
- the judging module 202 analyzes the spliced picture to obtain a picture overlapping area. If the picture overlapping area only belongs to the left and right or the upper and lower symmetrical distribution, the splicing picture conforming to 360 degrees is determined, and if the overlapping area of the picture is closed, The area is judged to match the 720-degree stitched picture.
- the 360-degree spliced picture is processed by using a krpano engine, and an html page is generated, and the html page is used to display the panoramic image when the mobile terminal accesses .
- the krpano engine is the Krpano panoramic display engine, but it adopts xml data structure and flash-based 3D panoramic display technology, and is a powerful 3D panoramic display engine.
- 360-degree stitching images can be quickly rendered to produce a panoramic image.
- an html page is generated, which can enable the mobile terminal to quickly access and display the panoramic image to the user.
- the 720-degree spliced picture is transmitted back to the mobile terminal, and the mobile terminal uses the open source opengl technology to render the 720-degree spliced picture for 720 degrees.
- the corresponding rendering process is not performed on the server side, but the stitched picture is directly transmitted back to each mobile terminal that needs to browse the service, and the mobile terminal directly spliced the picture through the opengl technology.
- Rendering shows 720 degrees.
- the amount of 720-degree stitched picture data is larger than the data amount of the general 360-degree stitched picture, by transferring the post-render processing to the mobile terminal, the running pressure of the server end can be effectively reduced, and the picture display efficiency is improved.
- the indoor image processing method proposed by the present application firstly receives a picture uploaded by the mobile terminal; secondly, determining, according to the picture uploaded by the mobile terminal, the collection angle of the picture collected by the mobile terminal And again, selecting a corresponding image splicing algorithm according to the collection angle, and splicing the image uploaded by the mobile terminal by using the image splicing algorithm; and finally, displaying the spliced image.
- the real-time scene image can be obtained according to the mobile terminal, and the 360-degree or 720-degree panoramic image of the target indoor space can be quickly obtained and displayed to the end user, which can make the display of the target indoor space more stereoscopic on the line, so that the consumer can obtain more.
- the first sensory information is provided.
- FIG. 4 it is a schematic flowchart of the implementation of the second embodiment of the indoor image processing method of the present application.
- the order of execution of the steps in the flowchart shown in FIG. 4 may be changed according to different requirements, and some steps may be omitted.
- FIG. 4 shows a further description of step S301 in FIG.
- step S401 the mobile terminal is controlled to turn on the camera application.
- the server 1 can send a control command to the mobile terminal through the communication network, and then remotely control the mobile terminal to open the camera application, thereby preparing to acquire an image.
- a picture acquisition angle is selected.
- the mobile terminal selects a picture acquisition angle according to the user's touch operation, for example, selecting a 360-degree shooting or a 720-degree shooting, wherein the 720-degree shooting selection is generally a wide-angle lens, and the viewing angle range is larger than a general lens.
- step S403 a shooting guide is displayed on the camera interface according to the selected collection angle, and the shooting guide is used to guide the user to perform picture collection by using the mobile terminal.
- the shooting guidance of the corresponding angle is displayed on the shooting interface of the camera, for example, the 360-degree acquisition angle is guided by an arrow indicating the XOY surface in the stereo coordinate system.
- the guidance is indicated by two XOY faces in the three-dimensional coordinate system and YOZ arrows.
- the image is acquired by slowly moving the shooting angle according to the shooting guide, thereby providing material for subsequent image stitching.
- step S405 the captured picture is selected for uploading.
- the mobile terminal uploads the acquired picture or image to the server through the wireless network, for further processing by the server 1.
- the indoor image processing method proposed by the present application can receive the scene picture uploaded by the mobile terminal according to the user demand.
- the present application further provides another embodiment, that is, a computer readable storage medium storing an indoor image processing program, the indoor image processing program being executable by at least one processor, The at least one processor is caused to perform the steps of the indoor image processing method as described above.
- the foregoing embodiment method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be through hardware, but in many cases, the former is better.
- Implementation Based on such understanding, the technical solution of the present application, which is essential or contributes to the prior art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk,
- the optical disc includes a number of instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the methods described in various embodiments of the present application.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Tourism & Hospitality (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Description
本申请要求于2018年3月19日提交中国专利局、申请号为201810226472.7、发明名称为“室内图像处理方法、服务器及计算机可读存储介质”的中国专利申请的优先权,其全部内容通过引用结合在申请中。The present application claims priority to Chinese Patent Application No. 201101226472.7, entitled "Indoor Image Processing Method, Server and Computer Readable Storage Media", filed on March 19, 2018, the entire contents of which are incorporated by reference. Combined in the application.
本申请涉及计算机技术领域,尤其涉及一种室内图像处理方法、服务器及计算机可读存储介质。The present application relates to the field of computer technologies, and in particular, to an indoor image processing method, a server, and a computer readable storage medium.
随着互联网的发展,越来越多的产业开始跟互联网结合,房地产电子商务也应运而生。相对于传统的房地产居间经营模式,房产电商已经从最基础的房地产信息电子化,涵盖到房地产经纪代理机构的地面服务和内部运营管理的各个领域。With the development of the Internet, more and more industries have begun to integrate with the Internet, and real estate e-commerce has emerged. Compared with the traditional real estate intermediation mode, real estate e-commerce has been electronically transformed from the most basic real estate information, covering the real estate brokerage agency's ground services and internal operations management.
作为房地产电子商务平台的主要产品,房产在电商业务中占据举足轻重的地位。对一套房产而言,视觉将会是客户对房产最直观的体验。在当今的互联网时代,客户通常会选择在电商平台上对房产进行基本的筛选,再在线下对筛选过的房产进行考察。在此种情况下,如果不能更好地在电商平台上将房产的室内图像细节显示给客户,客户可能不会对相应的房产做进一步的考察,进而丧失了可能的交易,影响相应的业绩表现。As the main product of the real estate e-commerce platform, real estate plays an important role in the e-commerce business. For a property, vision will be the most intuitive experience for customers. In today's Internet age, customers often choose to conduct a basic screening of the property on the e-commerce platform, and then examine the filtered property online. In this case, if the indoor image details of the property cannot be better displayed to the customer on the e-commerce platform, the customer may not further investigate the corresponding property, thereby losing the possible transaction and affecting the corresponding performance. which performed.
发明内容Summary of the invention
有鉴于此,本申请提出一种室内图像处理方法、服务器及计算机可读存储介质,可以根据移动终端获取实时的室内场景图片,快速的获取室内360度 或720度全景影像,并显示给最终用户,据此可以使得目标房产的室内情况在线上的展示更加立体,使消费者获得更多的第一感官资料。In view of this, the present application provides an indoor image processing method, a server, and a computer readable storage medium, which can acquire a real-time indoor scene image according to the mobile terminal, and quickly acquire an indoor 360-degree or 720-degree panoramic image and display it to the end user. According to this, the indoor situation of the target property can be more stereoscopically displayed on the online, so that the consumer obtains more first sensory information.
首先,为实现上述目的,本申请提出一种服务器,所述服务器包括存储器、处理器,所述存储器上存储有可在所述处理器上运行的图像处理程序,所述图像处理程序被所述处理器执行时实现如下步骤:接收所述移动终端上传的图片;根据所述移动终端上传的图片判断所述移动终端采集所述图片的采集角度;根据所述采集角度选择对应的图片拼接算法,利用所述图片拼接算法对所述移动终端上传的图片进行拼接;将拼接后的图片进行展示。First, in order to achieve the above object, the present application provides a server, which includes a memory, a processor, and an image processing program executable on the processor, where the image processing program is The processor performs the following steps: receiving a picture uploaded by the mobile terminal; determining, according to the picture uploaded by the mobile terminal, the collection angle of the picture by the mobile terminal; and selecting a corresponding picture splicing algorithm according to the collection angle, The picture uploaded by the mobile terminal is spliced by using the picture splicing algorithm; and the spliced picture is displayed.
此外,为实现上述目的,本申请还提供一种室内图像处理方法,应用于服务器,所述服务器连接移动终端,所述方法包括:接收所述移动终端上传的图片;根据所述移动终端上传的图片判断所述移动终端采集所述图片的采集角度;根据所述采集角度选择对应的图片拼接算法,利用所述图片拼接算法对所述移动终端上传的图片进行拼接;及将拼接后的图片进行展示。In addition, in order to achieve the above object, the present application further provides an indoor image processing method, which is applied to a server, and the server is connected to a mobile terminal, and the method includes: receiving a picture uploaded by the mobile terminal; and uploading according to the mobile terminal. Determining, by the picture, the collection angle of the picture by the mobile terminal, selecting a corresponding picture splicing algorithm according to the collection angle, splicing the picture uploaded by the mobile terminal by using the picture splicing algorithm, and performing splicing the picture Show.
进一步地,为实现上述目的,本申请还提供一种计算机可读存储介质,所述计算机可读存储介质存储有室内图像处理程序,所述室内图像处理程序可被至少一个处理器执行,以使所述至少一个处理器执行如上所述的室内图像处理方法的步骤。Further, in order to achieve the above object, the present application further provides a computer readable storage medium storing an indoor image processing program, the indoor image processing program being executable by at least one processor, so that The at least one processor performs the steps of the indoor image processing method as described above.
相较于现有技术,本申请所提出的室内图像处理方法、服务器及计算机可读存储介质,首先,接收所述移动终端上传的图片;其次,根据所述移动终端上传的图片判断所述移动终端采集所述图片的采集角度;再次,根据所述采集角度选择对应的图片拼接算法,并利用所述图片拼接算法对所述移动终端上传的图片进行拼接;最后,将拼接后的图片进行展示。这样,可以根据移动终端获取实时场景图片,快速的获取目标室内空间的360度或720度全景影像,并显示给最终用户,可以使得目标在线上的展示更加立体,使消费者获得更多的第一感官资料。Compared with the prior art, the indoor image processing method, the server, and the computer readable storage medium proposed by the present application firstly receive a picture uploaded by the mobile terminal; secondly, determine the mobile according to the picture uploaded by the mobile terminal. The terminal collects the collection angle of the picture; again, selects a corresponding picture splicing algorithm according to the collection angle, and uses the picture splicing algorithm to splicing the picture uploaded by the mobile terminal; finally, displaying the spliced picture . In this way, the real-time scene image can be obtained according to the mobile terminal, and the 360-degree or 720-degree panoramic image of the target indoor space can be quickly obtained and displayed to the end user, which can make the display on the target line more stereoscopic, so that the consumer obtains more A sensory data.
图1是本申请服务器一可选的硬件架构的示意图;1 is a schematic diagram of an optional hardware architecture of the server of the present application;
图2是本申请图像处理程序第一实施例的程序模块图;Figure 2 is a block diagram showing the program of the first embodiment of the image processing program of the present application;
图3为本申请室内图像处理方法第一实施例的流程图;3 is a flowchart of a first embodiment of a method for processing indoor images of the present application;
图4为本申请室内图像处理方法第二实施例的流程图。4 is a flow chart of a second embodiment of the indoor image processing method of the present application.
本申请目的的实现、功能特点及优点将结合实施例,参照附图做进一步说明。The implementation, functional features and advantages of the present application will be further described with reference to the accompanying drawings.
为了使本申请的目的、技术方案及优点更加清楚明白,以下结合附图及实施例,对本申请进行进一步详细说明。应当理解,此处所描述的具体实施例仅用以解释本申请,并不用于限定本申请。基于本申请中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。In order to make the objects, technical solutions, and advantages of the present application more comprehensible, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the application and are not intended to be limiting. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.
需要说明的是,在本申请中涉及“第一”、“第二”等的描述仅用于描述目的,而不能理解为指示或暗示其相对重要性或者隐含指明所指示的技术特征的数量。由此,限定有“第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。另外,各个实施例之间的技术方案可以相互结合,但是必须是以本领域普通技术人员能够实现为基础,当技术方案的结合出现相互矛盾或无法实现时应当认为这种技术方案的结合不存在,也不在本申请要求的保护范围之内。It should be noted that the descriptions of "first", "second" and the like in the present application are for the purpose of description only, and are not to be construed as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. . Thus, features defining "first" or "second" may include at least one of the features, either explicitly or implicitly. In addition, the technical solutions between the various embodiments may be combined with each other, but must be based on the realization of those skilled in the art, and when the combination of the technical solutions is contradictory or impossible to implement, it should be considered that the combination of the technical solutions does not exist. Nor is it within the scope of protection required by this application.
参阅图1所示,是服务器1一可选的硬件架构的示意图。Referring to FIG. 1, it is a schematic diagram of an optional hardware architecture of the server 1.
所述服务器1可以是机架式服务器、刀片式服务器、塔式服务器或机柜式服务器等计算设备,该服务器1可以是独立的服务器,也可以是多个服务器所组成的服务器集群。The server 1 may be a computing device such as a rack server, a blade server, a tower server, or a rack server. The server 1 may be a standalone server or a server cluster composed of multiple servers.
本实施例中,所述服务器1可包括,但不仅限于,可通过系统总线相互通 信连接存储器11、处理器12、网络接口13。In this embodiment, the server 1 may include, but is not limited to, a memory 11, a processor 12, and a network interface 13 that can communicate with each other through a system bus.
所述服务器1通过网络接口13连接网络,获取资讯。所述网络可以是企业内部网(Intranet)、互联网(Internet)、全球移动通讯系统(Global System of Mobile communication,GSM)、宽带码分多址(Wideband Code Division Multiple Access,WCDMA)、4G网络、5G网络、蓝牙(Bluetooth)、Wi-Fi、通话网络等无线或有线网络。The server 1 connects to the network through the network interface 13 to obtain information. The network may be an intranet, an Internet, a Global System of Mobile communication (GSM), a Wideband Code Division Multiple Access (WCDMA), a 4G network, or a 5G network. Wireless or wired networks such as networks, Bluetooth, Wi-Fi, and call networks.
需要指出的是,图1仅示出了具有组件11-13的服务器1,但是应理解的是,并不要求实施所有示出的组件,可以替代的实施更多或者更少的组件。It is pointed out that Figure 1 only shows the server 1 with the components 11-13, but it should be understood that not all illustrated components are required to be implemented, and more or fewer components may be implemented instead.
其中,所述存储器11至少包括一种类型的存储介质(计算机可读存储介质),所述存储介质包括闪存、硬盘、多媒体卡、卡型存储器(例如,SD或DX存储器等)、随机访问存储器(RAM)、静态随机访问存储器(SRAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、可编程只读存储器(PROM)、磁性存储器、磁盘、光盘等。在一些实施例中,所述存储器11可以是所述服务器1的内部存储单元,例如该服务器1的硬盘或内存。在另一些实施例中,所述存储器11也可以是所述服务器1的外部存储设备,例如该服务器1配备的插接式硬盘,智能存储卡(Smart Media Card,SMC),安全数字(Secure Digital,SD)卡,闪存卡(Flash Card)等。当然,所述存储器11还可以既包括所述服务器1的内部存储单元也包括其外部存储设备。本实施例中,所述存储器11通常用于存储安装于所述服务器1的操作系统和各类应用软件,例如图像处理程序200的程序代码等。此外,所述存储器11还可以用于暂时地存储已经输出或者将要输出的各类数据。The memory 11 includes at least one type of storage medium (computer readable storage medium) including a flash memory, a hard disk, a multimedia card, a card type memory (eg, SD or DX memory, etc.), a random access memory. (RAM), static random access memory (SRAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), programmable read only memory (PROM), magnetic memory, magnetic disk, optical disk, and the like. In some embodiments, the memory 11 may be an internal storage unit of the server 1, such as a hard disk or memory of the server 1. In other embodiments, the memory 11 may also be an external storage device of the server 1, such as a plug-in hard disk equipped with the server 1, a smart memory card (SMC), and a secure digital (Secure Digital). , SD) card, flash card (Flash Card), etc. Of course, the memory 11 can also include both the internal storage unit of the server 1 and its external storage device. In this embodiment, the memory 11 is generally used to store an operating system installed in the server 1 and various types of application software, such as program code of the
所述处理器12在一些实施例中可以是中央处理器(Central Processing Unit,CPU)、控制器、微控制器、微处理器、或其他数据处理芯片。该处理器12通常用于控制所述服务器1的总体操作,例如执行数据交互或者通信相关的控制和处理等。本实施例中,所述处理器12用于运行所述存储器11中存储的程序代码或者处理数据,例如运行所述的图像处理程序200等。The processor 12 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 12 is typically used to control the overall operation of the server 1, such as performing data interaction or communication related control and processing, and the like. In this embodiment, the processor 12 is configured to run program code or process data stored in the memory 11, such as running the
所述网络接口13可包括无线网络接口或有线网络接口,该网络接口13通常用于在所述服务器1与其他电子设备之间建立通信连接。The network interface 13 may comprise a wireless network interface or a wired network interface, which is typically used to establish a communication connection between the server 1 and other electronic devices.
本实施例中,所述服务器1内安装并运行有图像处理程序200,当所述图像处理程序200运行时,所述服务器1首先,接收所述移动终端上传的图片;其次,根据所述移动终端上传的图片判断所述移动终端采集所述图片的采集角度;再次,根据所述采集角度选择对应的图片拼接算法;然后,利用所述图片拼接算法对所述移动终端上传的图片进行拼接;最后,将拼接后图片进行展示。这样,可以根据移动终端获取实时场景图片,快速的获取目标空间的360度或720度全景影像,并显示给最终用户,可以使得目标在线上的展示更加立体,使用消费者获得更多的第一感官资料。In this embodiment, an
至此,己经详细介绍了本申请各个实施例的应用环境和相关设备的硬件结构和功能。下面,将基于上述应用环境和相关设备,提出本申请的各个实施例。So far, the application environment of the various embodiments of the present application and the hardware structure and functions of related devices have been described in detail. Hereinafter, various embodiments of the present application will be proposed based on the above-described application environment and related devices.
首先,本申请提出一种室内图像处理程序200。First, the present application proposes an indoor
参阅图2所示,是本申请室内图像处理程序200第一实施例的程序模块图。Referring to Fig. 2, it is a program block diagram of the first embodiment of the indoor
本实施例中,服务器1,包括一系列的存储于存储器11上的计算机程序指令,即所述室内图像处理程序200,当该计算机程序指令被处理器12执行时,可以实现本申请各实施例的图像处理操作。在一些实施例中,基于该计算机程序指令各部分所实现的特定的操作,所述室内图像处理程序200可以被划分为一个或多个模块。例如,在图2中,所述室内图像处理程序200可以被分割成接收模块201、判断模块202、算法选择模块203、拼接模块204及展示模块205。其中:In this embodiment, the server 1 includes a series of computer program instructions stored in the memory 11, that is, the indoor
所述接收模块201,用于接收所述移动终端上传的图片。The receiving module 201 is configured to receive a picture uploaded by the mobile terminal.
在本实施方式中,所述移动终端(图中未标出)为具有图像获取功能及数据传输功能的便携式终端,比如智能手机、个人数字助理、平板电脑等等。其中,移动终端上传的图片是用户根据需求对场景进行拍照获取的。具体在 拍照过程中,如果单次拍照后,效果不满意,可以选择取消本次拍照结果,该照片不会上传至服务器以供拼接,如果效果满意,则需选择保存本次拍照。对于上述接收所述移动终端上传的图片的步骤,在图4中会做具体的说明。In the present embodiment, the mobile terminal (not shown) is a portable terminal having an image acquisition function and a data transmission function, such as a smart phone, a personal digital assistant, a tablet computer, and the like. The picture uploaded by the mobile terminal is obtained by taking a picture of the scene according to the requirement of the user. Specifically, during the photo taking process, if the effect is not satisfactory after a single photo, you can choose to cancel the photo result. The photo will not be uploaded to the server for splicing. If the effect is satisfactory, you need to save the photo. The steps of receiving the picture uploaded by the mobile terminal are described in detail in FIG. 4.
所述判断模块202,用于根据所述移动终端上传的图片判断所述移动终端采集所述图片的采集角度。The determining module 202 is configured to determine, according to the picture uploaded by the mobile terminal, the collection angle of the picture collected by the mobile terminal.
在本实施方式中,对于图片获取过程,若选择360度或者720度进行图像采集,那么在获取的图片中将会嵌入一个采集角度数字标识,隐藏在图片像素集合中。在后续的图片处理过程中,所述判断模块203会解析图片的像素集合,则可以判断出当前分析的图片的采集角度。In the present embodiment, for the image acquisition process, if 360 degrees or 720 degrees are selected for image acquisition, an acquisition angle digital identifier will be embedded in the acquired image and hidden in the image pixel set. In the subsequent image processing process, the determining module 203 parses the pixel set of the image, and then can determine the collection angle of the currently analyzed image.
所述算法选择模块203,用于根据所述采集角度选择对应的图片拼接算法。The algorithm selection module 203 is configured to select a corresponding image splicing algorithm according to the collection angle.
在本实施方式中,所述图片拼接算法包括opencv图片360度全景拼接算法和opencv图片720度全景拼接算法。具体的,所述opencv图片360度全景拼接算法包括如下步骤:1.对每幅图进行特征点提取;2.对特征点进行匹配,其中在特征点提取即匹配的时候,只需要提取或匹配待拼接图片在图像采集方向上的对称两侧,比如一幅相对拍摄者左右方向的360度图片采集,只需要提取及匹配左右对称两边缘的特征点;3.进行图像配准;4.把图像拷贝到另一幅图像的特定位置,所述特定位置位于所述另一幅图像距离边缘一定距离的范围内,所述距离的设定可以根据用户需要进行设定,本文并不做限定;5.对重叠边界进行特殊处理,比如去除亮度、色调相同的像素点,去除图像噪点等。In this embodiment, the picture stitching algorithm includes an opencv picture 360 degree panoramic stitching algorithm and an opencv picture 720 degree panoramic stitching algorithm. Specifically, the 360-degree panoramic splicing algorithm of the opencv picture includes the following steps: 1. performing feature point extraction on each picture; 2. matching feature points, where only feature extraction or matching is needed, only need to extract or match The spliced picture is symmetrical on both sides of the image acquisition direction, such as a 360-degree picture acquisition relative to the left and right direction of the photographer, only need to extract and match the feature points of the left and right symmetrical edges; 3. Perform image registration; 4. Put The image is copied to a specific position of another image, and the specific position is located within a certain distance from the edge of the other image, and the setting of the distance may be set according to user needs, which is not limited herein; 5. Special treatment of overlapping boundaries, such as removing pixels with the same brightness and hue, and removing image noise.
所述opencv图片720度全景拼接算法包括如下步骤:1.对每幅图进行特征点提取;2.对特征点进行匹配,在进行特征点提取即匹配的时候,需要针对待拼接图片的所有边缘区域进行特征点的提取及匹配;3.进行图像配准;4.把图像拷贝到另一幅图像的特定位置,所述特定位置位于所述另一幅图像距离边缘一定距离的范围内,所述距离的设定可以根据用户需要进行设定,本文 并不做限定;5.对重叠边界进行特殊处理,比如去除亮度、色调相同的像素点,去除图像噪点等。The 720 degree panoramic splicing algorithm of the opencv picture includes the following steps: 1. performing feature point extraction on each picture; 2. matching feature points, when performing feature point extraction or matching, all edges of the picture to be spliced are needed The area performs feature point extraction and matching; 3. performs image registration; 4. copies the image to a specific position of another image, the specific position is within a certain distance from the edge of the other image, The setting of the distance can be set according to the needs of the user, and is not limited herein; 5. Special processing is performed on the overlapping boundary, such as removing pixels with the same brightness and hue, and removing image noise.
所述拼接模块204,用于利用所述图片拼接算法对所述移动终端上传的图片进行拼接。The splicing module 204 is configured to splicing a picture uploaded by the mobile terminal by using the picture splicing algorithm.
在本实施方式中,当选择的是360度对应的图片拼接算法时,则对所采集图像在采集方向上的对称两侧区域进行特征点采集与匹配,进而执行图片拼接。当选择的是720度对应的图片拼接算法时,则对所采集图像的周围预定范围内的部分进行特征点的提取及匹配,进而执行上述图片拼接。In the embodiment, when the image splicing algorithm corresponding to 360 degrees is selected, the feature points are collected and matched on the symmetrical two sides of the acquired image in the acquisition direction, and then the image splicing is performed. When the image splicing algorithm corresponding to 720 degrees is selected, the feature points are extracted and matched to the portions within the predetermined range around the acquired image, thereby performing the above-mentioned picture splicing.
展示模块205,用于将拼接后的图片进行展示。The display module 205 is configured to display the stitched picture.
在本实施方式中,在将拼接后所述图片进行展示之前,判断模块202还会对拼接后的图片做进一步的解析,进而判断拼接后的图片是否符合360度拼接图片或720度图片。具体的,判断模块202会对拼接后的图片进行分析,获取图片重叠区域,如果图片重叠区域仅仅只属于左右或上下对称分布,则判断符合360度的拼接图片,如果图片重叠区域呈多个封闭的区域,则判断符合720度的拼接图片。In this embodiment, before the splicing of the picture is displayed, the determining module 202 further analyzes the spliced picture to determine whether the spliced picture conforms to the 360-degree spliced picture or the 720-degree picture. Specifically, the judging module 202 analyzes the spliced picture to obtain a picture overlapping area. If the picture overlapping area only belongs to the left and right or the upper and lower symmetrical distribution, the splicing picture conforming to 360 degrees is determined, and if the overlapping area of the picture is closed, The area is judged to match the 720-degree stitched picture.
更进一步的,当所述拼接结果是360度拼接图片时,展示模块205利用krpano引擎渲染处理所述360度拼接图片,并生成html页面,所述html页面用于所述移动终端进行访问时进行展示全景图像。其中krpano引擎是Krpano全景显示引擎,但采用xml数据结构和基于flash的三维全景显示技术,是一个功能强大的三维全景显示引擎。通过使用krpano引擎,展示模块205可以使得360度拼接图片得以快速渲染生成一幅全景图像。并在生成了全景图像之后,生成html页面,进而可以使得移动终端快速的访问并展示全景图像给用户。Further, when the splicing result is a 360-degree spliced picture, the display module 205 processes and processes the 360-degree spliced picture by using a krpano engine, and generates an html page, where the html page is used for access by the mobile terminal. Show panoramic images. The krpano engine is the Krpano panoramic display engine, but it adopts xml data structure and flash-based 3D panoramic display technology, and is a powerful 3D panoramic display engine. By using the krpano engine, the presentation module 205 can cause a 360 degree stitched picture to be quickly rendered to produce a panoramic image. After the panoramic image is generated, an html page is generated, which can enable the mobile terminal to quickly access and display the panoramic image to the user.
更进一步的,当所述拼接结果是720度拼接图片时,展示模块205将720度拼接图片回传至所述移动终端,所述移动终端利用开源的opengl技术对所述720度拼接图片进行渲染进行720度展示。其中对于720度的拼接图片,相应的渲染处理并不在服务端进行,而是直接将拼接后的拼接图片回传至各个需要 浏览服务的移动终端,由移动终端通过opengl技术自行对拼接后的图片进行渲染展现720度的效果。由于720度拼接图片数据量要大于一般的360度拼接图片的数据量,故通过将后期渲染处理转移至移动终端,可以有效减轻服务器1端的运行压力,提高图片显示效率。Further, when the splicing result is a 720-degree spliced picture, the display module 205 returns a 720-degree spliced picture to the mobile terminal, and the mobile terminal renders the 720-degree spliced picture by using an open source opengl technology. Perform a 720 degree display. For the 720-degree stitched picture, the corresponding rendering process is not performed on the server side, but the stitched picture is directly transmitted back to each mobile terminal that needs to browse the service, and the mobile terminal directly spliced the picture through the opengl technology. Rendering shows 720 degrees. Since the amount of 720-degree stitched picture data is larger than the data amount of the general 360-degree stitched picture, by transferring the post-render processing to the mobile terminal, the running pressure of the server 1 end can be effectively reduced, and the picture display efficiency is improved.
通过上述程序模块201-205,本申请所提出的服务器,接收所述移动终端上传的图片;根据所述移动终端上传的图片判断所述移动终端采集所述图片的采集角度;根据所述采集角度选择对应的图片拼接算法,并利用所述图片拼接算法对所述移动终端上传的图片进行拼接;将拼接后图片进行展示。这样,可以根据移动终端获取实时场景图片,快速的获取目标室内空间的360度或720度全景影像,并显示给最终用户,可以使得目标的室内空间在线上的展示更加立体,使消费者获得更多的第一感官资料。The server of the present application receives the picture uploaded by the mobile terminal, and determines, according to the picture uploaded by the mobile terminal, the collection angle of the picture by the mobile terminal, according to the program module 201-205; Selecting a corresponding image splicing algorithm, and splicing the image uploaded by the mobile terminal by using the image splicing algorithm; and displaying the spliced image. In this way, the real-time scene image can be obtained according to the mobile terminal, and the 360-degree or 720-degree panoramic image of the target indoor space can be quickly obtained and displayed to the end user, which can make the indoor space of the target appear more stereoscopic on the line, so that the consumer can obtain more. More first sensory information.
此外,本申请还提出一种室内图像处理方法。In addition, the present application also proposes an indoor image processing method.
参阅图3所示,是本申请室内图像处理方法第一实施例的实施流程示意图。在本实施例中,根据不同的需求,图3所示的流程图中的步骤的执行顺序可以改变,某些步骤可以省略。Referring to FIG. 3, it is a schematic flowchart of the implementation of the first embodiment of the indoor image processing method of the present application. In this embodiment, the order of execution of the steps in the flowchart shown in FIG. 3 may be changed according to different requirements, and some steps may be omitted.
步骤S301,接收所述移动终端上传的图片。Step S301: Receive a picture uploaded by the mobile terminal.
在本实施方式中,所述移动终端为具有图像获取功能及数据传输功能的便携式终端,比如智能手机、个人数字助理、平板电脑等等。其中,移动终端上传的图片是用户根据需求对场景进行拍照获取的。具体在拍照过程中,如果单次拍照后,效果不满意,可以选择取消本次拍照结果,该照片不会上传至服务器1以供拼接,如果效果满意,则需选择保存本次拍照。其中,对于上述接收所述移动终端上传的图片的步骤,在本申请室内图像处理方法第二实施例中(如图4所示)会做具体的说明。In this embodiment, the mobile terminal is a portable terminal having an image acquisition function and a data transmission function, such as a smart phone, a personal digital assistant, a tablet computer, and the like. The picture uploaded by the mobile terminal is obtained by taking a picture of the scene according to the requirement of the user. Specifically, during the photo taking process, if the effect is not satisfactory after a single photo taking, you can choose to cancel the photo result. The photo will not be uploaded to the server 1 for splicing. If the effect is satisfactory, you need to save the photo. The step of receiving the picture uploaded by the mobile terminal is specifically described in the second embodiment of the indoor image processing method of the present application (as shown in FIG. 4).
步骤S302,根据所述移动终端上传的图片判断所述移动终端采集所述图片的采集角度。Step S302: Determine, according to the picture uploaded by the mobile terminal, the collection angle of the picture collected by the mobile terminal.
在本实施方式中,对于图片获取过程,若选择360度或者720度进行图 像采集,那么在获取的图片中将会嵌入一个采集角度数字标识,隐藏在图片像素集合中。在后续的图片处理过程中,所述判断模块203会解析图片的像素集合,则可以判断出当前分析的图片的采集角度。In this embodiment, for the image acquisition process, if 360 degree or 720 degrees is selected for image acquisition, an acquisition angle digital identifier will be embedded in the acquired image and hidden in the image pixel set. In the subsequent image processing process, the determining module 203 parses the pixel set of the image, and then can determine the collection angle of the currently analyzed image.
步骤S303,根据所述采集角度选择对应的图片拼接算法。Step S303, selecting a corresponding picture stitching algorithm according to the collection angle.
在本实施方式中,所述图片拼接算法包括opencv图片360度全景拼接算法和opencv图片720度全景拼接算法。In this embodiment, the picture stitching algorithm includes an opencv picture 360 degree panoramic stitching algorithm and an opencv picture 720 degree panoramic stitching algorithm.
具体的,所述opencv图片360度全景拼接算法包括如下步骤:1.对每幅图进行特征点提取;2.对特征点进行匹配,其中在特征点提取即匹配的时候,只需要提取或匹配待拼接图片在图像采集方向上的对称两侧,比如一幅相对拍摄者左右方向的360度图片采集,只需要提取及匹配左右对称两边缘的特征点;3.进行图像配准;4.把图像拷贝到另一幅图像的特定位置,所述特定位置位于所述另一幅图像距离边缘一定距离的范围内,所述距离的设定可以根据用户需要进行设定,本文并不做限定;5.对重叠边界进行特殊处理,比如去除亮度、色调相同的像素点,去除图像噪点等。Specifically, the 360-degree panoramic splicing algorithm of the opencv picture includes the following steps: 1. performing feature point extraction on each picture; 2. matching feature points, where only feature extraction or matching is needed, only need to extract or match The spliced picture is symmetrical on both sides of the image acquisition direction, such as a 360-degree picture acquisition relative to the left and right direction of the photographer, only need to extract and match the feature points of the left and right symmetrical edges; 3. Perform image registration; 4. Put The image is copied to a specific position of another image, and the specific position is located within a certain distance from the edge of the other image, and the setting of the distance may be set according to user needs, which is not limited herein; 5. Special treatment of overlapping boundaries, such as removing pixels with the same brightness and hue, and removing image noise.
所述opencv图片720度全景拼接算法包括如下步骤:1.对每幅图进行特征点提取;2.对特征点进行匹配,在进行特征点提取即匹配的时候,需要针对待拼接图片的所有边缘区域进行特征点的提取及匹配;3.进行图像配准;4.把图像拷贝到另一幅图像的特定位置,所述特定位置位于所述另一幅图像距离边缘一定距离的范围内,所述距离的设定可以根据用户需要进行设定,本文并不做限定;5.对重叠边界进行特殊处理,比如去除亮度、色调相同的像素点,去除图像噪点等。The 720 degree panoramic splicing algorithm of the opencv picture includes the following steps: 1. performing feature point extraction on each picture; 2. matching feature points, when performing feature point extraction or matching, all edges of the picture to be spliced are needed The area performs feature point extraction and matching; 3. performs image registration; 4. copies the image to a specific position of another image, the specific position is within a certain distance from the edge of the other image, The setting of the distance can be set according to the needs of the user, and is not limited herein; 5. Special processing is performed on the overlapping boundary, such as removing pixels with the same brightness and hue, and removing image noise.
步骤S304,利用所述图片拼接算法对所述移动终端上传的图片进行拼接。In step S304, the picture uploaded by the mobile terminal is spliced by using the picture splicing algorithm.
在本实施方式中,当选择的是360度对应的图片拼接算法时,则对所采集图像在采集方向上的对称两侧区域进行特征点采集与匹配,进而执行图片拼接。当选择的是720度对应的图片拼接算法时,则对所采集图像的周围预定范围内的部分进行特征点的提取及匹配,进而执行图片拼接。In the embodiment, when the image splicing algorithm corresponding to 360 degrees is selected, the feature points are collected and matched on the symmetrical two sides of the acquired image in the acquisition direction, and then the image splicing is performed. When the image splicing algorithm corresponding to 720 degrees is selected, the feature points are extracted and matched to the portions within the predetermined range around the acquired image, thereby performing image splicing.
步骤S305,将拼接后图片进行展示。In step S305, the stitched picture is displayed.
在本实施方式中,在将拼接后所述图片进行展示的步骤之前,还会对拼接后的图片做进一步的解析,进而判断拼接后的图片是否符合360度拼接图片或720度图片。具体的,判断模块202会对拼接后的图片进行分析,获取图片重叠区域,如果图片重叠区域仅仅只属于左右或上下对称分布,则判断符合360度的拼接图片,如果图片重叠区域呈多个封闭的区域,则判断符合720度的拼接图片。In this embodiment, before the step of displaying the picture after splicing, the spliced picture is further analyzed to determine whether the spliced picture conforms to a 360-degree spliced picture or a 720-degree picture. Specifically, the judging module 202 analyzes the spliced picture to obtain a picture overlapping area. If the picture overlapping area only belongs to the left and right or the upper and lower symmetrical distribution, the splicing picture conforming to 360 degrees is determined, and if the overlapping area of the picture is closed, The area is judged to match the 720-degree stitched picture.
更进一步的,当所述拼接结果是360度拼接图片时,利用krpano引擎渲染处理所述360度拼接图片,并生成html页面,所述html页面用于所述移动终端进行访问时进行展示全景图像。其中krpano引擎是Krpano全景显示引擎,但采用xml数据结构和基于flash的三维全景显示技术,是一个功能强大的三维全景显示引擎。通过使用krpano引擎,可以使得360度拼接图片得以快速渲染生成一幅全景图像。并在生成了全景图像之后,生成html页面,进而可以使得移动终端快速的访问并展示全景图像给用户。Further, when the splicing result is a 360-degree spliced picture, the 360-degree spliced picture is processed by using a krpano engine, and an html page is generated, and the html page is used to display the panoramic image when the mobile terminal accesses . The krpano engine is the Krpano panoramic display engine, but it adopts xml data structure and flash-based 3D panoramic display technology, and is a powerful 3D panoramic display engine. By using the krpano engine, 360-degree stitching images can be quickly rendered to produce a panoramic image. After the panoramic image is generated, an html page is generated, which can enable the mobile terminal to quickly access and display the panoramic image to the user.
更进一步的,当所述拼接结果是720度拼接图片时,将720度拼接图片回传至所述移动终端,所述移动终端利用开源的opengl技术对所述720度拼接图片进行渲染进行720度展示。其中对于720度的拼接图片,相应的渲染处理并不在服务端进行,而是直接将拼接后的拼接图片回传至各个需要浏览服务的移动终端,由移动终端通过opengl技术自行对拼接后的图片进行渲染展现720度的效果。由于720度拼接图片数据量要大于一般的360度拼接图片的数据量,故通过将后期渲染处理转移至移动终端,可以有效减轻服务器端的运行压力,提高图片显示效率。Further, when the splicing result is a 720-degree spliced picture, the 720-degree spliced picture is transmitted back to the mobile terminal, and the mobile terminal uses the open source opengl technology to render the 720-degree spliced picture for 720 degrees. Show. For the 720-degree stitched picture, the corresponding rendering process is not performed on the server side, but the stitched picture is directly transmitted back to each mobile terminal that needs to browse the service, and the mobile terminal directly spliced the picture through the opengl technology. Rendering shows 720 degrees. Since the amount of 720-degree stitched picture data is larger than the data amount of the general 360-degree stitched picture, by transferring the post-render processing to the mobile terminal, the running pressure of the server end can be effectively reduced, and the picture display efficiency is improved.
通过上述步骤S301-305,本申请所提出的室内图像处理方法,首先,接收所述移动终端上传的图片;其次,根据所述移动终端上传的图片判断所述移动终端采集所述图片的采集角度;再次,根据所述采集角度选择对应的图片拼接算法,并利用所述图片拼接算法对所述移动终端上传的图片进行拼接; 最后,将拼接后图片进行展示。这样,可以根据移动终端获取实时场景图片,快速的获取目标室内空间的360度或720度全景影像,并显示给最终用户,可以使得目标室内空间在线上的展示更加立体,使消费者获得更多的第一感官资料。Through the above steps S301-305, the indoor image processing method proposed by the present application firstly receives a picture uploaded by the mobile terminal; secondly, determining, according to the picture uploaded by the mobile terminal, the collection angle of the picture collected by the mobile terminal And again, selecting a corresponding image splicing algorithm according to the collection angle, and splicing the image uploaded by the mobile terminal by using the image splicing algorithm; and finally, displaying the spliced image. In this way, the real-time scene image can be obtained according to the mobile terminal, and the 360-degree or 720-degree panoramic image of the target indoor space can be quickly obtained and displayed to the end user, which can make the display of the target indoor space more stereoscopic on the line, so that the consumer can obtain more. The first sensory information.
参阅图4所示,是本申请室内图像处理方法第二实施例的实施流程示意图。在本实施例中,根据不同的需求,图4所示的流程图中的步骤的执行顺序可以改变,某些步骤可以省略。图4所示为图3中步骤S301的进一步说明。Referring to FIG. 4, it is a schematic flowchart of the implementation of the second embodiment of the indoor image processing method of the present application. In this embodiment, the order of execution of the steps in the flowchart shown in FIG. 4 may be changed according to different requirements, and some steps may be omitted. FIG. 4 shows a further description of step S301 in FIG.
在步骤S401,控制所述移动终端开启相机应用。In step S401, the mobile terminal is controlled to turn on the camera application.
具体的,服务器1可以通过通信网络发送控制指令至移动终端,进而远端控制移动终端开启相机应用,进而准备获取图像。Specifically, the server 1 can send a control command to the mobile terminal through the communication network, and then remotely control the mobile terminal to open the camera application, thereby preparing to acquire an image.
在步骤S402,选择图片采集角度。At step S402, a picture acquisition angle is selected.
具体的,开启相机应用后,移动终端根据用户的触碰操作进而选择图片采集角度,比如选择360度拍摄还是720度拍摄,其中720度拍摄选择的一般则是广角镜头,视角范围要大于一般镜头。Specifically, after the camera application is turned on, the mobile terminal selects a picture acquisition angle according to the user's touch operation, for example, selecting a 360-degree shooting or a 720-degree shooting, wherein the 720-degree shooting selection is generally a wide-angle lens, and the viewing angle range is larger than a general lens.
在步骤S403,根据选择采集角度在相机界面显示拍摄指引,所述拍摄指引用于指导用户利用所述移动终端进行图片采集。In step S403, a shooting guide is displayed on the camera interface according to the selected collection angle, and the shooting guide is used to guide the user to perform picture collection by using the mobile terminal.
具体的,当选择了采集角度后,在相机的拍摄界面显示相应角度的拍摄指引,比如360度的采集角度,则以一个环绕立体坐标系中的XOY面的箭头表示进行指引。而对于720度的采集角度,则分别以两个环绕立体坐标系中的XOY面和YOZ的箭头表示进行指引。具体的,根据拍摄指引缓慢移动拍摄视角进行图像获取,进而为后续图片拼接提供素材。Specifically, when the acquisition angle is selected, the shooting guidance of the corresponding angle is displayed on the shooting interface of the camera, for example, the 360-degree acquisition angle is guided by an arrow indicating the XOY surface in the stereo coordinate system. For the 720-degree acquisition angle, the guidance is indicated by two XOY faces in the three-dimensional coordinate system and YOZ arrows. Specifically, the image is acquired by slowly moving the shooting angle according to the shooting guide, thereby providing material for subsequent image stitching.
在步骤S405,选择采集的图片进行上传。In step S405, the captured picture is selected for uploading.
具体的,移动终端通过无线网络将获取的图片或图像进行上传至服务器,1,以供服务器1做进一步的处理。Specifically, the mobile terminal uploads the acquired picture or image to the server through the wireless network, for further processing by the server 1.
通过上述步骤S401-405,本申请所提出的室内图像处理方法,可以根据用户需求接收所述移动终端上传的场景图片。Through the above steps S401-405, the indoor image processing method proposed by the present application can receive the scene picture uploaded by the mobile terminal according to the user demand.
本申请还提供了另一种实施方式,即提供一种计算机可读存储介质,所述计算机可读存储介质存储有室内图像处理程序,所述室内图像处理程序可被至少一个处理器执行,以使所述至少一个处理器执行如上述的室内图像处理方法的步骤。The present application further provides another embodiment, that is, a computer readable storage medium storing an indoor image processing program, the indoor image processing program being executable by at least one processor, The at least one processor is caused to perform the steps of the indoor image processing method as described above.
上述本申请实施例序号仅仅为了描述,不代表实施例的优劣。The serial numbers of the embodiments of the present application are merely for the description, and do not represent the advantages and disadvantages of the embodiments.
通过以上的实施方式的描述,本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现,当然也可以通过硬件,但很多情况下前者是更佳的实施方式。基于这样的理解,本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中,包括若干指令用以使得一台终端设备(可以是手机,计算机,服务器,空调器,或者网络设备等)执行本申请各个实施例所述的方法。Through the description of the above embodiments, those skilled in the art can clearly understand that the foregoing embodiment method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be through hardware, but in many cases, the former is better. Implementation. Based on such understanding, the technical solution of the present application, which is essential or contributes to the prior art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk, The optical disc includes a number of instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the methods described in various embodiments of the present application.
以上仅为本申请的优选实施例,并非因此限制本申请的专利范围,凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换,或直接或间接运用在其他相关的技术领域,均同理包括在本申请的专利保护范围内。The above is only a preferred embodiment of the present application, and is not intended to limit the scope of the patent application, and the equivalent structure or equivalent process transformations made by the specification and the drawings of the present application, or directly or indirectly applied to other related technical fields. The same is included in the scope of patent protection of this application.
Claims (20)
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810226472.7A CN108492251A (en) | 2018-03-19 | 2018-03-19 | Off-the-air picture processing method, server and computer readable storage medium |
| CN201810226472.7 | 2018-03-19 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2019179023A1 true WO2019179023A1 (en) | 2019-09-26 |
Family
ID=63318424
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2018/102079 Ceased WO2019179023A1 (en) | 2018-03-19 | 2018-08-24 | Indoor image processing method, server and computer readable storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN108492251A (en) |
| WO (1) | WO2019179023A1 (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113888739B (en) * | 2019-05-24 | 2025-01-17 | 创新先进技术有限公司 | Image acquisition guidance method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103091966A (en) * | 2011-10-31 | 2013-05-08 | 高旭鹃 | 720-degree automatic panoramic imaging system |
| CN204681518U (en) * | 2015-05-26 | 2015-09-30 | 深圳市圆周率软件科技有限责任公司 | A kind of panorama image information collecting device |
| CN105472345A (en) * | 2015-12-28 | 2016-04-06 | 北京极图科技有限公司 | Six-way splicing type real-time panoramic imaging device |
| CN105827946A (en) * | 2015-11-26 | 2016-08-03 | 维沃移动通信有限公司 | Panoramic image generating method, panoramic image playing method and mobile terminal |
| CN106791396A (en) * | 2016-12-20 | 2017-05-31 | 宇龙计算机通信科技(深圳)有限公司 | Panoramic picture shooting method, system and electronic equipment |
-
2018
- 2018-03-19 CN CN201810226472.7A patent/CN108492251A/en active Pending
- 2018-08-24 WO PCT/CN2018/102079 patent/WO2019179023A1/en not_active Ceased
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103091966A (en) * | 2011-10-31 | 2013-05-08 | 高旭鹃 | 720-degree automatic panoramic imaging system |
| CN204681518U (en) * | 2015-05-26 | 2015-09-30 | 深圳市圆周率软件科技有限责任公司 | A kind of panorama image information collecting device |
| CN105827946A (en) * | 2015-11-26 | 2016-08-03 | 维沃移动通信有限公司 | Panoramic image generating method, panoramic image playing method and mobile terminal |
| CN105472345A (en) * | 2015-12-28 | 2016-04-06 | 北京极图科技有限公司 | Six-way splicing type real-time panoramic imaging device |
| CN106791396A (en) * | 2016-12-20 | 2017-05-31 | 宇龙计算机通信科技(深圳)有限公司 | Panoramic picture shooting method, system and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108492251A (en) | 2018-09-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2015307358B2 (en) | Photographing method and electronic device | |
| EP2328125B1 (en) | Image splicing method and device | |
| US9898856B2 (en) | Systems and methods for depth-assisted perspective distortion correction | |
| CN109525674B (en) | System and method for making house panorama | |
| US10165201B2 (en) | Image processing method and apparatus and terminal device to obtain a group photo including photographer | |
| US10051180B1 (en) | Method and system for removing an obstructing object in a panoramic image | |
| CA2722924A1 (en) | 3d content aggregation built into devices | |
| CN108564612A (en) | Model display method, device, storage medium and electronic equipment | |
| CN110278366B (en) | A panoramic image blurring method, terminal and computer-readable storage medium | |
| CN104320590A (en) | Split-screen image processing method and device and client terminal | |
| US20250053287A1 (en) | Systems, methods, and computer program products for digital photography | |
| US9792021B1 (en) | Transitioning an interface to a neighboring image | |
| GB2554925A (en) | Display of visual data with a virtual reality headset | |
| US20210352206A1 (en) | Digital media frame and method for configuring a field of view of a digital media frame | |
| JP2016504828A (en) | Method and system for capturing 3D images using a single camera | |
| US9088720B2 (en) | Apparatus and method of displaying camera view area in portable terminal | |
| CN109271123B (en) | Picture display method and picture display device | |
| CN111866383A (en) | An image processing method, terminal and storage medium | |
| CN116017129A (en) | Method, device, system, equipment and medium for adjusting angle of light supplementing lamp | |
| WO2019179023A1 (en) | Indoor image processing method, server and computer readable storage medium | |
| CN115268650B (en) | Screen capturing method, device, head-mounted virtual reality equipment and storage medium | |
| CN112087612A (en) | Multi-scene-based panoramic imaging method and device, computer equipment and storage medium | |
| CN108920598B (en) | Panorama browsing method and device, terminal equipment, server and storage medium | |
| CN114627000B (en) | Image processing method, device and electronic device | |
| CN113822899B (en) | Image processing methods, apparatus, computer equipment and storage media |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 18911225 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 13.01.2021) |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 18911225 Country of ref document: EP Kind code of ref document: A1 |