TWI469064B - Video recognizing system and method - Google Patents
Video recognizing system and method Download PDFInfo
- Publication number
- TWI469064B TWI469064B TW98139319A TW98139319A TWI469064B TW I469064 B TWI469064 B TW I469064B TW 98139319 A TW98139319 A TW 98139319A TW 98139319 A TW98139319 A TW 98139319A TW I469064 B TWI469064 B TW I469064B
- Authority
- TW
- Taiwan
- Prior art keywords
- image
- area
- camera
- key area
- key
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims description 13
- 239000000284 extract Substances 0.000 claims description 5
- 238000001514 detection method Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000001815 facial effect Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
Description
本發明係關於一種影像辨識系統及方法。 The present invention relates to an image recognition system and method.
習知數位影像監控系統針對同一場景僅以單一攝影機進行拍攝,並根據該單一攝影機拍攝所得的影像進行辨識。為了更精確的進行辨識,一般對攝影機的解析度具有較高的要求。然,由於解析度較高的攝影機拍攝所得到的影像的解析度亦較高,從而使得攝影機所傳回的影像檔較大,此時則需要對高解析度的影像進行壓縮等處理之後方回傳至後端處理系統。如此將會導致較長的處理時間。 The conventional digital image monitoring system captures only a single camera for the same scene, and recognizes the image captured by the single camera. In order to carry out the identification more accurately, the resolution of the camera generally has high requirements. However, since the resolution of the image obtained by the camera with high resolution is also high, the image file returned by the camera is large, and at this time, it is necessary to compress the high-resolution image and then return it. Pass to the backend processing system. This will result in a longer processing time.
鑒於上述內容,有必要提供一種速度較快的影像辨識系統及方法。 In view of the above, it is necessary to provide a faster image recognition system and method.
一種影像辨識系統,包括:一第一攝影機,用於對一被攝場景進行拍攝以得到一第一圖像;一第二攝影機,用於對該被攝場景進行拍攝以得到一第二圖像,該第二圖像的解析度高於該第一圖像的解析度,該第一圖像與第二圖像具有相同的內容且第一圖像的尺寸小於第二圖像的尺寸;一偵測單元,用於對該第一圖像進行偵測以得到第一圖像中關鍵 區域的圖像;一座標判定單元,用於得到第一圖像中關鍵區域所在的座標範圍,並根據第二圖像與第一圖像之間的尺寸比例對應得到第二圖像中關鍵區域所在的座標範圍;一攫取單元,用於根據第二圖像中關鍵區域的座標範圍從第二圖像中攫取對應區域的圖像,並將其根據第一圖像與第二圖像之間的尺寸比例對應縮小,該被縮小的關鍵區域的圖像用於替換第一圖像中的關鍵區域以得到一第三圖像;以及一辨識單元,用於對該第三圖像進行辨識。 An image recognition system includes: a first camera for photographing a scene to obtain a first image; and a second camera for photographing the scene to obtain a second image The resolution of the second image is higher than the resolution of the first image, the first image has the same content as the second image, and the size of the first image is smaller than the size of the second image; a detecting unit, configured to detect the first image to obtain a key in the first image An image of the region; a landmark determining unit configured to obtain a coordinate range of the key region in the first image, and obtain a key region in the second image according to a size ratio between the second image and the first image a coordinate range; a capture unit configured to extract an image of the corresponding region from the second image according to a coordinate range of the key region in the second image, and according to the first image and the second image The size ratio is correspondingly reduced, the image of the reduced key area is used to replace the key area in the first image to obtain a third image, and an identification unit is configured to identify the third image.
一種影像辨識方法,包括:一第一攝影機及第二攝影機在同一時間對同一場景進行拍攝,以分別得到第一圖像及第二圖像,其中,第一圖像相對第二圖像具有較低的解析度,該第一圖像與第二圖像具有相同的內容且第一圖像的尺寸小於第二圖像的尺寸;該第一圖像被傳送至一影像處理系統;該影像處理系統對該第一圖像進行偵測,以得到該第一圖像中關鍵區域的圖像;該影像處理系統對應得到第一圖像中該關鍵區域所對應的座標範圍,並根據第二圖像與第一圖像之間的尺寸比例對應得到第二圖像中關鍵區域所在的座標範圍;該影像處理系統根據得到的座標範圍對應從第二圖像中攫取關鍵區域的圖像,並將其根據第一圖像與第二圖像之間的尺寸比例對 應縮小,該被縮小的關鍵區域的圖像用於替換該第一圖像中關鍵區域的圖像,以得到該第三圖像;以及該影像處理系統對該第三圖像進行辨識。 An image recognition method includes: a first camera and a second camera simultaneously shooting the same scene to obtain a first image and a second image, wherein the first image has a comparison with the second image Low resolution, the first image has the same content as the second image and the size of the first image is smaller than the size of the second image; the first image is transmitted to an image processing system; the image processing The system detects the first image to obtain an image of a key area in the first image; the image processing system correspondingly obtains a coordinate range corresponding to the key area in the first image, and according to the second figure Corresponding to the size ratio between the image and the first image, the coordinate range of the key region in the second image is obtained; the image processing system correspondingly extracts the image of the key region from the second image according to the obtained coordinate range, and It is based on the ratio of the size between the first image and the second image Should be reduced, the image of the reduced key area is used to replace the image of the key area in the first image to obtain the third image; and the image processing system recognizes the third image.
前述影像辨識系統透過對具有較低解析度的第一圖像進行偵測,以得到其中關鍵區域的座標範圍,並根據得到的座標範圍對應得到第二圖像中對應區域的座標範圍。之後再攫取第二圖像中對應座標範圍的區域,並將其對應縮小。該被縮小的區域的圖像用於替換第一圖像中關鍵區域的圖像。由於第二圖像具有較高的解析度,故,使得後續辨識過程中可更加精確,且同時僅僅只攫取第二圖像中對應的區域又避免了將整個第二圖像全部傳送至影像處理系統所導致的處理時間太長。 The image recognition system detects the first image with a lower resolution to obtain a coordinate range of the key region, and obtains a coordinate range of the corresponding region in the second image according to the obtained coordinate range. Then, the area of the corresponding coordinate range in the second image is captured and reduced accordingly. The image of the reduced area is used to replace the image of the key area in the first image. Since the second image has a higher resolution, the subsequent identification process can be more accurate, and at the same time, only the corresponding region in the second image is captured, and the entire second image is prevented from being transmitted to the image processing. The processing time caused by the system is too long.
10‧‧‧第一攝影機 10‧‧‧First camera
20‧‧‧第二攝影機 20‧‧‧Second camera
30‧‧‧影像處理系統 30‧‧‧Image Processing System
301‧‧‧偵測單元 301‧‧‧Detection unit
302‧‧‧座標判定單元 302‧‧‧ coordinate determination unit
303‧‧‧攫取單元 303‧‧‧Capture unit
304‧‧‧辨識單元 304‧‧‧ Identification unit
50‧‧‧被攝場景 50‧‧‧Photographed scene
51‧‧‧第一圖像 51‧‧‧ first image
52‧‧‧第二圖像 52‧‧‧ second image
圖1是本發明影像辨識系統的較佳實施方式的方框圖。 1 is a block diagram of a preferred embodiment of an image recognition system of the present invention.
圖2是圖1中影像處理系統的方框圖。 2 is a block diagram of the image processing system of FIG. 1.
圖3是一被攝場景的示意圖。 Figure 3 is a schematic illustration of a scene taken.
圖4A及圖4B是利用第一及第二攝影機對被攝場景進行拍攝所得到的圖像的示意圖。 4A and 4B are schematic views of an image obtained by photographing a scene to be taken by the first and second cameras.
圖5是圖2中偵測單元對圖4中由第一攝影機得到的圖像進行偵測的示意圖。 FIG. 5 is a schematic diagram of the detection unit of FIG. 2 detecting an image obtained by the first camera in FIG. 4. FIG.
圖6是本發明影像辨識方法的較佳實施方式的流程圖。 6 is a flow chart of a preferred embodiment of the image recognition method of the present invention.
請參閱圖1,本發明影像辨識系統的較佳實施方式包括一第一攝 影機10、一第二攝影機20、一影像處理系統30。其中,該第一攝影機10相對第二攝影機20而言具有較低的解析度,其拍攝所得的影像相對攝影機20亦具有較低的解析度。本實施方式中,該第一攝影機10及第二攝影機20用於在同一時間對同一場景進行拍攝,以得到兩幅內容相同但解析度不同的圖像。其中,由該第一攝影機10所拍攝得到的第一圖像的解析度低於由該第二攝影機20所拍攝得到的第二圖像的解析度,第一圖像的尺寸同樣小於第二圖像的尺寸。 Referring to FIG. 1, a preferred embodiment of the image recognition system of the present invention includes a first shot. The camera 10, a second camera 20, and an image processing system 30. The first camera 10 has a lower resolution than the second camera 20, and the captured image has a lower resolution than the camera 20. In the present embodiment, the first camera 10 and the second camera 20 are configured to capture the same scene at the same time to obtain two images having the same content but different resolutions. The resolution of the first image captured by the first camera 10 is lower than the resolution of the second image captured by the second camera 20, and the size of the first image is also smaller than the second image. The size of the image.
請一併參閱圖2,該影像處理系統30包括一偵測單元301、一座標判定單元302、一攫取單元303及一辨識單元304。 As shown in FIG. 2 , the image processing system 30 includes a detecting unit 301 , a label determining unit 302 , a capturing unit 303 , and an identifying unit 304 .
該第一攝影機10將拍攝得到的第一圖像傳送至該影像處理系統30。 The first camera 10 transmits the captured first image to the image processing system 30.
該偵測單元301用於對該第一圖像進行偵測以得到第一圖像中關鍵區域的圖像。本實施方式中,該偵測單元301為一臉部偵測單元,其對第一圖像進行臉部偵測以得到其中臉部區域的圖像。其他實施方式中,該偵測單元301亦可為車牌偵測單元,此時該偵測單元301則得到車牌區域的圖像。 The detecting unit 301 is configured to detect the first image to obtain an image of a key area in the first image. In this embodiment, the detecting unit 301 is a face detecting unit that performs face detection on the first image to obtain an image of the face region. In other embodiments, the detecting unit 301 can also be a license plate detecting unit. At this time, the detecting unit 301 obtains an image of the license plate area.
該座標判定單元302用於對由偵測單元301所偵測得到的臉部區域的圖像進行處理,以得到第一圖像中臉部區域所在的座標範圍,並對應得到該第二攝影機20中存儲的第二圖像中臉部區域的座標範圍。其中,本實施方式中,假定該第一圖像及第二圖像中心點的座標為(0,0)。由於第一圖像與第二圖像的內容相同,該座標判定單元302可根據第一圖像與第二圖像之間的尺寸比例對應得到第二圖像中臉區域部的座標範圍。比如若第一圖像與第二圖 像之間的尺寸比例為1:3,則該第二圖像中臉部區域的座標與第一圖像中臉部區域的座標之間的關係為3:1。 The coordinate determining unit 302 is configured to process the image of the face region detected by the detecting unit 301 to obtain a coordinate range of the face region in the first image, and correspondingly obtain the second camera 20 The coordinate range of the face region in the second image stored in . In the present embodiment, it is assumed that the coordinates of the center point of the first image and the second image are (0, 0). Since the content of the first image and the second image are the same, the coordinate determining unit 302 can obtain the coordinate range of the face region portion in the second image according to the size ratio between the first image and the second image. For example, if the first image and the second image The ratio between the dimensions of the image is 1:3, and the relationship between the coordinates of the face region in the second image and the coordinates of the face region in the first image is 3:1.
該攫取單元303用於根據座標範圍在第二圖像中攫取對應的區域,並將第二圖像中被攫取的區域對應第一圖像中臉部區域按比例縮小後替換第一圖像中的臉部區域,以得到一第三圖像。此時所得到的第三圖像的臉部區域則具有較高的解析度,而其他區域則具有較低的解析度。 The capturing unit 303 is configured to capture a corresponding area in the second image according to the coordinate range, and replace the captured area in the second image with the face area in the first image, and then replace the first image. The face area to get a third image. The face region of the third image obtained at this time has a higher resolution, while the other regions have a lower resolution.
該辨識單元304用於對第三圖像中的臉部特徵進行辨識,如與一預存的圖像進行比對,以判斷由第一攝影機10及第二攝影機20所拍攝的人物是否為合法用戶。其中,辨識技術已屬本領域內習知技術,如透過比對第三圖像與預存的圖像之間的相同區域的圖元即可得知兩圖像之間的相似度,當相似度超過某一預定值時即可認為第三圖像中的人物與預存圖像中的人物相同。 The identification unit 304 is configured to identify a facial feature in the third image, such as comparing with a pre-stored image, to determine whether the person photographed by the first camera 10 and the second camera 20 is a legitimate user. . Among them, the identification technology is a well-known technique in the art, for example, by comparing the primitives of the same area between the third image and the pre-stored image, the similarity between the two images can be known, when the similarity is When a certain predetermined value is exceeded, the person in the third image is considered to be the same as the person in the pre-stored image.
前述影像辨識系統透過對具有較低解析度的第一圖像進行偵測,以得到其中臉部區域的座標範圍,並對應得到第二圖像中臉部區域的座標範圍。之後再根據得到的座標範圍攫取第二圖像中的對應區域以合成第三圖像。第三圖像中臉部區域具有較高的解析度,故,使得後續辨識過程中可更加精確,且同時僅僅只攫取第二圖像中對應的區域又避免了將整個第二圖像全部傳送至影像處理系統所導致的處理時間太長。 The image recognition system detects the first image having a lower resolution to obtain a coordinate range of the face region, and correspondingly obtains a coordinate range of the face region in the second image. Then, corresponding regions in the second image are extracted according to the obtained coordinate range to synthesize the third image. The face region in the third image has a higher resolution, so that the subsequent identification process can be more accurate, and at the same time, only the corresponding region in the second image is captured, and the entire second image is avoided. The processing time caused by the image processing system is too long.
下面將以一實例來說明本發明影像辨識系統的具體工作原理。 The specific working principle of the image recognition system of the present invention will be described below by way of an example.
請一併參閱圖3及圖4A-4B,該第一攝影機10及第二攝影機20同時對一被攝場景50進行拍攝,得到該第一圖像51及第二圖像52。其 中,該第一圖像51及第二圖像52內容相同,然,第一圖像51相對第二圖像52具有較低的解析度及尺寸。該第一圖像51被傳送至後端的影像處理系統30。 Referring to FIG. 3 and FIG. 4A-4B together, the first camera 10 and the second camera 20 simultaneously capture a scene 50 to obtain the first image 51 and the second image 52. its The first image 51 and the second image 52 have the same content. However, the first image 51 has a lower resolution and size than the second image 52. The first image 51 is transmitted to the image processing system 30 at the back end.
請一併參閱圖5,該偵測單元301對該第一圖像51進行偵測,得到該第一圖像51中具有三個臉部區域,如圖5中虛線框所示。該座標判定單元302則對應得到該三個臉部區域所對應的座標範圍。之後,該座標判定單元302根據第一圖像51與第二圖像52之間的大小比例對應得到第二圖像52中三個臉部區域的座標範圍。 Referring to FIG. 5 together, the detecting unit 301 detects the first image 51, and obtains three face regions in the first image 51, as shown by the dashed box in FIG. The coordinate determining unit 302 correspondingly obtains the coordinate range corresponding to the three face regions. Thereafter, the coordinate determining unit 302 obtains the coordinate range of the three face regions in the second image 52 according to the size ratio between the first image 51 and the second image 52.
該攫取單元303根據座標判定單元302所得到的座標範圍對應從存儲於該第二攝影機20中的第二圖像52中攫取三個臉部區域,並將該被攫取的三個臉部區域的圖像對應縮小。該被縮小的三個臉部區域的圖像即用於替換該第一圖像51中的三個臉部區域,以得到該第三圖像。此時,該第三圖像中臉部區域具有較高的解析度,其他區域則具有較低的解析度。該辨識單元304對該第三圖像進行辨識。 The capturing unit 303 extracts three face regions from the second image 52 stored in the second camera 20 according to the coordinate range obtained by the coordinate determining unit 302, and extracts the three face regions of the captured The image corresponds to zoom out. The image of the reduced three face regions is used to replace the three face regions in the first image 51 to obtain the third image. At this time, the face area in the third image has a higher resolution, and the other areas have a lower resolution. The identification unit 304 identifies the third image.
請參閱圖6,本發明影像辨識方法的較佳實施方式包括以下步驟: Referring to FIG. 6, a preferred embodiment of the image recognition method of the present invention includes the following steps:
步驟S1:該第一攝影機10及第二攝影機20在同一時間對同一場景50進行拍攝,以分別得到第一圖像51及第二圖像52。其中,該第一圖像51及第二圖像52內容相同,然,第一圖像51相對第二圖像52具有較低的解析度及較小的尺寸。 Step S1: The first camera 10 and the second camera 20 capture the same scene 50 at the same time to obtain the first image 51 and the second image 52, respectively. The first image 51 and the second image 52 have the same content. However, the first image 51 has a lower resolution and a smaller size than the second image 52.
步驟S2:該第一圖像51被傳送至後端的影像處理系統30。 Step S2: The first image 51 is transmitted to the image processing system 30 at the back end.
步驟S3:該偵測單元301對該第一圖像51進行偵測,得到該第一 圖像51中具有三個臉部區域。 Step S3: the detecting unit 301 detects the first image 51, and obtains the first There are three face regions in the image 51.
步驟S4:該座標判定單元302對應得到該三個臉部區域所對應的座標範圍,並根據第一圖像51和第二圖像52之間的大小比例對應得到第二圖像52中三個臉部區域的座標範圍。 Step S4: the coordinate determining unit 302 correspondingly obtains the coordinate range corresponding to the three face regions, and obtains three of the second images 52 according to the size ratio between the first image 51 and the second image 52. The range of coordinates of the face area.
步驟S5:該攫取單元303根據座標判定單元302所得到的座標範圍對應從存儲於該第二攝影機20中的第二圖像52中攫取三個臉部圖像,並將得到的三個臉部圖像根據第一圖像51和第二圖像52之間的大小比例對應縮小。該被縮小的三個臉部圖像即用於替換該第一圖像51中的三個臉部區域對應的座標範圍的圖像,以得到該第三圖像。此時,該第三圖像中臉部區域具有較高的解析度,其他區域則具有較低的解析度。 Step S5: The capturing unit 303 extracts three facial images from the second image 52 stored in the second camera 20 according to the coordinate range obtained by the coordinate determining unit 302, and obtains three facial images. The image is correspondingly reduced in accordance with the size ratio between the first image 51 and the second image 52. The reduced three face images are images for replacing the coordinate ranges corresponding to the three face regions in the first image 51 to obtain the third image. At this time, the face area in the third image has a higher resolution, and the other areas have a lower resolution.
步驟S6:該辨識單元304對該第三圖像進行辨識。 Step S6: The identification unit 304 identifies the third image.
綜上所述,本發明確已符合發明專利的要件,爰依法提出專利申請。惟,以上所述者僅為本發明的較佳實施方式,本發明的範圍並不以上述實施方式為限,舉凡熟悉本案技藝的人士爰依本發明的精神所作的等效修飾或變化,皆應涵蓋於以下申請專利範圍內。 In summary, the present invention has indeed met the requirements of the invention patent, and has filed a patent application according to law. The above is only the preferred embodiment of the present invention, and the scope of the present invention is not limited to the above-described embodiments, and equivalent modifications or variations made by those skilled in the art in accordance with the spirit of the present invention are It should be covered by the following patent application.
10‧‧‧第一攝影機 10‧‧‧First camera
20‧‧‧第二攝影機 20‧‧‧Second camera
30‧‧‧影像處理系統 30‧‧‧Image Processing System
Claims (6)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW98139319A TWI469064B (en) | 2009-11-19 | 2009-11-19 | Video recognizing system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW98139319A TWI469064B (en) | 2009-11-19 | 2009-11-19 | Video recognizing system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| TW201118762A TW201118762A (en) | 2011-06-01 |
| TWI469064B true TWI469064B (en) | 2015-01-11 |
Family
ID=44935841
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| TW98139319A TWI469064B (en) | 2009-11-19 | 2009-11-19 | Video recognizing system and method |
Country Status (1)
| Country | Link |
|---|---|
| TW (1) | TWI469064B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107071300A (en) * | 2017-03-22 | 2017-08-18 | 努比亚技术有限公司 | A kind of image combining method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060187305A1 (en) * | 2002-07-01 | 2006-08-24 | Trivedi Mohan M | Digital processing of video images |
| TW200905577A (en) * | 2007-07-31 | 2009-02-01 | Univ Nat Taiwan Science Tech | Iris recognition system |
| US20090046157A1 (en) * | 2007-08-13 | 2009-02-19 | Andrew Cilia | Combined wide-angle/zoom camera for license plate identification |
| US7505074B2 (en) * | 2004-02-06 | 2009-03-17 | Canon Kabushiki Kaisha | Image sensing apparatus and its control method |
-
2009
- 2009-11-19 TW TW98139319A patent/TWI469064B/en not_active IP Right Cessation
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060187305A1 (en) * | 2002-07-01 | 2006-08-24 | Trivedi Mohan M | Digital processing of video images |
| US7505074B2 (en) * | 2004-02-06 | 2009-03-17 | Canon Kabushiki Kaisha | Image sensing apparatus and its control method |
| TW200905577A (en) * | 2007-07-31 | 2009-02-01 | Univ Nat Taiwan Science Tech | Iris recognition system |
| US20090046157A1 (en) * | 2007-08-13 | 2009-02-19 | Andrew Cilia | Combined wide-angle/zoom camera for license plate identification |
Also Published As
| Publication number | Publication date |
|---|---|
| TW201118762A (en) | 2011-06-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102063610B (en) | Image identification system and method thereof | |
| KR102213328B1 (en) | Video processing apparatus, video processing method, and program | |
| JP5567853B2 (en) | Image recognition apparatus and method | |
| JP4642128B2 (en) | Image processing method, image processing apparatus and system | |
| JP5390943B2 (en) | Image processing apparatus and image processing method | |
| WO2020094091A1 (en) | Image capturing method, monitoring camera, and monitoring system | |
| WO2020034083A1 (en) | Image processing apparatus and method for feature extraction | |
| CA3147418A1 (en) | Living body detection method and system for human face by using two long-baseline cameras | |
| CN101840578B (en) | Camera device and dynamic detection method thereof | |
| CN109905641B (en) | A target monitoring method, device, equipment and system | |
| US20130083965A1 (en) | Apparatus and method for detecting object in image | |
| WO2017221644A1 (en) | Image processing device, image processing system, image processing method, and program | |
| CN108182682A (en) | A kind of detection method based on Computer Image Processing and pattern-recognition | |
| CN111582118A (en) | Face recognition method and device | |
| US7835552B2 (en) | Image capturing apparatus and face area extraction method | |
| CN114743264A (en) | Shooting behavior detection method, device, equipment and storage medium | |
| CN112507767B (en) | Face recognition method and related computer system | |
| JP2013156671A (en) | Image monitoring device | |
| CN104966283A (en) | Imaging layered registering method | |
| TWI469064B (en) | Video recognizing system and method | |
| JP2014191685A (en) | Image processing apparatus and image processing method | |
| TWI620147B (en) | Image synthesis method for synthesizing people | |
| JP2008035301A (en) | Mobile tracking device | |
| US9049382B2 (en) | Image processing apparatus and image processing method | |
| JP2009068935A (en) | Stereo measurement device, stereo measurement method, and stereo measurement processing program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| MM4A | Annulment or lapse of patent due to non-payment of fees |