[go: up one dir, main page]

WO2019236328A1 - Correction d'une image d'empreinte digitale - Google Patents

Correction d'une image d'empreinte digitale Download PDF

Info

Publication number
WO2019236328A1
WO2019236328A1 PCT/US2019/034032 US2019034032W WO2019236328A1 WO 2019236328 A1 WO2019236328 A1 WO 2019236328A1 US 2019034032 W US2019034032 W US 2019034032W WO 2019236328 A1 WO2019236328 A1 WO 2019236328A1
Authority
WO
WIPO (PCT)
Prior art keywords
darkfield
image
candidate image
sensor
fingerprint
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2019/034032
Other languages
English (en)
Inventor
Bruno Flament
Daniela HALL
Etienne De Foras
Harihar NARASIMHA-IYER
Romain Fayolle
Jonathan Baudot
Abbas ATAYA
Sina AKHBARI
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
InvenSense Inc
Original Assignee
InvenSense Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US16/270,516 external-priority patent/US20190188442A1/en
Application filed by InvenSense Inc filed Critical InvenSense Inc
Publication of WO2019236328A1 publication Critical patent/WO2019236328A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/12—Fingerprints or palmprints
    • G06V40/13—Sensors therefor
    • G06V40/1306—Sensors therefor non-optical, e.g. ultrasonic or capacitive sensing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/30—Noise filtering

Definitions

  • Fingerprint sensors have become ubiquitous in mobile devices as well as other applications for authenticating a user’s identity. They provide a fast and convenient way for the user to unlock a device, provide authentication for payments, etc.
  • Current fingerprint sensors are typically area sensors that obtain a two-dimensional image of the users finger area presented to the sensor. Different technologies can be used to image the finger such as capacitive, ultrasound, and optical sensing. Once an image is obtained, that image is processed by a matcher to extract features and to compare against stored images to
  • FIG. 1A is a diagram illustrating a piezoelectric micromachined ultrasonic transducer (PMUT) device having a center pinned membrane, according to some embodiments.
  • FIG. 1 B is a diagram illustrating a PMUT device having an unpinned membrane, according to some embodiments.
  • FIG. 2 is a diagram illustrating an example of membrane movement during activation of a PMUT device having a center pinned membrane, according to some embodiments.
  • FIG. 3 is a top view of the PMUT device of FIG. 1A, according to some embodiments.
  • FIG. 4 is a simulated map illustrating maximum vertical displacement of the membrane of the PMUT device shown in FIGs. 1A, 2, and 3, according to some embodiments.
  • FIG. 5 is a top view of an example PMUT device having a circular shape, according to some embodiments.
  • FIG. 6 illustrates an example array of square-shaped PMUT devices, according to some embodiments.
  • FIG. 7 A illustrates an example of an operational environment for sensing of human touch, according to some embodiments.
  • FIG. 7B illustrates an example fingerprint sensor, in accordance with various aspects
  • FIG. 8 illustrates a flow diagram of an example process for darkfieid acquisition, according to some embodiments.
  • FIG. 9 illustrates a flow diagram of an example method for capturing a darkfieid image, according to some embodiments
  • FIG. 10 illustrates example operation of void detection associated with a two- dimensional array of ultrasonic transducers, according to some embodiments.
  • FIGs. 11 A-11 E illustrate graphs of pixel values relative to time during void detection at a sensor, according to an embodiment.
  • FIG. 12 illustrates a flow diagram of a method for void detection, according to various embodiments
  • FIG. 13 illustrates a flow diagram of a method for darkfield tracking, according to various embodiments.
  • FIG. 14 illustrates a flow diagram an example method for determining whether an object is interacting with the sensor.
  • FIG. 15 illustrates a flow diagram another example method for determining whether an object is interacting with the sensor.
  • FIG. 16 illustrates an example fingerprint sensor comprising multiple layers, according to embodiments.
  • FIG. 17 illustrates a flow diagram of the procedures to predict and reconstruct the darkfield image over varying temperature, according to embodiments.
  • FIGs. 18A and 18B illustrate an example modeling of the background image of FIG. 17, according to embodiments.
  • FIG. 19 illustrates an example graph of the Darkfield Field Quality (DFQ) Spectral improvement over temperature, according to embodiments.
  • FIG. 20 illustrates a flow diagram of a process for modeling a darkfield image over varying temperature, according to an embodiment.
  • FIG. 21 Illustrates an example system for modeling a darkfield image based on a best fit model, according to embodiments.
  • FIG. 22 illustrates a flow diagram of a process for modeling a darkfield image using a best fit algorithm corresponding to the example system of FIG. 21 , according to embodiments.
  • F!G. 23 illustrates a flow diagram of an example method for determining darkfieid contamination and performing dynamic updates of the fingerprint templates of a fingerprint authentication system, according to embodiments.
  • FIG. 24 Illustrates a flow diagram of an example method for evaluating a darkfieid image for contamination, according to embodiments.
  • FIG. 25 illustrates a flow diagram of an example method for performing a darkfieid contamination verification, according to embodiments
  • FIG. 26 illustrates an example system for evaluating a darkfieid image for contamination based on a best fit model, according to embodiments.
  • FIG. 27 shows an example of an example defined temperature range for an allowed variance in darkfieid changes, according to embodiments.
  • Embodiments described herein may be discussed in the general context of processor- executable instructions residing on some form of non-transitory processor-readable medium, such as program modules, executed by one or more computers or other devices.
  • program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types.
  • the functionality of the program modules may be combined or distributed as desired in various embodiments.
  • a single block may be described as performing a function or functions; however, in actual practice, the function or functions performed by that block may be performed in a single component or across multiple components, and/or may be performed using hardware, using software, or using a combination of hardware and software.
  • various illustrative components, blocks, modules, logic, circuits, and steps have been described generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.
  • the example fingerprint sensing system and/or mobile electronic device described herein may include components other than those shown, including well-known components.
  • the techniques may be realized at least in part by a non-transitory processor-readable storage medium comprising instructions that, when executed, perform one or more of the methods described herein.
  • the non-transitory processor-readable data storage medium may form part of a computer program product, which may include packaging materials.
  • the non-transitory processor-readable storage medium may comprise random access memory (RAM) such as synchronous dynamic random access memory (SDRAM), read only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), FLASH memory, other known storage media, and the like.
  • RAM synchronous dynamic random access memory
  • ROM read only memory
  • NVRAM non-volatile random access memory
  • EEPROM electrically erasable programmable read-only memory
  • FLASH memory other known storage media, and the like.
  • the techniques additionally, or alternatively, may be realized at least in part by a processor-readable communication medium that carries or communicates code in the form of instructions or data structures and that can be accessed, read, and/or executed by a computer or other processor.
  • processors such as one or more motion processing units (MRUs), sensor processing units (SPUs), host processor(s) or core(s) thereof, digital signal processors (DSPs), general purpose
  • MRUs motion processing units
  • SPUs sensor processing units
  • DSPs digital signal processors
  • processors application specific integrated circuits (ASICs), application specific instruction set processors (ASiPs), field programmable gate arrays (FPGAs), a programmable logic controller (PLC), a complex programmable logic device (CPLD), a discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein, or other equivalent integrated or discrete logic circuitry.
  • ASICs application specific integrated circuits
  • ASiPs application specific instruction set processors
  • FPGAs field programmable gate arrays
  • PLC programmable logic controller
  • CPLD complex programmable logic device
  • discrete gate or transistor logic discrete hardware components, or any combination thereof designed to perform the functions described herein, or other equivalent integrated or discrete logic circuitry.
  • processor can refer to substantially any computing processing unit or device comprising, but not limited to comprising, single-core processors; single-processors with software mu!fiihread execution capability; multi-core processors; multi-core processors with software multithread execution capability; multi-core processors with hardware multithread technology; parallel platforms; and parallel platforms with distributed shared memory.
  • processors can exploit nano-scale architectures such as, but not limited to, molecular and quantum-dot based transistors, switches and gates, in order to optimize space usage or enhance performance of user equipment.
  • a processor may also be implemented as a combination of computing processing units.
  • a general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine.
  • a processor may also be implemented as a combination of computing devices, e.g., a combination of an SPU/MPU and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with an SPU core, MPU core, or any other such configuration.
  • Example piezoelectric micromachined ultrasonic transducer begins with a description of an example piezoelectric micromachined ultrasonic transducer (PMUT), in accordance with various embodiments.
  • Example sensors including arrays of ultrasonic transducers are then described.
  • Example darkfieid capture is then described.
  • Example operations for the darkfieid capture are then described.
  • Embodiments described herein provide a method and device for darkfieid capture at a sensor. It is determined whether an object is interacting with the sensor. Provided an object is not interacting with the sensor, a determination is made that a darkfieid candidate image can be captured at the sensor it is determined whether to capture a darkfieid candidate image at the sensor based at least in part on the determination that a darkfieid candidate image can be captured at the sensor. Responsive to making a determination to capture the darkfieid candidate image, the darkfieid candidate image is captured at the sensor, wherein the darkfieid candidate image is an image absent an object interacting with the sensor. A darkfieid estimate is updated with the darkfieid candidate image.
  • Systems and methods disclosed herein, in one or more aspects provide efficient structures for an acoustic transducer (e.g., a piezoelectric actuated transducer or PMUT).
  • an acoustic transducer e.g., a piezoelectric actuated transducer or PMUT.
  • the term“or” is intended to mean an inclusive“or” rather than an exclusive“or”. That is, unless specified otherwise, or clear from context,“X employs A or B” is intended to mean any of the natural inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then“X employs A or B” is satisfied under any of the foregoing instances in addition, the articles“a” and“an” as used in this application and the appended claims should generally be construed to mean“one or more” unless specified otherwise or clear from context to be directed to a singular form.
  • the word“coupled” is used herein to mean direct or indirect electrical or mechanical coupling in addition
  • the word “example” is used herein to mean serving as an example, instance, or illustration.
  • FIG. 1A is a diagram illustrating a PMUT device 100 having a center pinned membrane, according to some embodiments.
  • PMUT device 100 includes an interior pinned membrane 120 positioned over a substrate 140 to define a cavity 130.
  • membrane 120 is attached both to a surrounding edge support 102 and interior support 104.
  • edge support 102 is connected to an electric potential.
  • Edge support 102 and interior support 104 may be made of electrically conducting materials, such as and without limitation, aluminum, molybdenum, or titanium.
  • Edge support 102 and interior support 104 may also be made of dielectric materials, such as silicon dioxide, silicon nitride or aluminum oxide that have electrical connections the sides or in vias through edge support 102 or interior support 104, electrically coupling lower electrode 106 to electrical wiring in substrate 140.
  • dielectric materials such as silicon dioxide, silicon nitride or aluminum oxide that have electrical connections the sides or in vias through edge support 102 or interior support 104, electrically coupling lower electrode 106 to electrical wiring in substrate 140.
  • both edge support 102 and interior support 104 are attached to a substrate 140.
  • substrate 140 may include at least one of, and without limitation, silicon or silicon nitride. It should be appreciated that substrate 140 may include electrical wirings and connection, such as aluminum or copper.
  • substrate 140 includes a CMOS logic wafer bonded to edge support 102 and interior support 104.
  • the membrane 120 comprises multiple layers in an example embodiment, the membrane 120 includes lower electrode 106, piezoelectric layer 110, and upper electrode 108, where lower electrode 106 and upper electrode 108 are coupled to opposing sides of piezoelectric layer 110.
  • PMUT device 100 is a microelectromechanical (MEMS) device.
  • membrane 120 also includes a mechanical support layer 112 (e.g., stiffening layer) to mechanically stiffen the layers in various embodiments, mechanical support layer 112 may include at least one of, and without limitation, silicon, silicon oxide, silicon nitride, aluminum, molybdenum, titanium, etc.
  • PMUT device 100 also includes an acoustic coupling layer 114 above membrane 120 for supporting transmission of acoustic signals. It should be appreciated that acoustic coupling layer can include air, liquid, gel-like materials, or other materials for supporting transmission of acoustic signals in one
  • PMUT device 100 also includes platen layer 116 above acoustic coupling layer 114 for containing acoustic coupling layer 114 and providing a contact surface for a finger or other sensed object with PMUT device 100. It should be appreciated that, in various embodiments, in various combinations of platen layer 116 above acoustic coupling layer 114 for containing acoustic coupling layer 114 and providing a contact surface for a finger or other sensed object with PMUT device 100. It should be appreciated that, in various combinations thereof.
  • acoustic coupling layer 114 provides a contact surface, such that platen layer 116 is optional. Moreover, it should be appreciated that acoustic coupling layer 114 and/or platen layer 116 may be included with or used in conjunction with multiple PMUT devices. For example, an array of PMUT devices may be coupled with a single acoustic coupling layer 114 and/or platen layer 116.
  • FIG. 1 B is identical to FIG. 1A in ever way, except that the PMUT device 100’ of FIG. 1 B omits the interior support 104 and thus membrane 120 is not pinned (e.g., is“unpinned”). There may be instances in which an unpinned membrane 120 is desired. However, in other instances, a pinned membrane 120 may be employed.
  • FiG. 2 is a diagram illustrating an example of membrane movement during activation of pinned PMUT device 100, according to some embodiments. As illustrated with respect to FIG.
  • the electrodes 106 and 108 deliver a high frequency electric charge to the piezoelectric layer 110, causing those portions of the membrane 120 not pinned to the surrounding edge support 102 or interior support 104 to be displaced upward into the acoustic coupling layer 114.
  • This generates a pressure wave that can be used for signal probing of the object.
  • Return echoes can be detected as pressure waves causing movement of the membrane, with compression of the piezoelectric material in the membrane causing an electrical signal proportional to amplitude of the pressure wave.
  • the described PMUT device 100 can be used with almost any electrical device that converts a pressure wave into mechanical vibrations and/or electrical signals.
  • the PMUT device 100 can comprise an acoustic sensing element (e g., a piezoelectric element) that generates and senses ultrasonic sound waves.
  • An object in a path of the generated sound waves can create a disturbance (e.g., changes in frequency or phase, reflection signal, echoes, etc.) that can then be sensed.
  • the interference can be analyzed to determine physical parameters such as (but not limited to) distance, density and/or speed of the object.
  • the PMUT device 100 can be utilized in various applications, such as, but not limited to, fingerprint or physiologic sensors suitable for wireless devices, industrial systems, automotive systems, robotics, telecommunications, security, medical devices, etc.
  • the PMUT device 100 can be part of a sensor array comprising a plurality of ultrasonic transducers deposited on a wafer, along with various logic, control and communication electronics.
  • a sensor array may comprise homogenous or identical PMUT devices 100, or a number of different or heterogenous device structures.
  • the PMUT device 100 employs a piezoelectric layer 110, comprised of materials such as, but not limited to, aluminum nitride (AIN), lead zirconate titanate (PZT), quartz, polyvinylidene fluoride (PVDF), and/or zinc oxide, to facilitate both acoustic signal production and sensing.
  • the piezoelectric layer 110 can generate electric charges under mechanical stress and conversely experience a mechanical strain in the presence of an electric field.
  • the piezoelectric layer 110 can sense mechanical vibrations caused by an ultrasonic signal and produce an electrical charge at the frequency (e.g., ultrasonic frequency) of the vibrations.
  • the piezoelectric layer 110 can generate an ultrasonic wave by vibrating in an oscillatory fashion that might be at the same frequency (e.g., ultrasonic frequency) as an input current generated by an alternating current (AC) voltage applied across the piezoelectric layer 110.
  • AC alternating current
  • the piezoelectric layer 110 can include almost any material (or combination of materials) that exhibits piezoelectric properties, such that the structure of the material does not have a center of symmetry and a tensile or compressive stress applied to the material alters the separation between positive and negative charge sites in a ceil causing a polarization at the surface of the material.
  • the polarization is directly proportional to the applied stress and is direction dependent so that compressive and tensile stresses results in electric fields of opposite polarizations.
  • the PMUT device 100 comprises electrodes 106 and 108 that supply and/or collect the electrical charge to/from the piezoelectric layer 110.
  • electrodes 106 and 108 can be continuous and/or patterned electrodes (e.g., in a continuous layer and/or a patterned layer).
  • electrode 106 is a patterned electrode
  • electrode 108 is a continuous electrode.
  • electrodes 106 and 108 can be comprised of almost any metal layers, such as, but not limited to, aluminum (A!)/titanium (Ti), molybdenum (Mo), etc., which are coupled with an on opposing sides of the piezoelectric layer 110.
  • the acoustic impedance of acoustic coupling layer 114 is selected to be similar to the acoustic impedance of the platen layer 116, such that the acoustic wave is efficiently propagated to/from the membrane 120 through acoustic coupling layer 114 and platen layer 116
  • the platen layer 116 can comprise various materials having an acoustic impedance in the range between 0.8 to 4 Mega Rayleigh (MRayi), such as, but not limited to, plastic, resin, rubber, Teflon, epoxy, etc.
  • the platen layer 116 can comprise various materials having a high acoustic impedance (e.g., an acoustic impendence greater than 10 MRayi), such as, but not limited to, glass, aluminum-based alloys, sapphire, etc.
  • the platen layer 116 can be selected based on an application of the sensor. For instance, in fingerprinting applications, platen layer 116 can have an acoustic impedance that matches (e.g., exactly or approximately) the acoustic impedance of human skin (e.g., 1.6x10 ® Rayl).
  • the platen layer 116 can further include a thin layer of anti-scratch material.
  • the anti-scratch layer of the platen layer 116 is less than the wavelength of the acoustic wave that is to be generated and/or sensed to provide minimum interference during propagation of the acoustic wave.
  • the anti-scratch layer can comprise various hard and scratch-resistant materials (e.g., having a Mohs hardness of over 7 on the Mohs scale), such as, but not limited to sapphire, glass, titanium nitride (TIN), silicon carbide (SiC), diamond, etc.
  • PMUT device 100 can operate at 20 MHz and accordingly, the wavelength of the acoustic wave propagating through the acoustic coupling layer 114 and platen layer 116 can be 70-150 microns.
  • insertion loss can be reduced and acoustic wave propagation efficiency can be improved by utilizing an anti-scratch layer having a thickness of 1 micron and the platen layer 116 as a whole having a thickness of 1-2 millimeters.
  • anti-scratch material as used herein relates to a material that is resistant to scratches and/or scratch-proof and provides substantial protection against scratch marks.
  • the PMUT device 100 can include metal layers (e.g., aluminum (A!)/titanium (Ti), molybdenum (Mo), etc.) patterned to form electrode 106 in particular shapes (e.g., ring, circle, square, octagon, hexagon, etc.) that are defined in plane with the membrane 120. Electrodes can be placed at a maximum strain area of the membrane 120 or placed at close to either or both the surrounding edge support 102 and interior support 104. Furthermore, in one example, electrode 108 can be formed as a continuous layer providing a ground plane in contact with mechanical support layer 112, which can be formed from silicon or other suitable mechanical stiffening material. In still other embodiments, the electrode 106 can be routed along the interior support 104, advantageously reducing parasitic capacitance as compared to routing along the edge support 102.
  • metal layers e.g., aluminum (A!)/titanium (Ti), molybdenum (Mo), etc.
  • electrode 106 in particular shapes (e.g.,
  • the membrane 120 when actuation voltage is applied to the electrodes, the membrane 120 will deform and move out of plane. The motion then pushes the acoustic coupling layer 114 it is in contact with and an acoustic (ultrasonic) wave is generated. Oftentimes, vacuum is present inside the cavity 130 and therefore damping contributed from the media within the cavity 130 can be ignored. However, the acoustic coupling layer 114 on the other side of the membrane 120 can substantially change the damping of the PMUT device 100.
  • a quality factor greater than 20 can be observed when the PMUT device 100 is operating in air with atmosphere pressure (e.g., acoustic coupling layer 114 is air) and can decrease lower than 2 if the PMUT device 100 is operating in water (e.g., acoustic coupling layer 114 is water)
  • FIG. 3 is a top view of the PMUT device 100 of FIG. 1A having a substantially square shape, which corresponds in part to a cross section along dotted line 101 in FIG. 3. Layout of surrounding edge support 102, interior support 104, and lower electrode 106 are illustrated, with other continuous layers not shown.
  • the term“substantially” in “substantially square shape” is intended to convey that a PMUT device 100 is generally square shaped, with allowances for variations due to manufacturing processes and tolerances, and that slight deviation from a square shape (e.g., rounded corners, slightly wavering lines, deviations from perfectly orthogonal corners or intersections, etc.) may be present in a manufactured device. While a generally square arrangement PMUT device is shown, alternative
  • embodiments including rectangular, hexagon, octagonal, circular, or elliptical are contemplated.
  • more complex electrode or PMUT device shapes can be used, including irregular and non-symmetric layouts such as chevrons or pentagons for edge support and electrodes.
  • FIG. 4 is a simulated topographic map 400 illustrating maximum vertical displacement of the membrane 120 of the PMUT device 100 shown in F!Gs. 1A-3. As indicated, maximum displacement generally occurs along a center axis of the lower electrode, with corner regions having the greatest displacement. As with the other figures, FIG 4 is not drawn to scale with the vertical displacement exaggerated for illustrative purposes, and the maximum vertical displacement is a fraction of the horizontal surface area comprising the PMUT device 100. In an example PMUT device 100, maximum vertical displacement may be measured in
  • nanometers while surface area of an individual PMUT device 100 may be measured in square microns.
  • FIG. 5 is a top view of another example of the PMUT device 100 of FIG. 1A having a substantially circular shape, which corresponds in part to a cross section along dotted line 101 in FIG. 5.
  • Layout of surrounding edge support 102, inferior support 104, and lower electrode 106 are illustrated, with other continuous layers not shown.
  • the term“substantially” in“substantially circular shape” is intended to convey that a PMUT device 100 is generally circle-shaped, with allowances for variations due to manufacturing processes and tolerances, and that slight deviation from a circle shape (e.g., slight deviations on radial distance from center, etc.) may be present in a manufactured device.
  • FIG. 6 illustrates an example two-dimensional array 600 of square-shaped PMUT devices 601 formed from PMUT devices having a substantially square shape similar to that discussed in conjunction with FIGs. 1A, 1 B, 2, and 3.
  • array 600 includes columns of square-shaped PMUT devices 601 that are in rows and columns it should be appreciated that rows or columns of the square-shaped P UT devices 601 may be offset.
  • square-shaped PMUT devices 601 may contact each other or be spaced apart.
  • adjacent square-shaped PMUT devices 601 are electrically isolated.
  • groups of adjacent square-shaped PMUT devices 601 are electrically connected, where the groups of adjacent square-shaped PMUT devices 601 are electrically isolated.
  • selected sets of PMUT devices in the two-dimensional array can transmit an acoustic signal (e.g., a short ultrasonic pulse) and during sensing, the set of active PMSJT devices in the two-dimensional array can detect an interference of the acoustic signal with an object (in the path of the acoustic wave).
  • the received interference signal e.g., generated based on reflections, echoes, etc. Of the acoustic signal from the object
  • an image of the object, a distance of the object from the sensing component, a density of the object, a motion of the object, etc. can all be determined based on comparing a frequency and/or phase of the interference signal with a frequency and/or phase of the acoustic signal.
  • results generated can be further analyzed or presented to a user via a display device (not shown).
  • an image is obtained, that image is processed by a matcher to extract features and to compare against stored images to authenticate the user.
  • accuracy of captured images is essential to the performance of image matching for user authentication.
  • any background image or contributions to the image other than from the fingerprint should be removed or corrected for.
  • the background image is the image obtained by the sensor when no finger is present. This background image is also referred to as the darkfield or offset.
  • the embodiments described herein provide for capturing the darkfield image and correcting the fingerprint image for the darkfield image.
  • Various embodiments described herein provide a finger detection mode for identifying if an object has been placed on a fingerprint sensor.
  • a darkfield candidate image can be captured at the sensor, where the darkfield candidate image is an image absent an object interacting with the sensor.
  • the disclosure recognizes and addresses, in at least certain embodiments, the issue of power consumption and a power efficient always-on approach to void detection for determining when to capture a darkfield image at the sensor.
  • a void is detected when it is determined that an object is not interacting with the sensor such that a darkfield image can be captured.
  • the void defection stage is implemented continuously or nearly continuously and allows for the appropriate acquisition of the darkfield Image.
  • implementation of the low-power detection stage permits removal of physical actuation device (e.g., buttons or the like) while maintaining low power consumption. Absence of a physical actuation device does not hinder low-power consumption and does simplify user- device interaction when sensing human touch. While embodiments of the disclosure are illustrated with reference to a mobile electronic device, the embodiments are not limited in this respect and the embodiments can be applied to any device (mobile or otherwise) having a surface that is sensitive to touch and permits or otherwise facilitates control of the device by an end-user. Such a touch-sensitive surface can embody or can constitute, for example, a fingerprint sensor.
  • Mobile devices can be embodied in or can include consumer electronics devices (e.g., smartphones, portable gaming devices); vehicular devices (such as navigation and/or entertainment system device); medical devices; keys (e.g., for locking and gaining access to buildings, storage receptacles, cars, etc.); and the like.
  • consumer electronics devices e.g., smartphones, portable gaming devices
  • vehicular devices such as navigation and/or entertainment system device
  • medical devices e.g., for locking and gaining access to buildings, storage receptacles, cars, etc.
  • keys e.g., for locking and gaining access to buildings, storage receptacles, cars, etc.
  • splitting the sensing of human touch into a low power, always-on detection stage and a triggered, full-power analysis stage permits sensing human touch continuously or nearly continuously, without causing battery drainage or other inefficiencies.
  • utilizing a low power, always-on detection stage allows for the detection of voids for capturing darkfield images using the sensor.
  • FIG. 7A illustrates an example of an operational environment 700 for sensing of human touch in accordance with one or more embodiments of the disclosure.
  • a device 710 includes a fingerprint sensor 715 or other type of surface sensitive to touch in one embodiment, fingerprint sensor 715 is disposed beneath a touch-screen display device of device 710. In another embodiment, fingerprint sensor 715 is disposed adjacent or close to a touch-screen display device of device 710. In another embodiment, fingerprint sensor 715 is comprised within a touch-screen display device of device 710. in another embodiment, fingerprint sensor 715 is disposed on the side or back of the device it should be appreciated that device 710 includes a fingerprint sensor 715 for sensing a fingerprint of a finger interacting with device 710.
  • a human finger can touch or interact with a specific area of device 710 proximate fingerprint sensor 715.
  • fingerprint sensor 715 can be hard and need not include movable parts, such as a sensor button configured to detect human touch or otherwise cause the device 710 to respond to human touch.
  • the device 710 can include circuitry that can operate in response to touch (human or otherwise) of the touch-screen display device and/or fingerprint sensor 715 (or, in some embodiments, the other type of touch sensitive surface).
  • device 710 includes always-on circuitry 730 and system circuitry 740. It should be appreciated that components of always-on circuitry 730 and system circuitry 740 might be disposed within the same componentry, and are conceptually distinguished herein such that always-on circuitry 730 includes components that are always-on, or mostly always-on, and system circuitry 740 includes components that are powered off until they are powered on, for example, in response to an activation signal received from always-on circuitry 730.
  • such circuitry can be operatively coupled (e.g., electrically coupled, communicative coupled, etc.) via a bus architecture 735 (or bus 735) or conductive conduits configured to permit the exchange of signals between the always-on circuitry 730 and the system circuitry 740.
  • a printed circuit board (RGB) placed behind a touch-screen display device can include the a!ways-on circuitry 730, the system circuitry 740, and the bus 735.
  • the always-on circuitry 730 and the system circuitry 740 can be configured or otherwise arranged in a single semiconductor die.
  • always-on circuitry 730 can be configured or otherwise arranged in a first semiconductor die and the system circuitry 740 can be configured or otherwise arranged in a second semiconductor die.
  • the bus 735 can be embodied in or can include a dedicated conducting wire or a dedicated data line that connects the always- on circuitry 730 and the system circuitry 740.
  • Always-on circuitry 730 may be a sensor processor (or included within a sensor processor) that also controls the fingerprint sensor, and system circuitry 740 may be the host processor or application processor or included within the host processor or application processor of device 710.
  • always-on circuitry 730 and/or system circuitry 740 includes a temperature sensor for sensing a
  • the always-on circuitry 730 can operate as sensor for human touch and the system circuitry 740, or a portion thereof, can permit or otherwise facilitate analysis of the human touch.
  • always-on circuitry 730 includes fingerprint sensor 715. For example, responsive to capturing an image of a fingerprint, fingerprint sensor 715 can transmit the captured image to system circuitry 740 for analysis.
  • the analysis can include fingerprint recognition or other types of biometric evaluations.
  • the always-on circuitry 730 can be energized or otherwise power-on continuously or nearly continuously and can be configured to monitor touch of fingerprint sensor 715. in addition, in response to human touch (e.g., touch by a human finger or other human body part), the always- on circuitry 730 can be further configured to trigger detection and/or another type of analysis of elements of the human touch or a human body associated therewith. To at least that end, the always-on circuitry 730 can be configured to implement void detection for determining when to capture a darkfield image at the sensor.
  • FIG. 7B illustrates an example fingerprint sensor 715, in accordance with various embodiments.
  • fingerprint sensor 715 includes an array 750 of ultrasonic transducers (e.g., PMUT devices), a processor 760, and a memory 770.
  • ultrasonic transducers e.g., PMUT devices
  • processor 760 performs certain operations in accordance with instructions stored within memory 770. It should be appreciated that components of fingerprint sensor 715 are examples, and that certain components, such as processor 760 and/or memory 770 may not be located within fingerprint sensor 715. For example, always-on circuitry 730 or system circuitry 740 may include a processor and/or memory for performing certain operations.
  • fingerprint sensor 715 includes processor 760 for performing the pixel capture, where pixel capture is performed using subsets of ultrasonic transducers (e.g., PMUTs) of fingerprint sensor 715.
  • processor 760 can perform at least some signal analysis, e.g., void detection, to determine whether an object has interacted with fingerprint sensor 715.
  • void detection e.g., determining whether an object has interacted with fingerprint sensor 715.
  • a darkfieid image can be capture at fingerprint sensor 715.
  • system circuitry 740 is activated in response to detecting a void for capturing the darkfieid image. Subsequent the capture of the darkfieid image, system circuitry 740 can be deactivated.
  • FIG. 7B While the embodiment of FIG. 7B includes processor 760 and memory 770, as described above, it should be appreciated that various functions of processor 760 and memory 770 may reside in other components of device 710 (e.g., within always-on circuitry 730 or system circuitry 740). Moreover, it should be appreciated that processor 760 may be any type of processor for performing any portion of the described functionality (e.g., custom digital logic).
  • a power supply can energize at least a portion of the system circuitry 740 according with trigger signaling (or other type of control signal) provided (e.g., generated and transmitted) by the always-on circuitry 730.
  • system circuitry 740 can include a power controller that can receive trigger signaling (e.g., a control instruction) and, in response, can energize at least one processor of the system circuitry 740 from a power-save state to a full-power state.
  • the at least one processor that transitions from the power-save state to the full power state can execute a darkfieid image capture operation in response to the void detection of always-on circuitry 730.
  • the analysis of a darkfieid image or an image of a fingerprint can include computer-accessible instruction (e.g., computer-readable instructions and/or computer-executable instructions) that in response to execution by a processor can permit or otherwise facilitate the device 710 to implement a defined algorithm (or process) for darkfieid image correction or fingerprint identification or analysis.
  • computer-accessible instruction e.g., computer-readable instructions and/or computer-executable instructions
  • fingerprint sensor 715 can include ultrasonic transducers (e.g., PMUTs) or capacitive micromachined ultrasonic transducers (CMUTs)) able to generate and detect acoustic/pressure waves. Examples of PMUT devices and arrays of PMUT devices are described in accordance with F!Gs. 1A-6 above.
  • a device 710 includes fingerprint sensor 715 comprised of an array of ultrasonic transducers that can facilitate ultrasonic signal generation and sensing.
  • fingerprint sensor 715 can include a silicon wafer having a two-dimensional (or one-dimensional) array of ultrasonic transducers.
  • the darkfield image is used for improving or correction of captured fingerprint images.
  • the darkfield image also referred to as the background image or offset
  • the background image or offset is the image obtained by the sensor when no finger is present in general
  • any image captured by a sensor may include temporal noise or spatial noise in addition to a signal related to an object being imaged.
  • Temporal noise is considered as varying with time (e.g., successive images captured under similar conditions)
  • spatial noise is an offset that contributes to a captured signal (e.g., a ridge/valley signal).
  • To get the best possible image for the fingerprint matcher any background image or contributions to the image other than from the fingerprint should be removed or corrected for.
  • the embodiments described herein provide for capturing the darkfield image and using the darkfield image to correct the fingerprint image.
  • the fingerprint authentication or identification generates some constraints to the overall fingerprint sensing system. For instance, in some embodiments, a user is able to enter the authentication or identification system with no specific preparation steps. The fingerprint sensing system is in an always-on mode, waiting for the user to put his or her finger on the sensor, with no opportunities for a priori preparation prior to the user’s finger interacting with the sensor. It is also of importance to consider that in many scenarios the fingerprint sensing system is autonomous, therefore the desire to consume as less power as possible despite the always-on mode.
  • the darkfield estimate used for correction should be as current as possible to provide for the best correction of the fingerprint image in other words, in an ideal case, the darkfield image is captured just prior to a user placing their finger on the sensor.
  • the described embodiments provide for monitoring the darkfield such that a most current version of the darkfield estimate is captured, for improving the quality of the fingerprint image, thus providing for improved performance of fingerprint matching.
  • the background image or sensor offset also referred to herein as the darkfield
  • the darkfield is subject to change over time.
  • changes to the darkfield in the short term may be due to the influence of external parameters such as temperature, and in the longer term due to evolution of the sensor itself
  • the integration process of the sensor in the fingerprint sensing system may comprise several manufacturing steps and module construction steps that may have an impact on the background image.
  • An overall comprehensive calibration and storage of multiple darkfield images requires many resources in terms of manufacturing calibration steps, and would also require a huge memory amount which is usually costly and not compatible with the consumer market. Moreover, this is also likely to fail technically as darkfield will evolve during the product life cycle. Therefore, there is a need for a continuous darkfield tracker that would constantly update the darkfield as time goes by.
  • FIG. 8 illustrates a flow diagram 800 of an example process for darkfield acquisition, according to some embodiments.
  • void detection is performed.
  • a void is detected, as shown at procedure 830, a darkfield candidate image is captured.
  • darkfield contamination detection is performed.
  • FIGs. 23-27 Various embodiments of darkfield contamination detection are described in accordance with FIGs. 23-27 below.
  • flow diagram 800 proceeds to procedure 840, where a darkfield candidate image is modeled.
  • a darkfield candidate image is modeled.
  • the darkfield estimate is updated (e.g., with a captured darkfield candidate image or a modeled darkfield candidate image).
  • FIG. 9 illustrates a flow diagram 900 of an example method for capturing a darkfield image, according to some embodiments.
  • Flow diagram 900 shows an overview of the different procedures involved in tracking the darkfield and then correcting a fingerprint image for the darkfield.
  • the term darkfield and darkfield image may both be used below and represent the image captured by the sensor when no finger or object is touching the sensor. In an ideal situation, a sensor will provide a uniform darkfield when no object is present on the sensor.
  • the darkfield may be caused by ultrasound reflections within the sensor stack, and not due to any object on the sensor surface.
  • the sensor surface e.g., platen layer 116 of FIGs. 1A, 1 B, and 2 is the part of the device where the user is expected to put his or her finger in order to measure the fingerprint.
  • void detection includes a determination as to whether an object is present on the sensor surface or not. in some embodiments, void detection is discussed on further detail below (e.g., as described in FIGs. 10 through 15). in other embodiments, an additional detection sensor is used for performing void detection.
  • the fingerprint sensor may include one or more additional electrodes on fop of the fingerprint sensor or platen that detects the presence of an object or finger.
  • a signal is received at the additional detection sensor, and a determination as to whether an object is interacting with the fingerprint sensor is based at least in part on the signal.
  • an additional sensor may be used alone or in combination with the other embodiments of void detection described herein.
  • a decision to capture a darkfield candidate image is made, as shown at procedure 920.
  • the decision to capture the darkfield candidate image is optional and may be based on the monitoring of various parameters such as time and temperature.
  • a darkfield candidate image is captured, where the darkfield candidate image includes an image (full or partial) acquired by the sensor with no object contacting or otherwise interacting with the sensor.
  • the darkfield estimate (e.g , a previously stored darkfield image) is update with the darkfield candidate image.
  • the darkfield candidate image is merged with the estimated darkfield.
  • the darkfield candidate image is stored as the darkfield estimate.
  • a resulting darkfield estimate is then stored, as shown at procedure 950, as an output of the merge process, and is made available for the fingerprint imaging process.
  • the imaging process can use the so stored darkfield estimate to correct an acquired fingerprint image.
  • a fingerprint image is acquired and at procedure 970, the fingerprint image is corrected using the stored darkfield estimate.
  • the corrected fingerprint image may then be sent to a matcher for authentication.
  • the matcher is the part of the fingerprint sensing system that compares the fingerprint image to the fingerprint images acquired during enrollment of the user and authenticates the user of the sensor.
  • the fingerprint sensor is an always-on sensor, meaning that the fingerprint sensor should always be ready to acquire a fingerprint, without the user, or another part of the system instructing the fingerprint sensor to do so (e.g., as describe in FIGs. 7A and 7B).
  • the fingerprint sensor is constantly in operation and constantly checking if a finger is present on the sensor surface. Accordingly, there should always be a correct darkfield stored and available so that as soon as a finger is detected, the fingerprint image can be acquired and corrected for the darkfield.
  • Many techniques exist to determine if there is a finger present on the surface but they consist of acquiring a complete image and then determining if the image has the characteristics of a fingerprint. For an always- on sensor such techniques would require too much power and processing resources because a complete image is constantly acquired.
  • the void detection of the described embodiments does not require complete image acquisition and subsequent image analysis.
  • the void detection as described herein determines whether an object is in contact with, or otherwise interacting with, the sensor surface based only on a limited number of pixels (e.g., 5-50) of the fingerprint sensor.
  • the pixels are distributed over the fingerprint sensor. This subset of pixels can form a patch or one or multiple profile(s), or can be spatially distributed on the image without any image or profile cohesion.
  • the pixels used to in void detection may be referred to as the darkfieid pixels it should be appreciated that the more pixels used in performing void detection, the performance of void detection improves and the power consumption of the void detection increases. The decision is a trade-off and may be adapted automatically based on the available power resources or power mode.
  • the subset of pixels can be analyzed, e.g., using either their temporal pattern or behavior, and/or the spatial pattern or behavior when the subset can form a patch image or a profile.
  • the smaller the subset of pixels the better it is for power consumption.
  • a 1 A always on darkfieid tracker is so consuming 24mAh per 24 hours, to be compared to a lOOOmAh battery for instance.
  • it is also of paramount importance that the darkfieid tracker system does always maintain a good enough estimate of the current darkfieid to allow at any time a proper correction of the image formed on the sensor.
  • FIG. 10 illustrates example operation void detection associated with a two-dimensional array 1000 of ultrasonic transducers, according to some embodiments.
  • the void detection includes the activation of a first subset of ultrasonic transducers for capturing single pixels (e.g., pixel 1010) within a block (e.g., block 1020) of two-dimensional array 1000.
  • two-dimensional array 1000 includes twelve blocks of 24x24 ultrasonic devices.
  • blocks 1020 are independently controllable regions of two- dimensional array 1000 such that pixel acquisition can be performed concurrently for all blocks 1020.
  • the usage of blocks 1020 is optional for allowing for concurrent pixel acquisition, and other methods of concurrent pixel acquisition can be utilized in the described embodiments.
  • the first phase includes activation of ultrasonic devices of the middle eight 24x24 blocks 1020 of ultrasonic transducers for capturing a single pixel or a closely grouped plurality of pixels within each activated block. While the illustrated embodiment shows only eight of the twelve blocks activated, and only ultrasonic transducers activated for capturing a single pixel within the activated blocks, it should be appreciated that any number of blocks may be activated, that the pixel may be located at any position within a block, that any number of ultrasonic transducers may be activated for capturing any number of pixels, and that the illustrated embodiment is an example of many different possibilities.
  • the pixels may be at the same position within the blocks since this may simplify driving electronics, or different pixels may be used in the different blocks.
  • the two-dimensional array can include any number of ultrasonic transducers, and the two-dimensional array may be divided into any number of independently operable blocks.
  • embodiments described herein provide for utilizing multiple ultrasonic transducers, some of which may be time-delayed relative to each other, to focus a transmit beam to capture a pixel of an image.
  • the pixels and blocks that are activated during void detection may also depend on the size of the sensor, the size of the finger, or the most likely position the user will touch the sensor.
  • pixels and blocks covering the entire surface may be activated.
  • pixels and blocks covering the entire surface may be activated.
  • pixels and blocks covering the entire surface may be activated.
  • only a central section of the sensor may be activated to save power resources.
  • the central section may be adapted to the user, or the context of the device, e.g., the device orientation.
  • pixel 1010 is periodically captured during void detection. Although a single pixel is illustrated, it will be understood that multiple pixels can be used, either grouped together or distributed throughout the array. Also, each pixel may be imaged by activating a plurality of ultrasonic transducers around the pixel. Wtien a significant change in ultrasonic wave receive intensity occurs due to the presence of an object positioned near a sensor platen (not shown), circuitry is activated to indicate that a void is not detected in one embodiment, when a void is detected, a darkfield image can be captured at the fingerprint sensor using the two-dimensional array 1000.
  • void detection includes activating a small subset of the pixels in the array in a highly duty-cycled manner. For example, as illustrated, the 8-pixel pattern illustrated in FIG. 10 is activated in various embodiments, these pixels are operated at a rate of 10-100 samples/second.
  • the threshold defines an expected signal range such that signals falling within the expected signal range are indicative of no object interacting with the sensor.
  • the signal from each pixel would be compared to a mean pixel value plus/minus a threshold (e.g., an offset plus/minus a range), where the mean pixel value plus/minus the threshold is an example of the expected signal range. For example, if the signal on M or more pixels exceeds a single value, (where‘M’ is a programmable setting), the system determines that an object is interacting with the fingerprint sensor, and that a void is not detected.
  • a mean pixel value plus/minus a threshold e.g., an offset plus/minus a range
  • the system determines that an object is interacting with the fingerprint sensor, and that a void is not detected. Otherwise, the system determines that a void is detected. For example, in another embodiment, a sum of the received signals may be compared with a threshold, the received signals may be divided info groups and compared to a threshold, etc.
  • the signal and mean pixel value are gray scale levels
  • the threshold is plus/minus number of gray scale levels.
  • the total variance of gray scale levels could be fifteen levels
  • the threshold could be plus/minus four gray scale levels.
  • a signal that fails outside four gray scale levels of the mean pixel value would trigger an event indicating an object is interacting with the fingerprint sensor, where a signal falling within four gray scale levels would not trigger an event (e.g., assumed to be a result of signal noise).
  • the mean pixel value is tracked over time, allowing for slow changes in signal values to not impact the determination of an object contacting the fingerprint sensor. For example, a gradual
  • temperature change of a fingerprint sensor may impact the value of a received signal.
  • Embodiments described herein provide for tracking the mean pixel value over time, and adjusting the threshold range accordingly, to avoid false accepts.
  • FIG. 11A illustrates a graph 1100 of pixel values relative to time during void detection at a sensor, according to an embodiment.
  • graph 1100 is the pixel value for pixel 1010 of FIG. 10, where the pixel value for each pixel captured in FIG. 10 is received and tracked.
  • Pixel value 1110 is received and tracked over time, where pixel value 1110 is the signal value for the pixel received from the sensor and moving average 1112 is the mean pixel value over time (e.g., running average).
  • a first threshold also referred to as the void flags threshold 1102
  • a pixel value 1110 falling outside of the range defined by void flags threshold 1102 is indicative of an event 1114 (e.g., interaction of an object with the fingerprint sensor).
  • the threshold value may be pre-defined, or may be adaptive, e.g., based on the noise level or variance of the signal, the available resources, or the required latency.
  • the threshold may be expressed in pixel gray levels.
  • the first threshold may be of the order of 1- 10 gray levels.
  • a void flag may be set for the pixel. This means that, based on this pixel, there is an indication that there is no object on the sensor surface.
  • the void flags threshold 1102 may be fixed in gray levels, or may be set as a relative value of the sensor signal.
  • the void flags threshold 1102 may also depend on other factors, e.g., the noise level, signal-to-noise level (SNR), or contrast-to-noise-ievel (CNR).
  • the void flags threshold 1102 may be identical for ail pixels, or may be adapted individually for each pixel.
  • the void flags threshold 1102 can be set according to the pixel temporal noise behavior. It should be appreciated that one of ordinary skill in the art of signal processing may be able to set void flags threshold 1102 according to the probability density or distribution function of the pixel noise.
  • a void flags threshold 1102 can be set to allow for a 99% probability that the current pixel value Is indeed compliant for the“noise only” assumption it should be appreciated that one of ordinary skill in the art of signal processing may utilize other practical experimental methods to get a correct setting of such a void flags threshold 1102 it should be appreciated that the objective is to get an overall void flag only when no object is on the sensor, to lower the probability of acquisition of a wrong darkfield candidate image, e.g., to avoid false reads of voids or objects.
  • the void false reads rate (e.g., a void is declared and an object is on the sensor) should be kept very low, and void false rejection rate (e.g., a void is not declared and no object is on the sensor) can be kept high, as long as from time to time, a void is correctly declared.
  • Moving average 1112 is updated over time to account for environmental or other changes that impact signal values (e.g., temperature change).
  • moving average 1112 is updated by averaging the pixel value 1110 with the previous moving average 1112, resulting in an updated moving average 1112.
  • pixel value 1110 can be averaged with a number of previous values of moving average 1112, so as to control the impact of pixel value 1110 on moving average 1112. Updating moving average 1112, and thus void flags threshold 1102, allows for (e.g., slow) temporal evolution of factors impacting pixel signal values (e.g., temperature change).
  • the determination whether there is no object on the sensor surface, e.g., a void is detected, may be based on a combination of the plurality of pixels. For example, a void may be detected only if all the plurality of pixels have a void flag, meaning that for all pixels, the sensor signal for that pixel was within the void flags threshold 1102 of the moving average 1112. This will limit the void false alarm rate. Instead of all pixels have a void flag, it may be required that only a certain percentage of the pixels have a void flag (e.g., 50%-1GQ%). In some
  • this is a voting scheme, where all pixels have a vote on the void status, and a final decision is made.
  • Other techniques fusing the multiple pixel information can be used.
  • the void flag can be updated for each sample of the pixels temporal input.
  • the system may then acquire an image of the complete sensor by activating pixel acquisition over the entire two-dimensional array to capture the pixels of the image (or a decimated image), resulting in the capture of a darkfield candidate image.
  • the decision to capture a darkfield candidate image may also depend on other factors, e.g., the time passed since the last darkfield candidate image acquisition, the temperature change since the last darkfield candidate image acquisition, the available power resources or power mode of the device.
  • the settings of the darkfield tracker may also depends on an application running on the system that may have certain requirements on fingerprint image quality. For example, a banking application may need a high- quality fingerprint image, and the darkfield tracker may be configured accordingly.
  • the different factors may also be combined to determine an urgency of the darkfield acquisition, and their weight may be different.
  • temperature change is known to affect the darkfield, so a change in temperature may be an important factor in determining whether a new darkfield Is acquired. The temperature may be measured by a temperature sensor build into the fingerprint sensor, and may come from another temperature sensor in the device containing the fingerprint sensor.
  • FIG. 11A also shows a second threshold, also referred to as the moving average threshold 1104.
  • Moving average 1112 is updated over time to account for environmental or other changes that impact signal values (e.g., temperature change).
  • the moving average threshold 1104 range e.g., 5-50
  • the moving average threshold 1104 range is shown here to have a larger value than the void flags threshold 1102 range.
  • the sensor signal passes outside the moving average threshold 1104 range (e.g. as indicated by event 1114), it may be an indication that there may be an object on the sensor surface.
  • moving average 1112 is updated by averaging the pixel value 1110 with the previous moving average 1112, resulting in an updated moving average 1112 and if a pixel value falls outside of moving average threshold 1104 (e.g , at event 1114), moving average 1112 is not updated.
  • pixel value 1110 can be averaged with a number of previous values of moving average 1112, so as to control the impact of pixel value 1110 on moving average 1112. Updating moving average 1112, and thus void flags threshold 1102, allows for (e.g , slow) temporal evolution of factors impacting pixel signal values (e.g., temperature change).
  • the change in signal due to this event is much faster than any slow temporal evolution.
  • the timescale of the averaging should therefore be larger than the timescale of the signal change due to the sudden presence of an object.
  • the time scale of the averaging is adapted to the expected timescale of the temporal evolutions, e.g., due to temperature changes. After event 1114, the average may be reset so as to obtain an accurate average as soon as possible.
  • determination of moving average 1112 is stopped so as to avoid influencing or modifying moving average 1112 by any signal due to an object on the sensor surface.
  • the void flags threshold 1102 is set to capture an object being put on the sensor and the system can use a priori knowledge that whenever an object is set on the sensor, the only possible variation of the pixel level is either negative or positive. This a priori information can be used to build a non-symmetrical threshold detector.
  • Moving average 1112 may be stopped for ail darkfield pixels when at least one pixel indicates that the signal was outside moving average threshold 1104. Alternatively, this decision may be made when a predefined portion (e.g. ⁇ 20%) of the darkfield pixels indicated that the pixel value 1110 was outside moving average threshold 1104. Similar to void flags threshold 1102, moving average threshold 1104 may be defined by several methods. It is beneficial that moving average threshold 1104 is larger than void flags threshold 1102. As explained above, the moving average thresholding method may be symmetrical compared to the moving average, or may be asymmetrical.
  • the signal intensity is high because the signal reflected from the sensor surface is high due to the high acoustic impedance at the boundary with the air.
  • some of the acoustic waves may enter into the object, and thus less signal is reflected.
  • this is a matter of convention it is usual that a ridge or any object on the sensor generates a darker value compared to air. Therefore, when an object is present on the sensor surface, the signal decreases, as indicated in FIG. 11A.
  • FIG. 11A also shows that when the object/finger is removed, as indicated at event 1116, the signal intensity increases again.
  • the moving average 1112 is updated again, as illustrated.
  • the void flags threshold 1102 the void flags are regenerated.
  • the void flags threshold 1102 and moving average threshold 1104 may be different whether the sensor signal is within or outside the threshold, thereby creating some sort of hysteresis it may be beneficial to allow for an evolving threshold for at least the moving average threshold 1104, to make it more probable to catch the moment when the sensor is void again.
  • FIG. 11 B shows an example graph 1120 of a signal value when a finger maintains contact with the sensor.
  • Pixel value 1122 is received and tracked over time, where pixel value 1122 is the signal value for the pixel received from the sensor and moving average 1124 is the mean pixel value over time (e.g., running average).
  • the pixel value 1122 of the sensor signal decreases over time, starting at event 1126 (finger contacts surface), due to changes in the sensor system caused by a temperature change.
  • the temperature change may change the properties of the material of the sensor and may change the speed of sound in the materials, and/or may create mechanical stress in the materials and so big enough mechanical distortions in these material, which can thereby impact reflections and the timing characteristics of the signal acquisition.
  • the temperature does not change immediately back to the value before the contact.
  • the pixel value 1122 increases again, but the increase is not sufficient to get the signal within moving average threshold 1132.
  • the moving average 1124 is not activated, and the algorithm is blocked/frozen since the moving average is used for determining voids using void flags threshold 1130.
  • the pixel value 1122 may slowly get back to within moving average threshold 1132 as the temperature slowly changes, however, these may cause the algorithm to not work properly for an unacceptable time period.
  • the effect of the temperature on the moving average is predicted, as illustrated in graph 1140 of FIG. 11C.
  • the system may monitor the pixel value 1142 and the temperature change, and determine any possible correlation.
  • the expected pixel value 1144 for the darkfield pixels as a function of temperature is determined and stored. This correlation scheme can be obtained during normal use of the system or prior to the sensor system is launched in its final stage through a manufacturing calibration analysis step. If a strong enough correlation is found, the expected pixel value 1144 may be corrected for a change in temperature during the period of finger touch (e.g., between events 1146 and 1148).
  • Void flags threshold 1150 is determined based on the expected pixel value 1144 when a finger is touching the sensor and is based on the moving average 1143 when a finger is not touching the sensor. When pixel value 1142 falls outside of moving average threshold 1152, moving average 1143 is not determined. The application of the correction may depend on the confidence in the correlation. These effects may be determined on a pixel basis, or on a sensor basis, e.g., by averaging effect for the different pixels. In such an embodiment, the expected pixel value 1144 and the associated void flags threshold 1150 would not be horizontal during the period of finger touch (between events 1146 and 1148), but would be modified based on the observed correlation, and follow the temperature dependence of the pixel value (sensor signal) 1142 more closely. Once the finger is lifted, as shown at event 1148, the algorithm would not be frozen because, if the correction was accurate enough, the pixel value 1142 would again be within the void flags threshold 1150.
  • FIG. 11 D illustrates an example graph 1160 of a signal value when a finger maintains contact with the sensor including a timer, according to embodiments in some embodiment, a timer (e.g., 5-20 seconds) is introduced. It should be appreciated that the timer may be based on a predetermined time, e.g., the maximum time a user would be expected to keep their finger on the sensor, and may be dependent on the type of application being used and the context of the electronic device, e.g., activity of user, motion of device including the sensor. This timer is activated as soon as an object is detected on the sensor surface (at event 1166), as shown in FIG. 11C.
  • a timer e.g., 5-20 seconds
  • This timer allows to reactivate the moving average 1164 after a predefined period, irrespective of the pixel value 1162.
  • the object is removed from the sensor surface, but pixel value 1162 remains outside moving average threshold 1174.
  • moving average 1164 acquisition is restarted, thereby adjusting void flags threshold 1172 and moving average threshold 1174.
  • a verification operation may be performed to analyze the image for characteristics of a fingerprint or object. If the verification operation indicates that a finger or other object is interacting with the sensor, the timer can be restarted or set for a longer time period to allow for the finger or object to be removed from the sensor.
  • an Object up (or Finger up) detector is implemented within the sensor. This is to compensate for the possible drift in the pixel values during which an object has been put on the sensor as explained above and the need for a watchdog mechanism.
  • the Object or Finger up detector is a reversed function of the Object or Finger down detector, that stops the moving average mechanism as soon as the current pixel value is outside the boundaries.
  • a second moving average is initialized so as to capture the pixel average value while the first moving average is stopped and so when an Object or Finger is assumed on the sensor.
  • the current pixel value is then compared, to the second moving average with a threshold principle similar to the thresholding explained above in FIGs. 11A through 11 D.
  • an Object or Finger up detection event is considered and can unblock the first moving average.
  • a change in the second moving average (e.g., due to temperature variation) may also be used to correct to first moving average after the Object or Finger up event is detected.
  • the boundaries/thresholds benefit again to be asymmetrical that is use the a priori information that Object or Finger lift will only increase the pixel value (according to the convention where a finger or object put on the sensor can only decrease the pixel value). So, with this option, either the pixel value back to the moving average threshold (as explained above) or the Object or Finger up detector explained here can reactivate the moving average and the void detector.
  • the Object or Finger up detector will also benefit from a voting scheme for all the pixels that are used in the void detector.
  • temperature calibration of the subset of darkfield pixels used in the void detector is implemented. While the present embodiment is described as using temperature calibration, it should be appreciated that other parameters may be used in a similar manner.
  • the expected pixel value of the darkfield pixels may be determined as a function of temperature.
  • a table or other type of register is generated into which the expected darkfield values of the pixels are stored for each temperature or some temperature steps. This table consumes memory space, on one side, but remains very small compared to the more conventional principle to store ail the darkfield values for ail the pixels in the image for each temperature step.
  • the expected darkfield pixels values as a function of temperature can be used to determine if an object is interacting with the sensor.
  • FIG. 11 E illustrates example graph 1180.
  • the figure shows expected pixel value 1184 which changes over time as the temperature of the sensor changes.
  • Expected pixel value 1184 is determined by measuring the temperature of the sensor, and then retrieving expected pixel value 1184 from memory based on the previously determined relation.
  • the figure shows that when the finger is on the sensor, the temperature changes, e.g. by thermal conduction of the heat of the finger, which causes the expected pixel value to change.
  • Void flags threshold 1190 is used in a similar matter as above to determine if an object is interacting with the sensor. When the pixel value is within void flags threshold 1190 around expected pixel value 1184, it is determined that no object is interacting with the sensor.
  • the pixel value is outside void flags threshold 1190 around expected pixel value 1184, for example at event 1186, it is determined that an object is interacting with the sensor. Because the expected pixel value is tracked as a function of temperature, when the finger is removed from the sensor, at event 1188, the pixel value falls again within void flags threshold 1190 around expected pixel value 1184, as seen in the figure.
  • the void detector is so made robust to temperature changes, and the back to void state is also made robust.
  • the temperature darkfield values can be filled in at some manufacturing calibration stage, or be filled in and updated during the product’s life cycle.
  • the moving average value can be stored into the temperature pixel darkfield table. It can then be used either only for the back to void detection or whenever available for the non-void detection step. Combinations of the two techniques may be used depending on the context, application, and the reliability of the prediction of the expected darkfield pixel values. When reliable predictions can be made using the temperature, or even additional parameters, this technique can be very robust in other embodiments, other techniques may be used to determine the expected darkfield value dependence on temperature, and other factors if needed. For example, in an initial stage imaging techniques may be used to determine if an object is interacting with the sensor, while the expected darkfield value dependence on temperature is determined.
  • the void detector there should be a very high certainty that there is no object on the sensor before deciding to measure a darkfield. It is better to be conservative, and that when there is doubt, to decide not to declare a void is detected. This aspect is also illustrated by the fact that all darkfield pixels must be within the void flags threshold to indicate if a void is detected. Furthermore, it takes only one, or a few, pixels to be above the moving average threshold to stop the moving average in order to limit the change that an object on the sensor corrupts the darkfield Image determination.
  • the settings of the different threshold and conditions should reflect this principle. The settings may be adapted, e.g , to the different users, or the usage context
  • FIG. 12 illustrates a flow diagram 1200 of a method for void detection, according to various embodiments. It should be appreciated that not all procedures may be required, additional procedures may be performed, or the procedures may be performed in a different order.
  • the pixel value (sensor signal) is acquired for the darkfield pixels, e.g., in a continuous temporal manner.
  • procedure 1250 it is determined whether the pixel value is outside the void flags threshold if the sensor signal is outside the void flags threshold, as shown at procedure 1260, no void is detected (an object is detected). Flow diagram 1200 then returns to procedure 1210. If the sensor signal is not outside the void flags threshold, as shown at procedure 1270, a void is detected. Flow diagram 1200 then returns to procedure 1210. In one embodiment, a darkfield image is captured.
  • the darkfield may be merged with a previously determined darkfield image it should be appreciated that capturing the darkfield candidate image may also be subject to satisfying a time delay since last darkfield capture or temperature change.
  • a test that the darkfield candidate image is indeed a darkfield image can be performed. This test can look for structures in the image to distinguish an actual darkfield image from a fingerprint image or an object image.
  • An additional darkfield quality verification step may be applied before merging the recently acquired darkfield candidate image. For example, an image analysis may be applied to scan for any image contribution that are not likely to constitute a darkfield. The image analysis may comprise looking for features resembling a fingerprint, or spatial frequencies related to a fingerprint If such features are present, the darkfield candidate image may not be used, or used with a lesser weight. A darkfield quality factor may be determined, and the weight of the candidate darkfield in the merger may depend on the quality factor.
  • the quality factor may also express a confidence in the fact that no object was detected it may also be determined if the quality of the darkfield estimate will be negatively affected by the merger of the darkfield candidate image, and based on this determination, the weight of the darkfield candidate image may be adapted.
  • the stored darkfield estimate may be subtracted from the recently acquired darkfield candidate image, since this represent the latest acquired image of the sensor if the darkfield procedure is working properly, the so obtained corrected image should be nearly uniform but for a small contribution.
  • the uniformity of quality of the image may be determined to analysis the quality of the darkfield correction, and any issue or errors may be used as feedback to automatically adapt the darkfield correction process.
  • the darkfield estimate is updated.
  • only the last acquired darkfield candidate image may be used, without performing any merging.
  • the darkfield candidate image is merged with previously recorded darkfield images as the darkfield estimate.
  • the darkfield estimate provides a gradual evolution of the darkfield and allows for a reduction of the temporal noise that is captured with the darkfield candidate image.
  • the merging may be implemented as averaging the darkfield candidate image into the darkfield estimate. This may reduce, or remove, the temporal noise contribution. Many different types of averaging or merging may be used. For example, a recursive average filter may be used where the latest darkfield candidate image contribute with more weight than older darkfield candidate images.
  • the stored darkfield estimate may be used in the darkfield correction of acquired images containing a fingerprint.
  • the stored darkfield estimate may be subtracted from the fingerprint image to perform the darkfield correction.
  • a quality verification may be applied to make sure the darkfield correction actually improves the image quality of the fingerprint image. For example, the CNR of the ridge/valley pattern should improve due to the darkfield correction.
  • FIGs. 13 through 15 illustrate flow diagrams of example methods for darkfield tracking according to various embodiments. Procedures of these methods will be described with reference to elements and/or components of various figures described herein it is appreciated that in some embodiments, the procedures may be performed in a different order than described, that some of the described procedures may not be performed, and/or that one or more additional procedures to those described may be performed.
  • flow diagram 1300 illustrates an example method for darkfield tracking, according to various embodiments.
  • the method of flow diagram 1300 is performed at a fingerprint sensor.
  • a pixel value is received.
  • procedure 1320 is performed according to flow diagram 1400 of FIG. 14. in another embodiment, procedure 1320 is performed according to flow diagram 1500 of FIG. 15.
  • FIG. 14 illustrates an example method for determining whether an object is interacting with the sensor.
  • signals are transmitted at ultrasonic transducers of the ultrasonic sensor.
  • reflected signals are received at ultrasonic transducers of the ultrasonic sensor.
  • the reflected signals are compared to a void flags threshold around a moving average.
  • the moving average is updated provided the signals are within a moving average threshold, wherein the moving average threshold is larger than the void flags threshold in one embodiment, provided the reflected signals are within the void flags threshold, a determination is made that an object is not interacting with the ultrasonic sensor and that a darkfield candidate image can be captured at the sensor. Similar embodiments are described above with reference to flow diagram 1200 of FIG. 12. in one embodiment, provided an object is interacting with the sensor, an object lifting signal is generated when it is determined the object is no longer interacting with the sensor.
  • a timer is activated. Responsive to a predetermined period of the timer lapsing, the determining whether an object is interacting with the sensor is repeated in another
  • a signal received at the sensor is compared to a sensor temperature correlation scheme. Signal thresholds are modified according to the sensor temperature correlation scheme during the determining whether an object is interacting with the sensor. In one embodiment, an expected signal value as described in FIG. 11C can be used.
  • Flow diagram 1500 of FIG. 15 illustrates another example method for determining whether an object is interacting with the sensor.
  • a temperature of the sensor is measured.
  • the pixel value e.g., received at procedure 1310
  • a pixel value is received for a plurality of pixels.
  • procedure 1330 it is determined whether to capture a darkfield candidate image at the sensor based at least in part on the determination that a darkfield candidate image can be captured at the sensor. In one embodiment, the determination to capture the darkfield candidate image is also based at least in part on making a determination that a minimum amount of time has passed since a most recent darkfield candidate image capture. For example, if a darkfield candidate image was recently captured, it may not be necessary to capture another darkfield candidate image as there would be none or negligible changes.
  • the determination to capture the darkfield candidate image is also based at least in part on making a determination that a temperature change since a most recent darkfield candidate image capture has exceeded a temperature threshold. For example, if the temperature has been relatively constant, it may not be necessary to capture another darkfield candidate image as there would be none or negligible changes. It should be appreciated that in some embodiments, the determination to capture the darkfield candidate image may be based on a combination of time passed since the last darkfield candidate image was captured and the temperature of the sensor.
  • a temperature of the sensor is determined.
  • the temperature is associated with a corresponding pixel value.
  • the temperature and pixel value pair are then stored, e.g., as an expected pixel value.
  • the temperature and pixel value pair can be used in determining an expected pixel value of flow diagram 1500.
  • flow diagram 1300 if it is not determined that a darkfield candidate image should be captured, flow diagram 1300 returns to procedure 1310. In one embodiment, flow diagram 1300 delays the performance of procedure 1310 a predetermined time period, e.g., to ensure that enough time has passed that a darkfield candidate image could be captured, given satisfaction of other conditions. If it is determined that a darkfield candidate image should be captured, flow diagram 1300 proceeds to procedure 1340.
  • a darkfield image is captured as a darkfield candidate image, where a darkfield image is an image absent an object interacting with the sensor.
  • the darkfield estimate is updated with the darkfield candidate image.
  • the darkfield candidate image is merged with the darkfield estimate.
  • the darkfield candidate image is stored as the darkfield estimate.
  • flow diagram 1300 if it is determined that an object is interacting with the sensor at procedure 1320, flow diagram 1300 also proceeds to procedure 1370, where an image of the object is captured. In one embodiment, where the object is a finger, a fingerprint image is captured. At procedure 1380, the image is corrected using the darkfield estimate.
  • one way to determine the darkfield image for a sensor is to capture an image when there is no finger present on the sensor.
  • the darkfield image can be subtracted from a fingerprint image to correct for the variations from the background.
  • a darkfield image would be captured immediately before the user puts his or her finger on the sensor, and this background image would then be used to correct the fingerprint image.
  • darkfield images are dependent on the operational conditions or operating parameters of the sensor, e g., the temperature of the sensor.
  • the temperature of the sensor e g., the temperature of the sensor.
  • one factor causing the temperature dependence is the fact that the speed of sound is temperature dependent. When the speed of sound changes, this may impact the timing of the sensor image capture. Therefore, when the user puts his or her finger on the sensor, the sensor temperature changes due to thermal conduction, and consequently the darkfield image may change.
  • the darkfield image cannot be captured while the user finger is present.
  • Embodiments provided herein provide for the prediction of the darkfield image for situations when capturing a background image is not possible, e.g., when an object or finger is present on the sensor and when the temperature change is significant resulting in a change to the darkfield image.
  • Embodiments described herein provide an ultrasonic sensor comprised of a two- dimensional array of ultrasonic transducers (e.g., PMUTs).
  • the ultrasonic fingerprint sensor is comprised of multiple layers, e.g., a CMOS layer, an ultrasonic transducer layer, an epoxy layer, and adhesion layer, and a contact layer where the user presses the finger.
  • the layers are combined to form a package or stack forming the sensor.
  • An example fingerprint sensor package is described above in accordance with FIGs. 1A through 7B.
  • the principles discussed below may also be applied to other type of multi-layered sensors, where the two-dimensional array of ultrasonic transducers is replaced by other means of generating and or receiving ultrasonic signals.
  • piezoelectric films or piezoelectric bulb material may be used instead of the two-dimensional array of ultrasonic transducers.
  • Non-uniformities of the darkfield image may be caused by the package variations that result in pixel-to-pixei acoustic path difference during capture of the darkfield image.
  • the transmitted wave may be reflected off any interface (material change in the path, or acoustic impedance change) that is in the acoustic path of the signal.
  • any interface material change in the path, or acoustic impedance change
  • the two layers on each side of the interface have different acoustic properties that results in a large acoustic impedance mismatch, a significant portion of the signal may be reflected.
  • Such reflections can create non-uniformity within the background images because the interface may not be uniform.
  • the time-of-f!ight (ToF) for the different interfaces/layers may change with the temperature variations, e.g., in the same direction or in opposite directions.
  • the superposition of these interfacial reflection backgrounds creates backgrounds with phases and amplitudes that vary over temperature.
  • the embodiments described herein provide methods and techniques to reconstruct background images, and their temperature dependence, based on interfaciai reflections.
  • FIG. 16 illustrates an example fingerprint sensor 1600 comprising multiple layers. It should be appreciated that some or ail of the different layers may have different acoustic properties, e.g., different acoustic impedances interfaciai reflections may occur when a difference in acoustic impedance exist, e.g , an acoustic impedance mismatch occurs. When the impedance difference is small, no significant interfaciai reflections occur FIG. 16 illustrates an example fingerprint sensor 1600 where three different interfaces between layers are indicated which may give result in interfaciai reflections and contributing to non-uniformities in the background image during darkfield capture operations
  • Fingerprint sensor 1600 includes substrate 1610, CMOS layer 1620, sensing layer 1630, acoustic coupling layer 1640, and contact layer 1650.
  • fingerprint sensor 1600 also includes adhesion layer 1612 at the interface 1614 of substrate 1610 and CMOS layer 1620 and adhesion layer 1642 at the interface 1644 of acoustic coupling layer 1640 and contact layer 1650.
  • a third interface 1654 is illustrated at the interface between contact layer 1650 and ambient air.
  • Embodiments described herein provide a sensing layer 1830 comprised of a two- dimensional array of ultrasonic transducers (e.g., PMUTs).
  • sensing layer 1630 may be comprised of multiple layers, where one layer is for transmitting ultrasonic signals and another layer is for receiving reflected ultrasonic signals.
  • sensing layer 1830 transmits ultrasonic signals through acoustic coupling layer 1840, adhesion layer 1642, and contact layer 1650.
  • ultrasonic signals are also transmitted through CMOS layer 1820, adhesion layer 1612, and substrate 1610.
  • Sensing layer 1630 then receives reflected ultrasonic signals via the same transmission paths, passing through the same layers and interfaces.
  • the transmission properties of each layer of fingerprint sensor 1600 may be different, and that the ultrasonic signals can be reflected at the interfaces, due to different materials of the different layers and/or the acoustic impedance change between the layers at the interfaces.
  • the ToF for the different layers and interfaces may be impacted by temperature changes of fingerprint sensor 1600, thus impacting the timing of the sensor. For example, a finger being placed on contact layer 1650 may cause a change in the temperature of fingerprint sensor 1600.
  • FIG. 17 illustrates a flow diagram 1700 of the procedures to predict and reconstruct the darkfield image over varying temperature.
  • a plurality of base darkfield images are determined for different conditions in one embodiment, the plurality of darkfield images are captured at different ToF windows. This may be done at a single fixed temperature, e.g., at room temperature, or for a series of temperatures covering a desired temperature range. Alternatively, the plurality of darkfield images at different ToF windows are determined for different temperatures, and are then averaged over a temperature range. Based on the intensity of the reflections and ToF the main contributions representing interfacial reflection pattern can be determined with minimum correlation in one embodiment, the minimum number of darkfield images is equal to the number of interfaces of the sensor.
  • the full ToF window is divided into different segments that cover the different interfacial reflections.
  • other reflection within the layers e.g., due to impurities, may also be covered in the different ToF segments in general, embodiments described herein divide the reflections that contribute to the darkfield image into various reflection components such that the contributions of these different reflection components can be predicted over temperature to reconstruct the darkfieid image at any temperature. It should be appreciated that prior to capturing the plurality of darkfieid images, it shouid be confirmed that no object is on the surface (e.g , void detection).
  • the determination to model the darkfieid candidate image is also based at least in part on making a determination that a minimum amount of time has passed since a most recent darkfieid candidate image modeling. For example, if a darkfieid candidate image was recently modeled, it may not be necessary to capture another darkfieid candidate image as there would be none or negligible changes. In another embodiment, the
  • determination to model the darkfieid candidate image is also based at least in part on making a determination that a temperature change since a most recent darkfieid candidate image model has exceeded a temperature threshold. For example, if the temperature has been relatively constant, it may not be necessary to model another darkfieid candidate image as there would be none or negligible changes. It should be appreciated that in some embodiments, the determination to model the darkfieid candidate image may be based on a combination of time passed since the last darkfieid candidate image was modeled and the temperature of the sensor.
  • the darkfieid image I MGBG may be defined as a sum of a series of background images caused by different interfacia! reflections (linear superposition):
  • IMG BG ⁇ C-i (T) * IMG BG I (1)
  • I MGBGJ is the contribution to the darkfieid Image due to interfacial reflection at interface /
  • Ci(T) represents the contribution coefficient for I MGBG , i to the complete or estimated darkfieid image I MGBG. It is assumed herein that the darkfieid image I MGBG , . does not change over temperature (e.g., the base pattern remains the same), only its contribution to the complete darkfieid image changes, represented by temperature dependence of C,. Although not shown, different of additional operational conditions or parameters may be used to mode! the contribution coefficients. Alternative embodiments exist where the darkfieid images I MGBGJ may also have a temperature dependence. Procedure 1720 can be done by an optimization algorithm to minimize the difference between the measured darkfieid image and the
  • I MGBG may represent the full sensor image, or I MGBG may represent a partial sensor or subset of transducers/pixeis, so that the process is performed in parallel for different sections of the sensor.
  • procedure 1730 is performed by repeating the procedure 1720 at different temperatures.
  • different of additional operational conditions or parameters may be used.
  • the darkfield image is modeled (e g. , reconstructed) based on the interfacial background images.
  • the typical range of temperature under which the device may be operated maybe from -20 degrees Celsius to 60 degrees Celsius.
  • FIGs. 18A and 18B illustrate an example modeling of the background image described in flow diagram 1700.
  • the first row of images in FIGs. 18A and 18B show the actual measured darkfield over temperature
  • the second row shows the reconstructed/modeled darkfield image over temperature
  • the third row is the difference between measured darkfield image and the reconstructed/modeled darkfield image. The third row thus represents the remaining
  • the difference between the modeled and measured darkfield image is small, and the difference image between the modeled and measured darkfield image is nearly uniform. If the darkfield image is not correctly modeled, the difference between the modeled and measured darkfield Image increases, and the difference image between the modeled and measured darkfield image is increasingly nonuniform. Therefore, the nonuniformity of the difference image can be used as a measure for the quality of the darkfield modeling.
  • FIG. 19 illustrates the Darkfield Field Quality (DFQ) Spectral improvement over temperature under the described embodiments, where the DFQ spectral parameter is a measure of the nonuniformity of the image.
  • FIG. 19 shows the decrease of the nonuniformity after the darkfield correction.
  • the minimum at 25 degrees Celsius is due to the fact that the base darkfield images IMGB G .I have been determined at 25 degrees Celsius, and thus the best reconstruction can be performed at that temperature. If, according to some embodiments, the base darkfield images IMGB G, I are averaged over a temperature range, the spectral
  • a nonuniformity parameter may be used as an indication of the accuracy of the modeling and thus the accuracy of, and/or confidence in, the darkfield correction.
  • the confidence factor may then be used to control the application of the
  • the above described principles may be used to correct for the darkfield when a finger is present.
  • the above described principles may not be sufficiently accurate, and the modeled darkfield may not be used at all, or more weight may be given to the last captured darkfield in a combination.
  • the base darkfield images may not represent the different interfacial contributions, but rather represent darkfield images acquired at different
  • FIG. 20 illustrates a flow diagram 2000 of a process for modeling a darkfield image over varying temperature, according to another embodiment.
  • the base darkfield images at different temperatures may be determined in a controlled calibration process, or may be determined opportunistically.
  • base darkfield images may be determined over a range from -20 degrees Celsius to 60 degrees Celsius, e.g., with a step of 5 or 10 degrees Celsius.
  • the contribution of the different darkfield images to the modeled darkfield image is determined. For example, if base darkfield images exist at 20 degrees Celsius and 30 degrees Celsius, the darkfield can be modeled for a temperature in between by a weighted combination of both darkfield images (or other darkfield images at other
  • the weights/coefficient may be determined and verified by comparing modeled and measured darkfield images. If darkfield images at different temperature resemble too much (small difference), some of the darkfield images at some temperature may not need to be used because it would require additional image storages with no significant benefit.
  • the modeled darkfield is generated based on the measured darkfield images at different temperatures.
  • the darkfield images are not acquired and/or stored at a full resolution, and the pixels not measured/stored may be estimated based on surrounding pixels when the darkfield correction is performed.
  • the temperature dependence of the different contributions is determined so that this information can be used to predict and model the darkfield image at any temperature while considering any temperature change when the user is touching the sensor.
  • the temperature dependence can be determined using controlled temperature condition, e.g., during a manufacturing, characterization, or calibration stage using a temperature chamber where the temperature of the sensor can be controlled.
  • the contribution calculations at different temperatures can be done in an opportunistic way.
  • flow diagram 1700 can be performed to determine the contributions of the different interfacial reflections at that temperature. To avoid measuring while there are temperature gradients over the sensor package, the temperature change should not be too fast or large. As more and more situations with the sensor at different temperatures are encountered, the temperature dependence is determined over a larger range, or with more accuracy. If the darkfieid image needs to be modeled at a temperature that has not been measured yet, the contributions may be
  • the contribution coefficients and their temperature dependence may be determined for a collection of sensors, e.g., for one or a few sensors in a production batch, and then the coefficient may be applied to all the sensors in the batch if the coefficients are comparable for the different sensors. This means that the interfacia! reflections will be determined for each sensor, but that their relative contributions and the temperature dependences will be assumed to be similar for all sensors in a production batch.
  • the darkfieid modeling may be based on fitting the best darkfieid image to the captured fingerprint image.
  • FIG. 21 illustrates an example system 2100 for modeling a darkfieid image based on a best fit model, according to embodiments.
  • a best fit model may be used where the best fit is determined between the captured fingerprint image and a combination of darkfieid images (verified to be free of contamination).
  • These darkfieid images are selected from a database 2110 of darkfieid images, which may have been verified to be free of contamination, or may have been generated in a (controlled) calibration process.
  • the database 2110 may comprise darkfieid images captured at regular temperature intervals, and a plurality of contamination-free darkfieid images captured at different temperature may serve as a basis for the fit.
  • the captured fingerprint image 2120 may be processed before calculating the best fit.
  • the best darkfieid information may be present in the valleys of the fingerprint, and as such the process may only take into account these areas to determine the best fit. Therefore, in some embodiments, the valley information is extracted from the captured fingerprint images. Selecting the best areas of the fingerprint to be used may be performed through filtering or more complicated image processing techniques.
  • a best fit algorithm 2130 may then be applied to the (processed) fingerprint image.
  • the best fit algorithm 2130 may select a single image from the darkfield database 2110, or may select any combination for the darkfield database 2110.
  • the darkfield images for the fit may also be selected based on the operating condition of the sensor. For example, if the sensor is at a certain temperature, darkfield images from the database for a temperature range around the determined temperature may be selected. As shown in FIG. 21 , once the best fit darkfield image/model 2140 is determined, the darkfield correction 2150 may be applied by correcting the fingerprint image 2120 with the best fit darkfield image 2140, generating corrected fingerprint image 2160
  • FIG. 22 Illustrates a flow diagram 2200 of a process for modeling a darkfield image using a best fit algorithm corresponding to example system 2100 of FIG. 21 , according to an embodiment.
  • a fingerprint image is received.
  • darkfield information is extracted from the fingerprint image (e.g., the valley information).
  • a best fit algorithm is performed on the darkfield information from the fingerprint image to select a darkfield image (e.g., from a database of darkfield images).
  • darkfield correction is performed by applying the selected darkfield image to the fingerprint image.
  • the modeled darkfield image can be compared to a previously captured darkfield (e.g., a most recent captured darkfield image). Based on this comparison, a confidence can be determined in the accuracy of the modeled darkfield image.
  • a confidence threshold may be used, and if the determined confidence is below this threshold it may be that the modeled darkfield Is not accurate enough for use in correcting a fingerprint image. It should be appreciated that the confidence threshold may vary over time.
  • the confidence threshold may be less restrictive as the time since the darkfield image was captured, such that a darkfield model generated closer to the time of capture of the captured darkfield image should be closer to the captured darkfield image than a darkfield model generated farther from the time of capture of the captured darkfield image.
  • the modeling of the darkfield image presented herein may be used to determine the darkfield image when the user puts his or her finger on the sensor. It should be appreciated that the darkfield image modeling can work in combination with another algorithm or process that measures the darkfield image when there is nothing touching the sensor, e.g., the darkfield fracking as described above in FiGs. 9 through 15.
  • the darkfield image can only be captured when it has been verified that nothing is touching the sensor surface, which can be performed, e.g., by the above-described void detection.
  • the captured darkfield image (or darkfield estimate) may be used when the user puts his or her finger on the sensor and the temperature of the sensor does not change significantly.
  • the switch between darkfieid capturing and darkfieid modeling may be based on detecting contact with the sensor, a temperature, a temperature change, a time interval, or any other relevant parameter.
  • the temperature data may come from temperature sensor build into the sensor, or may come from a temperature sensor of the host device.
  • the darkfieid image of a sensor is used to correct captured fingerprint images for variations from the background in order to effectively use a darkfieid image (or darkfieid estimate) to correct a fingerprint image
  • the darkfieid image should be free from contamination.
  • the darkfieid image should only remove the undesired background from the image, and not otherwise influence the fingerprint image or other aspects of the authentication, such as the matching. Therefore, it is important that the darkfieid image only includes information related to the background, and does not get contaminated with other information, e.g., fingerprint-related or resembling features or any material contamination present on the sensor surface.
  • Embodiments described herein provide systems and methods to detect and prevent possible darkfieid contamination. As such, the dynamic updates of the fingerprint templates and the determination of the darkfieid image become more robust, more accurate, and has a higher precision and performance.
  • flow diagram 800 of an example method for capturing a darkfieid image is shown.
  • Flow diagram 800 results in the capturing of a darkfieid image (e.g., a darkfieid candidate image) when no object is present on the sensor, e.g., a void is detected.
  • Flow diagram 1200 of FIG. 12 describes an example method for void detection.
  • a darkfieid candidate image is captured.
  • the darkfieid candidate image is merged with a previously stored darkfieid image, e.g., a darkfieid estimate.
  • the darkfieid image may include non uniformities, e.g., due to ultrasound reflections within the sensor stack and not due to any object on the sensor surface.
  • the sensor surface is the part of the device where the user is expected to put his or her finger in order to acquire the fingerprint.
  • a captured fingerprint image is corrected using a darkfield image or darkfieid estimate to correct for these non-uniformities, resulting in a corrected fingerprint image.
  • the darkfield should be free of any contamination. For example, subsequent capturing a darkfield candidate image and prior to merging with the existing darkfield estimate, the darkfield candidate image is evaluated for quality, defects, and/or possible contamination. If a contaminated darkfieid candidate image is merged into the darkfieid estimate, the darkfield estimate is contaminated, resulting in potential authentication and performance errors of the matcher.
  • the darkfieid candidate image may be merged with a previously determined darkfield image (e.g., darkfield estimate).
  • a previously determined darkfield image e.g., darkfield estimate
  • the merging may be implemented as averaging the darkfield candidate image into the recorded darkfield estimate. This may reduce, or remove, the temporal noise contribution.
  • averaging or merging may be used. For example, a recursive average filter may be used where the latest darkfieid candidate image contribute with more weight than older darkfield images. Merging of the darkfieid candidate image and the darkfieid estimate may also be referred to as performing the darkfield update.
  • An additional darkfield quality verification step and/or contamination verification step may be applied before merging the recently acquired darkfield candidate image, e.g., before performing the darkfield update.
  • an image analysis may be applied to scan for any image contributions that are not likely to constitute a darkfieid image.
  • the image analysis may comprises looking for features resembling a fingerprint, spatial frequencies related to a fingerprint, or any other fingerprint characteristics. If such features are present, the darkfield candidate image may not be used, or used with a lesser weight.
  • a darkfieid quality factor may be determined, and the weight of the darkfield candidate image in the merger may depend on the quality factor.
  • the quality factor may also express a confidence in the fact that no object was detected it may also be determined if the quality of the darkfield estimate will be negatively affected by the merger of the darkfieid candidate image, and based on this determination, the weight of the darkfield candidate image may be adapted.
  • the stored darkfield image (e.g , darkfield estimate) after the merger may be subtracted from the recently acquired fingerprint image, since this represents the latest acquired darkfield image of the sensor. If the darkfield capture procedure is working properly, the corrected fingerprint image should be nearly uniform in situations where no object, such as a finger, is present on the sensor surface. The uniformity of the image may be determined to analyze the quality of the darkfield correction, and any issue or errors may be used as feedback to automatically adapt the darkfield correction process.
  • the corrected fingerprint image may then be sent to a matcher for authentication.
  • the matcher is the part of the fingerprint authentication system that compares the corrected fingerprint image to the stored fingerprint images acquired during enrollment of the user for authenticating the user of the sensor.
  • These stored fingerprint images acquired during enrollment may be referred to as the fingerprint templates, or templates.
  • These images are used as reference images for authentication, and may be referred to as authentication references images in some embodiments, these fingerprint templates may be updated after enrollment, referred to herein as a dynamic update of the fingerprint templates in order to ensure the accuracy of the authentication, the dynamic update should be performed using corrected fingerprint images, such that fingerprint images that include or are impacted by the use of a contaminated darkfield image are to be avoided.
  • FIG. 23 illustrates a flow diagram 2300 of an example method for determining darkfield contamination and performing dynamic updates of the fingerprint templates of a fingerprint authentication system, according to embodiments.
  • a darkfield image is determined or captured.
  • the determined darkfield image may be considered a darkfield candidate image for the darkfield update in some embodiments, this is performed using a single darkfield image acquired by the sensor, or this may be accomplished through a combination of multiple darkfield images acquired over time.
  • darkfield contamination evaluation process is initiated where a check is performed if the darkfield Is contaminated in any way. Embodiments of the darkfield contamination verification are discussed in detail below.
  • the darkfield contamination verification may be performed using many different methods.
  • the contamination may be investigated based on image analysis techniques, for example, by searching for image characteristics of the various possible contaminations.
  • the contamination investigation may be performed by comparing the darkfield candidate images to a database containing possible contaminations, and determine the resemblance between the darkfield candidate and the images of the database.
  • the darkfieid contamination may be investigated by monitoring changes in the darkfieid candidate. These changes may be monitored as a function of the operating conditions of the sensor.
  • the darkfieid contamination verification may also fake into consideration a contamination threshold such contamination is determined if an amount of contamination exceeds a contamination threshold. This allows for minor contamination to be disregarded.
  • multiple contamination thresholds may be utilized for different applications. For example, a first contamination threshold may be used for determining whether to use a darkfieid candidate image for correcting a fingerprint image during authentication (e.g., at procedure 2380) and a second contamination threshold may be used for determining whether to perform a dynamic update of the fingerprint template (e.g., procedure 2390), where the second contamination threshold is lower than (e.g., stricter than) the first contamination threshold.
  • corrective action may include removing or reducing the contamination from the darkfieid image, e.g., using a filter or other image enhancement techniques.
  • preventive actions may include determining the conditions under which the current darkfieid image was determined and trying to avoid acquired darkfieid images under future similar conditions. Such conditions may include any data and/or measurements from the fingerprint sensor, or any other sensor in the device that can help determine the condition or context of the device and/or user in some embodiments, the corrective action can include adjusting the determination of whether and object is interacting with the fingerprint sensor in some embodiments, the corrective action is adjusting the capture of the fingerprint image.
  • the corrective action may be to select another darkfieid image, which may be, e.g., a previously captured darkfieid image or a modeled darkfieid image.
  • the corrective action may be to adjust the weight of the darkfieid image in the update in order to limit the effect of the contamination on the darkfieid estimate.
  • the weight may be based on a determination of the quantify, quality, or type of contamination. For example, contamination resembling fingerprint patterns with ridge/valley type structures and/or spatial frequencies may have a more detrimental effect, than other type of contamination, for example, with spatial frequencies lower than typical fingerprints.
  • darkfieid candidate images with contamination may still be used in the darkfieid update, once it has been determined their incorporation does not negatively affect the sensor performance.
  • the darkfieid candidate image when it is determined that the darkfieid image is contaminated, e.g., the contamination quantity is above a threshold, the darkfieid candidate image may be replaced by a modeled darkfleld image, as described above in accordance with the described darkfield modeling.
  • the modeled darkfield image may then be merged with the darkfield estimate, or used instead of the darkfield estimate.
  • the contamination threshold may depend on the need for a new darkfield images. For example, if a long time has passed since the last darkfield capture was performed, the threshold may be lower
  • the darkfield update consists of merging the darkfield candidate image with the darkfield estimate, as discussed above.
  • the darkfield estimate is then used in combination with a fingerprint image captured at procedure 2380, to correct the fingerprint image for the darkfield in procedure 2370.
  • the fingerprint image may be captured before the darkfield image or after the darkfield image.
  • the corrected fingerprint image may be used to perform user authentication.
  • the authentication may be performed by comparing the corrected fingerprint image to the stored finger template(s) of users with allowed access. If the user authentication is successful, the corrected fingerprint image may be used for the dynamic update of the fingerprint templates.
  • a dynamic update of the fingerprint templates may be performed using the corrected fingerprint image.
  • Dynamic update of fingerprint templates adds a recent fingerprint image to the templates.
  • the dynamic update may take into consideration the quality of the fingerprint image. For example, any results of the darkfleld contamination determination may be used to determine the weight of the corrected fingerprint image in the dynamic update, similar to the darkfield update discussed above.
  • Dynamically updating the fingerprint template allows for improved fingerprint recognition, thereby improving reliability and performance. This means that when the darkfield Image is not contaminated it will not negatively influence the actual fingerprint image.
  • the allowed contamination content for using images in the dynamic update may be lower than the allowed contamination content for validating user access in other words, the quality and accuracy requirements before any fingerprint images and/or darkfield images, are used in the dynamic update are strict so as to protect against any contamination impacting the fingerprint images and/or darkfield images.
  • FIG. 23 shows the different steps of the process in an example order, and many different variations with a different order, with additional steps, or with omitted steps are possible.
  • the capture of the fingerprint image may be performed before the darkfield contamination verification in this case, the contamination verification is not performed every time a darkfield is captured, but only after the user puts a finger on the sensor and a fingerprint image is acquired. This may reduce the use of system resources in FIG 23, the darkfield contamination verification is shown at procedure 2320 that is before the darkfield update of procedure 2350, but, as an alternative, or in addition, the darkfield contamination verification may also be performed after the update to investigate if a contamination is present in the darkfield estimate.
  • the darkfield contamination verification of procedure 2320 is shown before the user authentication of procedure 2380.
  • the contamination verification may take a certain amount of time, and may therefore add additional latency to the user authentication, which may be undesired. Therefore, in some embodiments, the
  • contamination verification may be performed after, or in parallel, with the user authentication process. Should authenticated access be provided, and afterwards it is determined that a contamination is detected, access may be revoked, especially if the contamination contains fingerprint images (from the user). This means that the system may block the user and require a new authentication procedure. For example, the system may ask the user to take a new fingerprint image in some embodiments, a new darkfield image may be acquired, and this may comprise asking the user to remove his or her finger in order to acquire the darkfield image.
  • the darkfield contamination verification of procedure 2320 is performed once candidate dynamic updates of the fingerprint templates are generated, e.g., after procedure 2390.
  • a candidate dynamic update of the fingerprint templates is generated by combining the newly acquired fingerprint image with the existing fingerprint templates.
  • the candidate dynamic update of the fingerprint templates and/or the applied darkfield is then checked for possible contamination if no contamination is detected, the candidate dynamic update can be validated and used as the new updated fingerprint template.
  • the contamination verification may also comprises comparing the darkfield image to the candidate dynamic update, and if there is a match is means that the darkfield is contaminated with fingerprint pattern, possible of the authenticated user.
  • a decision whether or not to allow authentication may be based on the level of contamination (e.g., at procedure 2340). For example, for minor contamination, the authentication may be allowed, but the dynamic update may not be allowed. When a serious contamination is detected, other measures may be taken. For example, it may be decided not to do any darkfield correction, because no correction may yield better results than a correction with an incorrect darkfield. Alternatively, a different darkfield may be selected, e.g., an older darkfield. This different darkfield may be selected from a database of darkfield images acquired under similar conditions as the current operating conditions in some embodiments, a new darkfield may be determined, through measurement or
  • the system may ask the user the remove his or her finger in order to acquire a correct darkfield.
  • FIG. 24 illustrates a flow diagram 2400 of an example method for evaluating a darkfield image for contamination, according to embodiments.
  • flow diagram 2400 is performed subsequent capturing a darkfield candidate image and prior to updating a darkfield estimate with the darkfield candidate image, e.g., between procedures 830 and 840 of flow diagram 800 or between procedures 1240 and 1250 of flow diagram 1200, to confirm the darkfield candidate image is free of contamination.
  • a darkfield image is determined or captured.
  • this selected darkfield image may be the last acquired darkfield image or may be a merged darkfield image.
  • the selected darkfield Image is compared to a reference database.
  • This reference database may be a database containing any possible contamination artifacts. Different types of contamination may result in typical artifacts in the darkfield.
  • the reference database may include images or partial images of fingerprints or other contamination from other objects and/or artifacts on the sensor surface, such as, and without limitation: dust, dirt, (sticky) liquids such as water, or (sticky) solids with adhesive properties.
  • the reference database may include information for identifying any type of contamination artifact that might appear on the contact surface of the sensor.
  • the reference database can include images, models, or descriptions of such artifacts to identify contamination. Some contamination of the darkfield image may occur if the void detector is not working properly. For example, if the void detector indicates a void is detected, but in reality, the user has a finger on the sensor surface, a (partial) fingerprint of the user may be captured in the darkfield candidate image. Embodiments dealing with this problem are described below in more detail.
  • the reference database may be a single reference database with ail possible types of contamination, or several reference databases may be used for different types of contamination. In the latter case, a series for comparisons may be performed.
  • any filtering or image processing may be used to prepare the darkfield for the comparison. For example, high-pass or low-pas filtering, contrast enhancement, level correction, inversion, or any similar or other techniques may be used.
  • the darkfield image can be used as is or merged with a darkfield estimate in one embodiment, responsive to determining that contamination of the selected darkfield image is not detected, a dynamic update of the fingerprint template may be performed. Newly detected contaminations, which are not yet in the reference database, may be added to update the database of contaminations.
  • the contamination may be quantified and/or classified, e.g., to indicate the parts of the sensor surface affected, and the seriousness/importance of the contamination.
  • This information may then be used to decide whether or not to perform the darkfield update and/or the information may be used in the darkfield update process. For example, the importance of the contamination may be used as a weight in the darkfield update.
  • FIG. 25 illustrates a flow diagram 2500 of an example method for performing a darkfield contamination verification, according to embodiments.
  • the fingerprint templates are used as a reference database such that the darkfield image is compared to the fingerprint images acquired during enrollment or from a previous dynamic update procedure.
  • flow diagram 2500 detects and prevents any incorrectly determined darkfield images, where a darkfield image was acquired while the finger of the user was on the sensor. Any fingerprint images that incorrectly get included in the darkfield images, e.g., merged, can lead to false positives and related types of security risks.
  • a darkfield image is determined or captured. In some embodiments, this is performed using a single darkfield image acquired by the sensor, or this may be accomplished through a combination of multiple darkfield images acquired over time.
  • the darkfield image is compared to the fingerprint images acquired during enrollment or from a previous dynamic update procedure.
  • any other image analysis may be used to test for the presence of fingerprint like features, e.g., frequency or spatial analysis to detect the typical ridge/vai!ey pattern of a fingerprint.
  • the darkfieid image matches the fingerprint template, e.g., of an authorized user.
  • the darkfieid image may be run through the matcher to verify if the darkfieid image matches the fingerprint template of any authorized user.
  • the darkfieid image matches or is substantially similar to the fingerprint template, contamination is detected in one embodiment, when contamination is detected, the darkfieid image can be discarded and not used for updating the darkfieid estimate.
  • the darkfieid image is merged with the darkfieid estimate.
  • the darkfieid candidate image may be replaced by a modeled darkfieid image.
  • the modeled darkfieid image may then be merged with the darkfieid estimate, or used instead of the darkfieid estimate.
  • contamination in a darkfieid candidate image is evaluated by comparing the darkfieid candidate image to a best fit model of previously acquired darkfieid images. For example, in one embodiment the darkfieid candidate image is compared to a best fit model based on eight previously acquired darkfieid images. It should be appreciated that the previously acquired darkfieid images are assumed as not including contamination. The best fit model is then compared to the darkfieid estimate, and the residue is a measure of how good the fit is. Any contamination will increase the residue.
  • the darkfieid candidate image can be add to the registered darkfieid images for use of the correction of the fingerprint image.
  • the darkfieid contamination is determined by comparing the captured darkfieid image with a predicted or modeled darkfieid images. By analyzing a difference between the captured darkfieid Image and the modeled darkfieid image the contamination is determined if the difference is small, and both images are similar, no contamination is detected. If the difference is large, contamination is detected.
  • the modeling can be performed according to the embodiments described above. For example, the operating condition of the sensor is determined, for example the temperature, and the darkfieid is modeled based on the operating condition. [0182] In another embodiment, a best fit mode!
  • FIG. 26 illustrates an example system 2600 for evaluating a darkfieid image for contamination based on a best fit model, according to embodiments.
  • a best fit model may be used where the best fit is determined between the captured darkfieid image and a combination of darkfieid images (verified to be free of contamination).
  • These darkfieid images are selected from a database 2610 of darkfieid images, which may have been verified to be free of contamination, or may have been generated in a (controlled) calibration process.
  • the database 2610 may comprise darkfieid images captured at regular temperature intervals, and a plurality of contamination-free darkfieid images captured at different temperature may serve as a basis for the fit.
  • the captured darkfieid image 2620 may be processed before calculating the best fit.
  • the best darkfieid information may be present in the valleys of the fingerprint, and as such the process may only take into account these areas to determine the best fit. Selecting the best areas of the fingerprint to be used may be performed through filtering or more complicated image processing techniques.
  • a best fit algorithm may be applied to the captured darkfieid image in procedure 2630.
  • the best fit algorithm 2630 may select a single image from the darkfieid database 2610, or may select any combination for the darkfieid database 2610.
  • the difference between the best fit 2640 and the captured darkfieid image 2620 may be determined in 2650.
  • This difference, or residue 2660 may be quantified to determine the extent of any possible contamination, and may be compared to a residue threshold to determine if a contamination is present.
  • the residue exceeds the residue threshold, the darkfieid candidate image is modeled.
  • the difference or difference image may be analyzed to analyze the (type of) contamination. Similar to the embodiment discussed above, the difference image may be compared to a reference database.
  • the difference image may be compared to a fingerprint database to determine if the darkfieid image is contaminated with a fingerprint (from an authorized use). If it is determined that a contamination is present, the modeled darkfieid or best-fit darkfieid may be used for the darkfieid correction instead of the captured darkfieid.
  • the darkfieid contamination check is performed by monitoring the changes and evolution of the darkfieid image over time. Since the darkfieid image should only change gradually over time, any fast or abrupt change can be an indication of contamination. Furthermore, the changes and evolution of the darkfieid image may also be monitored as a function of the operating conditions of the sensor. Again, when the operating conditions change gradually, it is expected that there is no abrupt change in the darkfieid image. Therefore, darkfield contamination can be checked by comparing changes in the darkfieid image to changes in the operating condition, and/or by determining the speed of darkfieid changes. For the latter, a darkfieid image difference may be determined with one or more previous captured darkfieid images.
  • the darkfieid image difference may be normalized for the time since the previous capture. If the darkfieid difference is above a threshold, contamination may be present.
  • the darkfieid image difference may be expressed, for example, as the variance of the difference image, or as a difference of the variance of the different images. Any other quantification of the image difference may be used.
  • the operating condition comprises the temperature of the sensor
  • the darkfieid changes are monitored as a function of the temperature or as a function of the temperature change.
  • the change in darkfieid image with respect to a previous darkfieid image may be determined as a function of the temperature change since the previous darkfieid image. If the change in darkfieid image is within a certain range, it may be determined that the darkfieid is contaminated.
  • FIG. 27 shows an example of an example defined
  • temperature range for an allowed variance in darkfieid changes where the difference between the darkfieid image candidate (e.g., current darkfieid image) and the darkfieid image estimate (e.g., a previous darkfieid image), is shown as a function of the temperature difference between the candidate and the estimate.
  • the difference is expressed, in the example, as the difference between the variance of the candidate and the variance of the estimate.
  • FIG. 27 shows that for small temperature changes the boundary is set at a small variance difference, while the boundary increases as the temperature difference increases. If the variance difference is above the boundary, it is determined that there is a high likelihood that a contamination is present.
  • FIG. 27 also shows that above a certain temperature change no decision about the
  • contamination can be made because the amount of change in darkfieid due to change in the operating conditions may be comparable to the amount of change due to contamination.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Image Input (AREA)

Abstract

L'invention concerne un procédé de correction d'une image d'empreinte digitale, selon lequel il est déterminé si un objet interagit avec un capteur d'empreintes digitales. Si un objet n'est pas en interaction avec le capteur d'empreintes digitales, il est déterminé s'il faut capturer une image candidate de fond noir au niveau du capteur d'empreintes digitales, l'image candidate de fond noir étant une image dépourvue d'un objet interagissant avec le capteur d'empreintes digitales. Quand il est déterminé qu'il faut capturer l'image candidate de fond noir, l'image candidate de fond noir est capturée au niveau du capteur d'empreintes digitales. Si un objet interagit avec le capteur d'empreintes digitales, il est déterminé s'il faut modéliser une image candidate de fond noir au niveau du capteur d'empreintes digitales. Quand il est déterminé qu'il faut modéliser l'image candidate de fond noir, l'image candidate de fond noir est modélisée au niveau du capteur d'empreintes digitales. Une estimation de fond noir est mise à jour avec l'image candidate de fond noir. Une image d'empreinte digitale est capturée au niveau du capteur d'empreintes digitales. L'image d'empreinte digitale est corrigée à l'aide de l'estimation de fond noir.
PCT/US2019/034032 2018-06-07 2019-05-24 Correction d'une image d'empreinte digitale Ceased WO2019236328A1 (fr)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US201862682065P 2018-06-07 2018-06-07
US62/682,065 2018-06-07
US201862724200P 2018-08-29 2018-08-29
US62/724,200 2018-08-29
US16/270,516 2019-02-07
US16/270,516 US20190188442A1 (en) 2017-12-01 2019-02-07 Correcting a fingerprint image

Publications (1)

Publication Number Publication Date
WO2019236328A1 true WO2019236328A1 (fr) 2019-12-12

Family

ID=66821517

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2019/034032 Ceased WO2019236328A1 (fr) 2018-06-07 2019-05-24 Correction d'une image d'empreinte digitale

Country Status (1)

Country Link
WO (1) WO2019236328A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112101194A (zh) * 2020-01-21 2020-12-18 神盾股份有限公司 电子装置及其操作方法
CN112330595A (zh) * 2020-10-13 2021-02-05 浙江华睿科技有限公司 一种绊丝检测方法、装置、电子设备及存储介质
CN114327203A (zh) * 2021-12-28 2022-04-12 傲网信息科技（厦门）有限公司 一种用户操作的还原方法及装置、记录方法及装置

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160358003A1 (en) * 2015-06-05 2016-12-08 Synaptics Incorporated Finger detection with auto-baseline tracking
US20170231534A1 (en) * 2016-02-15 2017-08-17 Qualcomm Incorporated Liveness and spoof detection for ultrasonic fingerprint sensors
US20170255338A1 (en) * 2016-03-03 2017-09-07 Invensense, Inc. Determining force applied to an ultrasonic sensor

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160358003A1 (en) * 2015-06-05 2016-12-08 Synaptics Incorporated Finger detection with auto-baseline tracking
US20170231534A1 (en) * 2016-02-15 2017-08-17 Qualcomm Incorporated Liveness and spoof detection for ultrasonic fingerprint sensors
US20170255338A1 (en) * 2016-03-03 2017-09-07 Invensense, Inc. Determining force applied to an ultrasonic sensor

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
PAPAGEORGION C ET AL: "Self-calibration of ultrasonic transducers inan intelligent data acquisition system", INTELLIGENT DATA ACQUISITION AND ADVANCED COMPUTING SYSTEMS: TECHNOLOG Y AND APPLICATIONS, 2003. PROCEEDINGS OF THE SECOND IEEE INTERNATIONAL WORKSHOP ON SEPT. 8-10, 2003, PISCATAWAY, NJ, USA,IEEE, PISCATAWAY, NJ, USA, 8 September 2003 (2003-09-08), pages 2 - 6, XP010671615, ISBN: 978-0-7803-8138-4, DOI: 10.1109/IDAACS.2003.1249504 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112101194A (zh) * 2020-01-21 2020-12-18 神盾股份有限公司 电子装置及其操作方法
CN112330595A (zh) * 2020-10-13 2021-02-05 浙江华睿科技有限公司 一种绊丝检测方法、装置、电子设备及存储介质
CN112330595B (zh) * 2020-10-13 2024-04-02 浙江华睿科技股份有限公司 一种绊丝检测方法、装置、电子设备及存储介质
CN114327203A (zh) * 2021-12-28 2022-04-12 傲网信息科技（厦门）有限公司 一种用户操作的还原方法及装置、记录方法及装置
CN114327203B (zh) * 2021-12-28 2023-08-04 傲网信息科技（厦门）有限公司 一种用户操作的还原方法及装置、记录方法及装置

Similar Documents

Publication Publication Date Title
US10997388B2 (en) Darkfield contamination detection
US20190188442A1 (en) Correcting a fingerprint image
US10984209B2 (en) Darkfield modeling
US10936841B2 (en) Darkfield tracking
US20200257875A1 (en) Darkfield tracking
US11016186B2 (en) Defective ultrasonic transducer detection in an ultrasonic sensor
US10846502B2 (en) Ultrasonic fingerprint sensor with a non-uniform contact layer
US12002282B2 (en) Operating a fingerprint sensor comprised of ultrasonic transducers
US10936843B2 (en) Segmented image acquisition
US10452887B2 (en) Operating a fingerprint sensor comprised of ultrasonic transducers
US11651611B2 (en) Device mountable packaging of ultrasonic transducers
US10656255B2 (en) Piezoelectric micromachined ultrasonic transducer (PMUT)
US20170328866A1 (en) Sensing device with a temperature sensor
US11301552B2 (en) Medical device with integrated ultrasonic authentication
JP6619508B2 (ja) 超音波皮下プローブによるスプーフ検出
US11682228B2 (en) Ultrasonic fingerprint sensor with a contact layer of non-uniform thickness
WO2019236328A1 (fr) Correction d'une image d'empreinte digitale
KR20160108390A (ko) 음향 공진 캐비티를 통한 초음파 이미징
US11243300B2 (en) Operating a fingerprint sensor comprised of ultrasonic transducers and a presence sensor
US11216632B2 (en) Ultrasonic fingerprint sensor with a contact layer of non-uniform thickness
US11216641B2 (en) Latent fingerprint detection
CN115953811A (zh) 一种超声波传感器阵列自适应控制方法及装置
KR20230128294A (ko) 잠재 지문들의 회피를 위해 구성된 지문 센서
Gao et al. Ultrathin Multilayer P (VDF-TrFE) Film-Based Piezoelectric Resonator for High-Quality Under-Display Fingerprint Recognition

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19730094

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19730094

Country of ref document: EP

Kind code of ref document: A1