WO2025143564A1 - Electronic device for controlling brightness of display by using metadata of image, and method thereof - Google Patents

Abstract

An electronic device according to an embodiment may acquire a first image having a first dynamic range, map information related to the brightness of at least a part of the first image or the gradation for each color, and display control information related to the brightness level at the time of display. The electronic device may generate a second image on the basis of the first image and the map information. The electronic device may determine the brightness level of at least a part of the display on the basis of the control information of the display. The electronic device may display the second image according to a second dynamic range that is wider than the first dynamic range, on the basis of the determined brightness level.

Description

Electronic device and method for controlling brightness of display using metadata of image

The present disclosure relates to an electronic device and method for controlling the brightness of a display using metadata of an image.

Digital information generated for visualizing images and/or videos (e.g., image files in the format of the joint photographic experts group (JPEG) and/or moving picture experts group (MPEG)) can be generated to represent colors using a limited number of bits. For example, within the digital information, the brightness of a particular primary color (e.g., one of red, green, or blue) can be stored using 8 bits. In the above example, the digital information can be generated to represent the brightness of a particular primary color using a total of 256 levels (= 2 ⁸ ) of brightness levels.

The above information may be provided as related art for the purpose of helping to understand the present disclosure. None of the above is claimed to be prior art related to the present disclosure or can be used in making decisions related to prior art.

In one embodiment, an electronic device may include a display, at least one processor including a processing circuit, and a memory including one or more storage media storing instructions. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to detect an event for displaying a first image having a first dynamic range while a brightness level of the display is set to a first brightness level. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to, in response to the event, obtain, from metadata in a file including the first image, map information for another portion of the first image that is visually emphasized with respect to a portion of the first image, and display control information for changing a brightness level of the display in relation to the display of the first image. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to generate a second image by applying the map information to the first image. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to set a brightness level of the display from the first brightness level to a second brightness level higher than the first brightness level, based on the display control information. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display the second image on the display set to the second brightness level, such that the first image having a second dynamic range wider than the first dynamic range is shown on the display.

In one embodiment, a method of an electronic device including a display may be provided. The method may include detecting an event for displaying a first image having a first dynamic range while a brightness level of the display is set to a first brightness level. The method may include, in response to the event, obtaining, from metadata in a file including the first image, map information for another portion of the first image visually emphasized with respect to a portion of the first image, and display control information for changing a brightness level of the display in relation to displaying the first image. The method may include generating a second image by applying the map information to the first image. The method may include setting a brightness level of the display from the first brightness level to a second brightness level higher than the first brightness level based on the display control information. The method may include displaying the second image on the display set to the second brightness level such that the first image having a second dynamic range wider than the first dynamic range is shown on the display.

In one embodiment, an electronic device may include at least one camera, at least one processor including a processing circuit, and a memory including one or more storage media storing instructions. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to control the at least one camera to acquire a plurality of images in response to a photographing input. The plurality of images may include a first image acquired according to a first dynamic range and a second image having a second dynamic range at least partially different from the first dynamic range. A bit depth of the first dynamic range may be the same as a bit depth of the second dynamic range. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to acquire a third image having a third dynamic range using at least a portion of the plurality of images. A bit depth of the third dynamic range may be the same as a bit depth of the first dynamic range. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine display control information for changing a brightness level of the display device such that the third image is displayed as an image having a fourth dynamic range. A bit depth of the fourth dynamic range may be greater than the bit depth of the third dynamic range. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to generate a file including metadata including the map information and the display control information, and the third image.

In one embodiment, a method of an electronic device including at least one camera may be provided. The method may include an operation of controlling the at least one camera to acquire a plurality of images in response to a photographing input. The plurality of images may include a first image acquired according to a first dynamic range and a second image having a second dynamic range at least partially different from the first dynamic range, wherein a bit depth of the first dynamic range may be the same as a bit depth of the second dynamic range. The method may include an operation of acquiring a third image having a third dynamic range using at least a portion of the plurality of images. The bit depth of the third dynamic range may be the same as a bit depth of the first dynamic range. The method may include an operation of generating map information for another portion of the third image that is visually emphasized for a portion of the third image using the plurality of images. The method may include an operation of determining display control information for changing a brightness level of a display device so that the third image is displayed as an image having a fourth dynamic range. The bit depth of the fourth dynamic range may be greater than the bit depth of the third dynamic range. The method may include an operation of generating a file including metadata including the map information and the display control information and the third image.

In one embodiment, an electronic device may include a display, at least one processor including a processing circuit, and a memory including one or more storage media storing instructions. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to obtain a first image having a first dynamic range, map information related to brightness or color gradation of at least a portion of the first image, and display control information related to a brightness level during display. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to generate a second image based on the first image and the map information. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine a brightness level of at least a portion of the display based on the display control information. The above instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display the second image according to a second dynamic range that is wider than the first dynamic range, based on the determined brightness level.

In one embodiment, a non-transitory computer-readable storage medium storing instructions may be provided. The instructions may be executed by an electronic device including a display. The instructions, when executed by the electronic device, may cause the electronic device to obtain a first image having a first dynamic range, map information related to brightness or color gradation of at least a portion of the first image, and display control information related to a brightness level during display. The instructions, when executed by the electronic device, may cause the electronic device to generate a second image based on the first image and the map information. The instructions, when executed by the electronic device, may cause the electronic device to determine a brightness level of at least a portion of the display based on the display control information. The above instructions, when executed by the electronic device, may cause the electronic device to display the second image according to a second dynamic range that is wider than the first dynamic range, based on the determined brightness level.

FIG. 1 illustrates an exemplary operation of an electronic device for displaying an image from a file, according to one embodiment.

FIG. 2 illustrates a block diagram of an electronic device according to one embodiment.

FIG. 3 illustrates an exemplary operation of an electronic device for controlling at least one camera to generate information including an image supporting HDR effects and SDR.

FIG. 4 illustrates an exemplary operation of an electronic device for generating a file using a plurality of images acquired from at least one camera.

FIG. 5 illustrates an exemplary structure of a file generated by an electronic device according to one embodiment.

Figure 6 illustrates exemplary brightness levels of a display of an electronic device displaying an image contained in a file.

Figure 7 illustrates an exemplary operation of an electronic device for generating information to be transmitted from a file to a display.

Figure 8 illustrates exemplary states of an electronic device displaying an image in HDR mode.

FIG. 9 is a block diagram of an electronic device within a network environment according to various embodiments.

FIG. 10 is a block diagram illustrating a camera module according to various embodiments.

FIG. 11 is a block diagram of a display module according to various embodiments.

FIGS. 12A, 12B, and 12C illustrate exemplary operations performed by an electronic device including a housing foldable by a folding axis to display an image in HDR mode.

FIG. 13 illustrates an exemplary operation performed by an electronic device including multiple displays to display an image in HDR mode.

FIGS. 14A and 14B illustrate exemplary operations performed by an electronic device including a flexible display to display an image in HDR mode.

FIG. 15 illustrates exemplary operations performed by a head-mounted display (HMD) device, according to one embodiment, to display an image in HDR mode.

FIG. 16 illustrates an exemplary operation of an electronic device that uses user-related information to at least partially enhance the brightness level of a display when displaying an image in HDR mode.

Below, various embodiments of this document are described with reference to the attached drawings.

The various embodiments of this document and the terminology used herein are not intended to limit the technology described in this document to the specific embodiments, but should be understood to encompass various modifications, equivalents, and/or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar components. The singular expressions may include plural expressions unless the context clearly indicates otherwise. In this document, expressions such as "A or B", "at least one of A and/or B", "A, B, or C" or "at least one of A, B and/or C" can include all possible combinations of the items listed together. Expressions such as "first", "second", "first" or "second" can modify the corresponding components, regardless of order or importance, and are only used to distinguish one component from another and do not limit the corresponding components. When it is said that a certain (e.g., a first) component is "(functionally or communicatively) connected" or "connected" to another (e.g., a second) component, said certain component may be directly connected to said other component, or may be connected through another component (e.g., a third component).

The term "module" as used in this document includes a unit composed of hardware or firmware, and may be used interchangeably with terms such as logic, block, component, or circuit. A module may be an integrally composed component or a minimum unit or part thereof that performs one or more functions. For example, a module may be composed of an application-specific integrated circuit (ASIC).

FIG. 1 illustrates an exemplary operation of an electronic device (101) for displaying an image from a file (110), according to one embodiment. The electronic device (101) may have various form factors, such as a smart phone, a laptop personal computer (PC), a tablet PC, a head-mounted display (HMD) device, a watch, and other similar computing devices (not shown). The electronic device (101) may also be referred to as a mobile device, a user terminal, a user equipment (UE), a multi-function device, a portable communication device, and/or a handheld device. The form factor of the electronic device (101) is not limited to the exemplary form factors illustrated in FIG. 1.

In one embodiment, the electronic device (101) may generate a file (110), and/or visualize or display media content (e.g., media content referred to as photographs and/or images) of the generated file (110). The file (110) may include a JPEG file, a high efficiency image file format (HEIF) file, a high efficiency image container (HEIC) file, a file for storing raw data (e.g., a digital negative image (DNG) file), a portable network graphic (PNG) file, and/or a graphics interchange format (GIF) file. An exemplary hardware configuration of the electronic device (101) for executing functions related to generating and/or processing the file (110) is described with reference to FIG. 2.

In the present disclosure, the dynamic range of an image may mean a ratio between a minimum brightness and a maximum brightness represented by a color distribution of the image. The dynamic range may include a contrast ratio. Referring to FIG. 1, a file (110) may include an image (112) related to a low dynamic range (LDR) and/or a standard dynamic range (SDR). The brightness (or luma value) of pixels of an image (112) having SDR may be represented as a binary value having a bit depth of 8 bits. Using a bit depth of 8 bits, the image (112) having SDR may represent brightness of 256 levels (= 2 ⁸ ). For example, an image displayed using SDR may have a contrast ratio of approximately 250:1. Referring to FIG. 1, a contrast ratio between the minimum brightness l1 and the maximum brightness l2 of pixels included in an image (112) may be about 250:1. Light generated to display an image (112) having SDR may have a deviation of up to 100 nits (e.g., l2 - l1) in different parts of the image (112). An image (112) associated with SDR may have color information based on, for example, an sRGB (standard RGB) color space.

In one embodiment, the electronic device (101) can perform operations related to a wider dynamic range than SDR (e.g., high dynamic range (HDR)). For example, the electronic device (101) can control the display (120) supporting the HDR mode to output light having a contrast ratio different from the 250:1 described above (e.g., 10,000:1 and/or 20,000:1). For example, the electronic device (101) can synthesize (e.g., bracket) images acquired from a camera to generate a file (110) that can be processed in both SDR and HDR. For example, the file (110) can include an image (112) that can be displayed by another electronic device (or display device) that supports only SDR among SDR or HDR. For example, the electronic device (101) may include information that can scale visual information of the image (112) to HDR while including an image (112) of SDR that is compatible with a legacy display device that only supports SDR. An exemplary operation of the electronic device (101) generating a file (110) is described with reference to FIGS. 3 and 4. An exemplary structure of a file (110) generated by the electronic device (101) is described with reference to FIG. 5.

Referring to FIG. 1, according to an embodiment, a file (110) generated by an electronic device (101) may include metadata required for an electronic device (101) (or a display device) supporting an HDR mode to synthesize an image (130) having an HDR effect from an image (112) having an SDR. The metadata may include additional information related to the image (112) having an SDR (e.g., photographing information based on EXIF (EXchangeable Image File)), map information (114), and/or display control information (116). The map information (114) may be a two-dimensional array having a width less than or equal to the width and height of the image (112), respectively, and a height. Elements of the two-dimensional array (e.g., pixels of the map information (114)) may include numerical values (e.g., coefficients and/or increments) used to change the brightness of different portions of the image (112). By changing the values (e.g., brightness values) of pixels of the image (112) using the map information (114), the electronic device (101) can obtain an image (130) for HDR effect. For example, the image (130) synthesized from the image (112) using the map information (114) can include brightness values expressed using a bit depth greater than the bit depth of the image (112).

According to one embodiment, the electronic device (101) may control the display (120) based on HDR when displaying an image (130) on the display (120). The electronic device (101) may display an image (130) for HDR effect, synthesized from an image (112) having SDR, included in a file (110), on the display (120). When displaying the image (130) using the display (120), a contrast ratio between the minimum brightness l3 and the maximum brightness l4 of the image (130) may exceed a contrast ratio associated with SDR (e.g., approximately 250:1) (e.g., 10,000:1). When displaying an image (130) through a display (120), the electronic device (101) can display the image (130) based on a color space wider than SDR (e.g., DCI (digital cinema initiatives)-P3 and/or REC.2020).

In one embodiment, the display control information (116) included in the file (110) may be used to control the display (120). For example, the display control information (116) may include a numerical value related to a maximum brightness l4 required to display an image (130) having an HDR effect based on the file (110). For example, the display control information (116) may include a ratio between the maximum brightness l2 of pixels of the image (112) of SDR and the maximum brightness l4 of pixels of the image (130) having the HDR effect. An exemplary operation of the electronic device (101) to control the display (120) to display the image (130) is described with reference to FIGS. 6 to 8.

As described above, according to one embodiment, the electronic device (101) can clearly visualize an image (130) having an HDR effect in order to at least partially increase (or enhance) the brightness of the display (120). The electronic device (101) can generate a file (110) including all of the information for synthesizing the image (130) of the HDR effect (e.g., the map information (114) and/or the display control information (116)) and the image (112) of the SDR. Using the image (112) of the SDR, the electronic device (101) can generate a file (110) that is compatible with a legacy image rendering pipeline that uses a bit depth of 8 bits (e.g., a hardware accelerator that performs rendering on a file having a format of JPEG, and/or a software application configured to emulate the hardware accelerator). Furthermore, the electronic device (101) can add information used to synthesize an image (130) supporting HDR effect to the file (110), thereby allowing a modern display device supporting HDR (e.g., an electronic device (101) including a display (120)) to display the image (130) using a wider dynamic range than SDR.

Below, with reference to FIG. 2, an exemplary hardware configuration of the electronic device (101) of FIG. 1 is described.

FIG. 2 illustrates a block diagram of an electronic device (101) according to one embodiment. The electronic device (101) of FIG. 1 may include the electronic device (101) of FIG. 1. Referring to FIG. 2, the electronic device (101) may include a processor (210) (e.g., processor 920 of FIG. 9) and a display (120), a memory (215), at least one camera (225), and a battery (235). The hardware configuration of the electronic device (101) is not limited to the embodiment of FIG. 2. For example, the electronic device (101) may further include electronic components described with reference to FIG. 9. For example, some of the electronic components of FIG. 2 (e.g., battery (235) and/or at least one camera (225)) may be excluded from the electronic device (101).

For example, the processor (210) may be operably coupled with the display (120) or the display driver circuit (122) within the display (120). For example, the processor (210) being operably coupled with the display (120) (or the display driver circuit (122)) may indicate that the processor (210) is directly coupled with the display (120) (or the display driver circuit (122)). For example, the processor (210) being operably coupled with the display (120) (or the display driver circuit (122)) may indicate that the processor (210) is coupled with the display (120) (or the display driver circuit (122)) via another component of the electronic device (101). For example, the fact that the processor (210) is operatively coupled with the display (120) (or the display driving circuit (122)) may indicate that the state of the processor (210) is a state that can control the display (120) (or the display driving circuit (122)). For example, the fact that the processor (210) is operatively coupled with the display (120) (or the display driving circuit (122)) may indicate that operation of the display (120) (or the display driving circuit (122)) is caused based on information, data, signals, or commands obtained from the processor (210). However, the present invention is not limited thereto.

For example, the processor (210) of the electronic device (101) may include a circuit (e.g., a processing circuit) for processing data based on one or more instructions. The circuit for processing data may include, for example, an arithmetic and logic unit (ALU), a floating point unit (FPU), a field programmable gate array (FPGA), a central processing unit (CPU), a graphic processing unit (GPU), a neural processing unit (NPU), and/or an application processor (AP). For example, the number of processors may be one or more. The processing circuit of the processor that loads (or fetches) an instruction and performs a calculation corresponding to the loaded instruction may be referred to as a core circuit (or core). For example, the processor may have a multi-core processor structure including a plurality of core circuits, such as a dual core, a quad core, a hexa core, or an octa core. The functions and/or operations described with reference to the present disclosure may be performed individually or collectively by one or more processing circuits included in the processor (210).

For example, the display (120) of the electronic device (101) can output visualized information (e.g., the screen of FIG. 1) to the user. For example, the display (120) can be controlled by a controller such as a GPU (graphic processing unit) to output visualized information to the user. The display (120) can include a liquid crystal display (LCD), a plasma display panel (PDP), and/or one or more light emitting diodes (LEDs). The LEDs can include organic LEDs (OLEDs). The display (120) can include a flat panel display (FPD) and/or electronic paper. The embodiment is not limited thereto, and the display (120) can have an at least partially curved shape or a deformable shape. A display (120) having a deformable shape can be referred to as a flexible display.

For example, the display (120) of the electronic device (101) may include a sensor (e.g., a touch sensor panel (TSP)) for detecting an external object (e.g., a user's finger) on the display (120). For example, based on the TSP, the processor (210) may detect an external object that is in contact with the display (120) or floating on the display (120). In response to detecting the external object, the processor (210) may execute a function related to a specific visual object corresponding to a location of the external object on the display (120) among visual objects displayed on the display (120).

For example, the display (120) may include a display driver circuit (122) (e.g., a display driver IC (1130) of FIG. 11) and a display panel (124) (e.g., a display (1110) of FIG. 11). For example, the display driver circuit (122) may be operatively coupled with the display panel (124). For example, when the display panel (124) includes a plurality of LEDs arranged in a two-dimensional matrix form, the display driver circuit (122) may be configured to control at least one LED included in a corresponding row or column among the plurality of LEDs. The display driver circuit (122) controlling the at least one LED may include an operation of adjusting the luminance (or light amount, brightness) of the LEDs. In the present disclosure, luminance may mean the intensity of light emitted from pixels of the display (120) (e.g., the intensity of light measured in units of nits (or cd/m ² )). In the present disclosure, brightness may include a relative value expressed in bits of a bit depth, referred to as a brightness level, as the brightness of a pixel of an image to be displayed through a display (120).

For example, the memory (215) of the electronic device (101) may include a circuit and/or a storage medium for storing data and/or instructions input to and/or output from the processor (210). The memory may include, for example, volatile memory such as random-access memory (RAM) and/or non-volatile memory such as read-only memory (ROM). The non-volatile memory may be referred to as storage. The volatile memory may include, for example, at least one of dynamic RAM (DRAM), static RAM (SRAM), Cache RAM, and pseudo SRAM (PSRAM). The non-volatile memory may include, for example, at least one of programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, a hard disk, a compact disc, a solid state drive (SSD), and an embedded multi media card (eMMC). The processor (210) of the electronic device (101) can execute instructions of the memory (215) within the electronic device (101) to perform functions and/or operations indicated by the instructions. For example, when the electronic device (101) includes at least one processor, the at least one processor can be configured to collectively or individually execute the instructions.

For example, at least one camera (225) of the electronic device (101) may include one or more optical sensors (e.g., a CCD (charged coupled device) sensor, a CMOS (complementary metal oxide semiconductor) sensor) that generate an electrical signal representing a color and/or brightness of light. The plurality of optical sensors included in the at least one camera (225) may be arranged in the form of a two-dimensional array. The at least one camera (225) may acquire the electrical signals of each of the plurality of optical sensors substantially simultaneously to generate two-dimensional frame data corresponding to light reaching the optical sensors of the two-dimensional array. For example, photographic data captured using the at least one camera (225) may mean one (a) two-dimensional frame data acquired from the at least one camera (225). For example, video data captured using the at least one camera (225) may mean a sequence of a plurality of two-dimensional frame data acquired from the at least one camera (225). At least one camera (225) of FIG. 2 may include a camera module (980) of FIG. 9 and/or FIG. 10.

In one embodiment, the electronic device (101) may include a battery (235). The battery (235) may be any one of a lithium ion (Li-ion) battery, a lithium ion polymer (Li-ion polymer) battery, a lead-acid battery, a nickel-cadmium (NiCd) battery, and a nickel-metal hydride (NiMH) battery. To manage charging and/or discharging of the battery (235), the electronic device (101) may include a power management integrated circuit (PMIC) (e.g., a power management module (988) of FIG. 9 ). Using the PMIC, the processor (210) of the electronic device (101) may obtain or identify parameters indicative of the state of the battery (235), such as a state of charge (SOC), an open circuit voltage (OCV), and/or a state of health (SOH)).

Referring to FIG. 2, information (e.g., file (110)) stored in a memory (215) of an electronic device (101) and/or programs (e.g., image renderer (216) and/or display brightness determiner (217)) are illustrated. A processor (210) executing instructions included in a program may perform functions and/or operations indicated by the instructions. The file (110) may include color information representing colors of pixels of an image (e.g., image (112) of FIG. 1) according to a color space such as YUV, RGB, and/or HSV. For example, a file (110) based on a color space of RGB may represent a color of a specific pixel using intensities of three primary colors of red, green, and blue. For example, a file (110) based on the color space of YUV can represent the color of a specific pixel using three components including a brightness component (e.g., Y component) and chrominance components (e.g., Cb component and/or Cr component).

For example, at least three channels may be used to represent colors of pixels of an image. From a file (110) in JPEG format, the processor (210) may identify colors of a plurality of pixels expressed by three channels having a bit depth of 8 bits. The file (110) may additionally include map information (e.g., map information (114) of FIG. 1) corresponding to a specific component (e.g., brightness component). The map information corresponding to a specific component may be used to increase a bit depth of the specific component (e.g., a bit number greater than 8 bits).

By executing the image renderer (216), the processor (210) can generate or synthesize an image having an HDR effect (e.g., an image (130) of FIG. 1) from an image having an SDR included in the file (110) (e.g., an image (112) of FIG. 1). For example, the processor (210) can restore an image having an HDR effect from an image of the SDR by using map information (e.g., the map information (114) of FIG. 1) included in the metadata of the file (110). The restoration can include scaling (e.g., amplifying and/or attenuating) the brightness of pixels of the image of the SDR by using information included in the map information. The processor (210) executing the image renderer (216) can generate or obtain information for displaying an image having a bit depth of HDR.

By executing the display brightness determiner (217), the processor (210) can increase, at least partially, the brightness of the pixels of the display (120). The processor (210) executing the display brightness determiner (217) can control the display driving circuit (122) to at least partially increase the brightness of the display (120). For example, in order to visually emphasize a part of an image to be displayed on the display (120) over another part, the processor (210) can cause the brightness of at least one pixel of the display (120) corresponding to the part to exceed the brightness of at least one pixel of the display (120) corresponding to the other part. With the display brightness determiner (217) executing, the processor (210) can determine an amount of brightness increase of the entire or at least a part of the display (120) by using map information (e.g., map information (114) of FIG. 1). Using the increased luminance increase, the processor (210) can perform operations such as gamma adjustment.

The processor (210) executing the display brightness determiner (217) can adjust the gamma of the image rendered by the image renderer (216). The gamma can mean the relationship between the brightness of pixels of the image and the brightness of pixels of the display (120) displaying the image. The gamma can be expressed as a function referred to as a gamma curve. When displaying an HDR image synthesized from a file (110), the processor (210) can control the display driving circuit (122) to display the image by using gamma related to metadata of the file (110) (e.g., display control information (116)).

As described above, according to one embodiment, the electronic device (101) may, when displaying a file (110) based on SDR, restore or synthesize an HDR image using metadata of the file (110). When displaying a synthesized image, the electronic device (101) may control the display (120) so that brightness of the display (120) is at least partially increased using metadata of the file (110). The electronic device (101) may control a camera to generate or store the file (110) using images acquired. Hereinafter, with reference to FIGS. 3 and 4 , an exemplary operation of the electronic device (101) that generates a file (110) using at least one camera (225) will be described.

FIG. 3 illustrates an exemplary operation of an electronic device (101) that generates information including an image supporting HDR effects and SDR by controlling at least one camera. The electronic device (101) of FIGS. 1 and 2, and/or the processor (210) of FIG. 2, may perform the operation of the electronic device (101) described with reference to FIG. 3. At least one camera of FIG. 3 may correspond to at least one camera (225) of FIG. 2.

Referring to FIG. 3, an exemplary state of an electronic device (101) displaying a preview image (320) using at least one camera (e.g., at least one camera (225) of FIG. 2) is illustrated. The electronic device (101) may display the screen of FIG. 3 on the display (120) while executing a software application (e.g., a camera application) for controlling at least one camera. The electronic device (101) may display, on the display (120), a preview image (320) based on at least a portion of an image acquired from at least one camera.

Referring to FIG. 3, together with the preview image (320), the electronic device (101) may display a visual object (310) mapped to a function for changing an option related to at least one camera. An image having a shape of a gear and/or an icon is illustrated as an example, but the embodiment is not limited thereto. In response to an input related to the visual object (310) (e.g., a touch input to a portion of the display (120) on which the visual object (310) is displayed), the electronic device (101) may display a screen (350) on the display (120). The screen (350) may include a settings screen provided by a software application for controlling at least one camera.

Referring to an exemplary screen (350) of FIG. 3, the electronic device (101) may provide an option (352) for checking whether to generate a file including information for displaying an image with an HDR effect when saving a file related to an image (e.g., a file (110) of FIG. 1). Using a visual object (354) (e.g., a radio button and/or a toggle switch) corresponding to the option (352), the electronic device (101) may receive an input for activating or deactivating the option (352). When the option (352) is activated by the visual object (354), the electronic device (101) may determine whether to save an image with an HDR effect based on a deviation in the amount of light in the external environment detected by at least one camera and/or a distribution.

Referring to FIG. 3, while displaying a preview image (320) based on at least a portion of an image provided from at least one camera, the electronic device (101) may display a visual object (330) indicating that an image supporting an HDR effect can be captured or acquired. The visual object (330) may include designated text, such as “HDR.” While the visual object (330) is displayed, in response to an input related to the visual object (330) (e.g., a touch input on a portion of the display (120) on which the visual object (330) is displayed), the electronic device (101) may stop displaying the visual object (330) on the display (120), or at least temporarily stop storing information for displaying an image in HDR according to the capturing input (e.g., map information (114) and/or display control information (116) of FIG. 1).

In the exemplary state of FIG. 3, the electronic device (101) may receive a shooting input. The shooting input may include an input for saving a file related to images that are being continuously acquired through at least one camera. For example, the shooting input may be detected by a gesture (e.g., a tap gesture) toward a visual object (340) displayed on the display (120). The visual object (340) may be referred to as a shooting button. Although a visual object (340) having a circular shape is illustrated as an example, the embodiment is not limited thereto.

For example, the photographing input may be detected by a gesture of pressing a button (342) (e.g., a button (342) for adjusting the volume of the electronic device (101)) exposed externally through one side of the electronic device (101) (e.g., a front side and a side connecting a rear side opposite to the front side).

For example, the photographing input may be detected in response to a body part (e.g., a palm) detected by at least one camera while displaying the preview image (320), and/or a user gesture related to the body part. For example, when a palm having an extended posture is detected using an image acquired from at least one camera, the electronic device (101) may determine that a gesture indicating a photographing input has been detected. The electronic device (101) may display an indicator (e.g., a visual object in the form of a rectangular line) indicating a location where the palm is detected within the preview image (320). In order to detect the palm from the image, the electronic device (101) may perform an algorithm for object recognition.

For example, the photographing input may be detected based on an audio signal acquired from a microphone of the electronic device (101). For example, if the electronic device (101) acquires a natural language sentence (e.g., “let’s take a picture” and/or “smile”) representing the photographing input from the audio signal, the electronic device (101) may determine that a voice command representing the photographing input has been detected. In order to recognize the natural language sentence, the electronic device (101) may process the audio signal acquired from the microphone by performing an algorithm such as STT (speech to text) while displaying the preview image (320) of FIG. 3.

An electronic device (101) that receives a photographing input can generate or store a file (e.g., a file (110) of FIG. 1 and/or FIG. 2) corresponding to the photographing input based on the operations described with reference to FIGS. 1 and 2. Hereinafter, an exemplary operation of an electronic device (101) that generates a file corresponding to a photographing input is described with reference to FIG. 3.

FIG. 4 illustrates an exemplary operation of an electronic device (101) that generates a file (110) using a plurality of images (411, 412, 413) acquired from at least one camera (225). The electronic device (101) of FIGS. 1 and 2, and/or the processor (210) of FIG. 2, may perform the operation of the electronic device (101) described with reference to FIG. 4. At least one camera (225) of FIG. 4 may correspond to at least one camera (225) of FIG. 2.

Referring to FIG. 4, in response to a photographing input, the electronic device may control at least one camera (225) to acquire a plurality of images (411, 412, 413). In one embodiment of acquiring a plurality of images (411, 412, 413) using a single camera, in order to acquire a plurality of images (411, 412, 413) having different brightness levels, the electronic device may change or adjust properties of the camera (e.g., exposure, ISO (International Standardization Organization) sensitivity, aperture opening degree (e.g., f-stop) and/or shutter speed) each time it acquires each of the plurality of images (411, 412, 413) from the camera.

In one embodiment of acquiring a plurality of images (411, 412, 413) using each of a plurality of cameras, the electronic device can assign different properties (e.g., exposure, ISO sensitivity, aperture opening degree, and/or shutter speed) to the plurality of cameras having at least partially overlapping FoVs, so as to acquire a plurality of images (411, 412, 413) from the plurality of cameras substantially simultaneously. The plurality of images (411, 412, 413) can have the same dynamic range (e.g., a dynamic range configured to represent 256 brightness levels based on a bit depth of 8 bits) while including information about colors at different brightness levels. For example, after acquiring a first image (411), in order to acquire a second image (412) having a different brightness level from the first image (411), the electronic device can control the camera so that the camera has a different property from at least one of a shutter speed, an ISO sensitivity, an aperture opening degree, or an exposure value of the camera at the time the camera acquired the first image (411). Using the camera having the above property, the electronic device can acquire the second image (412).

Referring to FIG. 4, brightness distributions of a plurality of images (411, 412, 413) acquired by at least one camera (225) are illustrated. When the electronic device acquires each of the plurality of images (411, 412, 413) while changing a property of the camera related to exposure (e.g., exposure value), the plurality of images (411, 412, 413) may have different brightness levels. For example, brightness values of the plurality of images (411, 412, 413) all have binary values between 0 and 255 because they have a consistent dynamic range, but may be mapped to external light of different intensities. When combining multiple images (411, 412, 413), an image (e.g., an image based on HDR) that expresses colors with a wider dynamic range than the dynamic range corresponding to each of the multiple images (411, 412, 413) can be generated.

Referring to FIG. 4, brightness levels of each of a plurality of images (411, 412, 413) are illustrated according to the intensity of external light. For example, the plurality of images (411, 412, 413) may include a first image (411) based on a brightness level having a minimum intensity of ea1 and a maximum intensity of ea2, a second image (412) based on a brightness level having a minimum intensity of eb1 and a maximum intensity of eb2, and a third image (413) based on a brightness level having a minimum intensity of ec1 and a maximum intensity of ec2. Referring to FIG. 4, ranges of brightness levels of the plurality of images (411, 412, 413) may overlap each other. According to one embodiment, at least one of the bit depths of the dynamic range of the first image (411), the second image (412), or the third image (413) may be formed to have the same value.

Referring to FIG. 4, the first image (411) may be acquired from at least one camera (225) that is controlled to have a shutter speed slower than a reference shutter speed, an exposure value greater than a reference exposure value, an aperture opening greater than a reference opening degree, and/or an ISO sensitivity more sensitive than a reference ISO sensitivity, in one embodiment of acquiring images (411, 412, 413) using one camera. The reference shutter speed, the reference exposure value, the reference opening degree, and/or the reference ISO sensitivity may be properties applied to at least one camera (225) to display the preview image (320) of FIG. 3. Since the first image (411) has a relatively large exposure value, a specific portion of the first image (411) that received relatively strong light (e.g., a portion corresponding to the sky and/or the sun) may be saturated, and only other portions of the first image (411) that received relatively weak light (e.g., a portion corresponding to the ground) may not be saturated. The specific portion of the first image (411) may be uniformly filled with a brightness value (e.g., 255) representing the maximum brightness. For example, within the first image (411), light stronger than ea2 may be mapped to a brightness value representing the maximum brightness.

Referring to FIG. 4, the second image (412) may be acquired from at least one camera (225) that is controlled to have a reference shutter speed, a reference exposure value, a reference aperture value, and/or a reference ISO sensitivity, in one embodiment of acquiring images (411, 412, 413) using one camera. Since the second image (412) has a lower exposure value than the first image (411), a portion that was saturated in the first image (411) may not be saturated in the second image (412). For example, a portion of the second image (412) corresponding to the sky may not be saturated. In the second image (412), a portion that received light stronger than eb2 may be filled with a brightness value representing the maximum brightness (e.g., 255), and a portion that received light weaker than eb1 may be filled with a brightness value representing the minimum brightness (e.g., 0). For example, a portion of the second image (412) corresponding to the ground may be darker than a corresponding portion of the first image (411), and a portion of the second image (412) corresponding to the sky may represent an external image that is clearer than a corresponding portion of the first image (411).

Referring to FIG. 4, the third image (413) may be acquired from at least one camera (225) that is controlled to have a shutter speed faster than a reference shutter speed, an exposure value smaller than a reference exposure value, an aperture opening smaller than a reference opening degree, and/or an ISO sensitivity less sensitive than a reference ISO sensitivity, in one embodiment of acquiring the images (411, 412, 413) using one camera. Since the third image (413) has a lower exposure value than the first image (411) and the second image (412), a portion (e.g., a portion corresponding to the sun) that was saturated in the first image (411) and the second image (412) may not be saturated in the third image (413). In the third image (413), a portion that received light stronger than ec2 may be filled with a brightness value representing the maximum brightness (e.g., 255), and a portion that received light weaker than ec1 may be filled with a brightness value representing the minimum brightness (e.g., 0). For example, in the third image (413), the portions corresponding to the sky and/or the ground may be uniformly filled with a brightness value representing the minimum brightness (e.g., 0) as they receive light weaker than ec1.

Referring to FIG. 4, by synthesizing a plurality of images (411, 412, 413) having a bit depth of SDR, the electronic device can synthesize an image (112) having a bit depth of SDR. For example, brightness values (or values of each channel) expressed using the same bit depth (e.g., 8 bits) may be assigned or stored to pixels of the images (411, 412, 413, 112). In one embodiment, the electronic device can generate the image (112) by synthesizing other images (e.g., the second image (412) and/or the third image (413)) with the darkest image (e.g., the first image (411) having the largest exposure value) among the plurality of images (411, 412, 413), or can obtain information (e.g., map information (114)) linked to the image (112).

In one embodiment, the electronic device may determine the first image (411) as an image (112) of a file (110) to be stored in response to a photographing input, and may obtain or generate map information (114) by synthesizing a plurality of images (411, 412, 413). Elements (e.g., pixel values) of the map information (114) may correspond to different portions of the image (112) and may include values related to brightness levels of the corresponding portions. Referring to FIG. 4, as the first image (411) is determined as the image (112), the image (112) may include brightness values that distinguish light that is stronger than es1 and weaker than es2. The electronic device may additionally acquire map information (114) that is stronger than es2 and is used to distinguish light weaker than the strongest light intensity eh that can be distinguished from multiple images (411, 412, 413). The map information (114) may have a form of a two-dimensional array. A width (w2) and/or a height (h2) of the two-dimensional array may be smaller than a width (w1) and/or a height (h1) of the image (112).

For example, differences between brightness values of pixels of the second image (412) and brightness values of pixels of the first image (411) may be stored in map information (114). For example, the electronic device may store differences between brightness values of pixels of the third image (413) and brightness values of pixels of the first image (411) in map information (114). For example, the electronic device may multiply brightness values of relatively bright second image (412) (e.g., having relatively small exposure value) by a weight to make brightness values of the second image (412) relatively large. The weight may be determined by properties of cameras used to acquire each of the first image (411) and the second image (412).

By performing a pixelwise subtraction difference operation (e.g., subtraction) on the brightness values of the weighted second image (412) and the brightness values of the first image (411), the electronic device can obtain the map information (114). In this case, in the map information (114), a numerical value representing the difference in brightness values of the second image (412) and the first image (411) can be stored in a non-saturated portion (e.g., a portion corresponding to the sky and/or the sun) in the second image (412), and a numerical value representing that a relatively dark portion (e.g., a portion corresponding to the ground) in the second image (412) maintains the brightness value of the first image (411) can be stored. As described above, the electronic device can use the multiple images (411, 412, 413) to generate map information (114) for a portion of the image (112) that is visually highlighted (e.g., a portion corresponding to the ground) while another portion of the image (112) is visually highlighted (e.g., a portion corresponding to the sky and/or the sun).

In one embodiment where three or more images (411, 412, 413) are acquired, the electronic device may generate or store map information indicating a difference in brightness value with respect to the first image (411), for each of the second image (412) and the third image (413). Since the map information (114) indicating only the brightness difference is generated, the resolution, the width (w2), and/or the height (h2) of the map information (114) may be smaller than the resolution, the width (w1), and/or the height (h1) of the image (112), respectively. The embodiment is not limited thereto, and the resolution, the width (w2), and/or the height (h2) of the map information (114) may be equal to the resolution, the width (w1), and/or the height (h1) of the image (112), respectively.

Although the operation of generating map information (114) representing an area for providing relatively high luminance has been described, the embodiment is not limited thereto. The electronic device may generate map information that emphasizes or represents a low luminance area. Alternatively, the electronic device may generate map information corresponding to an area in a luminance range between high luminance and low luminance.

Although an exemplary operation of obtaining map information (114) related to a brightness value (e.g., a Y value of a YUV color space) has been described, the embodiment is not limited thereto. In order to enhance color reproducibility, an electronic device that obtains a plurality of images (411, 412, 413) may perform the above-described operation for a chrominance component of a YUV color space (e.g., a chrominance component of a specified primary color such as green and/or blue) to obtain map information for the chrominance component. In this case, the map information obtained by the electronic device may be used to express a level of a primary color corresponding to a specific chrominance component using a relatively wide color space (e.g., color spaces of DCI-P3 and/or REC.2020 that are larger than a color space of sRGB). When the electronic device generates map information for each of a plurality of components of the color space, the map information corresponding to each of the components may have different resolutions, widths, heights, and/or bit depths.

The embodiment is not limited thereto, and the electronic device may perform the above-described operation for the primary color components of the RGB color space (e.g., the primary color components of red, green, and/or blue, respectively) to obtain map information for the primary color components. For example, for the image (112), the electronic device may independently generate map information for enhancing the brightness level of the blue component and map information for enhancing the brightness level of the red component. When additionally generating the map information for enhancing the brightness level of the blue component, the map information may be used to visualize the blue component of the image (112) in more detail. The embodiment is not limited thereto, and the electronic device may obtain map information for the color component of a specific wavelength (or a combination of a plurality of wavelengths). The map information may be used to display the image (112) using a color space extended from the color space corresponding to the image (112).

Based on the above-described operation, together with the generated map information (114), the electronic device may determine display control information (116) for changing the brightness level of a display device (e.g., a display device supporting HDR) when displaying a composite image (e.g., an image (130) having a dynamic range of HDR of FIG. 1) obtained by synthesizing the image (112) and the map information (114) and having a wider dynamic range than the dynamic range of SDR. The display control information (116) may be information for controlling the display device so that the composite image having a relatively wide dynamic range is displayed through pixels of the display device that are driven in a luminance range based on the relatively wide dynamic range. For example, the electronic device may determine display control information (116) for changing the brightness level of the display device so that the composite image is viewed as an image having a relatively wide dynamic range.

In one embodiment, the electronic device may generate display control information (116) indicating a degree to which the brightness of pixels of the display is increased (or boosted) when displaying an image (130). In terms of information used to increase the brightness of the pixels, the map information (114) may be referred to as a gain map, and/or the display control information (116) may be referred to as gain information. For example, the electronic device may generate or determine the display control information (116) by using properties (e.g., exposure value, shutter speed, ISO sensitivity, and/or aperture value) of a third image (413) having the highest exposure value (or the lowest aperture value) among the plurality of images (411, 412, 413). For example, the electronic device (101) may generate or determine the display control information (116) by using an aperture value of a camera controlled to capture the brightest subject among the images (411, 412, 413) (e.g., a maximum value among the aperture values corresponding to each of the images (411, 412, 413)).

The above aperture value may be increased by a specified multiple (e.g., 2 times) of the intensity of light received through the aperture each time the aperture value is increased by a unit stop. For example, as the aperture value increases, the intensity of light received through the aperture may increase, and the brightness of an image expressed by the light may increase. For example, as the aperture value decreases, the intensity of light received through the aperture may decrease, and the brightness of an image expressed by the light may decrease.

For example, a ratio between the size of the overall brightness levels of a plurality of images (411, 412, 413) acquired from a plurality of at least one camera (225) (e.g., a range between light intensities of ea1 to ec2) and the size of the brightness levels of an image (112) synthesized from the plurality of images (411, 412, 413) (e.g., a size of SDR, a range between light intensities of es1 to es2) can be determined as display control information (116). For example, the display control information (116) can be referred to as a ratio between dynamic ranges (e.g., an SDR-to-HDR ratio). The display control information (116) can be used to determine the luminance of a pixel of a display that will display an image having an HDR effect synthesized by the map information (114).

In one embodiment, the electronic device may generate or store a file (110) including display control information (116), metadata including map information (114), and an image (112). The map information (114) stored in the file (110) may be applied to the image (112) to be displayed by a display device supporting HDR, based on an event associated with the image (112) and/or the file (110). The display control information (116) may be used to change a brightness level of the display device displaying the image (112) to which the map information (114) is applied, based on the event. By generating auxiliary information (e.g., map information (114) and/or display control information (116)) to be used with the image (112) having a bit depth based on SDR (e.g., 8 bits), the electronic device may generate a file (110) supporting HDR effects while having a relatively small size. For example, since the resolution of the map information (114) is smaller than the resolution of the image (112), the size of the file (110) may be smaller than a file with increased bit depth.

In one embodiment, for compatibility, the file (110) may have a format such as JPEG. Below, with reference to FIG. 5, an exemplary structure of a file (110) including an image (112), map information (114), and display control information (116) is described.

FIG. 5 illustrates an exemplary structure of a file (110) generated by an electronic device according to one embodiment. The electronic device (101) of FIGS. 1 to 2 and/or the processor (210) of FIG. 2 may perform the operation of the electronic device (101) described with reference to FIG. 5. The file (110) of FIG. 5 may be generated by the operation of the electronic device (101) described with reference to FIGS. 3 to 4.

Referring to FIG. 5, a structure of a file (110) based on a format of ISO (International Standardization Organization) referred to as EXIF is illustrated. A file (110) stored in a memory (e.g., memory (215) of FIG. 2) may start from an area (M1) in which a designated value (e.g., a value of Table 1 below) for indicating the start of the file (110) is stored. After the area (M1), application areas (APPlication segments) (e.g., M2, M3, M4, ..., M10) may be formed in the file (110).

In the first application area (M2) of the file (110), tag information may be stored. The tag information may include one or more character strings used for indexing the file (110). In the second application area (APP2) of the file (110), content list data (M3) and/or stream data (M4 to M10) may be stored. Within the file (110), after the application areas (M2, M3, ..., M10), a JPEG table area (M11, M12, ..., M15) may be formed. The JPEG table area may include a Define-Quantization-Tables (DQT) area (M11) (e.g., a variable area starting with a value in Table 1 below), a Define-Huffman-Tables (DHT) area (M12) (e.g., a variable area starting with a value in Table 1 below), a Define-Restart-Interval (DRI) area (M13) (e.g., a fixed area starting with a value in Table 1 below), a Start of Frame (SOF) area (M14), and/or a Start-Of-Scan (SOS) area (M15) (e.g., a variable area starting with a value in Table 1 below). Within the file (110), a JPEG compressed data area (M16) may be formed after the JPEG table area. The file (110) may include, after the JPEG compressed data area (M16), an area (M17) (e.g., a fixed area starting with the value of Table 1 below) in which a designated value is stored to indicate the end of the file (110).

In one embodiment, information stored in a file (110) based on the format of EXIF is not limited to the example of FIG. 5. For example, the file (110) may include information having a name in Table 1.

Abbreviation Value in file (110) (hexadecimal based) Length of information (or payload) designation SOI 0xFF, 0xD8 doesn't exist Starting point of the image SOF0 0xFF, 0xC0 Variable Starting point of the frame (baseline DCT (discrete cosine transform)) SOF2 0xFF, 0xC2 Variable Starting point of the frame (progressive DCT) DHT 0xFF, 0xC4 Variable Definition of Huffman table DQT 0xFF, 0xDB Variable Definition of quantization table DRI 0xFF, 0xDD 4 bytes Definition of restart cycle SOS 0xFF, 0xDA Variable Starting point of scan RSTn 0xFF, 0xDn(n = 0, ..., 7) doesn't exist Restart point APPn 0xFF, 0xEn Variable Application areas (EXIF, APP1, etc.) COM 0xFF, 0xFE Variable Comment EOI 0xFF, 0xD9 doesn't exist End point of the image

In one embodiment, a composite image (e.g., image (112) of FIG. 1) obtained from images (e.g., multiple images (411, 412, 413) of FIG. 4) obtained from at least one camera (e.g., at least one camera (225) of FIG. 2) may be stored in a JPEG compressed data area (M16) of a file (110). Metadata may be stored in another area (e.g., M2 to M10) of the file (110) that is different from the JPEG compressed data area (M16). For example, shooting information of a camera used to restore and/or display an HDR image, map information (114), and/or display control information (116) may be stored in a second application area (APP2) of the file (110).

Below, an exemplary operation of an electronic device for controlling a display (e.g., the display (120) of FIG. 1 and/or FIG. 2) to display an image having an HDR effect is described, using an exemplary file (110) of FIG. 5.

FIG. 6 illustrates exemplary brightness levels of a display (120) of an electronic device (101) that displays an image included in a file. The electronic device (101) of FIGS. 1 to 2, and/or the processor (210) of FIG. 2, may perform the operations of the electronic device (101) described with reference to FIG. 6. The file of FIG. 6 may correspond to the file (110) of FIGS. 1 to 2, and/or FIG. 5.

Referring to FIG. 6, different states (601, 602) of an electronic device (101) displaying a screen for searching and/or displaying an image are illustrated. The electronic device (101) may display the screen of FIG. 6 on the display (120) while executing a software application (e.g., a gallery application) for viewing images and/or videos stored in a memory (e.g., memory (215) of FIG. 2). Referring to FIG. 6, the electronic device (101) may display visual objects (611, 612, 613) for switching the screen displayed on the display (120) along the bottom edge of the display (120). The visual object (611) may be mapped to a function for displaying a list of software applications that have been executed by the electronic device (101). The visual object (612) may be mapped to a function for switching to a designated screen referred to as a home screen (or launcher screen). A visual object (613) may be mapped to a function for switching to another screen that was displayed before the screen currently being displayed on the display (120).

In an exemplary state (601) of FIG. 6, the electronic device (101) can display thumbnail images corresponding to each of the images stored in the memory. In one embodiment in which the file (110) described with reference to FIGS. 1 to 5 is stored, the electronic device (101) can display a thumbnail image (619) corresponding to the file (110). The thumbnail image (619) can be displayed using pixels of the display (120) operating in SDR. For example, the electronic device (101) can generate or display the thumbnail image (619) using at least a portion of the image (112) having the SDR in the file (110). The thumbnail image (619) can have a smaller resolution, width, and/or height than the image (112) included in the file (110).

Within a state (601) of displaying a screen including a thumbnail image (619), the brightness of the display (120) of the electronic device (101) may be distributed within a relatively small size range based on SDR. Referring to FIG. 6, within the state (601), a minimum brightness (l5) and a maximum brightness (l7) of pixels of the display (120) are illustrated. Between the minimum brightness (l5) and the maximum brightness (l7), the pixels (e.g., pixels corresponding to the thumbnail image (619)) may be controlled according to 256 brightness levels (e.g., brightness levels represented according to a bit depth of 8 bits of SDR).

Within the exemplary state (601) of FIG. 6, while displaying a thumbnail image (619) having a first dynamic range based on SDR, the electronic device (101) may receive an input related to the thumbnail image (619). The input may include a touch input (e.g., a tap gesture) on a portion of the display (120) on which the thumbnail image (619) is displayed. The input may be performed to display an image corresponding to the thumbnail image (619) (e.g., the file (110) of FIG. 1 , and/or the image (112) within the file (110). For example, in response to the input, the electronic device (101) may detect an event for displaying the image having SDR while a brightness level of the display (120) is set to a first brightness level associated with SDR.

In response to an input for a thumbnail image (619) corresponding to a file supporting HDR (e.g., file (110) of FIG. 1), the electronic device (101) may switch to a state (602) for controlling the display (120) within HDR. Within the state (602), the electronic device (101) may synthesize an image (130) having an HDR effect using an image of SDR and metadata included in the file. Within the state (602) for displaying an execution screen on which the image (130) is arranged, the electronic device (101) may control pixels of a first display area of the display (120) corresponding to the image (130) so that the pixels of the first display area operate at a brightness level associated with HDR. Within state (602), the electronic device (101) can control pixels of a second display area of the display (120) (or a remaining display area of the display (120) that is different from the first display area) corresponding to a running screen adjacent to the image (130) (or different from the image (130)) to operate at a brightness level associated with a dynamic range (e.g., SDR) different from HDR.

Referring to FIG. 6, in a state (602) of displaying an image (130) based on HDR, the minimum brightness (l6) and the maximum brightness (l8) of the pixels of the display (120) are illustrated. The difference (or contrast ratio) between the maximum brightness (l8) and the minimum brightness (l6) may be greater than the difference (or contrast ratio) between the maximum brightness (l7) and the minimum brightness (l5) in the state (601) in which the display (120) is controlled based on SDR. Between the maximum brightness (l8) and the minimum brightness (l6), the pixels of the display (120) may be controlled by brightness levels exceeding 256 steps (e.g., 2 ¹⁰ = 1024) (e.g., brightness levels represented according to a bit depth of 10 bits of HDR).

Referring to FIG. 6, in the luminance domain, the minimum luminance (l5) in the state (601) in which the display (120) is controlled based on SDR and the minimum luminance (l6) in the state (602) in which the display (120) is controlled based on HDR may have a relatively small difference or may be the same. In the luminance domain, the maximum luminance (l7) in the state (601) in which the display (120) is controlled based on SDR and the maximum luminance (l8) in the state (602) in which the display (120) is controlled based on HDR may have a relatively large difference. For example, while controlled based on HDR, the electronic device (101) may control the display (120) using metadata of a file corresponding to the image (130) (e.g., display control information (116) of FIG. 1) such that at least one pixel of the display (120) corresponding to a portion of the image (130) having a minimum brightness has a minimum brightness (l6) of a dynamic range based on HDR. For example, the electronic device (101) may control the display (120) using the metadata such that at least one pixel of the display (120) corresponding to a portion of the image (130) having a maximum brightness has a maximum brightness (l8) of a dynamic range based on HDR.

In a state (602) where the display (120) is controlled based on HDR, the electronic device (101) can display the image (130) using a color space that is wider than the color space used in the state (601) where the thumbnail image (619) is displayed using SDR. For example, after switching from the state (601) where the thumbnail image (619) is displayed using the color space of sRGB (e.g., the color space (811) of FIG. 8) to the state (602), the electronic device (101) can display the image (130) using a color space wider than the color space (e.g., the color space (812) of FIG. 8 based on DCI-P3). Since the image (130) is displayed using the extended color space, the electronic device (101) can express the color of the image (130) in more detail.

Within a state (602) of controlling a display (120) based on HDR, the electronic device (101) may display a visual object (624) to guide visualization of an image (130) using HDR on the display (120). For example, a visual object (624) including designated text such as “HDR ON” is illustrated, but the embodiment is not limited thereto. Within the state (602), the electronic device (101) may display an area (621) including visual objects providing functions related to the image (130) (e.g., visual objects corresponding to functions such as like, rename, share, and/or delete).

In order to generate and/or synthesize an image (130) having an HDR effect from a file corresponding to a thumbnail image (619), the electronic device (101) may perform image processing on the file. Hereinafter, with reference to FIG. 7, an exemplary operation of the electronic device (101) performing image processing is described.

FIG. 7 illustrates an exemplary operation of an electronic device for generating information to be transmitted from a file to a display. The electronic device (101) of FIGS. 1 to 2, and/or the processor (210) of FIG. 2, may perform the operation of the electronic device (101) described with reference to FIG. 7. For example, the electronic device (101) may execute the image renderer (216) of FIG. 2, and/or the display luminance determiner (217) to perform the operation of FIG. 7. The file of FIG. 7 may correspond to the file (110) of FIGS. 1 to 2, and/or FIG. 5. The display of FIG. 7 may correspond to the display (120) of FIGS. 1 to 6.

Referring to FIG. 7, in response to an event to display an image (112) having an SDR of a file (110), the electronic device may obtain, from the metadata of the file (110), map information (114) for another portion of the image (112) that is visually emphasized for a portion of the image (112), and/or display control information (116) for changing a brightness level of the display in relation to the display of the image (112). Referring to FIG. 7, at operation (711), the electronic device may linearize a brightness component (e.g., Y values) of the image (112). At operation (712), the electronic device may linearize a brightness component (e.g., G_Y values) of the map information (114). The linearization of the operations (711, 712) may be performed to compensate for distortion of brightness components due to a curved (or non-linear) gamma curve used by an image signal processor (ISP) of a camera when generating an image (112) and/or map information (114). The linearization of the operations (711, 712) may be performed by a specified transfer function (e.g., an electro-optical transfer function (EOTF)).

In one embodiment, when map information (114) having a resolution, width, and/or height smaller than the resolution, width, and/or height of the image (112) is stored in the file (110), the electronic device may perform upscaling on the map information (114) to obtain upscaled map information corresponding to the resolution, width, and/or height of the image (112). In this case, the electronic device may perform operation (712) on the upscaled map information.

Referring to FIG. 7, within operation (720), the electronic device may obtain intermediate values (X, Y, Z) representing colors and/or brightness of pixels of the display by using at least one of linearized brightness components (e.g., brightness components of the image (112) and/or map information (114)), chrominance components of the image (112) (e.g., Cb values and/or Cr values), display control information (116), and state information (722) of the electronic device (or display) based on operations (711, 712). Within operation (730), the electronic device may generate color values (R1, G1, B1) in an RGB color space to be transmitted to the display (or the display driving circuit (122) of FIG. 2) from the intermediate values (X, Y, Z). Conversion between intermediate values (X, Y, Z) and color values (R1, G1, B1) of the motion (730) can be performed by specified transfer functions and/or mapping tables.

Within the operation (720), the electronic device can apply the luminance values (e.g., linearized G_Y values) of the linearized map information (114) to the luminance values (e.g., linearized Y values) of the linearized image (112) to generate luminance values having an increased bit depth. The luminance values can be combined with the chrominance components (e.g., Cb values and/or Cr values) of the image (112) to calculate intermediate values (X, Y, Z). The electronic device can calculate or determine the intermediate values (X, Y, Z) based on a luminance difference of the display, as recommended by the display control information (116) (e.g., SDR-to-HDR ratio). For example, if the display control information (116) includes a value indicating a 5x brightness increase, the electronic device may control the display to output light at 5x the brightness (e.g., 500 nits) of the maximum brightness in SDR (e.g., 100 nits) for the pixel having the maximum brightness within the image (112), or may calculate intermediate values (X, Y, Z) to output the light.

In one embodiment, the electronic device may calculate or determine intermediate values (X, Y, Z) to be used to control pixels of the display using state information (722) of the electronic device and/or the display. The state information (722) may include at least one of a brightness of the display, a range of brightness (or brightness levels) that the display can output, a SOC of a battery (e.g., battery 235 of FIG. 2), a mode of the electronic device related to the SOC (e.g., a low power mode), and an on pixel ratio (OPR) of the display. The OPR may be a ratio between the number of pixels turned on to display an image (or pixels turned on to have maximum brightness, such as white) and the number of pixels in the entire display area of the display (or the number of pixels turned off).

Referring to FIG. 7, in operations (741, 742, 743), the electronic device (or the display driving circuit (122) of the electronic device) may apply transfer functions (e.g., EOTF1, EOTF2, and/or EOTF3) corresponding to each of the primary color components of the RGB color space to each of the color values (R1, G1, B1) generated by operation (730), to obtain voltage values (R1', G1', B1') to be input to at least one of the pixels of the display (e.g., the pixels arranged on the display panel (124) of FIG. 2). The EOTF may be a function that defines a conversion relationship between the color values (R1, G1, B1) generated by operation (730) and which are electrical information, and information for outputting optical brightness (e.g., voltage values (R1', G1', B1')). For example, for the same color values (R1, G1, B1), the voltage values calculated in operations (741, 742, 743) when the display operates in HDR may be larger than the voltage values calculated in operations (741, 742, 743) when the display operates in SDR. For example, voltage values (R1', G1', B1') corresponding to a pixel exhibiting maximum brightness in the image (112) (e.g., a pixel having a brightness value of 255) may be calculated or determined as relatively large voltage values when the display operates in HDR.

In operations (741, 742, 743), the electronic device (e.g., the display driving circuit (122) of FIG. 2) may calculate or determine the voltage values (R1', G1', B1') by applying a gamma curve based on the characteristics of the display and/or the display control information (116). The gamma curve may include a hybrid log gamma (HLG) curve and/or a perceptual quantization (PQ) curve. For example, the processor of the electronic device may transmit the gamma characteristic indicated by the display control information (116) to the display driving circuit, so that the display driving circuit applies the voltage values (R1', G1', B1') based on the gamma characteristic to at least one pixel.

As described above, according to one embodiment, the electronic device can perform an operation to visualize an image of an HDR effect having a bit depth greater than the bit depth from a file (110) including an image (112) having a bit depth of 8 bits. The operation can be performed using metadata stored in the file (110) to synthesize or restore an image of the HDR effect. Since the metadata is generated using a relatively small size (e.g., map information (114) having a relatively small resolution, width, and/or height), the electronic device can perform a function related to HDR using a file (110) having a relatively small capacity.

Hereinafter, with reference to FIG. 8, an exemplary operation of an electronic device that performs the operation of FIG. 7 to display an image with an HDR effect is described.

FIG. 8 illustrates exemplary states of an electronic device (101) displaying an image (130) in HDR mode. The electronic device (101) of FIGS. 1 to 2, and/or the processor (210) of FIG. 2, may perform the operation of the electronic device (101) described with reference to FIG. 8. Referring to FIG. 8, an exemplary state (602) of the electronic device (101) displaying an image (130) with HDR effect using the file (110) of FIG. 1 is illustrated. The state (602) of FIG. 8 may correspond to the state (602) of FIG. 6. Within the state (602), the electronic device may apply map information (e.g., map information (114) of FIG. 1) to an image of SDR stored in the file (110) to generate an image (130) for HDR effect. Within the state (602), the electronic device (101) can set the brightness level of the display to a second brightness level higher than the first brightness level of the SDR based on the display control information (e.g., the display control information (116) of FIG. 1) in the file (110), and display the image (130) using the second brightness level. For example, the electronic device (101) can display the image (130) set to the second brightness level so that an image having a wider dynamic range than the dynamic range of the SDR is shown on the display.

Referring to FIG. 8, the electronic device (101) may receive an input to display an image (112) having an SDR of a file (e.g., file (110) of FIG. 1) used to display the image (130) instead of an image (130) having an HDR effect. The input may include a touch input (e.g., a long touch gesture based on a finger in contact for more than about 1.5 seconds) on a portion of the display (120) on which the image (130) is displayed, within the state (602). The input may include an input to segment a portion associated with a subject within the image (130) (e.g., a portion representing a separable subject from the image (130), such as a tree and/or the sun, within the exemplary image (130) of FIG. 8). Within state (602) of FIG. 8, the electronic device (101) may switch to state (803) in response to the input.

Within the exemplary state (803) of FIG. 8, the electronic device (101) may display an image (112) having a dynamic range of SDR on the display (120). The image (112) may be displayed on a location of the display (120) where the image (130) was displayed within the state (602). The image (112) may be displayed on a portion of the display (120) that received the touch input. Within the state (803) of displaying the image (112) having SDR, the electronic device (101) may display a visual object (e.g., a visual object (831) including designated text such as “HDR OFF”) indicating that the image (112) of SDR is displayed on the display (120). In a state (803) of displaying an image (112) having SDR, the luminance range between the minimum luminance (l7) and the maximum luminance (l9) of the pixels of the display may be related to the dynamic range of the SDR.

The state (803) of displaying the image (112) having the SDR of Fig. 8 can be maintained by a touch input on the display (120) of the electronic device (101). For example, after detecting a long touch input on a portion of the display (120) on which the image (130) is displayed in the state (602), while the long touch input is maintained, the electronic device (101) can enter the state (803) and display the image (112) having the SDR. In the state (803), when the long touch input is released, the electronic device (101) can switch back to the state (602) and display the image (130) having the HDR effect again.

Referring to FIG. 8, color spaces used in each of a state (602) of displaying an image (130) having an HDR effect and a state (803) of displaying an image (112) having an SDR are illustrated. In the state (803) of displaying an image (112) having an SDR, the electronic device (101) can control the display (120) based on a color space (811) of sRGB. In the state (602) of displaying an image (130) having an HDR effect, the electronic device (101) can display the image (130) using a color space wider than the color space (811) of sRGB (e.g., a color space (812) based on DCI-P3 and/or a color space (813) wider than the color space (812) above).

As described above, according to one embodiment, the electronic device (101) may generate or store a file (e.g., file (110) of FIG. 1) including metadata (e.g., map information (114) of FIG. 1 and/or display control information (116)) used to synthesize an image of an HDR effect using images of SDR (e.g., multiple images (411, 412, 413) of FIG. 4). The electronic device (101) may restore or synthesize an image of an HDR effect (e.g., image (130) of FIG. 1) from the file. When displaying the image of the HDR effect, the electronic device (101) may use the metadata in the file to increase luminance and/or brightness levels of pixels of the display (e.g., display (120) of FIG. 1 and/or FIG. 2) to visually emphasize at least a portion of the image more than other portions.

FIG. 9 is a block diagram of an electronic device (901) in a network environment (900) according to various embodiments. Referring to FIG. 9, in the network environment (900), the electronic device (901) may communicate with the electronic device (902) via a first network (998) (e.g., a short-range wireless communication network) or may communicate with at least one of the electronic device (904) or the server (908) via a second network (999) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (901) may communicate with the electronic device (904) via the server (908). According to one embodiment, the electronic device (901) may include a processor (920), a memory (930), an input module (950), an audio output module (955), a display module (960), an audio module (970), a sensor module (976), an interface (977), a connection terminal (978), a haptic module (979), a camera module (980), a power management module (988), a battery (989), a communication module (990), a subscriber identification module (996), or an antenna module (997). In some embodiments, the electronic device (901) may omit at least one of these components (e.g., the connection terminal (978)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (976), the camera module (980), or the antenna module (997)) may be integrated into one component (e.g., the display module (960)).

The processor (920) may control at least one other component (e.g., a hardware or software component) of the electronic device (901) connected to the processor (920) by executing, for example, software (e.g., a program (940)), and may perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculations, the processor (920) may store a command or data received from another component (e.g., a sensor module (976) or a communication module (990)) in the volatile memory (932), process the command or data stored in the volatile memory (932), and store result data in the nonvolatile memory (934). According to one embodiment, the processor (920) may include a main processor (921) (e.g., a central processing unit or an application processor) or an auxiliary processor (923) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (921). For example, when the electronic device (901) includes the main processor (921) and the auxiliary processor (923), the auxiliary processor (923) may be configured to use less power than the main processor (921) or to be specialized for a given function. The auxiliary processor (923) may be implemented separately from the main processor (921) or as a part thereof.

The auxiliary processor (923) may control at least a portion of functions or states associated with at least one of the components of the electronic device (901) (e.g., the display module (960), the sensor module (976), or the communication module (990)), for example, on behalf of the main processor (921) while the main processor (921) is in an inactive (e.g., sleep) state, or together with the main processor (921) while the main processor (921) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (923) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (980) or a communication module (990)). In one embodiment, the auxiliary processor (923) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. Such learning may be performed, for example, in the electronic device (901) itself on which the artificial intelligence model is executed, or may be performed through a separate server (e.g., server (908)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers. The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), deep Q-networks, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may additionally or alternatively include a software structure.

The memory (930) can store various data used by at least one component (e.g., the processor (920) or the sensor module (976)) of the electronic device (901). The data can include, for example, software (e.g., the program (940)) and input data or output data for commands related thereto. The memory (930) can include a volatile memory (932) or a nonvolatile memory (934).

The program (940) may be stored as software in memory (930) and may include, for example, an operating system (942), middleware (944), or an application (946).

The input module (950) can receive commands or data to be used for components of the electronic device (901) (e.g., processor (920)) from an external source (e.g., a user) of the electronic device (901). The input module (950) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

The audio output module (955) can output an audio signal to the outside of the electronic device (901). The audio output module (955) can include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback. The receiver can be used to receive an incoming call. According to one embodiment, the receiver can be implemented separately from the speaker or as a part thereof.

The display module (960) can visually provide information to an external party (e.g., a user) of the electronic device (901). The display module (960) can include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (960) can include a touch sensor configured to detect a touch, or a pressure sensor configured to measure a strength of a force generated by the touch.

The audio module (970) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (970) can obtain sound through the input module (950), or output sound through an audio output module (955), or an external electronic device (e.g., an electronic device (902)) (e.g., a speaker or a headphone) directly or wirelessly connected to the electronic device (901).

The sensor module (976) can detect an operating state (e.g., power or temperature) of the electronic device (901) or an external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (976) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface (977) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (901) with an external electronic device (e.g., the electronic device (902)). In one embodiment, the interface (977) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal (978) may include a connector through which the electronic device (901) may be physically connected to an external electronic device (e.g., the electronic device (902)). According to one embodiment, the connection terminal (978) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

The haptic module (979) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that a user can perceive through a tactile or kinesthetic sense. According to one embodiment, the haptic module (979) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module (980) can capture still images and moving images. According to one embodiment, the camera module (980) can include one or more lenses, image sensors, image signal processors, or flashes.

The power management module (988) can manage power supplied to the electronic device (901). According to one embodiment, the power management module (988) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

The battery (989) can power at least one component of the electronic device (901). In one embodiment, the battery (989) can include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

The communication module (990) may support establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (901) and an external electronic device (e.g., the electronic device (902), the electronic device (904), or the server (908)), and performance of communication through the established communication channel. The communication module (990) may operate independently from the processor (920) (e.g., the application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (990) may include a wireless communication module (992) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (994) (e.g., a local area network (LAN) communication module or a power line communication module). Any of these communication modules may communicate with an external electronic device (904) via a first network (998) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (999) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (992) may use subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (996) to identify or authenticate the electronic device (901) within a communication network such as the first network (998) or the second network (999).

The wireless communication module (992) can support a 5G network after a 4G network and next-generation communication technology, for example, NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), terminal power minimization and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (992) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (992) may support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (992) may support various requirements specified in the electronic device (901), an external electronic device (e.g., the electronic device (904)), or a network system (e.g., the second network (999)). According to one embodiment, the wireless communication module (992) may support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL) each, or 1 ms or less for round trip) for URLLC realization.

The antenna module (997) can transmit or receive signals or power to or from the outside (e.g., an external electronic device). According to one embodiment, the antenna module (997) can include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (997) can include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (998) or the second network (999), can be selected from the plurality of antennas by, for example, the communication module (990). A signal or power can be transmitted or received between the communication module (990) and the external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) can be additionally formed as a part of the antenna module (997).

According to various embodiments, the antenna module (997) can form a mmWave antenna module. According to one embodiment, the mmWave antenna module can include a printed circuit board, an RFIC positioned on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) positioned on or adjacent a second side (e.g., a top side or a side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

At least some of the above components may be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, a general purpose input and output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)).

In one embodiment, commands or data may be transmitted or received between the electronic device (901) and an external electronic device (904) via a server (908) connected to a second network (999). Each of the external electronic devices (902, or 704) may be the same or a different type of device as the electronic device (901). In one embodiment, all or part of the operations executed in the electronic device (901) may be executed in one or more of the external electronic devices (902, 704, or 708). For example, when the electronic device (901) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (901) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform at least a part of the function or service. One or more external electronic devices that have received the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (901). The electronic device (901) may process the result as it is or additionally and provide it as at least a part of a response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (901) may provide an ultra-low latency service by using, for example, distributed computing or mobile edge computing. In one embodiment, the external electronic device (904) may include an IoT (Internet of Things) device. The server (908) may be an intelligent server using machine learning and/or a neural network. According to one embodiment, the external electronic device (904) or the server (908) may be included in the second network (999). The electronic device (901) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

FIG. 10 is a block diagram (1000) illustrating a camera module (980) according to various embodiments. Referring to FIG. 10, the camera module (980) may include a lens assembly (1010), a flash (1020), an image sensor (1030), an image stabilizer (1040), a memory (1050) (e.g., a buffer memory), or an image signal processor (1060). The lens assembly (1010) may collect light emitted from a subject that is a target of an image capture. The lens assembly (1010) may include one or more lenses. According to one embodiment, the camera module (980) may include a plurality of lens assemblies (1010). In this case, the camera module (980) may form, for example, a dual camera, a 360-degree camera, or a spherical camera. Some of the plurality of lens assemblies (1010) may have the same lens properties (e.g., angle of view, focal length, autofocus, f number, or optical zoom), or at least one lens assembly may have one or more lens properties that are different from the lens properties of the other lens assemblies. A lens assembly (1010) may include, for example, a wide-angle lens or a telephoto lens.

The flash (1020) can emit light used to enhance light emitted or reflected from a subject. According to one embodiment, the flash (1020) can include one or more light-emitting diodes (e.g., red-green-blue (RGB) LEDs, white LEDs, infrared LEDs, or ultraviolet LEDs), or a xenon lamp. The image sensor (1030) can acquire an image corresponding to the subject by converting light emitted or reflected from the subject and transmitted through the lens assembly (1010) into an electrical signal. According to one embodiment, the image sensor (1030) can include one image sensor selected from among image sensors having different properties, such as an RGB sensor, a black and white (BW) sensor, an IR sensor, or a UV sensor, a plurality of image sensors having the same property, or a plurality of image sensors having different properties. Each image sensor included in the image sensor (1030) may be implemented using, for example, a CCD (charged coupled device) sensor or a CMOS (complementary metal oxide semiconductor) sensor.

The image stabilizer (1040) can move at least one lens or image sensor (1030) included in the lens assembly (1010) in a specific direction or control the operating characteristics of the image sensor (1030) (e.g., adjusting read-out timing, etc.) in response to movement of the camera module (980) or the electronic device (901) including the same. This allows compensating for at least some of the negative effects of the movement on the captured image. In one embodiment, the image stabilizer (1040) can detect such movement of the camera module (980) or the electronic device (901) by using a gyro sensor (not shown) or an acceleration sensor (not shown) disposed inside or outside the camera module (980). In one embodiment, the image stabilizer (1040) can be implemented as, for example, an optical image stabilizer. The memory (1050) can temporarily store at least a portion of an image acquired through the image sensor (1030) for the next image processing task. For example, when image acquisition is delayed due to a shutter, or when a plurality of images are acquired at high speed, the acquired original image (e.g., a Bayer-patterned image or a high-resolution image) is stored in the memory (1050), and a corresponding copy image (e.g., a low-resolution image) can be previewed through the display module (960). Thereafter, when a specified condition is satisfied (e.g., a user input or a system command), at least a portion of the original image stored in the memory (1050) can be acquired and processed by, for example, the image signal processor (1060). According to one embodiment, the memory (1050) can be configured as at least a portion of the memory (930), or as a separate memory that operates independently therefrom.

The image signal processor (1060) can perform one or more image processing operations on an image acquired through an image sensor (1030) or an image stored in a memory (1050). The one or more image processing operations may include, for example, depth map generation, 3D modeling, panorama generation, feature point extraction, image synthesis, or image compensation (e.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softening). Additionally or alternatively, the image signal processor (1060) may perform control (e.g., exposure time control, or read-out timing control, etc.) for at least one of the components included in the camera module (980) (e.g., the image sensor (1030)). The image processed by the image signal processor (1060) may be stored back in the memory (1050) for further processing or provided to an external component of the camera module (980) (e.g., the memory (930), the display module (960), the electronic device (902), the electronic device (904), or the server (908)). According to one embodiment, the image signal processor (1060) may It may be configured as at least a part of the processor (920), or may be configured as a separate processor that operates independently of the processor (920). When the image signal processor (1060) is configured as a separate processor from the processor (920), at least one image processed by the image signal processor (1060) may be displayed through the display module (960) as is or after undergoing additional image processing by the processor (920).

According to one embodiment, the electronic device (901) may include a plurality of camera modules (980), each having different properties or functions. In this case, for example, at least one of the plurality of camera modules (980) may be a wide-angle camera and at least another may be a telephoto camera. Similarly, at least one of the plurality of camera modules (980) may be a front camera and at least another may be a rear camera.

FIG. 11 is a block diagram (1100) of a display module (960) according to various embodiments. Referring to FIG. 11, the display module (960) may include a display (1110) and a display driver IC (DDI) (1130) for controlling the display (1110). The DDI (1130) may include an interface module (1131), a memory (1133) (e.g., a buffer memory), an image processing module (1135), or a mapping module (1137). The DDI (1130) may receive image information including, for example, image data or an image control signal corresponding to a command for controlling the image data, from another component of the electronic device (901) through the interface module (1131). For example, according to one embodiment, image information may be received from a processor (920) (e.g., a main processor (921) (e.g., an application processor) or an auxiliary processor (923) (e.g., a graphic processing unit) that operates independently of the function of the main processor (921). The DDI (1130) may communicate with a touch circuit (1150) or a sensor module (976) through the interface module (1131). In addition, the DDI (1130) may store at least some of the received image information in the memory (1133), for example, in units of frames. The image processing module (1135) may perform preprocessing or postprocessing (e.g., resolution, brightness, or size adjustment) on at least some of the image data based on at least the characteristics of the image data or the characteristics of the display (1110), for example. The mapping module (1137) may generate a voltage value or a current value corresponding to the image data preprocessed or postprocessed through the image processing module (1135). There is. According to one embodiment, the generation of the voltage value or current value may be performed at least in part based on, for example, properties of pixels of the display (1110) (e.g., arrangement of pixels (RGB stripe or pentile structure), or size of each sub-pixel). At least some pixels of the display (1110) may be driven at least in part based on, for example, the voltage value or current value, so that visual information (e.g., text, image, or icon) corresponding to the image data may be displayed through the display (1110).

According to one embodiment, the display module (960) may further include a touch circuit (1150). The touch circuit (1150) may include a touch sensor (1151) and a touch sensor IC (1153) for controlling the same. The touch sensor IC (1153) may control the touch sensor (1151) to detect, for example, a touch input or a hovering input for a specific location of the display (1110). For example, the touch sensor IC (1153) may detect the touch input or the hovering input by measuring a change in a signal (e.g., voltage, light amount, resistance, or charge amount) for a specific location of the display (1110). The touch sensor IC (1153) may provide information (e.g., location, area, pressure, or time) about the detected touch input or hovering input to the processor (920). According to one embodiment, at least a portion of the touch circuit (1150) (e.g., the touch sensor IC (1153)) may be included as part of the display driver IC (1130), or as part of the display (1110), or as part of another component (e.g., the auxiliary processor (923)) positioned external to the display module (960).

According to one embodiment, the display module (960) may further include at least one sensor (e.g., a fingerprint sensor, an iris sensor, a pressure sensor, or an illuminance sensor) of the sensor module (976), or a control circuit therefor. In this case, the at least one sensor or the control circuit therefor may be embedded in a part of the display module (960) (e.g., the display (1110) or the DDI (1130)) or a part of the touch circuit (1150). For example, when the sensor module (976) embedded in the display module (960) includes a biometric sensor (e.g., a fingerprint sensor), the biometric sensor may obtain biometric information (e.g., a fingerprint image) associated with a touch input through a part of the display (1110). For another example, when the sensor module (976) embedded in the display module (960) includes a pressure sensor, the pressure sensor may obtain pressure information associated with a touch input through a part or the entire area of the display (1110). According to one embodiment, the touch sensor (1151) or the sensor module (976) may be disposed between pixels of a pixel layer of the display (1110), or above or below the pixel layer.

The electronic devices according to various embodiments disclosed in this document may be devices of various forms. The electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliance devices. The electronic devices according to embodiments of this document are not limited to the above-described devices.

It should be understood that the various embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly dictates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first", "second", or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first) is referred to as "coupled" or "connected" to another (e.g., a second) component, with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

The term "module" used in various embodiments of this document may include a unit implemented in hardware, and may be used interchangeably with terms such as logic, block, component, or circuit, for example. A module may be an integrally configured component, or a minimum unit of the component, or a part thereof, that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document may be implemented as software (e.g., a program (940)) including one or more instructions stored in a storage medium (e.g., an internal memory (936) or an external memory (938)) readable by a machine (e.g., an electronic device (901)). For example, a processor (e.g., a processor (920)) of the machine (e.g., the electronic device (901)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g. electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a commodity. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a part of the computer program product may be at least temporarily stored or temporarily generated in a machine-readable storage medium, such as a memory of a manufacturer's server, a server of an application store, or an intermediary server.

According to various embodiments, each component (e.g., a module or a program) of the above-described components may include a single or multiple entities, and some of the multiple entities may be separately arranged in other components. According to various embodiments, one or more of the components or operations of the above-described components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, the multiple components (e.g., a module or a program) may be integrated into one component. In this case, the integrated component may perform one or more functions of each of the multiple components identically or similarly to those performed by the corresponding component of the multiple components before the integration. According to various embodiments, the operations performed by the module, program, or other components may be sequentially, in parallel, repeatedly, or heuristically executed, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added. The electronic device (1001) of FIG. 9 may be an example of the electronic device (101) of FIGS. 1 to 8.

FIGS. 12A, 12B, and 12C illustrate exemplary operations performed by an electronic device (1201) including a housing foldable by a folding axis (f) to display an image (1230) in HDR mode. Referring to FIGS. 12A to 12C , the electronic device (1201) according to one embodiment may include a deformable housing (e.g., a foldable housing). The electronic device (1201) may be an example of the electronic device (101) described with reference to FIGS. 1 to 8 .

Referring to FIGS. 12A to 12C, the foldable housing of the electronic device (1201) may include a first housing part (1211), a second housing part (1212), and a hinge part (1213) configured to rotatably couple the first housing part (1211) and the second housing part (1212). The first housing part (1211) and the second housing part (1212) may be referred to as a rigid part (or rigid part) of the foldable housing. The hinge part (1213) may be referred to as a bendable part (or bendable part) of the foldable housing, or a foldable part (or foldable part). The electronic device (1201) may include a flexible display (1220) that is visible from a front side of the foldable housing.

Referring to FIGS. 12A to 12C, the flexible display (1220) may extend from the first housing part (1211) across the hinge part (1213) to the second housing part (1212). A display area (or active area) of the flexible display (1220), defined by pixels of the flexible display (1220), may include a first display area (1221) positioned on the first housing part (1211), a second display area (1222) positioned on the second housing part (1212), and a third display area (1223) positioned on the hinge part (1213).

For example, the first display area (1221) and the second display area (1222) can maintain a flat shape independently of rotation (or folding) at the hinge part (1213) because they are positioned on the rigid part of the foldable housing. The third display area (1223) can be folded or unfolded by rotation at the hinge part (1213) because it is positioned on the foldable part of the foldable housing. The first display area (1221) and the second display area (1222) can be referred to as a flat portion of the flexible display (1220). The third display area (1223) can be referred to as a folding portion of the flexible display (1220). The flexible display (1220) can be referred to as a foldable display.

Referring to FIG. 12A, an exemplary state (1291) of an electronic device (1201) including a flexible display (1220) is illustrated. The first housing part (1211) can be rotated relative to the second housing part (1212) by rotation at the hinge part (1213) including the folding axis (f). The electronic device (1201) can detect or calculate an angle (a) between the first housing part (1211), the hinge part (1213), and the second housing part (1212) that is rotated based on the folding axis (f) using a sensor, such as a Hall sensor and/or an inertial measurement unit (IMU).

In an exemplary state (1291) of FIG. 12A, the entire display area of the flexible display (1220) of the electronic device (1201) may have a form of a single plane. In the state (1291), the angle (a) between the first housing part (1211), the hinge part (1213), and the second housing part (1212) may be substantially flat (e.g., included in an angle range including 180º or 180º). The state (1291) in which the first housing part (1211), the hinge part (1213), and the second housing part (1212) are arranged such that the entire display area of the flexible display (1229) has a form of a single plane may be referred to as an unfolded state, an unfolded state, a flat state, an outspread state, and/or a fully unfolded state.

Referring to FIG. 12A, an electronic device (1201) can control a display (1220) supporting an HDR mode to display an image (1230) having an HDR effect. The electronic device (1201) can obtain a first image (e.g., an image (112) of FIG. 1) having a first dynamic range (e.g., a dynamic range of SDR), map information related to brightness and color gradation of at least a portion of the first image (e.g., map information (114) of FIG. 1), and/or display control information related to a brightness level of a display (e.g., a flexible display (1220)) when displaying the first image (e.g., display control information (116) of FIG. 1).

In one embodiment, the first image, the map information, and the display control information may be stored in one file (e.g., file (110) of FIG. 1). For example, the map information and/or the display control information may be included as metadata in a file including the first image. The embodiment is not limited thereto. For example, the map information and the display control information may be included in, or stored in, a different file from the file including the first image. For example, the electronic device (1201) may generate the display control information based on the map information.

According to one embodiment, the electronic device (1201) may generate or synthesize a second image (e.g., the image (130) of FIG. 1) based on the first image and the map information. The map information used to synthesize the second image may include information about differences in brightness and/or color gradation between at least a portion of the first image and at least a portion of a third image captured of the same object based on a different exposure value than when the first image was captured. For example, the map information may represent a two-dimensional array including values representing brightness levels of different portions included in the first image, and/or may be used to generate the two-dimensional array. The horizontal pixel number and the vertical pixel number of the two-dimensional array may be smaller than the horizontal pixel number and the vertical pixel number of the first image, respectively.

In one embodiment, the display control information for the first image may include at least one of: maximum brightness information required to display a second image synthesized from the first image to have a second dynamic range wider than the first dynamic range; and/or ratio information between the maximum brightnesses required to display the first image.

Based on display control information corresponding to a first image having a first dynamic range such as SDR, the electronic device (1201) can determine or calculate a brightness level of at least a portion of a display (e.g., a flexible display (1220)). The electronic device (1201) can control the display such that the at least a portion of the display has the determined brightness level, so that the first image (or a second image synthesized by applying map information to the first image) appears to have a second dynamic range wider than the first dynamic range. The electronic device (1201) can display the second image on the display, the at least a portion of which has the determined brightness level.

Referring to FIG. 12A, the electronic device (1201) can display an image (1230) having a second dynamic range. The image (1230) can correspond to a second image synthesized by applying map information to a first image. The electronic device (1201) can enter a state (1291) of displaying the image (1230) of FIG. 12A in response to an input related to a thumbnail image (e.g., thumbnail image (619) of FIG. 6) having a size smaller than that of the first image.

In one embodiment, while entering state (1291) based on an input related to a thumbnail image, the electronic device (1201) can display, at least temporarily, a first image having a first dynamic range on the display (1220). For example, while a second image having a second dynamic range is synthesized based on the first image and the map information, the electronic device (1201) can display the first image. Referring to FIG. 12A, the electronic device (1201) can display the first image on a portion of the flexible display (1220) on which the image (1230) is displayed. In response to synthesizing the second image, the electronic device (1201) can change the first image being displayed on the flexible display (1220) to the second image. By changing the first image to the second image, the electronic device (1201) can display the second image (e.g., image (1230)).

Referring to FIG. 12a, in a state (1291) of displaying an image (1230) having a second dynamic range, synthesized from a first image having a first dynamic range, the electronic device (1201) may display a designated visual object (1232) (e.g., a visual affordance including designated text such as “HDR ON”) indicating that the image (1230) is displayed based on an HDR mode. The visual object (1232) may be an example of the visual object (624) of FIG. 6.

Referring to FIG. 12A, brightness distributions of different portions of the flexible display (1220) (e.g., the first display area (1221), the second display area (1222), and the third display area (1223)) are illustrated. The electronic device (1201) can control the flexible display (1220) such that at least one pixel of the flexible display (1220), corresponding to a portion of the first image having a minimum brightness of the first dynamic range, has a minimum brightness la1 of the second dynamic range. Similarly, the electronic device (1201) can control the flexible display (1220) such that at least one other pixel of the flexible display (1220), corresponding to another portion of the first image having a maximum brightness of the first dynamic range, has a maximum brightness of the second dynamic range.

In one embodiment, within a state (1291) of displaying an image (1230) having a second dynamic range, the electronic device (1201) can control pixels of a portion of a display area of the flexible display (1220) on which the image (1230) is displayed, based on a brightness level indicated by map information and/or display control information. Within the state (1291), the electronic device (1201) can control pixels of a remaining portion of the display area of the flexible display (1220) on which the image (1230) is displayed, other than the portion, based on a different brightness level lower than the brightness level. A brightness level used to control pixels of a portion of a display area of a flexible display (1220) on which an image (1230) is displayed may be determined based on at least one of a range of brightness levels displayable by the flexible display (1220), the SOC of the electronic device (1201), and/or the OPR of the flexible display (1220).

According to one embodiment, the electronic device (1201) may determine or calculate different brightness levels of different portions of the flexible display (1220) based on a state of the foldable housing (e.g., a folding state) and/or information related to the image (1230) (e.g., display control information). Referring to FIG. 12A, the third display area (1223), which is a folding portion of the electronic device (1201), may be controlled within a range of brightness levels having a minimum brightness of la1 and a maximum brightness of la3. Meanwhile, the electronic device (1201) may control the first display area (1221) (and/or the second display area (1222)), which is a flat portion, within another range that is different from the range of brightness levels of the third display area (1223), which is a folding portion, or that at least partially overlaps with the range of brightness levels of the third display area (1223). For example, the electronic device (1201) can control the third display area (1223) within a range of brightness levels having a minimum brightness of la1 and a maximum brightness of la2, and can control the first display area (1221) and/or the second display area (1222) within a range of brightness levels having a minimum brightness of la1 and a maximum brightness of la3 (e.g., a maximum brightness exceeding la2). The embodiment is not limited thereto, and the electronic device (1201) can control the first display area (1221) (and/or the second display area (1222)), which is a flat surface, within a range of brightness levels having a minimum brightness of la1 and a maximum brightness of la2.

For example, the electronic device (1201) can determine ranges of brightness levels of portions of the display areas of the flexible display (1220), respectively, such that the brightness level of the third display area (1223) is higher than the brightness levels of the first display area (1221) and/or the second display area (1222). Using the determined ranges of brightness levels, the electronic device (1201) can display an image (1230) displayed through the portions of the display areas.

In one embodiment, the electronic device (1201) can at least partially change brightness levels (or dynamic ranges) of portions of a display area of the flexible display (1220) (e.g., the first display area (1221), the second display area (1222), and the third display area (1223)) based on an angle (a) between the first housing part (1211), the hinge part (1213), and the second housing part (1212) rotated about the folding axis (f). As the user rotates the second housing part (1212) relative to the first housing part (1211) within the state (1291) of FIG. 12A corresponding to the unfolded state, the state of the electronic device (1201) can be switched from the state (1291) to the state (1292) of FIG. 12B. The electronic device (1201) can determine that the state of the electronic device (1201) has been switched to the state (1292) based on detecting an angle (b) less than 180º (or a reference angle range associated with the unfolded state) using a sensor. The state (1292) of the electronic device (1201) can include a sub-folded state, a sub-folded state, a half-folded state, a half-folded state, a concave state, and/or an intermediate state.

Within the exemplary state (1292) of FIG. 12b, the electronic device (1201) can control pixels of the first display area (1221) (or the second display area (1222)) within a range of brightness levels having a minimum brightness of la1 and a maximum brightness of la4. Within the state (1292), the electronic device (1201) can control pixels of the third display area (1223) within a range of brightness levels having a minimum brightness of la1 and a maximum brightness of la2. Referring to the states (1291, 1292), during switching from the unfolded state to the intermediate state, the maximum brightness of pixels of the third display area (1223), which is a folded portion, may be reduced less than the maximum brightness of pixels of other display areas (e.g., the first display area (1221) and/or the second display area (1222)). As the angle (b) decreases, the electronic device (1201) can gradually reduce the maximum brightness (la2) used to control the pixels of the third display area (1213).

For example, as the angle (b) decreases, the curvature of the third display area (1213) may increase. When the curvature increases, the reflection characteristic of the third display area (1213) may change. The change in the reflection characteristic may cause the user to perceive the third display area (1213) as brighter than the other display areas. In one embodiment, the electronic device (1201) may control the brightness of the pixels of the third display area (1213) according to a maximum brightness (la2) that is lower than the maximum brightness (la4) of the pixels of the other display areas, in order to make the brightness distribution of the entire display area of the flexible display (1220) uniform despite the change in the reflection characteristic.

Meanwhile, when the angle (b) increases, the electronic device (1201) may gradually increase the maximum brightness (la2) used to control the pixels of the third display area (1223). The embodiment is not limited thereto, and the electronic device (1201) may gradually increase or decrease the range of brightness levels of the first display area (1221) and/or the second display area (1222) depending on the angle (b).

Within the exemplary state (1293) of FIG. 12c, the electronic device (1201) can control the pixels of the first display area (1221) (or the second display area (1222)) within a range of brightness levels having a minimum brightness of la1 and a maximum brightness of la2. Within the state (1291) of FIG. 12a, in response to identifying that the angle (b) is decreasing, the electronic device (1201) can switch to the state (1293) of FIG. 12c. Within the state (1293), the electronic device (1201) can control the pixels of the third display area (1223) within a range of brightness levels having a minimum brightness of la1 and a maximum brightness of la4. Referring to states (1291, 1293), the maximum brightness of the pixels of the third display area (1223), which is a folding member, can be reduced from la3 to la4, which is less than la3. As the angle (b) decreases, the electronic device (1201) can gradually reduce the maximum brightness (la4) used to control the pixels of the third display area (1223).

Meanwhile, when the angle (b) increases, the electronic device (1201) may gradually increase the maximum brightness (la4) used to control the pixels of the third display area (1223). The embodiment is not limited thereto, and the electronic device (1201) may gradually increase or decrease the range of brightness levels of the first display area (1221) and/or the second display area (1222) depending on the angle (b).

As described above, according to one embodiment, the electronic device (1201) may include a foldable housing. Depending on the shape (or state) of the foldable housing, the electronic device (1201) displaying the image (1230) in the HDR mode may at least partially change the brightness level of the flexible display (1220) on which the image (1230) is displayed. For example, the brightness of each pixel of a flat portion (e.g., the first display area (1221) and/or the second display area (1222)) and/or a folding portion (e.g., the third display area (1223)) of the flexible display (1220) may be differentially enhanced. The brightness levels of each portion of the display area of the flexible display (1220) may be decreased or increased depending on the folding or unfolding of the foldable housing.

FIG. 13 illustrates an exemplary operation performed by an electronic device (1201) including a plurality of displays (e.g., a flexible display (1220) and a cover display (1310)) to display an image (1230) in HDR mode. The electronic device (1201) of FIG. 13 may be an example of the electronic device (1201) described with reference to FIGS. 12A to 12C.

In one embodiment, the electronic device (1201) may include a cover display (1310) that is visible from a rear side of the foldable housing. Referring to FIG. 13, the cover display (1310) is illustrated positioned on a rear side of a first housing (1211) of the electronic device (1201). Since the cover display (1310) is positioned on a rear side opposite to a front side where the flexible display (1220) is positioned, in an unfolded state of the electronic device (1201), the cover display (1310) may face a second direction opposite to a first direction in which the flexible display (1220) is visually exposed.

For example, in one embodiment of an electronic device (1201) including a plurality of displays, such as a flexible display (1220) and a cover display (1310), the flexible display (1220) may be referred to as a front display and the cover display (1310) may be referred to as a rear display. The flexible display (1220) may be referred to as a main display and the cover display (1310) may be referred to as a sub display. The flexible display (1220) may be referred to as a first display and the cover display (1310) may be referred to as a second display.

Referring to FIG. 13, an exemplary state of the electronic device (1201) is illustrated in which the flexible display (1220) is visually completely hidden by the rotation of the first housing part (1211) and the second housing part (1212) about the hinge part (1213). The state may be referred to as a folded state, a folded state, and/or a fully folded state. The electronic device (1201) may detect or determine the folded state of the electronic device (1201) based on whether the angle of the folding axis (f) corresponds to substantially 0º (or a designated angular range including 0º).

Referring to FIG. 13, in a hidden folding state of a flexible display (1220), the electronic device (1201) can display an image (1320) in HDR mode through the cover display (1310). Together with the image (1320), the electronic device (1201) can display a visual object (1332) for indicating the HDR mode. The visual object (1332) of FIG. 13 can correspond to the visual object (1232) described with reference to FIGS. 12A to 12C.

For example, the image (1320) may correspond to a second image synthesized by applying map information and/or display control information to a first image having a first dynamic range of SDR. The electronic device (1201) may use the ratio between dynamic ranges indicated by the display control information to increase or strengthen a ratio between brightness levels of pixels of the cover display (1310) on which the image (1320) is displayed, to a second dynamic range wider than the first dynamic range (e.g., a second dynamic range of HDR).

Referring to FIG. 13, a range of brightness levels of an image (1230) displayed on a flexible display (1220) and a range of brightness levels of an image (1320) displayed on a cover display (1310) are illustrated. Although the minimum luminances of the two ranges are illustrated as being identical to each other (lb1), this is for convenience of explanation (or comparison) and the embodiment is not limited thereto.

According to one embodiment, the electronic device (1201) may, when displaying a second image synthesized using a first image having a first dynamic range, map information, and display control information, determine brightness levels of the flexible display (1220) and the cover display (1310) based on the display control information such that at least a portion of the flexible display (1220) has a first brightness level and at least a portion of the cover display (1310) has a second brightness level different from the first brightness level.

Referring to FIG. 13, the electronic device (1201) can display an image (1230) of a second dynamic range wider than the first dynamic range, based on a brightness level having a minimum brightness of lb1 and a maximum brightness of lb3, on the flexible display (1220). The electronic device (1201) can display an image (1320) of a second dynamic range wider than the first dynamic range, based on a brightness level having a minimum brightness of lb1 and a maximum brightness of lb2, on the cover display (1310). For example, the maximum brightness (lb2) of the image (1320) and the maximum brightness (lb3) of the image (1230) may be different from each other. An embodiment is shown in which the maximum brightness (lb2) of the pixels of the cover display (1310) set to display the image (1320) is lower than the maximum brightness (lb3) of the pixels of the flexible display (1220) set to display the image (1230), but the embodiment is not limited thereto.

As described above, when displaying an image having an HDR effect (e.g., images (1230, 1320)), the electronic device (1201) may determine a brightness level of the image depending on the display (e.g., the flexible display (1220) and/or the cover display (1310)) on which the image is displayed. For example, the degree to which the brightness of the pixels of the cover display (1310) is enhanced to display the image (1310) may be different from the degree to which the brightness of the pixels of the flexible display (1220) is enhanced to display the image (1230).

FIGS. 14A and 14B illustrate exemplary operations performed by an electronic device (1401) including a flexible display (1420) to display an image (1430) in HDR mode. The electronic device (1401) of FIGS. 14A and/or 14B may be an example of the electronic device (101) described with reference to FIGS. 1 to 8.

Referring to FIGS. 14A and 14B , the electronic device (1401) may include a deformable housing (e.g., a sliderable housing and/or a rollable housing). The housing of the electronic device (1401) may include a first housing part (1411) and a second housing part (1412) that is movable relative to the first housing part (1411). The housing of the electronic device (1401) may be configured to provide a collapsed state and/or an expanded state. For example, in the collapsed state, the second housing part (1412) may be at least partially retractable into the first housing part (1411). For example, in the expanded state, at least a portion of the second housing part (1412) that was retracted into the first housing part (1411) may be withdrawn out of the first housing part (1411).

Referring to FIGS. 14A and 14B , the electronic device (1401) may include a flexible display (1420) that is visible from the front of the housing. The flexible display (1420) may extend from a first housing part (1411) to a second housing part (1412). A first portion (1421) of the flexible display (1420) that is visible in a contracted state may be smaller than the entire display area of the flexible display (1420). In an expanded state, the first portion (1421) and a second portion (1422) of the flexible display (1420) that is hidden in the contracted state may be visible. The first portion (1421) of the flexible display (1420) may be referred to as a flat portion in terms of having a flat shape in both the contracted state and the expanded state. The second part (1422) of the flexible display (1420) may be referred to as a rolling part, from the perspective that it has a form that is at least partially rolled up in a contracted state. From the perspective that it includes the rolling part, the flexible display (1420) may be referred to as a rollable display.

Referring to FIG. 14A, according to one embodiment, an electronic device (1401) may display an image (1430) having an HDR effect on a flexible display (1420). Similar to the visual object (1232) of FIGS. 12A to 12C and/or the visual object (1332) of FIG. 13, the electronic device (1401) may display a visual object (1432) indicating that the image (1430) having an HDR effect is displayed. The image (1430) may correspond to a second image synthesized from a first image having a first dynamic range of SDR based on map information and/or display control information corresponding to the first image.

When displaying an image (1430) having an HDR effect, the electronic device (1401) may change a dynamic range and/or a brightness level of the image (1430) depending on a state of the flexible display (1420). Referring to FIG. 14A, a brightness distribution of pixels (e.g., visually exposed pixels) of the flexible display (1420) is illustrated. For example, within a contracted state, the electronic device (1401) may control pixels controlled to display the image (1430) within a brightness level range of a minimum brightness of lc1 and a maximum brightness of lc2. Within an expanded state, the electronic device (1401) may control the pixels within a brightness level range of a minimum brightness of lc1 and a maximum brightness of lc3 (e.g., a maximum brightness exceeding lc2). While changing from the contracted state to the expanded state, the electronic device (1401) can gradually change the maximum luminance from lc2 to lc3. While changing from the expanded state to the contracted state, the electronic device (1401) can gradually reduce the maximum luminance from lc3 to lc2. The embodiment is not limited thereto, and the luminance difference between the minimum luminance (e.g., lc1) and the maximum luminance (lc2) within the contracted state can be smaller than the luminance difference between the minimum luminance (e.g., lc1) and the maximum luminance (lc3) within the expanded state.

Referring to FIG. 14a, the electronic device (1401) may display an image (1430) along the direction of the height among the width or height of the flexible display (1420) based on the landscape mode. The embodiment is not limited thereto. Referring to FIG. 14b, the electronic device (1401) may display an image (1430) along the direction of the width of the flexible display (1420) based on the portrait mode. The electronic device (1401) may determine the mode of the electronic device (1401) among the portrait mode or the landscape mode by using sensor data of an acceleration sensor and/or a geomagnetic sensor. The portrait mode and the landscape mode may be modes defined to determine a reference direction of a UI displayed on the flexible display (1420). The electronic device (1401) can determine the mode of the electronic device (1401) among portrait mode or landscape mode so that the UI displayed on the flexible display (1420) can be easily recognized by the user (e.g., so that text is arranged along a horizontal direction to the user).

In one embodiment, when displaying an image (1430) having an HDR effect, the electronic device (1401) may determine or set brightness levels of different portions (e.g., the first portion (1421) and/or the second portion (1422)) of the flexible display (1420) differently based on the display control information. For example, based on the display control information, the electronic device (1401) may determine brightness levels of different portions of the flexible display (1420) such that, in an expanded state, the rolling portion (e.g., the second portion (1422)) of the flexible display (1420) has a first brightness level and the flat portion (e.g., the first portion (1421)) of the flexible display (1420) has a second brightness level different from the first brightness level. Within the expanded state, the electronic device (1401) can display an image (1430) on a flexible display (1420) including a rolling portion having a first brightness level and a flat portion having a second brightness level.

Referring to FIG. 14B, the brightness distribution of pixels located in parts of the flexible display (1420) (e.g., the first part (1421) and the second part (14222)) is illustrated. The pixels located in the first part (1421) can be controlled to have a minimum brightness of lc1 and a maximum brightness of lc2. The pixels located in the second part (1422) can be controlled to have a minimum brightness of lc1 and a maximum brightness of lc3. Since the maximum brightness (lc3) of the pixels of the second part (1422), which is a rolling part, is higher than the maximum brightness (lc2) of the pixels of the first part (1421), which is a flat part, the brightness level of a part of the image (1430) displayed through the second part (1422) can be higher than the brightness level of another part of the image (1430) displayed through the first part (1421). The examples are not limited thereto.

In one embodiment, the electronic device (1401) can be switched between a contracted state and an expanded state. The electronic device (1401) can switch, or toggle, between the contracted state and the expanded state in response to an input for changing the state of the electronic device (or for deforming the electronic device (1401)). While changing from the contracted state to the expanded state, the electronic device (1401) can gradually change or increase the brightness level of a flat portion (e.g., the first portion (1421)). For example, when switching from the contracted state to the expanded state, the maximum brightness (lc2) of the pixels of the flat portion, the first portion (1421), can (gradually) increase to the maximum brightness (lc3) of the pixels of the rolling portion, the second portion (1422).

Meanwhile, when changing from the expanded state to the contracted state, the electronic device (1401) can gradually change or reduce the brightness level of at least one of the flat portion (e.g., the first portion (1421)) or the rolling portion (e.g., the second portion (1422)). For example, when switching from the expanded state to the contracted state, the electronic device (1401) can reduce the maximum brightness (lc3) of the pixels of the second portion (1422), which is the rolling portion, to the maximum brightness (lc2) of the pixels of the first portion (1421), which is the flat portion.

Referring to FIG. 14B, within the contracted state, the electronic device (1401) can control the pixels of the flexible display (1420) within a brightness range having a minimum brightness of lc1 and a maximum brightness of lc2. When switching from the contracted state to the expanded state, the electronic device (1401) can increase the maximum brightness from lc2 to lc3 exceeding lc2. Meanwhile, when switching from the expanded state to the contracted state, the electronic device (1401) can decrease the maximum brightness from lc3 to lc2 less than lc3.

As described above, when displaying an image (1430) having an HDR effect, the electronic device (1401) may change, increase, or decrease the maximum brightness of the image (1430) depending on the state of the electronic device (1401) (e.g., a contracted state and/or an extended state). Alternatively, when displaying an image (1430) having an HDR effect, the electronic device (1401) may determine the degree to which the brightness of pixels of the flexible display (1420) on which the image (1430) is displayed is enhanced differently in parts of the flexible display (1420) (e.g., a first part (1421) which is a flat part and a second part (1422) which is a rolling part). In one embodiment including a rollable housing, the electronic device (1401) can incrementally increase the maximum brightness of pixels controlled to display an image (1430) as the viewable portion of the flexible display (1410) increases (e.g., when changing from a contracted state to an expanded state).

FIG. 15 illustrates an exemplary operation performed by a head-mounted display (HMD) device (1501), according to one embodiment, to display an image (1550) in HDR mode. The HMD device (1501) of FIG. 15 may be an example of the electronic device (101) described with reference to FIGS. 1 to 8. For example, the HMD device (1501) of FIG. 15 may perform the operation of the electronic device (101) described with reference to FIGS. 1 to 8.

In one embodiment, the HMD device (1501) may include a housing contoured to fit the user's two eyes (e.g., a left eye (1531) and a right eye (1532)). For example, the HMD device (1501) may include a housing wearable on the user's head. The HMD device (1501) may include displays configured to be positioned toward each of the user's two eyes when worn by the user. Referring to FIG. 15, the HMD device (1501) may include a first display (1511) configured to be positioned toward the left eye (1531) and a second display (1512) configured to be positioned toward the right eye (1532).

For example, the HMD device (1501) may include cameras (e.g., eye tracking (ET) cameras) configured to be positioned toward each of the user's two eyes when worn by the user. Referring to FIG. 15 , examples of the cameras include a first camera (1521) configured to be positioned toward the left eye (1531) and a second camera (1522) configured to be positioned toward the right eye (1532). Using images and/or videos acquired from the first camera (1521) and the second camera (1522), the HMD device (1501) may detect the gaze of the user wearing the HMD device (1501). The HMD device (1501) may acquire information related to the gaze. The information may include a gaze position, a gaze direction, and/or a gaze error of the user wearing the HMD device (1501).

According to one embodiment, the HMD device (1501) can display a screen (1540) including an image (1550) using the first display (1511) and the second display (1512). When displaying the image (1550) having an HDR effect, the HMD device (1501) can further display a visual object (1552) indicating that the image (1550) having an HDR effect is displayed within the screen (1540). As described above with reference to FIGS. 1 to 8, the HMD device (1501) can generate or synthesize a second image having a second dynamic range wider than the first dynamic range, using the first image having the first dynamic range, map information, and display control information.

The HMD device (1501) can control the first display (1511) and/or the second display (1512) to display the second image. For example, the HMD device (1501) can display an image (1550) having an HDR effect by enhancing the brightness of portions of the first display (1511) and the second display (1512) corresponding to a specific portion of the second image based on the display control information.

In one embodiment, the HMD device (1501) can render a region (e.g., a foveated region (fa)) within the screen (1540) that the user is staring at and the remaining region at different resolutions and/or different refresh rates. The HMD device (1501) can estimate or determine, using ET cameras (e.g., the first camera (1521) and/or the second camera (1522)), the foveated region (fal) within the first display (1511) stared at by the left eye (1531) and the foveated region (far) within the second display (1512) stared at by the right eye (1532). Using the foveated regions (fal, far), the HMD device (1501) can determine the foveated region (fa) within the screen (1540). The foveated area (fa) is an area within the screen (1540) that includes the gaze position and may correspond to a part of the screen (1540) that a user wearing the HMD device (1501) is looking at.

According to one embodiment, the HMD device (1501) may determine or change the brightness level of the foveated area (fa) to be a different brightness level from the brightness level of the remaining area. Referring to FIG. 15, the HMD device (1501) may determine the brightness level of the screen (1540) such that the foveated area (fa) toward which the gaze is directed has a first brightness level and the remaining area away from the gaze has a second brightness level different from the first brightness level. For example, the first brightness level of the foveated area (fa) may be determined within a brightness range having a minimum brightness of ld1 and a maximum brightness of ld3. For example, the second brightness level of the remaining area may be determined within a brightness range having a minimum brightness of ld1 and a maximum brightness of ld2. ld2 may be lower than ld3. For example, the second brightness level of the remaining area may be determined to be a level lower than the first brightness level of the foveated area (fa).

The HMD device (1501) can detect the movement of the gaze position while changing the brightness level of the foveated region (fa) to a higher level than the brightness levels of the remaining regions. The HMD device (1501) can (gradually) change the brightness level of the foveated region (fa') corresponding to the moved gaze position to a brightness level higher than the second brightness level (e.g., the first brightness level). For example, the HMD device (1501) can determine the brightness level of the foveated region (fa') within a brightness range having a minimum brightness of ld1 and a maximum brightness of ld3. In the above example, the HMD device (1501) can (gradually) change the first brightness level of the foveated region (fa) to a brightness level lower than the first brightness level (e.g., the second brightness level). For example, the HMD device (1501) can determine the brightness level of the foveated area (fa) within a brightness range having a minimum brightness of ld1 and a maximum brightness of ld2.

As described above, according to one embodiment, the HMD device (1501) can control the displays (e.g., the first display (1511) and the second display (1512)) to display an image (1550) having an HMD effect. While displaying the image (1550) having the HMD effect, the HMD device (1501) can detect or identify an area (e.g., a foveated area (fa)) within the screen (1540) corresponding to a gaze position of a user wearing the HMD device (1501). The HMD device (1501) can enhance the brightness level of pixels corresponding to the foveated area (fa) more than the brightness level of pixels corresponding to the remaining areas. The HMD device (1501) can identify or predict the movement of the foveated area (fa) based on the gaze direction and/or the gaze position. Based on the movement of the predicted foveated area (fa), the HMD device (1501) can increase or decrease the brightness level of the pixels.

FIG. 16 illustrates an exemplary operation of an electronic device (101) to at least partially enhance the brightness level of a display (120) using user-related information when displaying an image (1610) in HDR mode.

Referring to FIG. 16, an exemplary state of an electronic device (101) displaying an image (1610) having an HDR effect is illustrated. The electronic device (101) can apply map information and display control information to a first image having a first dynamic range (e.g., SDR) to generate or synthesize a second image having a second dynamic range (e.g., HDR) wider than the first dynamic range. By controlling the display (120) based on the second image, the electronic device (101) can display the image (1610) having an HDR effect.

In one embodiment, similar to the HMD device (1501) described above with reference to FIG. 15, the electronic device (101) may use gaze position to at least partially determine or change a brightness level of the display (120). The electronic device (101) may include a camera (e.g., a selfie camera and/or a front side camera) positioned to face the direction in which the display (120) is positioned. Using images and/or video acquired through the camera, the electronic device (101) may detect the two eyes of the user gazing at the display (120). Using information about the two eyes, the electronic device (101) may calculate or estimate the portion (1630) of the display (120) that is being gazed at by the user.

According to one embodiment, the electronic device (101) can control pixels arranged in a portion (1630) of the display (120) within a range between a minimum brightness of le1 and a maximum brightness of le3. The electronic device (101) can control pixels of a remaining portion of the display (120), which is different from the portion (1630), within a range between the minimum brightness of le1 and the maximum brightness of le2. For example, the maximum brightness (le3) that the pixels of the portion (1630) can have can be higher than the maximum brightness (le2) that the pixels of the remaining portion can have. For example, the electronic device (101) can change or determine brightness levels of the pixels of the display (120) such that a portion of the screen and/or image (1610) corresponding to the portion (1630) is expressed brighter than the other portions.

According to one embodiment, the electronic device (101) can change the size and/or position of the portion (1630) controlled based on a relatively high brightness level according to the movement of the gaze of the user gazing at the display (120). For example, based on the direction of the gaze movement, the electronic device (101) can translate the portion (1630) within the display (120). For example, based on the speed of the gaze movement, the electronic device (101) can increase or decrease the size of the portion (1630).

In one embodiment, a method may be required to display an image having a contrast ratio equal to or similar to the contrast ratio on a display by using information related to a contrast ratio of an external environment at a time when an image is captured, when displaying the image. In one embodiment, a method may be required to improve a front-of-screen appearance of an image. As described above, an electronic device (e.g., the electronic device (101) of FIG. 1 and/or the electronic device (901) of FIG. 9) according to an embodiment may include a display (e.g., the display (120) of FIG. 1 and/or FIG. 2), at least one processor including a processing circuit (e.g., the processor (210) of FIG. 2), and a memory including one or more storage media storing instructions (e.g., the memory (215) of FIG. 2). The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to detect an event for displaying a first image (e.g., image (112) of FIG. 1 ) having a first dynamic range while a brightness level of the display is set to a first brightness level. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device, in response to the event, to obtain, from metadata in a file (e.g., file (110) of FIG. 1 ) containing the first image, map information for another portion of the first image that is visually emphasized with respect to a portion of the first image (e.g., map information (114) of FIG. 1 ), and display control information for changing a brightness level of the display in connection with the display of the first image (e.g., display control information (116) of FIG. 1 ). The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to generate a second image (e.g., image (130) of FIG. 1) by applying the map information to the first image. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to set a brightness level of the display from the first brightness level to a second brightness level higher than the first brightness level based on the display control information. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display the second image on the display set to the second brightness level such that the first image having a second dynamic range wider than the first dynamic range is shown on the display. According to one embodiment, the electronic device can display an image having a contrast ratio equal to or similar to the contrast ratio on the display by using information related to the contrast ratio of the external environment at the time when the image is captured. According to one embodiment, the electronic device can improve the appearance of the front screen of the image.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to detect the event generated by an input related to the thumbnail image while displaying the thumbnail image on the display, the thumbnail image having a size smaller than a size of the first image and having the first dynamic range.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to control the display, using the display control information, such that at least one pixel of the display corresponding to a portion of the first image having a minimum brightness represented by the first dynamic range has a minimum brightness of the second dynamic range. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to control the display such that at least one other pixel of the display corresponding to a different portion of the first image having a maximum brightness represented by the first dynamic range has a maximum brightness of the second dynamic range.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to obtain display control information, wherein the display control information is a ratio between a size of a total of third dynamic ranges of a plurality of third images that were used to synthesize the first image in the file, and a size of the first dynamic range.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to obtain map information for extending a color space of the first image from the file.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to generate, from the map information obtained from the file, a two-dimensional array comprising values each representing brightness levels of different portions of the first image. A width and a height of the two-dimensional array may be less than a width and a height of the first image.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to control the display such that, when displaying an execution screen having the second image arranged on the display set to the second brightness level, pixels within a first display area of the display corresponding to the second image operate at the second brightness level, and pixels within a second display area of the display corresponding to the execution screen adjacent to the second image operate at the first brightness level.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine the second brightness level by using at least one of a range of brightness levels displayable by the display, a state of charge (SOC) of a battery of the electronic device, or an on pixel ratio (OPR) of the display when displaying the second image, together with the display control information obtained from the file.

In one embodiment, as described above, a method of an electronic device including a display may be provided. The method may include an operation of detecting an event for displaying a first image having a first dynamic range while a brightness level of the display is set to a first brightness level. The method may include an operation of, in response to the event, obtaining, from metadata in a file including the first image, map information for another portion of the first image visually emphasized with respect to a portion of the first image, and display control information for changing a brightness level of the display in relation to displaying the first image. The method may include an operation of generating a second image by applying the map information to the first image. The method may include an operation of setting a brightness level of the display from the first brightness level to a second brightness level higher than the first brightness level, based on the display control information. The method may include displaying the second image on the display set to the second brightness level such that the first image having a second dynamic range wider than the first dynamic range is shown on the display.

For example, the detecting operation may include an operation of detecting the event generated by an input related to the thumbnail image while displaying a thumbnail image on the display, the thumbnail image having a size smaller than the size of the first image and having the first dynamic range.

For example, the setting operation may include an operation of controlling the display, using the display control information, such that at least one pixel of the display corresponding to a portion of the first image having a minimum brightness represented by the first dynamic range has a minimum brightness of the second dynamic range. The setting operation may include an operation of controlling the display such that at least one other pixel of the display corresponding to another portion of the first image having a maximum brightness represented by the first dynamic range has a maximum brightness of the second dynamic range.

For example, the obtaining operation may include obtaining the display control information, which is a ratio between the size of the entire third dynamic ranges of the plurality of third images used in synthesizing the first image in the file and the size of the first dynamic range.

For example, the obtaining operation may include obtaining map information for extending a color space of the first image from the file.

For example, the generating operation may include generating a two-dimensional array, from the map information obtained from the file, each of which includes values representing brightness levels of different portions of the first image. A width and a height of the two-dimensional array may be smaller than a width and a height of the first image.

For example, the displaying operation may include an operation of controlling the display so that, when displaying an execution screen on which the second image is arranged on the display set to the second brightness level, pixels in a first display area of the display corresponding to the second image operate at the second brightness level. The displaying operation may include an operation of controlling the display so that pixels in a second display area of the display corresponding to the execution screen adjacent to the second image operate at the second brightness level.

For example, the setting operation may include an operation of determining the second brightness level by using at least one of a range of brightness levels displayable by the display, a state of charge (SOC) of a battery of the electronic device, or an on pixel ratio (OPR) of the display when displaying the second image, together with the display control information obtained from the file.

According to one embodiment of the present invention, an electronic device as described above may include at least one camera, at least one processor including a processing circuit, and a memory including one or more storage media storing instructions. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to control the at least one camera to acquire a plurality of images in response to a photographing input. The plurality of images may include a first image acquired according to a first dynamic range and a second image having a second dynamic range at least partially different from the first dynamic range, wherein a bit depth of the first dynamic range may be the same as a bit depth of the second dynamic range. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to acquire a third image having a third dynamic range using at least a portion of the plurality of images. The bit depth of the third dynamic range may be the same as the bit depth of the first dynamic range. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to generate map information for another portion of the third image visually emphasized with respect to a portion of the third image using the plurality of images. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine display control information for changing a brightness level of the display device so that the third image is displayed as an image having a fourth dynamic range. The bit depth of the fourth dynamic range may be greater than the bit depth of the third dynamic range. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to generate a file including metadata including the map information and the display control information and the third image.

For example, the map information may be applied to the third image to be displayed by a display device supporting HDR (high dynamic range), based on an event for displaying the third image.

For example, the display control information may be used to change a brightness level of the display device displaying the third image based on the event.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to control the at least one camera to have a property different from at least one of a shutter speed, an ISO sensitivity, an aperture opening, or an exposure value of the at least one camera at the time the first image was acquired, in order to acquire the first image and the second image in response to the photographing input, and to acquire the second image having the second dynamic range at least partially different from the first dynamic range after acquiring the first image. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to acquire the second image using the at least one camera having the property.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to generate the file including the display control information, the file including a size of the fourth dynamic range and a ratio between the third dynamic range and the overall dynamic range of the plurality of images.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to generate map information representing levels of chrominance components of specified primary colors of the third image.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to generate the map information having a form of a two-dimensional array, the two-dimensional array including values representing brightness levels of different portions of the third image. A width and a height of the two-dimensional array may be less than a width and a height of the third image.

In one embodiment as described above, a method of an electronic device including at least one camera may be provided. The method may include an operation of controlling the at least one camera to acquire a plurality of images in response to a photographing input. The plurality of images may include a first image acquired according to a first dynamic range and a second image having a second dynamic range at least partially different from the first dynamic range, wherein a bit depth of the first dynamic range may be the same as a bit depth of the second dynamic range. The method may include an operation of acquiring a third image having a third dynamic range using at least a portion of the plurality of images. The bit depth of the third dynamic range may be the same as a bit depth of the first dynamic range. The method may include an operation of generating map information for another portion of the third image visually emphasized for a portion of the third image using the plurality of images. The method may include an operation of determining display control information for changing a brightness level of a display device so that the third image is displayed as an image having a fourth dynamic range. A bit depth of the fourth dynamic range may be greater than the bit depth of the third dynamic range. The method may include an operation of generating a file including metadata including the map information and the display control information and the third image.

For example, the map information may be applied to the third image to be displayed by a display device supporting HDR (high dynamic range), based on an event for displaying the third image.

For example, the display control information may be used to change a brightness level of the display device displaying the third image based on the event.

For example, the controlling operation may include an operation of controlling the at least one camera to have a different property from at least one of a shutter speed, an aperture opening degree, an ISO sensitivity, or an exposure value of the at least one camera at a time when the first image was acquired, in order to acquire the second image having the second dynamic range at least partially different from the first dynamic range, after acquiring the first image, when acquiring the first image and the second image in response to the photographing input. The controlling operation may include an operation of acquiring the second image using the at least one camera having the property.

For example, the act of generating the file may include the act of generating the file including the display control information including a size of the fourth dynamic range corresponding to the entire dynamic ranges of the plurality of images and a ratio between the third dynamic range.

For example, the operation of generating the map information may include an operation of generating the map information representing the level of the chrominance component of the specified primary color of the third image.

For example, the operation of generating the map information may include the operation of generating the map information in the form of a two-dimensional array including values representing brightness levels of different parts of the third image. A width and a height of the two-dimensional array may be smaller than a width and a height of the third image.

According to one embodiment of the present invention, an electronic device as described above may include a display, at least one processor including a processing circuit, and a memory including one or more storage media storing instructions. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to obtain a first image having a first dynamic range, map information related to brightness or color gradation of at least a portion of the first image, and display control information related to a brightness level during display. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to generate a second image based on the first image and the map information. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine a brightness level of at least a portion of the display based on the display control information. The above instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display the second image according to a second dynamic range that is wider than the first dynamic range, based on the determined brightness level.

For example, the map information may include information about differences in brightness or color gradation between at least a portion of the first image and at least a portion of a third image captured of the same object based on a different exposure value when the first image was captured.

For example, the display control information may include at least one of maximum brightness information required for display of the second image, or ratio information between the maximum brightness required for display of the first image and the maximum brightness required for display of the second image.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to obtain a file including the map information as metadata together with the first image. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to generate the display control information based on the map information.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to obtain a file including the map information and the display control information as metadata together with the first image.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display a thumbnail image having a size smaller than a size of the first image on at least a portion of the display. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to perform an operation of displaying the second image based on an input related to the thumbnail image.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display the first image for at least some of the time based on the input related to the thumbnail image. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to perform an action of displaying the second image by changing the first image to the second image after the at least some of the time.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to control the display, based on the display control information, such that at least one pixel of the display corresponding to a portion of the first image having a minimum brightness of the first dynamic range has a minimum brightness of the second dynamic range. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to control the display such that at least one other pixel of the display corresponding to another portion of the first image having a maximum brightness of the first dynamic range has a maximum brightness of the second dynamic range.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to generate, based on the map information, a two-dimensional array including values each representing brightness levels of different portions included in the first image. A horizontal pixel count and a vertical pixel count of the two-dimensional array may be smaller than a horizontal pixel count and a vertical pixel count of the first image, respectively.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to control the display such that, when displaying an execution screen having the second image arranged on the display set to the brightness level, pixels within a first display area of the display corresponding to the second image operate at the brightness level, and pixels within a second display area of the display corresponding to the execution screen adjacent to the second image operate at a different brightness level lower than the brightness level.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine the brightness level based on at least one of a range of brightness levels displayable by the display, a state of charge (SOC) of a battery of the electronic device, or a ratio of pixels having a specified grayscale level or higher among all pixels of the display when displaying the second image (on pixel ratio (OPR)), together with the display control information obtained from the file.

For example, the electronic device may include a foldable housing. The display accommodated in the foldable housing may include a foldable display including a folding portion that is folded and unfolded as the foldable housing is folded and unfolded and a flat portion that is maintained flat. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to perform an operation of determining a brightness level based on a folding state of the foldable housing and the display control information, such that at least a portion of the folding portion has a first brightness level and at least a portion of the flat portion has a second brightness level different from the first brightness level. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to perform an operation of displaying the second image on the foldable display, such that at least a portion of the folding portion has the first brightness level and at least a portion of the flat portion has the second brightness level.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to perform the operations of determining the brightness level such that the first brightness level is higher than the second brightness level and the operations of displaying the second image.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to perform an operation of determining a brightness level and an operation of displaying the second image, such that at least one of the first brightness level and the second brightness level is gradually changed to a third brightness level lower than the first brightness level and the second brightness level as the foldable housing is unfolded and folded based on the folding state.

For example, the electronic device may include a foldable housing. The display accommodated in the foldable housing may include a foldable display visually exposed to face a first direction in an unfolded state, and a sub-display visually exposed to face a second direction opposite to the first direction. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to perform an operation of determining a brightness level based on the display control information, such that at least a portion of the foldable display has a first brightness level and at least a portion of the sub-display has a second brightness level different from the first brightness level. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to perform an operation of displaying the second image on at least a portion of the foldable display having the first brightness level or on at least a portion of the sub-display having the second brightness level.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to perform the operations of determining the brightness level such that the first brightness level is higher than the second brightness level and the operations of displaying the second image.

For example, the electronic device may include a housing configured to provide a contracted state and an expanded state. The display accommodated in the housing may include a rollable display including a rolling portion in which the housing is at least partially rolled in the contracted state and unfolded in the expanded state, and a flat portion that remains flat in the contracted state and the expanded state. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to perform an operation of determining a brightness level, based on the display control information, when the housing is in the expanded state, such that the rolling portion of the rollable display has a first brightness level and the flat portion has a second brightness level different from the first brightness level. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to perform an operation of displaying the second image on the rollable display, when the housing is in the expanded state, such that the rolling portion has the first brightness level and the flat portion has the second brightness level.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to perform the operations of determining the brightness level such that the first brightness level is higher than the second brightness level and the operations of displaying the second image.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to perform an operation of determining a brightness level and an operation of displaying the second image such that the second brightness level is gradually changed to a third brightness level higher than the second brightness level as the housing changes from the contracted state to the expanded state.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to perform an operation of determining a brightness level and an operation of displaying the second image, such that at least one of the first brightness level and the second brightness level is gradually changed to a third brightness level lower than the first brightness level and the second brightness level as the housing changes from the expanded state to the contracted state.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to detect a gaze of a user of the electronic device. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to perform an operation of determining a brightness level based on the gaze and the display control information, such that a first portion of the display toward which the gaze is directed has a first brightness level and a second portion of the display away from the gaze has a second brightness level different from the first brightness level. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to perform an operation of displaying a second image on the display, such that the first portion has the first brightness level and the second portion has the second brightness level.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to perform the operations of determining the brightness level such that the first brightness level is higher than the second brightness level and the operations of displaying the second image.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to perform an operation of determining the brightness level and an operation of displaying the second image, such that as the point to which the gaze is directed moves from the first portion to the second portion, the first portion is gradually changed to a third brightness level lower than the first brightness level, and the second portion is gradually changed to a fourth brightness level higher than the second brightness level.

For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to perform an operation of determining the brightness level such that the second brightness level is equal to the third brightness level and the first brightness level is equal to the fourth brightness level, and an operation of displaying the second image.

For example, the electronic device may include a housing that is mountable on the user's head and houses the display.

In one embodiment, as described above, a non-transitory computer-readable storage medium storing instructions may be provided. The instructions may be executed by an electronic device including a display. The instructions, when executed by the electronic device, may cause the electronic device to obtain a first image having a first dynamic range, map information related to brightness or color gradation of at least a portion of the first image, and display control information related to a brightness level during display. The instructions, when executed by the electronic device, may cause the electronic device to generate a second image based on the first image and the map information. The instructions, when executed by the electronic device, may cause the electronic device to determine a brightness level of at least a portion of the display based on the display control information. The above instructions, when executed by the electronic device, may cause the electronic device to display the second image according to a second dynamic range that is wider than the first dynamic range, based on the determined brightness level.

As used herein, the term "if" will be understood to mean "when, upon," "in response to determining," or "in response to detecting," as appropriate. Similarly, "if it is determined to," or "if [the stated condition or event] is detected," will be understood to mean, optionally, "upon determining," or "in response to determining," "upon detecting [the stated condition or event]," or "in response to detecting [the stated condition or event]."

The devices described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone, but those skilled in the art will appreciate that the processing device may include multiple processing elements and/or multiple types of processing elements. For example, the processing device may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

The software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device. The software and/or data may be embodied in any type of machine, component, physical device, computer storage medium, or device for interpretation by the processing device or for providing instructions or data to the processing device. The software may be distributed over network-connected computer systems and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. At this time, the medium may be one that continuously stores a program executable by a computer, or one that temporarily stores it for execution or downloading. In addition, the medium may be various recording means or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, and may also be distributed on a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and ROMs, RAMs, flash memories, etc., configured to store program commands. In addition, examples of other media may include recording media or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.

Although the embodiments have been described above by way of limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, appropriate results can be achieved even if the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or are replaced or substituted by other components or equivalents.

Therefore, other implementations, other embodiments, and equivalents to the claims are also included in the scope of the claims described below.

Claims (15)

In an electronic device (101; 901), display (120); A memory (215) including one or more storage media storing instructions; and At least one processor (210) comprising a processing circuit, The above instructions, when individually or collectively executed by the at least one processor, Obtaining a first image (112) having a first dynamic range, map information (114) related to brightness or color gradation of at least a part of the first image, and display control information (116) related to a brightness level at the time of display; Based on the first image and the map information, a second image (130) is generated; Based on the display control information, determining a brightness level of at least a portion of the display; Based on the determined brightness level, the second image is displayed according to a second dynamic range that is wider than the first dynamic range. causing said electronic device, Electronic devices. In claim 1, The above map information (114) is Containing information about differences in brightness or color gradation between at least a portion of the first image (112) and at least a portion of a third image captured of the same object based on a different exposure value than when the first image was captured. Electronic devices. In claims 1 and 2, The above display control information is, At least one of information on the maximum brightness required for displaying the second image, or information on the ratio between the maximum brightness required for displaying the first image and the maximum brightness required for displaying the second image, Electronic devices. In claims 1 to 3, The above instructions, when individually or collectively executed by the at least one processor, Obtain a file including the map information (114) as metadata together with the first image (112); Causing the electronic device to generate the display control information based on the above map information (114). Electronic devices. In claims 1 to 4, The above instructions, when individually or collectively executed by the at least one processor, Causing the electronic device to obtain a file including the map information (114) and the display control information (116) as metadata together with the first image (112). Electronic devices. In claims 1 to 5, The above instructions, when individually or collectively executed by the at least one processor, Displaying a thumbnail image having a size smaller than the size of the first image on at least a portion of the display; Causing the electronic device to perform an action of displaying the second image based on an input related to the thumbnail image; Electronic devices. In claim 6, The above instructions, when individually or collectively executed by the at least one processor, Displaying said first image for at least some of the time based on said input related to said thumbnail image; causing the electronic device to perform an action of displaying the second image by changing the first image to the second image after at least a portion of the time; Electronic devices. In claims 1 to 7, The above instructions, when individually or collectively executed by the at least one processor, Based on the above display control information: Controlling the display so that at least one pixel of the display corresponding to a portion of the first image having a minimum brightness of the first dynamic range has a minimum brightness of the second dynamic range; causing the electronic device to control the display so that at least one other pixel of the display corresponding to another portion of the first image having a maximum brightness of the first dynamic range has a maximum brightness of the second dynamic range; Electronic devices. In claims 1 to 8, The above instructions, when individually or collectively executed by the at least one processor, Causing the electronic device to generate a two-dimensional array containing values each representing brightness levels of different parts included in the first image based on the map information; The number of horizontal pixels and the number of vertical pixels of the two-dimensional array are each smaller than the number of horizontal pixels and the number of vertical pixels of the first image, Electronic devices. In claims 1 to 9, The above instructions, when individually or collectively executed by the at least one processor, When displaying an execution screen on which the second image is placed on the display set to the brightness level, the electronic device controls the display so that pixels in the first display area of the display corresponding to the second image operate at the brightness level, and pixels in the second display area of the display corresponding to the execution screen adjacent to the second image operate at a different brightness level lower than the brightness level. Electronic devices. In claims 1 to 10, The above instructions, when individually or collectively executed by the at least one processor, Causing the electronic device to determine the brightness level based on at least one of a range of brightness levels displayable by the display, a state of charge (SOC) of a battery of the electronic device, or a ratio of pixels having a specified grayscale or higher among all pixels of the display when displaying the second image (OPR; on pixel ratio), together with the display control information obtained from the file. Electronic devices. In claims 1 to 11, Includes a foldable housing, The display accommodated in the foldable housing includes a foldable display including a folding portion that folds and unfolds as the foldable housing is folded and unfolded and a flat portion that remains flat, The above instructions, when individually or collectively executed by the at least one processor, Based on the folding state of the foldable housing and the display control information, an operation of determining the brightness level such that at least a part of the folding portion has a first brightness level and at least a part of the flat portion has a second brightness level different from the first brightness level is performed; Causing the electronic device to perform an operation of displaying the second image on the foldable display, wherein at least a portion of the folding portion has the first brightness level and at least a portion of the flat portion has the second brightness level. Electronic devices. In claim 12, The above instructions, when individually or collectively executed by the at least one processor, Causing the electronic device to perform an operation of determining the brightness level and an operation of displaying the second image so that the first brightness level is higher than the second brightness level. Electronic devices. In claim 12, The above instructions, when individually or collectively executed by the at least one processor, Causing the electronic device to perform an operation of determining the brightness level and an operation of displaying the second image, based on the folding state, such that at least one of the first brightness level and the second brightness level is gradually changed to a third brightness level lower than the first brightness level and the second brightness level as the foldable housing is unfolded and folded. Electronic devices. A method of an electronic device including a display, An operation of obtaining a first image having a first dynamic range, map information related to brightness or color gradation of at least a portion of the first image, and display control information related to a brightness level at the time of display. An operation of generating a second image based on the first image and the map information; An operation of determining a brightness level of at least a portion of the display based on the display control information; and An operation of displaying the second image according to a second dynamic range wider than the first dynamic range, based on the determined brightness level. method.

Images