WO2015167525A1 - Automatic page layout for text items and image items - Google Patents
Automatic page layout for text items and image items Download PDFInfo
- Publication number
- WO2015167525A1 WO2015167525A1 PCT/US2014/036151 US2014036151W WO2015167525A1 WO 2015167525 A1 WO2015167525 A1 WO 2015167525A1 US 2014036151 W US2014036151 W US 2014036151W WO 2015167525 A1 WO2015167525 A1 WO 2015167525A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- items
- regions
- area
- region
- font size
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/103—Formatting, i.e. changing of presentation of documents
Definitions
- Digital content can include an arrangement of items that are arranged and presented to a user in many different formats, such as by a webpage, in a newspaper or magazine, blog, e-mail listing, photo album, among others.
- Displaying digital content can involve page layout decisions regarding
- Figure 1 is a diagram illustrating a computing system according to an example of the present disclosure.
- Figure 2 is a diagram illustrating recursive page bisecting according to an example of the present disclosure.
- Figure 3 is a diagram illustrating a page layout according to an example of the present disclosure.
- Figure 4 is a block diagram illustrating a system for performing an automatic page layout according to an example of the present disclosure.
- Figure 5 is a diagram illustrating control of area estimates in automatic page layout with a font optimization method according to an example of the present disclosure.
- Figure 6 is a flow diagram illustrating a method for performing an automatic page layout according to an example of the present disclosure.
- Deterministic methods are usually used for environments where documents are to be produced quickly and in large volumes (such as in VDP), given their efficiency and predictability of results.
- a guillotine-based method may be used in contexts such as VDP or Web content aggregation, since it can be efficient to generate automatically and is flexible in terms of content placement, as opposed to the use of templates.
- the guillotine format is commonly used in widespread publications such as newspapers, and thus it is a comfortable and familiar format for readers.
- a guillotine layout a page is hierarchically divided in smaller sections by horizontal or vertical lines that run across the extremities of the larger section without crossing any items in the way.
- One implementation is directed to an enhanced guillotine-based layout method that produces unique document layouts tailored to previously-selected content that can be used in different workflows for generating documents.
- An advantage of the approach is that each page of a document may hold an unspecified number of items with unknown sizes. The method relies on a set of constraints and finds an optimal solution based on reasonable aesthetic goals, including optimizing the use of areas by textual elements.
- One disadvantage of some previous approaches is that textual elements may be presented in a range of different text size configurations, which causes an undesired effect on the aesthetic appearance and legibility of a document.
- some implementations disclosed herein adapt areas on a page in an attempt to minimize the difference in font sizes, thus improving the legibility and appearance of a document while still benefiting from the flexibility of providing font size variations. Moreover, the role of images in this framework is considered as well.
- One implementation is directed to a mixed content automatic layout (MCAL) method, and more specifically to a method for automatically generating template-free document layouts with adaptable font sizes, minimal text size variation, and image inclusion support.
- MCAL mixed content automatic layout
- An example scenario is a Web-to-print application, where a user may collect a number of different parts of web pages and assemble them into a single document using automated layout technology. The user has no knowledge and does not interfere in how the document is assembled from the selected content.
- the use of template-based layouts is challenging because of the unknown number and size of the input.
- the present disclosure includes a system and method for automatic page layout.
- Computer software can be adapted for arranging text and images according to the user's direction.
- software can be used to manually place and arrange stored images, such as photographs, video frames, and blocks of text, on a page (e.g., a webpage, a printer output, a display).
- web page design programs and word processing program can include such capabilities.
- such programs typically enable a user to place images and text as a user directs.
- Other computer software such as e-mail packages and personal web page design software, can allow a user to designate areas to display received digital content.
- a user may designate a portion of an e-mail program within which to display received e-mail messages, or may design a personal web page to display streaming news articles received from selected sources within one area of a page, streaming blog entries of interest to display in another area of the same page, and streaming movies to display in yet another area.
- These programs often use a standard template or involve a user's input to set the various areas. As such, these programs do not constitute an automated page layout.
- Figure 1 is a diagram illustrating a computing system 120 according to an example of the present disclosure.
- the computing system 120 shown in Figure 1 is a networked computing system.
- the network 122 can be a private network such as a local area network or wide area network, or can be a public network, such as the Internet.
- examples of the present disclosure are not limited to a particular computing system configuration.
- the computing system 120 can be comprised of a number of computing resources communicatively coupled to the network 122.
- Figure 1 shows a server 124 having a data source 126, and a client 128.
- Server 124 and client 128 are computing devices.
- Client 128 includes processors 129
- the non-transitory computer-readable medium 130 is structured to store programs 133 that are executed by the processors 129 and/or data.
- Client 128 is further communicatively coupled to a production device 135 (e.g., electronic display, printer, etc.). Client 128 can also be communicatively coupled to an external computer-readable memory 151 .
- the client 128 can cause an output to the production device 135, for example, as a result of executing instructions of programs stored in non-transitory computer-readable medium 130, by at least one processor 129, to implement a method of automatic page layout according to the present disclosure.
- Causing an output can include, but is not limited to, displaying text and images to an electronic display and/or printing text and images to a tangible medium (e.g., paper). For example, a plurality of text articles and images, with or without captions, can be arranged to appear in an arrangement similar to a newspaper or magazine and caused to be output to a computer monitor or printed to a tangible medium.
- Server 124 and client 128 are communicatively coupled to one another through the network 122. While a single server is shown in Figure 1 , the computing system can be comprised of multiple interconnected computing resources, such as servers 124 and clients 128.
- a computing resource e.g., 124, 128) can include control circuitry such as a processor, a state machine, application specific integrated circuit (ASIC), controller, and/or similar machine.
- ASIC application specific integrated circuit
- the indefinite articles "a” and/or “an” can indicate more than one of the named object.
- a processor can include one processor or more than one processor, such as a parallel processing
- the control circuitry can have a structure that provides a given
- non-transitory computer-readable medium e.g., 126, 130
- the non-transitory computer-readable medium can be integral, or communicatively coupled, to a computing resource such as client 128, in either in a wired or wireless manner.
- the non-transitory computer-readable medium 130 can be an internal memory, a portable memory, a portable disk, or a memory located internal to another computing resource (e.g., enabling the computer-readable instructions to be downloaded over the Internet).
- the non-transitory computer- readable medium 130 can have computer-readable instructions stored thereon that are executed by the control circuitry (e.g., processor) to provide a particular functionality.
- the non-transitory computer-readable medium 130 can include volatile and/or non-volatile memory.
- Volatile memory can include memory that depends upon power to store information, such as various types of dynamic random access memory (DRAM), among others.
- Non-volatile memory can include memory that does not depend upon power to store information.
- Examples of non-volatile memory can include solid state media such as flash memory, EEPROM, phase change random access memory (PCRAM), among others.
- the non-transitory computer-readable medium 130 can include optical discs, digital video discs (DVD), high definition digital versatile discs (HD DVD), compact discs (CD), laser discs, and magnetic media such as tape drives, floppy discs, and hard drives, solid state media such as flash memory,
- EEPROM electrically erasable programmable read-only memory
- PCRAM phase change random access memory
- Figure 1 further illustrates a the client 128 receiving a stream 132 of digital content containing a plurality of graphical items (e.g., 134-1 , . . . , 134-N).
- the plurality of graphical items e.g., 134-1 , . . . , 134-N
- the plurality of graphical items have a particular order (e.g., from 1 to N).
- the plurality of graphical items (e.g., 134-1 , . . . , 134-N) may be received in the order, or may be received out of order but including an indication of the order so that the items may be ordered by the client 128 after receipt thereof.
- the plurality of graphical items may have been stored on the data source 126, and requested by client 128 from server 124, such as by requesting the content of a web page, newspaper page, book, or similar digital content.
- the plurality of graphical items (e.g., 134-1 , . . . , 134-N) may have been generated by computing devices (e.g., server 124) and copied to the client 128 in near real time, such as via an RSS feed where the ordering is chronological as determined from a time stamp accompanying the item.
- the item of digital content may be a graphical item, such as text and/or images.
- a graphical item is a discrete quantity of digital content, for example, a news article with or without a headline, or an image with or without a caption.
- a sequence of graphical items (e.g., a plurality of graphical items having an order) may be received at the client 128 shown in Figure 1 for display electronically or physically on a single page or multiple pages.
- the plurality of graphical items can be arranged into a large number of different arrangements and displayed electronically or physically on a single page or multiple pages. Many such arrangements do not provide a reading order corresponding to the order of the sequence of graphical items (e.g., received order, designated order).
- the present disclosure addresses the issue of automatically placing document items on pages of some output device by bisecting the page into regions where each item is to be placed.
- the page can be divided into columns, with the graphical items positioned in the columns according to an order, such as in a newspaper-type arrangement.
- Dividing the page into various regions provides the advantage of allowing for a wide range of publications (e.g., designed to be arranged in such a format) to be produced.
- Publications where a page is not divided into columns can easily be simulated by defining areas for each graphical item that do not necessarily correspond to a column layout.
- Figures 2 and 3 illustrate examples of automatic page layout in accordance with the present disclosure that include both column arrangements and non-column arrangements. Figures 2 and 3 are discussed further below.
- FIG. 1 is a diagram illustrating recursive page bisecting according to an example of the present disclosure.
- Figure 2 shows a space 240A having one region 242A.
- Bisection of a region can be made by horizontal or vertical sectioning, dividing the region into two new regions.
- a region can be bisected in two different ways to achieve two different results. For example, space 240B reflects region 242A being bisected horizontally into a first horizontal region 242B and a second horizontal region 244B, while space 240C reflects region 242A being bisected vertically into a first vertical region 242C and a second vertical region 244C.
- Bisection of one (or both) regions of spaces 240B and 240C can continue (e.g., recursively) to create additional regions having different configurations, sizes and positions.
- space 240D reflects region 242C being bisected horizontally (and region 244C not being bisected) resulting in a first region 242D, a second region 244D, and a third region 246D.
- Space 240E reflects region 244C being bisected horizontally (and region 242C not being bisected) to result in a first region 242E, a second region 244E, and a third region 246E, that provides a different arrangement than that indicated for space 240D.
- Space 240F reflects one of regions 242C or 244C being bisected vertically to result in a first region 242F, a second region 244F, and a third region 246F, that provides different region configurations and relative
- region 244D could be bisected to provide 4 regions in space 240D
- region 244F could be bisected to provide 4 regions in space 240F. It can be appreciated from the bisection examples illustrated by Figure 2 that recursive bisection can be used to configure a space in many different ways, including various sized regions positioned as desired within the space.
- graphical items can be displayed within the regions. For example, all the graphical items could be displayed in a space (e.g., a page) having a single region. Alternatively, half of the graphical items could be displayed in each region of a space bisected into two regions. Examples of the present disclosure are not limited to any particular quantity of graphical items corresponding to a particular region, and equal numbers of graphical items do not have to be displayed in each region, for at least because the regions could be of different size and/or the graphical items may involve more or less text, or larger or smaller images than other graphical items.
- the plurality of ordered graphical items is also divided into groups of graphical items, with each group corresponding to a region. Bisecting the portions of the space and dividing the plurality of ordered graphical items can occur simultaneously, or without any intentional delay (e.g., substantially simultaneously, or at different times.
- Figure 3 is a diagram illustrating a page layout according to an example of the present disclosure.
- the page layout illustrated in Figure 3 was
- Page 350 (e.g., space) includes multiple regions, including a first region 354, a second region 356, a third region 358, a forth region 360, a fifth region 362, a sixth region 366, a seventh region 368, and regions for each of first image 364 (image 1 ) and second image 370 (image 2).
- the page header 352 can also be considered a region.
- the automatic page layout method attempts to use all available space on a page; however, regions do not have to occupy the entire space 350. That is, for readability, regions can be separated from each other and/or the edge of the space by a margin.
- minimum and maximum aspect ratios can be specified (e.g., given with the sequence of graphical items as input), and an automatic page layout solution becomes acceptable when all regions have aspect ratios within those bounds.
- Such constraints assure that areas containing text are not too wide on the page or too narrow.
- Regions can be further partitioned into areas such that multiple graphical items could be arranged within a particular region.
- the first region 354 is partitioned into four areas, 372, 374, 376, 378 to accommodate four graphical items (e.g., an article headline, an article first portion, an article second portion, and an article third portion) where the headline and portions of the same article are received as discrete graphical items.
- a graphical item will comprise an entire article, including a headline and a body.
- a region can then include a single area equivalent to the region, and the area can be further partitioned into sub-areas.
- an area can be further partitioned into a headline sub-area and at least one column sub-area.
- a headline of the article can be associated with and displayed in the headline sub-area, and the body of an article can be associated with and displayed across the column sub-areas.
- second region 356 can include (i.e., is equivalent to) one area which can be further partitioned into a second headline area 380 and two column areas 382 and 384.
- Each of the third, fourth, fifth, and sixth regions can include (i.e., is equivalent to) one area which can be further partitioned into a headline area (e.g., 385, 393, 389) and two column areas (e.g., 386/388, 394/396, 390/392).
- Seventh region 368 can include (i.e., is equivalent to) one area which can be further partitioned into a seventh headline area 395 and four column areas 397, 398, 399 and 391 , as shown.
- the regions associated with first image 364 and second image 370 can include (i.e., is equivalent to) one area which is not further partitioned.
- regions associated with a graphical item that includes an image could be further partitioned, for example into an image sub-area and a caption sub- area.
- the regions are positioned in the space in a reading order that maintains the order of the plurality of graphical items as between groups.
- the approach of the present disclosure endeavors to preserve a reading order provided by the input, so that content is provided in a designated order to provide requisite context between graphical items, or that content provided by an editor is not reordered without his knowledge.
- a reading order is substantially a combination of left-to-right and top-to-bottom ordering, arranged to provide a first graphical item in the plurality of graphical items in a top-left-most position of a space (e.g., page) and a last graphical item in the plurality of graphical items in a bottom- right-most position of the space. In this manner, those graphical items in regions wholly to the right or below a particular region contain graphical items that occur in the order after those graphical items in the particular region.
- the automatic page layout method of the present disclosure is based on recursive page division. Item geometry is not known but area percentages for those items can be provided as input, or determined, for example, based on relative numbers of characters of text found in particular graphical items.
- a page can be divided accordingly, and graphical items are displayed later in their assigned regions.
- automatic page layout is accomplished by recursive
- partitioning Some areas are further structured into columns (as in most newspapers), with items using an integer number of columns, which can be easily configured.
- the input file to the automatic page layout method can be an extension of XML obtained by converting RSS feeds and/or Wiki formats, among others. Different outputs can be generated, such as PDF, LATEX and dvi.
- the approach used by the automatic page layout method involves automatically producing pages in a format that is akin to a newspaper, where the page can be recursively decomposed through successive horizontal or vertical bisections.
- This approach is also called guillotine partitioning, and involves dividing a page area of a specified size into smaller regions through recursive bisections to contain each input element, such that the region areas match each requested area as evenly as possible.
- the sequence of graphical items can be divided into groups of items and the page can be divided, producing regions where each group can be placed.
- the optimal number of groups is chosen to partition the sequence of graphical items. If a sequence of M graphical items is received, the M graphical items can be divided into from one single group placed as a single region, to M independently placed groups having one item each. First, the items in each group are chosen. There are (M-1 )!/(K-1 )!/(M-K)! ways of dividing a sequence of M graphical items into K groups. Evaluating all grouping possibilities for optimum placement is unfeasible in most cases. Therefore, some reasonable way of dividing a sequence of graphical items into groups is chosen instead.
- the size and position of the page region assigned to each group is chosen.
- the size has to take into account the amount of area for the group, which can be summed from the percentage of area recommendations that can accompany the received sequence of graphical items, or be determined therefrom.
- the position of the regions affects the reading order. Consecutive groups of graphical items to be placed into regions (and optionally areas and columnar sub-areas) can be distributed into these regions/areas/columns from top to bottom and/or left to right, assuring that reading order is respected.
- a best possible placement can be achieved by reducing an error measure (e.g., reducing the maximum error among all items in the page, so that all individual errors are as similar as possible, causing their font sizes to be as similar as possible).
- the error measure is used to describe how well placement of a graphical item uses the page, and the final automatic page layout tries to obtain the best possible placement by minimizing this error measure.
- the placement error of a single graphical item is a measure of how well it fits into its assigned area.
- the first step in determining an error measurement is to process and normalize each item area (e.g., the percentage of the total space occupied by an area associated with a particular graphical item).
- a graphical item can be comprised of an image and/or text. If the graphical item is an image, its error can be measured by the proportion between the region height and the image's expected height determined by its fixed aspect ratio based on the image's width across a number of columns (c). If the graphical item is text, its error can be measured by the proportion between the normalized region area and the recommended area (which can be received with the graphical image).
- FIG. 4 is a block diagram illustrating a system 400 for performing an automatic page layout according to an example of the present disclosure.
- the system 400 includes an area estimates component 404, an automatic page layout (APL) component 408, and a renderer component 412.
- APL automatic page layout
- the area estimates component 404 receives a series of graphical items 402 and produces normalized area information 406.
- the normalized area information 406 is received by APL component 408, which produces position and size information 410.
- Renderer component 412 receives the graphical items 402 and the position and size information 410, and produces an output document 414, which is a PDF document in the illustrated example.
- the system 400 generates guillotine layouts that preserve some of the main aesthetic qualities that define the look and feel of a newspaper, as many readers are comfortable with this format. For instance, newspapers usually divide the page in a vertical grid of equal-width columns, where each story takes an integer number of columns. This helps the reader to perceive structure on the page, and also allows the method to generate layouts that are well aligned.
- the input data are the following: (1 ) a tuple (W, H, C),
- c is the number of columns that will be taken up by item / ' and h; is the height.
- ⁇ , W ⁇ c, l C.
- a n of items 402 ( Figure 4) is normalized by area estimates component 404 before being used by the APL component 408.
- a new sequence of normalized areas Ni, . . . , N n (i.e., normalized areas 406) is defined such that:
- APL component 408 ( Figure 4) uses a divide- and-conquer approach, and successively reduces a problem L (i, j, c, h) into two smaller sub-problems L (i, k, c, h k ) and L (k + 1 , j, c, h - h k ) in the case of a horizontal partition, or L (i, k, Ck, h) and L (k + 1 , j, c - Ck, h) in the case of a vertical partition. Note that each division splits both the current sequence and the region in smaller pieces.
- the height of the page can be divided in any real-valued point in the open interval (0, h), for each index k in / ' ⁇ k ⁇ j.
- one approach is to choose a splitting point based on the sum of areas of the items that will be at each side of the split, and appropriately choose a proportional fraction of h.
- the column grid since the column grid is respected, there will be c - 1 possible splitting points in the region, but for each one, different partitions of the sequence are considered.
- error (i, c, h) which represents the placement error for a single text in a region
- Item range [i . . . j], number of columns c and height h
- Pseudo Code Example II refers to the body of the recursive function L(), which is discussed above. As this problem exhibits the optimal sub-structure property (i.e., an optimal solution to a problem is constructed from optimal solutions to its sub-problems) and overlapping sub-problems (i.e., the same layout sub-problem is reached several times from different partitions), it is possible to solve it more efficiently by turning Pseudo Code Example II into a dynamic programming method by introducing a look-up table for memorization.
- the look-up table also enables the reconstruction of the guillotine layout from the optimal solution, and derivation of other layout parameters such as the position (xicide yi) of item / ' , as the function in Pseudo Code Example II computes the minimal error solution instead of generating the actual layout.
- a separate method is then used to reconstruct the layout from the look-up table, but this method is straightforward given the function in Pseudo Code Example II.
- the result of this process is the generation of a layout comprising the positions and sizes information 410 ( Figure 4) by the APL component 408.
- the method proceeds to the renderer component 412 ( Figure 4), where a typesetting engine (such as TeX or iText) is instructed to place each region in a page and then insert the corresponding graphical items.
- a typesetting engine such as TeX or iText
- renderer component 412 uses the TeX typesetting engine along with the LaTeX system.
- TeX may be programmed to tailor the final output document 414 for a better look, such as determining better column presentations, changing fonts or adding a page header for a newspaper.
- the difference in font sizes should not be too large, as a goal of the APL component 408 is to produce a layout whose areas match the items as close as possible.
- the method allocates areas based on the lengths of each article, many factors such as white space generated by line- breaking, justification, hyphenation, individual kerning and leading are not taken into account by the APL component 408. This is because these parameters are computed for a text block during the run time of the renderer 412, and as such these parameters are not considered by the APL component 408.
- a font balancing scheme is used. Since the layout method works on area estimates and has no knowledge of typographical parameters, the method uses font size information obtained from the rendering step in TeX and uses it to improve the initial area estimates from the APL component 408. The areas are repeatedly improved until a convergence or stop criteria is met. This approach is based on the minimization of a new objective function, using the layout method as part of its evaluation. Figure 5 shows how the two functions are connected to minimize the font size difference in layouts.
- Figure 5 is a diagram illustrating control of area estimates in automatic page layout with a font optimization method according to an example of the present disclosure.
- a sequence 502 of n areas A 1; A 2 , . . ., A n corresponding to the lengths of items, / ' are provided as an input to APL component 408.
- the APL component 408 determines a layout including position and size information 410 that minimizes the error measure as represented by graph 504.
- renderer 412 e.g., TeX
- renderer 412 Based on the received information, renderer 412 generates font sizes 514, which are fed back to an optimizer or optimization process 516.
- Optimizer 516 explores the domain of areas for each item, as represented by graph 518, and feeds back the resulting information to APL component 408.
- the function can be described by:
- ⁇ is an unknown function that takes a real-valued set R - - that can be described by the software inside the typesetting engine -
- any viable optimization strategy should be conservative when evaluating points from domain A, since the evaluation of F is usually an expensive operation. In this sense, a heuristic is used that attempts to minimize f by making small changes in item areas, and finding the set of areas that produces the least noticeable difference in font sizes.
- the heuristic works iteratively, by repeatedly increasing the requested area of the item that produced the smallest font size during the last run. This allows the system to control the precise amount of change to be made in an area, and avoid constant changes in geometry. A cooling parameter is used that decreases as the font size differences are reduced, ensuring convergence. The result of the process is the final output document 414.
- Stretching or shrinking an image to fit a rectangle is also disallowed, as information in the image may become illegible and/or aesthetically
- An exhaustive backtracking approach is used for finding the best configuration(s), but other less expensive approaches may be used if the number of images is large, such as local search methods or reducing the search space using pruning techniques.
- image / is scaled to fit one of the lengths of R,, so that it fits entirely in R,. Since the aspect ratios may not match, the placement may leave columns or rows of white space in the page, producing a misalignment that is accounted for as an error.
- the height, h r of the region is larger than the height, h p , of the image, and the width, ⁇ ⁇ , of the region is the same as the width, ⁇ ⁇ , of the image.
- the height, h r , of the region is the same as the height, h p , of the image, and the width, ⁇ ⁇ , of the region is larger than the width, ⁇ ⁇ , of the image.
- the error for this example is given by ⁇ ⁇ h ⁇ ( ⁇ - i).
- the first approach involves fixed image areas.
- the size for the images is chosen first, allowing them to use the available space of the page first. Texts will then share the remaining area.
- an image / uses an integer number of columns c, in page (W, H, C) and has a fixed aspect ratio ⁇ , ⁇ .
- the requested amount of page areaA used by image / is given in Equation VIII, using the definitions in Equation VII.
- Equation X Equatio
- the second approach uses a text area percentage.
- the second approach does not allow images to use the page area freely, but rather constrains their relative areas to take at most a user-defined percentage of the available space, leaving the remaining amount for texts to accommodate.
- ⁇ as the percentage of the page area that is to be occupied by text items, the following Equation XII may be used:
- a method to find optimal values for ⁇ could be developed using optimization techniques, but may be inefficient, as an optimal layout would be computed for several different values for ⁇ , wherein a single evaluation is already considered expensive. Moreover, for minimizing font size differences, an optimal value for ⁇ is chosen while ensuring that no other value produces a smaller font size difference. Therefore, another approach may be used to properly distribute areas between texts and images such that the maximum font size difference is minimized, as discussed below.
- the threshold ⁇ value can be set by a user. However, deciding an appropriate value is easier than the previous ⁇ value as it directly involves the font size, instead of a possibly more difficult decision that provides no guarantees in regard to the font size. Nevertheless, the same method could be applied to the second normalization approach: assuming that larger values of ⁇ always produce larger font sizes, a binary search procedure can be used to find the exact value for ⁇ that produces the minimum requested font size ⁇ .
- Figure 6 is a flow diagram illustrating a method 600 for performing an automatic page layout according to an example of the present disclosure.
- system 400 ( Figure 4) is configured to perform method 600.
- a plurality of graphical items including text items and image items is received.
- a space is recursively bisected into regions, wherein each region corresponds to one of the graphical items.
- at least one of the regions and a font size of at least one of the text items are modified to produce a layout that reduces noticeable differences in font sizes for the text items.
- an output of the layout is caused in which each region has been filled with that region's corresponding one of the graphical items.
- method 600 is performed by at least one processor.
- Method 600 further includes providing an area estimate for each of the regions to a renderer component, and generating font size information with the renderer component based on the area estimates, wherein the modifying at least one of the regions comprises modifying at least one of the area estimates based on the font size information generated with the renderer component.
- the method 600 further includes generating new font size information with the renderer component based on the modified at least one area estimate, performing an additional modification to the modified at least one area estimate based on the new font size information, and repeating the generating new font size information and the performing an additional modification until a
- the modifying at least one of the regions in method 600 includes iteratively modifying area estimates for the regions until a set of area estimates is determined that produces a least noticeable difference in font sizes for the text items.
- the method 600 further includes identifying one of the area estimates that produced a smallest font size during a current iteration, and increasing the identified area estimate during a next iteration.
- Method 600 an aspect ratio of each of the image items is preserved, and content of each of the image items is preserved by disallowing cropping of the image items.
- Method 600 according to one example further includes calculating a placement error for each of the regions that correspond to one of the image items based on a difference in aspect ratios of the image items and their corresponding regions.
- method 600 further includes calculating area estimates for the regions that correspond to the image items prior to determining area estimates for the regions that correspond to the text items, and calculating area estimates for the regions that correspond to the text items based on an overall remaining area after the area estimates for the regions that correspond to the image items are calculated.
- the area estimates for the regions that correspond to the image items are constrained based on a predetermined percentage.
- method 600 further includes generating an area estimate for each of the regions, generating font size information for each of the text items based on the area estimates, identifying a smallest font size in the generated font size information, and rejecting the generated area estimates as a possible layout solution if the identified smallest font size is less than a minimum font size constraint.
- the system includes an area estimates component to receive a plurality of graphical items including text items and image items, and generate normalized area information for each of the graphical items.
- the system includes an automatic page layout (APL) component to bisect a space into regions based on the normalized area information, each region corresponding to one of the graphical items.
- the system includes a renderer component to cause an output in which each region has been filled with that region's corresponding one of the graphical items, wherein the renderer component feeds back font size information to the APL component, and wherein the APL component modifies at least one of the regions based on the font size information.
- APL automatic page layout
- the APL component iteratively modifies area estimates for the regions based on the font size information provided by the renderer component until a set of area estimates is determined that produces a least noticeable difference in font sizes for the text items.
- the APL component according to one example identifies one of the area estimates that produced a smallest font size during a current iteration, and increases the identified area estimate during a next iteration.
- Another example of the present disclosure is directed to a non- transitory computer-readable medium having computer-readable instructions stored thereon that, when executed by at least one processor, causes the at least one processor to: receive a plurality of graphical items including text items and image items; bisect a space into regions, each region corresponding to one of the graphical items; modify at least one of the regions based on font size information generated by a rendering process to produce a layout that reduces noticeable differences in font sizes for the text items; and cause an output of the layout in which each region has been filled with that region's corresponding one of the graphical items.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Processing Or Creating Images (AREA)
- Document Processing Apparatus (AREA)
Abstract
A method for performing an automatic page layout includes receiving a plurality of graphical items including text items and image items, and bisecting a space into regions, each region corresponding to one of the graphical items. The method includes modifying at least one of the regions and a font size of at least one of the text items to produce a layout that reduces noticeable differences in font sizes for the text items, and causing an output of the layout in which each region has been filled with that region's corresponding one of the graphical items.
Description
AUTOMATIC PAGE LAYOUT FOR TEXT ITEMS AND IMAGE ITEMS
Background
[0001] The amount and use of digital content is increasing, including text and/or images, and combinations thereof. Digital content can be displayed
electronically, such as by a computer monitor, television, or mobile device, or printed to physical print media, such as a newspaper, magazine, or single paper sheet. Digital content can include an arrangement of items that are arranged and presented to a user in many different formats, such as by a webpage, in a newspaper or magazine, blog, e-mail listing, photo album, among others.
Displaying digital content can involve page layout decisions regarding
placement of text and images on a page, including their relationship to one another.
Brief Description of the Drawings
[0002] Figure 1 is a diagram illustrating a computing system according to an example of the present disclosure.
[0003] Figure 2 is a diagram illustrating recursive page bisecting according to an example of the present disclosure.
[0004] Figure 3 is a diagram illustrating a page layout according to an example of the present disclosure.
[0005] Figure 4 is a block diagram illustrating a system for performing an automatic page layout according to an example of the present disclosure.
[0006] Figure 5 is a diagram illustrating control of area estimates in automatic page layout with a font optimization method according to an example of the present disclosure.
[0007] Figure 6 is a flow diagram illustrating a method for performing an automatic page layout according to an example of the present disclosure.
Detailed Description
[0008] In the following detailed description, reference is made to the
accompanying drawings which form a part hereof, and in which is shown by way of illustration specific examples in which the disclosure may be practiced. It is to be understood that other examples may be utilized and structural or logical changes may be made without departing from the scope of the present disclosure. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present disclosure is defined by the appended claims. It is to be understood that features of the various examples described herein may be combined, in part or whole, with each other, unless specifically noted otherwise.
[0009] The rapid growth of the Web and mobile devices as new forms of media targets allowed the emergence of new trends and models for publishing and delivering content to readers. For example, most news agencies today have already both a print and a digital form of their publication available on the Web to their customers. Along with new publication targets, the amount of available information has also grown immensely. This removed the centralized power of traditional publishers to a more distributed and personalized model for delivering information that is relevant to readers. However, traditional publications such as magazines still appeal to customers, mainly because of their aesthetic quality and readability. Digital documents on the other hand, have not been able to make full use of graphic design principles commonly found on printed
publications. One reason for this is that digital documents are constantly being changed and updated, both to new content and form factors, whereas traditional publications can be designed in advance, as their content is static.
[0010] To overcome such issues, recently there has been a growing interest in research of automated document layout technologies to deal more rapidly with a large volume of information and provide customization. Several methods have been proposed for automating the layout task. Most methods proposed in the recent literature are based on a collection of previously-generated page templates, which involve expensive graphic design work, and for some scenarios do not offer a reasonable compromise. Other methods use template- free generative methods, often based on constrained optimization techniques, in order to model a space of valid layouts, bound to a set of graphic design guidelines. The approach used largely depends on a trade-off between how often or quickly documents are to be produced and design uniqueness. For example, a monthly magazine is usually presented with a high-quality layout that conveys the branding and identity of that magazine. In this case, there will be few, if not at all, variations in a document's content, and a careful design crafted by a professional will be worth the investment. On the other hand, content that should be produced (and consumed) quickly, such as aggregated material from the Web, should be produced quickly and tailored to a large user base. This is the scenario where automated layout tools are typically more relevant.
[0011] Methods for automatic page layout have gained more attention from research in recent years, in a shift from lower-level typographical concerns such as word spacing and line-breaking. This involves distributing large pieces of content such as blocks of text and images into areas of a page, with respect to certain aesthetic criteria or design guidelines. Depending on the approach, optimizing layouts can be a computationally challenging task. Most known document layout problems are in NP-Hard. However, for some current document production workflows, such as Variable Data Printing (VDP), an expensive method is not well-suited.
[0012] To provide aesthetic quality while still generating layouts efficiently, existing methods usually rely on simpler layout models, or use a library of existing layout templates and enumerate possible adaptations to the existing layouts. When some characteristics regarding the content to be input are
known in advance, such as average text length or a fixed number of items, these models tend to work well. Conversely, it is very difficult to apply these same models when content length and input size are unknown beforehand. Because of such difficulty, several approaches rely on optimization heuristics to search a smaller random sample of possible documents for a satisfactory layout. However, these methods are also known for requiring long execution times to find approximations that are not guaranteed to be optimal.
Deterministic methods are usually used for environments where documents are to be produced quickly and in large volumes (such as in VDP), given their efficiency and predictability of results.
[0013] A guillotine-based method may be used in contexts such as VDP or Web content aggregation, since it can be efficient to generate automatically and is flexible in terms of content placement, as opposed to the use of templates. The guillotine format is commonly used in widespread publications such as newspapers, and thus it is a comfortable and familiar format for readers. In a guillotine layout, a page is hierarchically divided in smaller sections by horizontal or vertical lines that run across the extremities of the larger section without crossing any items in the way.
[0014] There seems to be a general agreement both from research in
typography and graphic design guides that excessive type size variation may harm the legibility and appearance of a document. However, by allowing font sizes to change slightly, an optimization method can find better layouts than methods that use fixed sizes if those changes are small enough to pass undetected by the reader's eyes.
[0015] One implementation is directed to an enhanced guillotine-based layout method that produces unique document layouts tailored to previously-selected content that can be used in different workflows for generating documents. An advantage of the approach is that each page of a document may hold an unspecified number of items with unknown sizes. The method relies on a set of constraints and finds an optimal solution based on reasonable aesthetic goals, including optimizing the use of areas by textual elements.
[0016] One disadvantage of some previous approaches is that textual elements may be presented in a range of different text size configurations, which causes an undesired effect on the aesthetic appearance and legibility of a document. To enhance document quality, some implementations disclosed herein adapt areas on a page in an attempt to minimize the difference in font sizes, thus improving the legibility and appearance of a document while still benefiting from the flexibility of providing font size variations. Moreover, the role of images in this framework is considered as well.
[0017] One implementation is directed to a mixed content automatic layout (MCAL) method, and more specifically to a method for automatically generating template-free document layouts with adaptable font sizes, minimal text size variation, and image inclusion support. By allowing changes in font sizes and optimizing the size differences between text blocks, better document layouts may be derived when compared to existing approaches, supporting both textual and graphical elements (e. g., images). Since the font size for a text is allowed to change (although such changes are minimized), the number of acceptable solutions is larger, thus allowing a non-designer to automatically produce layouts for very unpredictable inputs.
[0018] An example scenario is a Web-to-print application, where a user may collect a number of different parts of web pages and assemble them into a single document using automated layout technology. The user has no knowledge and does not interfere in how the document is assembled from the selected content. In this and other similar scenarios, the use of template-based layouts is challenging because of the unknown number and size of the input. Some previous works do not allow font sizes to change, and thus compensate by arbitrarily deleting content, which is undesirable in most cases. Moreover, few works provide support for inclusion of both images and texts in a template- free layout.
[0019] When page items have fixed sizes or the page itself is fixed (e.g., when the layout is template-based) it is difficult to find a feasible layout when the number and size of items are unknown. The present disclosure does not sacrifice the content (by deleting text or cropping images) to allow the
successful generation of a document. The present disclosure does not use arbitrary rules for removing content (such as removing the last paragraph of a text or removing the edges of a picture), as such rules may frequently cause undesired user surprise.
[0020] The present disclosure includes a system and method for automatic page layout. Computer software can be adapted for arranging text and images according to the user's direction. For example, software can be used to manually place and arrange stored images, such as photographs, video frames, and blocks of text, on a page (e.g., a webpage, a printer output, a display). For example, web page design programs and word processing program can include such capabilities. However, such programs typically enable a user to place images and text as a user directs. Other computer software, such as e-mail packages and personal web page design software, can allow a user to designate areas to display received digital content. For example, a user may designate a portion of an e-mail program within which to display received e-mail messages, or may design a personal web page to display streaming news articles received from selected sources within one area of a page, streaming blog entries of interest to display in another area of the same page, and streaming movies to display in yet another area. These programs often use a standard template or involve a user's input to set the various areas. As such, these programs do not constitute an automated page layout.
[0021] Figure 1 is a diagram illustrating a computing system 120 according to an example of the present disclosure. The computing system 120 shown in Figure 1 is a networked computing system. The network 122 can be a private network such as a local area network or wide area network, or can be a public network, such as the Internet. However, examples of the present disclosure are not limited to a particular computing system configuration.
[0022] The computing system 120 can be comprised of a number of computing resources communicatively coupled to the network 122. Figure 1 shows a server 124 having a data source 126, and a client 128. Server 124 and client 128 are computing devices. Client 128 includes processors 129
communicatively coupled to a non-transitory computer-readable medium 130.
The non-transitory computer-readable medium 130 is structured to store programs 133 that are executed by the processors 129 and/or data.
[0023] Client 128 is further communicatively coupled to a production device 135 (e.g., electronic display, printer, etc.). Client 128 can also be communicatively coupled to an external computer-readable memory 151 . The client 128 can cause an output to the production device 135, for example, as a result of executing instructions of programs stored in non-transitory computer-readable medium 130, by at least one processor 129, to implement a method of automatic page layout according to the present disclosure. Causing an output can include, but is not limited to, displaying text and images to an electronic display and/or printing text and images to a tangible medium (e.g., paper). For example, a plurality of text articles and images, with or without captions, can be arranged to appear in an arrangement similar to a newspaper or magazine and caused to be output to a computer monitor or printed to a tangible medium.
[0024] Server 124 and client 128 are communicatively coupled to one another through the network 122. While a single server is shown in Figure 1 , the computing system can be comprised of multiple interconnected computing resources, such as servers 124 and clients 128. A computing resource (e.g., 124, 128) can include control circuitry such as a processor, a state machine, application specific integrated circuit (ASIC), controller, and/or similar machine. As used herein, the indefinite articles "a" and/or "an" can indicate more than one of the named object. Thus, for example, "a processor" can include one processor or more than one processor, such as a parallel processing
arrangement.
[0025] The control circuitry can have a structure that provides a given
functionality, and/or execute computer-readable instructions that are stored on a non-transitory computer-readable medium (e.g., 126, 130). The non-transitory computer-readable medium can be integral, or communicatively coupled, to a computing resource such as client 128, in either in a wired or wireless manner. For example, the non-transitory computer-readable medium 130 can be an internal memory, a portable memory, a portable disk, or a memory located internal to another computing resource (e.g., enabling the computer-readable
instructions to be downloaded over the Internet). The non-transitory computer- readable medium 130 can have computer-readable instructions stored thereon that are executed by the control circuitry (e.g., processor) to provide a particular functionality.
[0026] The non-transitory computer-readable medium 130, as used herein, can include volatile and/or non-volatile memory. Volatile memory can include memory that depends upon power to store information, such as various types of dynamic random access memory (DRAM), among others. Non-volatile memory can include memory that does not depend upon power to store information. Examples of non-volatile memory can include solid state media such as flash memory, EEPROM, phase change random access memory (PCRAM), among others. The non-transitory computer-readable medium 130 can include optical discs, digital video discs (DVD), high definition digital versatile discs (HD DVD), compact discs (CD), laser discs, and magnetic media such as tape drives, floppy discs, and hard drives, solid state media such as flash memory,
EEPROM, phase change random access memory (PCRAM), as well as other types of machine-readable media.
[0027] Figure 1 further illustrates a the client 128 receiving a stream 132 of digital content containing a plurality of graphical items (e.g., 134-1 , . . . , 134-N). The plurality of graphical items (e.g., 134-1 , . . . , 134-N) have a particular order (e.g., from 1 to N). The plurality of graphical items (e.g., 134-1 , . . . , 134-N) may be received in the order, or may be received out of order but including an indication of the order so that the items may be ordered by the client 128 after receipt thereof.
[0028] The plurality of graphical items (e.g., 134-1 , . . . , 134-N) may have been stored on the data source 126, and requested by client 128 from server 124, such as by requesting the content of a web page, newspaper page, book, or similar digital content. The plurality of graphical items (e.g., 134-1 , . . . , 134-N) may have been generated by computing devices (e.g., server 124) and copied to the client 128 in near real time, such as via an RSS feed where the ordering is chronological as determined from a time stamp accompanying the item.
[0029] The item of digital content may be a graphical item, such as text and/or images. A graphical item is a discrete quantity of digital content, for example, a news article with or without a headline, or an image with or without a caption. A sequence of graphical items (e.g., a plurality of graphical items having an order) may be received at the client 128 shown in Figure 1 for display electronically or physically on a single page or multiple pages. Depending on the quantity of graphical items, the plurality of graphical items can be arranged into a large number of different arrangements and displayed electronically or physically on a single page or multiple pages. Many such arrangements do not provide a reading order corresponding to the order of the sequence of graphical items (e.g., received order, designated order).
[0030] The present disclosure addresses the issue of automatically placing document items on pages of some output device by bisecting the page into regions where each item is to be placed. According to some implementations, the page can be divided into columns, with the graphical items positioned in the columns according to an order, such as in a newspaper-type arrangement. Dividing the page into various regions (e.g., columns) provides the advantage of allowing for a wide range of publications (e.g., designed to be arranged in such a format) to be produced. Publications where a page is not divided into columns can easily be simulated by defining areas for each graphical item that do not necessarily correspond to a column layout. Figures 2 and 3 illustrate examples of automatic page layout in accordance with the present disclosure that include both column arrangements and non-column arrangements. Figures 2 and 3 are discussed further below.
[0031] An example method of the present disclosure for automatic page layout is based on a "divide and conquer" approach. For example, a client computing device (e.g., Figure 1 at 128) may receive information defining a page width and height, and a list of graphical elements to be placed, and optionally an indication of a percentage of the total page space that is recommended to be allocated to each graphical item. Through consecutive bisections, the space of the page can be divided into regions in an attempt to allocate the recommended portions of the space within which to display each graphical item.
[0032] Figure 2 is a diagram illustrating recursive page bisecting according to an example of the present disclosure. Figure 2 shows a space 240A having one region 242A. Bisection of a region can be made by horizontal or vertical sectioning, dividing the region into two new regions. A region can be bisected in two different ways to achieve two different results. For example, space 240B reflects region 242A being bisected horizontally into a first horizontal region 242B and a second horizontal region 244B, while space 240C reflects region 242A being bisected vertically into a first vertical region 242C and a second vertical region 244C.
[0033] Bisection of one (or both) regions of spaces 240B and 240C can continue (e.g., recursively) to create additional regions having different configurations, sizes and positions. For example, space 240D reflects region 242C being bisected horizontally (and region 244C not being bisected) resulting in a first region 242D, a second region 244D, and a third region 246D. Space 240E reflects region 244C being bisected horizontally (and region 242C not being bisected) to result in a first region 242E, a second region 244E, and a third region 246E, that provides a different arrangement than that indicated for space 240D. Space 240F reflects one of regions 242C or 244C being bisected vertically to result in a first region 242F, a second region 244F, and a third region 246F, that provides different region configurations and relative
positioning than either of the arrangements provided for spaces 240D or 240E.
[0034] Other bisections than those illustrated in Figure 2 could be additionally or alternatively made. For example, region 244D could be bisected to provide 4 regions in space 240D, or region 244F could be bisected to provide 4 regions in space 240F. It can be appreciated from the bisection examples illustrated by Figure 2 that recursive bisection can be used to configure a space in many different ways, including various sized regions positioned as desired within the space.
[0035] As mentioned, graphical items can be displayed within the regions. For example, all the graphical items could be displayed in a space (e.g., a page) having a single region. Alternatively, half of the graphical items could be displayed in each region of a space bisected into two regions. Examples of the
present disclosure are not limited to any particular quantity of graphical items corresponding to a particular region, and equal numbers of graphical items do not have to be displayed in each region, for at least because the regions could be of different size and/or the graphical items may involve more or less text, or larger or smaller images than other graphical items.
[0036] However, according to the present disclosure, as portions of the space (e.g., regions) are bisected, the plurality of ordered graphical items is also divided into groups of graphical items, with each group corresponding to a region. Bisecting the portions of the space and dividing the plurality of ordered graphical items can occur simultaneously, or without any intentional delay (e.g., substantially simultaneously, or at different times.
[0037] Figure 3 is a diagram illustrating a page layout according to an example of the present disclosure. The page layout illustrated in Figure 3 was
determined according to a method of the present disclosure. The page layout is briefly discussed first, followed by a further discussion of the methods used to arrive at a particular page layout. Page 350 (e.g., space) includes multiple regions, including a first region 354, a second region 356, a third region 358, a forth region 360, a fifth region 362, a sixth region 366, a seventh region 368, and regions for each of first image 364 (image 1 ) and second image 370 (image 2). The page header 352 can also be considered a region. The automatic page layout method attempts to use all available space on a page; however, regions do not have to occupy the entire space 350. That is, for readability, regions can be separated from each other and/or the edge of the space by a margin. To avoid very wide or narrow regions, minimum and maximum aspect ratios can be specified (e.g., given with the sequence of graphical items as input), and an automatic page layout solution becomes acceptable when all regions have aspect ratios within those bounds. Such constraints assure that areas containing text are not too wide on the page or too narrow.
[0038] Regions can be further partitioned into areas such that multiple graphical items could be arranged within a particular region. For example, the first region 354 is partitioned into four areas, 372, 374, 376, 378 to accommodate four graphical items (e.g., an article headline, an article first portion, an article
second portion, and an article third portion) where the headline and portions of the same article are received as discrete graphical items.
[0039] However, more usual, a graphical item will comprise an entire article, including a headline and a body. A region can then include a single area equivalent to the region, and the area can be further partitioned into sub-areas. For example, an area can be further partitioned into a headline sub-area and at least one column sub-area. A headline of the article can be associated with and displayed in the headline sub-area, and the body of an article can be associated with and displayed across the column sub-areas. For example, second region 356 can include (i.e., is equivalent to) one area which can be further partitioned into a second headline area 380 and two column areas 382 and 384. Each of the third, fourth, fifth, and sixth regions (e.g., 358, 360, 362, 366 and 368 respectively) can include (i.e., is equivalent to) one area which can be further partitioned into a headline area (e.g., 385, 393, 389) and two column areas (e.g., 386/388, 394/396, 390/392). Seventh region 368 can include (i.e., is equivalent to) one area which can be further partitioned into a seventh headline area 395 and four column areas 397, 398, 399 and 391 , as shown.
[0040] The regions associated with first image 364 and second image 370 can include (i.e., is equivalent to) one area which is not further partitioned.
However, regions associated with a graphical item that includes an image could be further partitioned, for example into an image sub-area and a caption sub- area.
[0041] The regions are positioned in the space in a reading order that maintains the order of the plurality of graphical items as between groups. The approach of the present disclosure endeavors to preserve a reading order provided by the input, so that content is provided in a designated order to provide requisite context between graphical items, or that content provided by an editor is not reordered without his knowledge. A reading order is substantially a combination of left-to-right and top-to-bottom ordering, arranged to provide a first graphical item in the plurality of graphical items in a top-left-most position of a space (e.g., page) and a last graphical item in the plurality of graphical items in a bottom- right-most position of the space. In this manner, those graphical items in
regions wholly to the right or below a particular region contain graphical items that occur in the order after those graphical items in the particular region.
[0042] The automatic page layout method of the present disclosure is based on recursive page division. Item geometry is not known but area percentages for those items can be provided as input, or determined, for example, based on relative numbers of characters of text found in particular graphical items.
According to one example, a page can be divided accordingly, and graphical items are displayed later in their assigned regions. According to some implementations, automatic page layout is accomplished by recursive
partitioning. Some areas are further structured into columns (as in most newspapers), with items using an integer number of columns, which can be easily configured.
[0043] The input file to the automatic page layout method can be an extension of XML obtained by converting RSS feeds and/or Wiki formats, among others. Different outputs can be generated, such as PDF, LATEX and dvi.
[0044] The approach used by the automatic page layout method according to one example involves automatically producing pages in a format that is akin to a newspaper, where the page can be recursively decomposed through successive horizontal or vertical bisections. This approach is also called guillotine partitioning, and involves dividing a page area of a specified size into smaller regions through recursive bisections to contain each input element, such that the region areas match each requested area as evenly as possible.
[0045] When a sequence of graphical items is available for placement (e.g., received at client 128 shown in Figure 1 ), decisions are made about how to partition the page. Many previous approaches for page layout with optimal fitting do not preserve the reading order established by the order of the graphical items (e.g., received order or indicated order). Readers are
comfortable with layouts based on rows and columns, and the automatic page layout method of the present disclosure seeks to achieve a similar result. If several graphical items are to be placed on a page, the sequence of graphical items can be divided into groups of items and the page can be divided, producing regions where each group can be placed. The optimal number of
groups is chosen to partition the sequence of graphical items. If a sequence of M graphical items is received, the M graphical items can be divided into from one single group placed as a single region, to M independently placed groups having one item each. First, the items in each group are chosen. There are (M-1 )!/(K-1 )!/(M-K)! ways of dividing a sequence of M graphical items into K groups. Evaluating all grouping possibilities for optimum placement is unfeasible in most cases. Therefore, some reasonable way of dividing a sequence of graphical items into groups is chosen instead.
[0046] The size and position of the page region assigned to each group is chosen. The size has to take into account the amount of area for the group, which can be summed from the percentage of area recommendations that can accompany the received sequence of graphical items, or be determined therefrom. The position of the regions affects the reading order. Consecutive groups of graphical items to be placed into regions (and optionally areas and columnar sub-areas) can be distributed into these regions/areas/columns from top to bottom and/or left to right, assuring that reading order is respected.
[0047] Finally, a choice is made among different region placement possibilities. As a region can be divided into rows and/or columns, the best choice for placing a region corresponding to a particular group will reduce some error measure. A best possible placement can be achieved by reducing an error measure (e.g., reducing the maximum error among all items in the page, so that all individual errors are as similar as possible, causing their font sizes to be as similar as possible). The error measure is used to describe how well placement of a graphical item uses the page, and the final automatic page layout tries to obtain the best possible placement by minimizing this error measure. The placement error of a single graphical item is a measure of how well it fits into its assigned area.
[0048] The first step in determining an error measurement is to process and normalize each item area (e.g., the percentage of the total space occupied by an area associated with a particular graphical item). A graphical item can be comprised of an image and/or text. If the graphical item is an image, its error can be measured by the proportion between the region height and the image's
expected height determined by its fixed aspect ratio based on the image's width across a number of columns (c). If the graphical item is text, its error can be measured by the proportion between the normalized region area and the recommended area (which can be received with the graphical image).
[0049] These steps will now be described in further detail with reference to Figure 4. Figure 4 is a block diagram illustrating a system 400 for performing an automatic page layout according to an example of the present disclosure. The system 400 includes an area estimates component 404, an automatic page layout (APL) component 408, and a renderer component 412. As shown in Figure 4, the area estimates component 404 receives a series of graphical items 402 and produces normalized area information 406. The normalized area information 406 is received by APL component 408, which produces position and size information 410. Renderer component 412 receives the graphical items 402 and the position and size information 410, and produces an output document 414, which is a PDF document in the illustrated example.
[0050] The system 400 according to one example generates guillotine layouts that preserve some of the main aesthetic qualities that define the look and feel of a newspaper, as many readers are comfortable with this format. For instance, newspapers usually divide the page in a vertical grid of equal-width columns, where each story takes an integer number of columns. This helps the reader to perceive structure on the page, and also allows the method to generate layouts that are well aligned.
[0051] Given a page and a sequence of graphical items (e.g., articles) to be put on the page, the input data are the following: (1 ) a tuple (W, H, C),
corresponding to the page width, height and number of vertical columns, respectively; and (2) a sequence of n areas A1; A2, . . ., An corresponding to the lengths of items, /', where 1≤ i≤ n.
[0052] In one example, each item, /', is mapped to a rectangular region R, = (c,; hi) of the page, where c, is the number of columns that will be taken up by item /' and h; is the height. Note that items use an integer number of columns, and thus the region width is implicitly computed as ω, = W■ c, l C. All columns on a page grid have the same width, i.e., ω0 = W 7 C.
[0053] To further characterize the guillotine layout model, the following constraints are accounted for:
[0054] 1 . The page is entirely and homogeneously covered by graphical items, i.e., ^g-^ft* = wm.
[0055] 2. The order of the input sequence is respected, as items may have a reading order determined by the input order.
[0056] 3. All items are placed on the page, i.e., no items are deleted.
[0057] 4. The regions R, = (c,; hi) are not rotated and no intersection between any two items is allowed.
[0058] 5. For legibility and aesthetic reasons, text regions are not allowed to be too thin or too wide. Thus, a pair of real values Amin and Amax restricts the minimum and maximum tolerated aspect ratios for each region R„ whose aspect ratios are given by ω .
[0059] 6. Any number of items may be placed on a page. For this, texts are allowed to shrink or expand their sizes in order to fit a particular region R,.
[0060] The input order preservation from the second constraint listed above is reflected in the document layout. In a guillotine layout, a sequence of items will be split either by a horizontal or a vertical line that runs across the page containing the sequence. Thus, the items in the sequence up to a split index k are allocated to the left or top side of the guillotine partition (depending on the split orientation), while the items after / are allocated to the right or bottom side. A western reader usually scans a page in a top-bottom and left-to-right fashion, and thus an item k appears higher or to the left of item k + 1 .
[0061] This, however, provides a partial reading order, and some ambiguities may occur in some layouts when scanning the items sequentially (i.e., an item following another one may appear below or to the right of the item being currently read, but not to the left and top simultaneously). Nevertheless, by disallowing changes in the input order when finding a layout, a more efficient method can be obtained, as testing different combinations of input order would involve a more computationally expensive approach that would not produce solutions in a practicable time.
[0062] To satisfy the sixth constraint listed above, items share the page using relative areas, taking a percentage of the page instead of a fixed value. As such, the sequence A1} . . . , An of items 402 (Figure 4) is normalized by area estimates component 404 before being used by the APL component 408. In this case, a new sequence of normalized areas Ni, . . . , Nn (i.e., normalized areas 406) is defined such that:
[0063] Equation I
[0065] For simplicity, the sum S (/', )) of all areas between A, and Aj are defined to be:
[0066] Equation II [0067] £¾
[0068] The relative area /V, of an item /' is thus defined as:
[0069] Equation III
[0071] Given the input description, the layout involves distributing a sequence [/' . . . , j] of items Λ/,, . . . , Λ/, to a set of regions R = R,, . . . , Rj contained in a larger region (c, h). In one example, APL component 408 (Figure 4) uses a divide- and-conquer approach, and successively reduces a problem L (i, j, c, h) into two smaller sub-problems L (i, k, c, hk) and L (k + 1 , j, c, h - hk) in the case of a horizontal partition, or L (i, k, Ck, h) and L (k + 1 , j, c - Ck, h) in the case of a vertical partition. Note that each division splits both the current sequence and the region in smaller pieces. The smallest sub-problem will then be the allocation of a single item /' into a region R, = (c, h), which can be trivially evaluated. However, at each problem subdivision step, options of values for k and k (or Ck) are considered. As such, the method explores different splitting possibilities, searching for a solution that minimizes a certain error measure for item placement.
[0072] In the horizontal splitting case, the height of the page can be divided in any real-valued point in the open interval (0, h), for each index k in /' < k < j. To
explore horizontal splitting possibilities, one approach is to choose a splitting point based on the sum of areas of the items that will be at each side of the split, and appropriately choose a proportional fraction of h. For vertical splits, since the column grid is respected, there will be c - 1 possible splitting points in the region, but for each one, different partitions of the sequence are considered.
[0073] The error measure measures the quality of the association between an item /' and a region r = (c, h). In the case of a single item, the allocation is considered to be as good as is the similarity between the requested area /V, and the relative area offered by the region r.
[0074] Equation IV
[0075] offered =
[0076] Given the assumption that texts may be resized to fit an area, offered areas that are smaller or larger than the requested area are penalized
accordingly by the error measure. Moreover, regions that have incompatible aspect ratios are discarded (e.g., by attributing an infinite error e = °°). The complete error function, called as error (i, c, h), which represents the placement error for a single text in a region, is given in the following example:
[0077] Pseudo Code Example I
Input: Item index i, number of columns c in the region and region height h Output: Placement error for text i
w ^ W■ c / C
ar <— w / h
if ar < Amin or ar > Amax then
return∞
end
offered <- c■ h / (C■ H)
Scale difference ratio by the text area
return w■ h ■ (max (offered/Nj,Nj/offered) - 1 )
[0078] Given the error function from Pseudo Code Example I, the next step is to find a sequence of partitions of the page whose area distribution is equally similar to the relative areas requested by the items. The choice of partitioning minimizes the largest placement error found in one of the two resulting sub- problems from that partition. The method that accomplishes this task and selects the minimal error among candidate layouts is presented in the following example:
[0079] Pseudo Code Example II
Input: Item range [i . . . j], number of columns c and height h
Output: minimal layout error
if i = j then
return error (i, c, h)
end
minerr <—∞
Try horizontal splits and select one with minimal error
for k <— i to j - 1 do
hk ^ h ■ S (i, k) / S (i, j)
The error of this partition attempt is the worse between the two sub- problems
err «- max (L (i, k, c, hk), L (k + 1 , j, c, h - hk))
minerr <— min (minerr, err)
end
Try vertical splits and select one with minimal error
for Ck <— 1 to c - 1 do
for k <— i to j - 1 do
err «- max (L (i, k, ck> h), L (k + 1 , j, c - ck> h))
minerr <— min (minerr, err)
end
end
return minerr
[0080] Pseudo Code Example II refers to the body of the recursive function L(), which is discussed above. As this problem exhibits the optimal sub-structure property (i.e., an optimal solution to a problem is constructed from optimal solutions to its sub-problems) and overlapping sub-problems (i.e., the same layout sub-problem is reached several times from different partitions), it is possible to solve it more efficiently by turning Pseudo Code Example II into a dynamic programming method by introducing a look-up table for memorization. The look-up table also enables the reconstruction of the guillotine layout from the optimal solution, and derivation of other layout parameters such as the position (x„ yi) of item /', as the function in Pseudo Code Example II computes the minimal error solution instead of generating the actual layout. A separate method is then used to reconstruct the layout from the look-up table, but this method is straightforward given the function in Pseudo Code Example II. The result of this process is the generation of a layout comprising the positions and sizes information 410 (Figure 4) by the APL component 408.
[0081] After generating the sizes and positions R, for each item /', the method proceeds to the renderer component 412 (Figure 4), where a typesetting engine (such as TeX or iText) is instructed to place each region in a page and then insert the corresponding graphical items. One implementation of renderer component 412 uses the TeX typesetting engine along with the LaTeX system. When generating the final output document 414, TeX may be programmed to tailor the final output document 414 for a better look, such as determining better column presentations, changing fonts or adding a page header for a newspaper.
[0082] However, given the input constraints and typical instances, there will seldom be a case where region areas will perfectly match an item's requested area. Some reasons for this include the addition of too many or too few items on a page, very large articles or very strict aspect ratio constraints. Since items are allowed by design to change their sizes in order to fit a particular area, the manipulation of the font size can be used to correctly fill a region. By using a binary search method and feedback information from the typesetting engine, the exact font size that fills up a region may be determined. The leading (i.e., spacing between baselines) is scaled together with the font size, in order to preserve the same proportional distance between lines for every item.
[0083] In general, the difference in font sizes should not be too large, as a goal of the APL component 408 is to produce a layout whose areas match the items as close as possible. However, since the method allocates areas based on the lengths of each article, many factors such as white space generated by line- breaking, justification, hyphenation, individual kerning and leading are not taken into account by the APL component 408. This is because these parameters are computed for a text block during the run time of the renderer 412, and as such these parameters are not considered by the APL component 408.
[0084] While some changes are small, this is not always the case. For example, if a text containing a single word is to be rendered with a larger font, its area estimate may be computed as the length of that word. However, if the text is rendered into a narrow column, that word may be broken by a hyphen into two lines, doubling the amount of area and forcing a reduction in font size. Given the interplay between these factors, a separate rendering process will not
always be able to produce the best results than if it were able to interact with the APL component 408. An approach is described below that uses the renderer 412 to feed back information back into the APL component 408 for improving area estimation and thus producing better results.
[0085] To solve the font size issues with the layout method discussed above, a font balancing scheme is used. Since the layout method works on area estimates and has no knowledge of typographical parameters, the method uses font size information obtained from the rendering step in TeX and uses it to improve the initial area estimates from the APL component 408. The areas are repeatedly improved until a convergence or stop criteria is met. This approach is based on the minimization of a new objective function, using the layout method as part of its evaluation. Figure 5 shows how the two functions are connected to minimize the font size difference in layouts.
[0086] Figure 5 is a diagram illustrating control of area estimates in automatic page layout with a font optimization method according to an example of the present disclosure. As shown in Figure 5, a sequence 502 of n areas A1; A2, . . ., An corresponding to the lengths of items, /', are provided as an input to APL component 408. The APL component 408 determines a layout including position and size information 410 that minimizes the error measure as represented by graph 504. The position and size information 410 corresponds to the set 506 of regions R = Ri , . . . , Rn, which is provided as an input to renderer 412 (e.g., TeX). Based on the received information, renderer 412 generates font sizes 514, which are fed back to an optimizer or optimization process 516. Optimizer 516 explores the domain of areas for each item, as represented by graph 518, and feeds back the resulting information to APL component 408. The function can be described by:
[0087] Equation V
[0088] f (A , A2, . . . , An) = max (F) - min (F)
[0089] Equation VI
[0090] F = r (f?i , /¾, . . . , Rn)
[0091] where: F is the set of resulting font sizes from the typesetting engine for rendered areas R = (f?i , R2, . . . , Rn)-
[0092] Considering that τ is an unknown function that takes a real-valued set R - - that can be described by the software inside the typesetting engine -, any viable optimization strategy should be conservative when evaluating points from domain A, since the evaluation of F is usually an expensive operation. In this sense, a heuristic is used that attempts to minimize f by making small changes in item areas, and finding the set of areas that produces the least noticeable difference in font sizes.
[0093] The heuristic works iteratively, by repeatedly increasing the requested area of the item that produced the smallest font size during the last run. This allows the system to control the precise amount of change to be made in an area, and avoid constant changes in geometry. A cooling parameter is used that decreases as the font size differences are reduced, ensuring convergence. The result of the process is the final output document 414.
[0094] The inclusion of images in the layout model involves a different approach than the one used for supporting texts. A difference between texts and images is that while the area for a text is specified by a single parameter (e.g., text length) and its containing rectangle may be freely distorted, the same is not true for images. Images are specified by a width and a height, and aspect ratio distortion or cropping are not allowed for aesthetic and legibility reasons. To support the inclusion of images, the method described above is modified to take images into account in several stages of the layout workflow. For this approach, the following factors are assumed:
[0095] 1 . Arbitrary image cropping is disallowed, as no information about the image is known to the layout method. Although methods exist for detecting and removing uninteresting parts of an image, this method may fail to preserve important details in some occasions.
[0096] 2. Stretching or shrinking an image to fit a rectangle is also disallowed, as information in the image may become illegible and/or aesthetically
unpleasing.
[0097] 3. Images may be presented in a number of different sizes. However, their aspect ratios are preserved.
[0098] 4. To preserve vertical alignment, images may assume widths that are multiples of the column width ω0 = W 7 C of a page, and not other widths. Along with the previous factor, this suggests that each image can be presented in C different ways, where C is the number of columns in a page.
[0099] Given these factors, one example for enabling image inclusion in APL involves the following steps:
[00100] 1 . As text areas were previously given relative percentages of the page, image areas are converted to the same scale for fairness of area distribution.
[00101] 2. As images can extend their widths to an integer number of columns, all different image configurations are evaluated in conjunction to find the best possible configuration of image sizes.
[00102] 3. For each different image configuration, the sequence of relative areas Λ/ι , A/2, . . . , Nn, is recomputed, as the method relies on these values for correctly distributing the page area between items.
[00103] 4. As images are not distorted or cropped, an image /, with aspect ratio Δ, placed in region R, with aspect ratio ¾ leaves some white space on R, in case Δ,·≠ &St . Thus, the evaluation of the placement error of an image takes into account the difference in the aspect ratios of image /, and its containing region R,.
[00104] Steps 2 and 3 above suggest a combinatorial search method to enumerate all different sizing possibilities for images, given the finite range of discrete sizes. For example, placing two images in a document divided into C = 2 columns involves the evaluation of configurations (ci = 1 ; C2 = 1 ), (ci = 1 ; C2 = 2), (ci = 2; C2 = 1 ) and (ci = 2; C2 = 2) for finding a layout with minimal error. An exhaustive backtracking approach is used for finding the best configuration(s), but other less expensive approaches may be used if the number of images is large, such as local search methods or reducing the search space using pruning techniques.
[00105] After enumerating a single column configuration (ci, C2, . . . , c;NJ for images /i , /2, . . . , the layout is evaluated using the method from Pseudo Code Example II. All configurations are then enumerated and evaluated, and
the configuration that generated the smallest error is recorded and used for the final layout.
[00106] To place an image /, in region R, and satisfy the first and second factors listed above, image /, is scaled to fit one of the lengths of R,, so that it fits entirely in R,. Since the aspect ratios may not match, the placement may leave columns or rows of white space in the page, producing a misalignment that is accounted for as an error.
[00107] The scaling of an image to a region will be described with reference to two examples. In one example, the height, hr, of the region is larger than the height, hp, of the image, and the width, ωΓ, of the region is the same as the width, ωρ, of the image. The error for this example is given by ω■ h■ (^- 1), where Ap = u)p/hp and Ar = coJhr. In another example, the height, hr, of the region is the same as the height, hp, of the image, and the width, ωΓ, of the region is larger than the width, ωρ, of the image. The error for this example is given by ω■ h■ ( ^- i).
%
[00108] To evaluate the placement error for images, the function from Pseudo Code Example I is modified to consider aspect ratio differences instead, as shown in the following Pseudo Code Example III, which gives an example of the placement error function imgerror (i, c, h) for an image in a region:
[00109] Pseudo Code Example III
Input: Item index i, number of columns c in the region and region height h Output: Placement error for image i
Δ w/h
Column numbers of image and region do not match, reject placement if c≠ c, then
return∞
end
Scale error by the region area to account for the unused white space
return w ■ h ■ (max (Δ/Δ,, Δ/Δ) - 1 )
[00110] It may not be immediately clear what may be the best criteria for deciding how a page's real estate should be distributed between texts and images: whereas images take up fixed amounts of area, texts may be freely scaled and take up any non-zero amount of space in the page. Moreover, the distribution should meet the goal of minimizing font size differences. Two
alternatives for obtaining the relative areas Λ/ι , N2, . . . , Nn of fixed-size images and text are described below. It will also be described how these approaches may be adapted to consider font size constraints and thereby meet the goal of minimizing font size differences.
[00111] The first approach involves fixed image areas. In the first approach, when setting areas for each item, the size for the images is chosen first, allowing them to use the available space of the page first. Texts will then share the remaining area. Recall from the description above that an image / uses an integer number of columns c, in page (W, H, C) and has a fixed aspect ratio Δ,·. Thus, the requested amount of page areaA used by image / is given in Equation VIII, using the definitions in Equation VII.
[00112] Equation VII
[00113] *¾ = s f ¾ =
[00114] Equation VIII
[00116] As images are scaled in the same units as the page area, text areas are converted from their lengths to the page unit as well for proper normalization. Consider T, and /, the requested amount of area by text item 1 < /' < nt and image item 1≤j≤ ng, respectively. l/ and H are the width and height of the page, respectively. It follows that:
[00122]
[00123] A drawback of this approach, however, is that the area distribution assumes that images will take full advantage of their given areas and the corresponding regions will be sized accordingly. As a consequence, for certain configurations of items, the optimal layout will contain very large images, leaving little room for text items. Texts will thus be forced to squeeze their font sizes to fit the remaining space, producing very small font sizes as a result.
Furthermore, a document with a large number of images and/or very tall images may not be produced if the total requested area by the images exceeds the page area, even though the images could theoretically be squeezed to fit the page. Another approach is to carefully balance the distribution of space between texts and images, which will be described next.
[00124] The second approach uses a text area percentage. The second approach does not allow images to use the page area freely, but rather constrains their relative areas to take at most a user-defined percentage of the available space, leaving the remaining amount for texts to accommodate. Thus, given an additional parameter σ as the percentage of the page area that is to be occupied by text items, the following Equation XII may be used:
[00125] Equation XII
[00127] where A is the requested area for item /'. Note that setting relative areas using Equation XII ensures that the sum of relative areas uses the entire page, as
**img **iKsr
[00129] A user may experiment with different values for σ until a
satisfactory layout is found. A method to find optimal values for σ could be developed using optimization techniques, but may be inefficient, as an optimal layout would be computed for several different values for σ, wherein a single evaluation is already considered expensive. Moreover, for minimizing font size
differences, an optimal value for σ is chosen while ensuring that no other value produces a smaller font size difference. Therefore, another approach may be used to properly distribute areas between texts and images such that the maximum font size difference is minimized, as discussed below.
[00130] By requiring a minimum font size constraint for a document, it can be ensured that no font will be too small to read and unlike the σ percentage discussed above, this constraint may be computed automatically without requiring user input. One example method for obtaining the best set of image sizes and meeting font size constraints, works as follows:
[00131] 1 . A minimum font size constraint β is introduced, and the approach described above for obtaining relative areas is used.
[00132] 2. The backtracking method described above is used to find image sizes and to minimize the font size differences as well, using the method described above.
[00133] 3. When testing for a configuration of image sizes, the minimum produced font size f is computed and the solution is rejected if f < β.
[00134] By rejecting solutions that produce texts with font sizes smaller than β, the method ensures that: the solution space is thoroughly explored; a solution with a minimal font β is found; and that font size differences are minimized using the method described above.
[00135] The threshold β value can be set by a user. However, deciding an appropriate value is easier than the previous σ value as it directly involves the font size, instead of a possibly more difficult decision that provides no guarantees in regard to the font size. Nevertheless, the same method could be applied to the second normalization approach: assuming that larger values of σ always produce larger font sizes, a binary search procedure can be used to find the exact value for σ that produces the minimum requested font size β.
[00136] Figure 6 is a flow diagram illustrating a method 600 for performing an automatic page layout according to an example of the present disclosure. In one example, system 400 (Figure 4) is configured to perform method 600. At 602 in method 600, a plurality of graphical items including text items and image items is received. At 604, a space is recursively bisected into regions, wherein
each region corresponds to one of the graphical items. At 606, at least one of the regions and a font size of at least one of the text items are modified to produce a layout that reduces noticeable differences in font sizes for the text items. At 608, an output of the layout is caused in which each region has been filled with that region's corresponding one of the graphical items. In one implementation, method 600 is performed by at least one processor.
[00137] Method 600 according to one example further includes providing an area estimate for each of the regions to a renderer component, and generating font size information with the renderer component based on the area estimates, wherein the modifying at least one of the regions comprises modifying at least one of the area estimates based on the font size information generated with the renderer component. In one form of this example, the method 600 further includes generating new font size information with the renderer component based on the modified at least one area estimate, performing an additional modification to the modified at least one area estimate based on the new font size information, and repeating the generating new font size information and the performing an additional modification until a
convergence criteria is satisfied.
[00138] In one example, the modifying at least one of the regions in method 600 includes iteratively modifying area estimates for the regions until a set of area estimates is determined that produces a least noticeable difference in font sizes for the text items. In one form of this example, the method 600 further includes identifying one of the area estimates that produced a smallest font size during a current iteration, and increasing the identified area estimate during a next iteration.
[00139] In one form of method 600, an aspect ratio of each of the image items is preserved, and content of each of the image items is preserved by disallowing cropping of the image items. Method 600 according to one example further includes calculating a placement error for each of the regions that correspond to one of the image items based on a difference in aspect ratios of the image items and their corresponding regions.
[00140] In one example, method 600 further includes calculating area estimates for the regions that correspond to the image items prior to determining area estimates for the regions that correspond to the text items, and calculating area estimates for the regions that correspond to the text items based on an overall remaining area after the area estimates for the regions that correspond to the image items are calculated. In one form of this example, the area estimates for the regions that correspond to the image items are constrained based on a predetermined percentage.
[00141] In another example, method 600 further includes generating an area estimate for each of the regions, generating font size information for each of the text items based on the area estimates, identifying a smallest font size in the generated font size information, and rejecting the generated area estimates as a possible layout solution if the identified smallest font size is less than a minimum font size constraint.
[00142] Another example of the present disclosure is directed to a system for performing an automatic page layout. The system includes an area estimates component to receive a plurality of graphical items including text items and image items, and generate normalized area information for each of the graphical items. The system includes an automatic page layout (APL) component to bisect a space into regions based on the normalized area information, each region corresponding to one of the graphical items. The system includes a renderer component to cause an output in which each region has been filled with that region's corresponding one of the graphical items, wherein the renderer component feeds back font size information to the APL component, and wherein the APL component modifies at least one of the regions based on the font size information.
[00143] In one form of this example, the APL component iteratively modifies area estimates for the regions based on the font size information provided by the renderer component until a set of area estimates is determined that produces a least noticeable difference in font sizes for the text items. The APL component according to one example identifies one of the area estimates
that produced a smallest font size during a current iteration, and increases the identified area estimate during a next iteration.
[00144] Another example of the present disclosure is directed to a non- transitory computer-readable medium having computer-readable instructions stored thereon that, when executed by at least one processor, causes the at least one processor to: receive a plurality of graphical items including text items and image items; bisect a space into regions, each region corresponding to one of the graphical items; modify at least one of the regions based on font size information generated by a rendering process to produce a layout that reduces noticeable differences in font sizes for the text items; and cause an output of the layout in which each region has been filled with that region's corresponding one of the graphical items.
[00145] Although specific examples have been illustrated and described herein, a variety of alternate and/or equivalent implementations may be substituted for the specific examples shown and described without departing from the scope of the present disclosure. This application is intended to cover any adaptations or variations of the specific examples discussed herein.
Therefore, it is intended that this disclosure be limited by the claims and the equivalents thereof.
Claims
1 . A method for performing an automatic page layout, comprising:
receiving a plurality of graphical items including text items and image items;
bisecting a space into regions, each region corresponding to one of the graphical items;
modifying at least one of the regions and a font size of at least one of the text items to produce a layout that reduces noticeable differences in font sizes for the text items;
causing an output of the layout in which each region has been filled with that region's corresponding one of the graphical items; and
wherein above steps are performed by at least one processor.
2. The method of claim 1 , and further comprising:
providing an area estimate for each of the regions to a renderer component;
generating font size information with the renderer component based on the area estimates; and
wherein the modifying at least one of the regions comprises modifying at least one of the area estimates based on the font size information generated with the renderer component.
3. The method of claim 2, and further comprising:
generating new font size information with the renderer component based on the modified at least one area estimate; and
performing an additional modification to the modified at least one area estimate based on the new font size information.
4. The method of claim 3, and further comprising:
repeating the generating new font size information and the performing an additional modification until a convergence criteria is satisfied.
5. The method of claim 1 , wherein modifying at least one of the regions comprises:
iteratively modifying area estimates for the regions until a set of area estimates is determined that produces a least noticeable difference in font sizes for the text items.
6. The method of claim 5, and further comprising:
identifying one of the area estimates that produced a smallest font size during a current iteration; and
increasing the identified area estimate during a next iteration.
7. The method of claim 1 , and further comprising:
preserving an aspect ratio of each of the image items;
preserving content of each of the image items by disallowing cropping of the image items.
8. The method of claim 1 , and further comprising:
calculating a placement error for each of the regions that correspond to one of the image items based on a difference in aspect ratios of the image items and their corresponding regions.
9. The method of claim 1 , and further comprising:
calculating area estimates for the regions that correspond to the image items prior to determining area estimates for the regions that correspond to the text items; and
calculating area estimates for the regions that correspond to the text items based on an overall remaining area after the area estimates for the regions that correspond to the image items are calculated.
10. The method of claim 9, and further comprising:
constraining the area estimates for the regions that correspond to the image items based on a predetermined percentage.
1 1 . The method of claim 1 , and further comprising:
generating an area estimate for each of the regions;
generating font size information for each of the text items based on the area estimates;
identifying a smallest font size in the generated font size information; and rejecting the generated area estimates as a possible layout solution if the identified smallest font size is less than a minimum font size constraint.
12. A system for performing an automatic page layout, comprising:
an area estimates component to receive a plurality of graphical items including text items and image items, and generate normalized area information for each of the graphical items;
an automatic page layout (APL) component to bisect a space into regions based on the normalized area information, each region corresponding to one of the graphical items;
a renderer component to cause an output in which each region has been filled with that region's corresponding one of the graphical items; and
wherein the renderer component feeds back font size information to the APL component, and wherein the APL component modifies at least one of the regions based on the font size information.
13. The system of claim 12, wherein the APL component iteratively modifies area estimates for the regions based on the font size information provided by the renderer component until a set of area estimates is determined that produces a least noticeable difference in font sizes for the text items.
14. The system of claim 13, wherein the APL component identifies one of the area estimates that produced a smallest font size during a current iteration, and increases the identified area estimate during a next iteration.
15. A non-transitory computer-readable medium having computer-readable instructions stored thereon that, when executed by at least one processor, causes the at least one processor to:
receive a plurality of graphical items including text items and image items;
bisect a space into regions, each region corresponding to one of the graphical items;
modify at least one of the regions based on font size information generated by a rendering process to produce a layout that reduces noticeable differences in font sizes for the text items; and
cause an output of the layout in which each region has been filled with that region's corresponding one of the graphical items.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/US2014/036151 WO2015167525A1 (en) | 2014-04-30 | 2014-04-30 | Automatic page layout for text items and image items |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/US2014/036151 WO2015167525A1 (en) | 2014-04-30 | 2014-04-30 | Automatic page layout for text items and image items |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2015167525A1 true WO2015167525A1 (en) | 2015-11-05 |
Family
ID=54359072
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/US2014/036151 Ceased WO2015167525A1 (en) | 2014-04-30 | 2014-04-30 | Automatic page layout for text items and image items |
Country Status (1)
| Country | Link |
|---|---|
| WO (1) | WO2015167525A1 (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108845961A (en) * | 2018-06-29 | 2018-11-20 | 武汉斗鱼网络科技有限公司 | A kind of creation vertex buffer method, apparatus and readable storage medium storing program for executing |
| CN111178447A (en) * | 2019-12-31 | 2020-05-19 | 北京市商汤科技开发有限公司 | Model compression method, image processing method and related device |
| US20220004698A1 (en) * | 2020-07-06 | 2022-01-06 | Canon Kabushiki Kaisha | Information processing apparatus, information processing method, and storage medium |
| CN113989404A (en) * | 2021-11-05 | 2022-01-28 | 北京字节跳动网络技术有限公司 | Image processing method, apparatus, device, storage medium and program product |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7561722B2 (en) * | 2005-12-14 | 2009-07-14 | Xerox Corporation | System and method for interactive document layout |
| US20090204891A1 (en) * | 2005-08-19 | 2009-08-13 | Vistaprint Technologies Limited | Automated product layout |
| US20090309894A1 (en) * | 2008-06-17 | 2009-12-17 | Canon Kabushiki Kaisha | Electronic layout generation based on visual context |
| US20120042240A1 (en) * | 2010-08-10 | 2012-02-16 | Oliveira Joao Batista Sousade | System and method for automatic page layout |
| US20130014008A1 (en) * | 2010-03-22 | 2013-01-10 | Niranjan Damera-Venkata | Adjusting an Automatic Template Layout by Providing a Constraint |
-
2014
- 2014-04-30 WO PCT/US2014/036151 patent/WO2015167525A1/en not_active Ceased
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090204891A1 (en) * | 2005-08-19 | 2009-08-13 | Vistaprint Technologies Limited | Automated product layout |
| US7561722B2 (en) * | 2005-12-14 | 2009-07-14 | Xerox Corporation | System and method for interactive document layout |
| US20090309894A1 (en) * | 2008-06-17 | 2009-12-17 | Canon Kabushiki Kaisha | Electronic layout generation based on visual context |
| US20130014008A1 (en) * | 2010-03-22 | 2013-01-10 | Niranjan Damera-Venkata | Adjusting an Automatic Template Layout by Providing a Constraint |
| US20120042240A1 (en) * | 2010-08-10 | 2012-02-16 | Oliveira Joao Batista Sousade | System and method for automatic page layout |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108845961A (en) * | 2018-06-29 | 2018-11-20 | 武汉斗鱼网络科技有限公司 | A kind of creation vertex buffer method, apparatus and readable storage medium storing program for executing |
| CN108845961B (en) * | 2018-06-29 | 2020-07-10 | 武汉斗鱼网络科技有限公司 | Method and device for creating vertex buffer area and readable storage medium |
| CN111178447A (en) * | 2019-12-31 | 2020-05-19 | 北京市商汤科技开发有限公司 | Model compression method, image processing method and related device |
| CN111178447B (en) * | 2019-12-31 | 2024-03-08 | 北京市商汤科技开发有限公司 | Model compression method, image processing method and related device |
| US20220004698A1 (en) * | 2020-07-06 | 2022-01-06 | Canon Kabushiki Kaisha | Information processing apparatus, information processing method, and storage medium |
| US11947895B2 (en) * | 2020-07-06 | 2024-04-02 | Canon Kabushiki Kaisha | Information processing apparatus for representing a web page using external fonts, and information processing method, and storage medium thereof |
| CN113989404A (en) * | 2021-11-05 | 2022-01-28 | 北京字节跳动网络技术有限公司 | Image processing method, apparatus, device, storage medium and program product |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9348801B2 (en) | System and method for automatic page layout | |
| JP6725714B2 (en) | System and method for automatic conversion of interactive sites and applications that support mobile and other viewing environments | |
| US8291314B2 (en) | Arranging graphic objects on a page | |
| US11727206B2 (en) | Systems and methods for applying layout to documents | |
| US9269323B2 (en) | Image layout for a display | |
| US20050071781A1 (en) | Single pass automatic photo album page layout | |
| CN111723555B (en) | Plane typesetting method and system | |
| US10789745B2 (en) | Information processing apparatus and information processing method | |
| JP6054330B2 (en) | Image layout generation apparatus, image product generation system, image layout generation method, image layout generation program, and recording medium | |
| CN106484669B (en) | A kind of automatic composing method of Classification Oriented informative advertising newspaper | |
| AU2013359428A1 (en) | Preserving layout of region of content during modification | |
| WO2015167525A1 (en) | Automatic page layout for text items and image items | |
| CN114282081A (en) | Media file display processing method and device, electronic equipment and readable storage medium | |
| US20220100913A1 (en) | Automated event detection and photo product creation | |
| KR20210060808A (en) | Document editing device to check whether the font applied to the document is a supported font and operating method thereof | |
| CN104462089A (en) | Data processing method and device | |
| JP7040255B2 (en) | Editing support program, editing support method and editing support device | |
| Piccoli et al. | Optimal pagination and content mapping for customized magazines | |
| CN115828883B (en) | Document content rearrangement method and device, electronic display equipment and medium | |
| de Oliveira | Two algorithms for automatic page layout and possible applications | |
| Piccoli et al. | Balancing font sizes for flexibility in automated document layout | |
| JP6474373B2 (en) | Shape extraction program, shape extraction method, and shape extraction apparatus | |
| CN116092097A (en) | Method and device for typesetting image content, electronic display equipment and medium | |
| CN107122345B (en) | Data typesetting method and device | |
| JP2018067153A (en) | Shape extraction program, shape extraction method, and shape extraction apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14890896 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 14890896 Country of ref document: EP Kind code of ref document: A1 |