HK1213705B - Document classification using multiscale text fingerprints - Google Patents

Description

Document Classification Using Multi-Scale Text Fingerprints

Background Art

The present invention relates to methods and systems for classifying electronic documents, and more particularly to systems and methods for screening unsolicited electronic communications (spam) and detecting fraudulent online documents.

Unsolicited electronic communications (also known as spam) constitute a significant portion of global communications traffic, affecting both computer and telephone messaging services. Spam can take many forms, from unsolicited email communications to spam messages disguised as user comments on various Internet sites (e.g., blogs and social networking sites). Spam consumes valuable hardware resources, impacts productivity, and is considered annoying and disruptive by many users of communications services and/or the Internet.

Online fraud, particularly in the form of phishing and identity theft, is posing an increasing threat to Internet users worldwide. Sensitive identifying information (such as user names, IDs, passwords, national identification numbers, and medical records, bank and credit card details) fraudulently obtained by international criminal networks operating on the Internet is used to withdraw private funds and/or further sold to third parties. In addition to causing direct financial losses to individuals, online fraud also has a range of harmful side effects, such as increased security costs for companies, higher retail prices and bank fees, falling stock values, lower wages, and decreased tax revenues.

In an exemplary phishing attempt, a fake website (also called a clone) may masquerade as a genuine webpage belonging to an online retailer or financial institution, asking the user to enter some personal information (e.g., username or password) or some financial information (e.g., credit card number, account number, or security code). Once the unsuspecting user submits this information, it can be collected by the fake website. In addition, the user may be directed to another webpage that can install malware on the user's computer. The malware (e.g., virus, Trojan horse) can continue to steal personal information by recording the keys typed by the user when visiting certain webpages, and can transform the user's computer into a platform for launching other phishing and spam attacks.

In the case of spam email or email scams, software running on a user's or email service provider's computer system can be used to classify email messages as spam/not spam (or scam/legitimate), and even differentiate between different types of messages, for example, product offers, adult content, and Nigerian scams. Spam/scam messages can then be directed to a special folder or deleted. Similarly, software running on a content provider's computer system can be used to intercept spam/scam messages posted to a website hosted by the corresponding content provider and prevent the corresponding message from being displayed, or display a warning to users of the website that the corresponding message may be scam or spam.

Several methods have been proposed for identifying spam and/or online scams, including matching the originating address of a message to lists of known illegal or trusted addresses (techniques known as blacklists and whitelists, respectively), searching for certain words or word patterns (e.g., "refinance," "stocks"), and analyzing message headers. Feature extraction/matching methods are sometimes used in conjunction with automated data classification methods (e.g., Bayesian screening, neural networks).

Some proposed methods use hashing to produce a compact representation of electronic text messages. Such representations allow efficient inter-message comparisons, which are used for spam or fraud detection purposes.

Spammers and online scammers attempt to evade detection by using various obfuscation methods, such as misspelling certain words, embedding spam and/or fraudulent content within larger blocks of text disguised as legitimate documents, and changing the form and/or content of messages from one distribution wave to another. Anti-spam and anti-fraud methods that use hashing are often susceptible to such obfuscation because small changes in text can produce substantially different hashes. Successful detection can therefore benefit from methods and systems that can identify polymorphic spam and fraud.

Summary of the Invention

According to one aspect, a client computer system includes at least one processor configured to determine a text fingerprint of a target electronic document such that a length of the text fingerprint is constrained between a lower limit and an upper limit, wherein the lower limit and the upper limit are predetermined. Determining the text fingerprint includes: selecting a plurality of text tokens of the target electronic document; and in response to selecting the plurality of text tokens, determining a fingerprint segment size based on the upper and lower limits and based on a count of the selected plurality of text tokens. Determining the text fingerprint further includes: determining a plurality of fingerprint segments, each of the plurality of fingerprint segments being determined based on a hash of a distinct text token in the selected plurality of text tokens, each fingerprint segment consisting of a character sequence whose length is selected to be equal to the fingerprint segment size; and concatenating the plurality of fingerprint segments to form the text fingerprint.

According to another aspect, a server computer system includes at least one processor configured to execute transactions with a plurality of client systems, wherein the transactions include: receiving a text fingerprint from a client system of the plurality of client systems, the text fingerprint determined for a target electronic document such that a length of the text fingerprint is constrained between a lower limit and an upper limit, wherein the lower limit and the upper limit are predetermined; and sending a target tag to the client system indicating a document category to which the target electronic document belongs. Determining the text fingerprint includes: selecting a plurality of text tags of the target electronic document; and in response to selecting the plurality of text tags, determining a fingerprint segment size based on the upper and lower limits and based on a count of the selected plurality of text tags. Determining the text fingerprint further includes: determining a plurality of fingerprint segments, each of the plurality of fingerprint segments being determined based on a hash of a distinct text tag from the selected plurality of text tags, each fingerprint segment consisting of a character sequence whose length is selected to be equal to the fingerprint segment size; and concatenating the plurality of fingerprint segments to form the text fingerprint. Determining the target label includes: retrieving a reference fingerprint from a database of reference fingerprints, the reference fingerprint being determined for a reference electronic document belonging to the category, the reference fingerprint being selected based on a length of the reference fingerprint such that the length of the reference fingerprint is between an upper limit and a lower limit; and determining whether the target electronic document belongs to the category based on a result of comparing the text fingerprint with the reference fingerprint.

According to another aspect, a method includes determining, using at least one processor of a client computer system, a text fingerprint of a target electronic document such that a length of the text fingerprint is constrained between a lower limit and an upper limit, wherein the lower limit and the upper limit are predetermined. Determining the text fingerprint includes: selecting a plurality of text tokens of the target electronic document; and, in response to selecting the plurality of text tokens, determining a fingerprint segment size based on the upper and lower limits and based on a count of the plurality of selected text tokens. Determining the text fingerprint further includes: determining a plurality of fingerprint segments, each of the plurality of fingerprint segments being determined based on a hash of a distinct text token in the plurality of selected text tokens, each fingerprint segment consisting of a character sequence whose length is selected to be equal to the fingerprint segment size; and concatenating the plurality of fingerprint segments to form the text fingerprint.

According to another aspect, a method includes using at least one processor of a server computer system configured to execute transactions with a plurality of client systems to: receive a text fingerprint from a client system of the plurality of client systems, the text fingerprint determined for a target electronic document such that a length of the text fingerprint is constrained between a lower limit and an upper limit, wherein the lower limit and the upper limit are predetermined; and send a target tag determined for the target electronic document to the client system, the target tag indicating a document category to which the target electronic document belongs. Determining the text fingerprint includes: selecting a plurality of text tags for the target electronic document; and in response to selecting the plurality of text tags, determining a fingerprint segment size based on the upper and lower limits and based on a count of the selected plurality of text tags. Determining the text fingerprint further includes: determining a plurality of fingerprint segments, each of the plurality of fingerprint segments being determined based on a hash of a distinct text tag from the selected plurality of text tags, each fingerprint segment consisting of a character sequence whose length is selected to be equal to the fingerprint segment size; and concatenating the plurality of fingerprint segments to form the text fingerprint. Determining the target label includes: retrieving a reference fingerprint from a database of reference fingerprints, the reference fingerprint being determined for a reference electronic document belonging to the category, the reference fingerprint being selected based on a length of the reference fingerprint such that the length of the reference fingerprint is between an upper limit and a lower limit; and determining whether the target electronic document belongs to the category based on a result of comparing the text fingerprint with the reference fingerprint.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing aspects and advantages of the present invention will be better understood after reading the following detailed description and after referring to the accompanying drawings, in which:

1 shows an exemplary anti-spam/anti-fraud system including a security server protecting multiple client systems, according to some embodiments of the present invention.

FIG2-A shows an exemplary hardware configuration of a client computer system according to some embodiments of the present invention.

FIG2-B shows an exemplary hardware configuration of a secure server computer system according to some embodiments of the present invention.

FIG2-C shows an exemplary hardware configuration of a content server computer system according to some embodiments of the present invention.

FIG. 3-A shows an exemplary spam email message including a block of text, according to some embodiments of the present invention.

FIG. 3-B shows an exemplary spam blog comment including a block of text, according to some embodiments of the present invention.

FIG. 3-C illustrates an exemplary fraudulent web page including multiple blocks of text, according to some embodiments of the present invention.

4-A illustrates an exemplary spam/scam detection transaction between a client computer and a secure server, according to some embodiments of the present invention.

4-B illustrates an exemplary spam/scam detection transaction between a content server and a security server according to some embodiments of the present invention.

5 shows exemplary target indicators for a target electronic document, including text fingerprints and other spam/fraud identification data, according to some embodiments of the present invention.

6 shows a diagram of an exemplary set of applications executing on a client system according to some embodiments of the invention.

FIG7 illustrates an exemplary sequence of steps performed by the fingerprint calculator of FIG6 according to some embodiments of the present invention.

FIG8 shows an exemplary determination of a text fingerprint of a target block of text according to some embodiments of the present invention.

9 shows multiple fingerprints determined for a target text block at various magnification and reduction factors, according to some embodiments of the present invention.

10 illustrates an exemplary sequence of steps performed by a fingerprint computer to determine a reduced fingerprint according to some embodiments of the present invention.

FIG. 11 shows an exemplary application executing on a secure server according to some embodiments of the present invention.

12 shows a diagram of an exemplary document classifier executed on a secure server according to some embodiments of the invention.

13 shows spam detection rates obtained in a computer experiment involving analysis of actual spam message streams, the analysis being performed according to some embodiments of the present invention; comparing the detection rates with those obtained by conventional methods.

DETAILED DESCRIPTION

In the following description, it should be understood that all enumerated connections between structures can be direct operational connections or indirect operational connections through intermediary structures. An element set includes one or more elements. Any enumeration of elements should be understood to refer to at least one element. A plurality of elements includes at least two elements. Unless otherwise required, any described method steps do not necessarily need to be performed in the specific order described. A first element (e.g., data) derived from a second element encompasses a first element that is equal to the second element, as well as a first element and optionally other data generated by processing the second element. Making a determination or decision based on a parameter encompasses making a determination or decision based on a parameter and optionally based on other data. Unless otherwise specified, an indicator of some quantity/data may be the quantity/data itself, or an indicator different from the quantity/data itself. Unless otherwise specified, a hash is the output of a hash function. Unless otherwise specified, a hash function is a mathematical transformation that maps a sequence of symbols (e.g., characters, bits) into a number or bit string. Computer-readable media encompass non-transitory media such as magnetic, optical, and semiconductor storage media (e.g., hard drives, optical disks, flash memories, DRAM), as well as communication links such as conductive cables and optical fiber links. According to some embodiments, the present invention provides, among other things, a computer system comprising hardware (eg, one or more processors) programmed to perform the methods described herein and a computer-readable medium encoding instructions to perform the methods described herein.

The following description illustrates embodiments of the invention by way of example and not necessarily by way of limitation.

FIG1 shows an exemplary anti-spam/anti-fraud system 10 according to some embodiments of the present invention. System 10 includes a content server 12, a sender system 13, a security server 14, and a plurality of client systems 16a-c, all of which are connected by a communication network 18. Network 18 may be a wide area network (e.g., the Internet), while portions of network 18 may also include local area networks (LANs).

In some embodiments, content server 12 is configured to receive user-contributed content (e.g., articles, blog entries, media uploads, comments, etc.) from a plurality of users and to organize, format, and distribute such content to third parties (e.g., client systems 16a-c). An exemplary embodiment of content server 12 is an email server that provides electronic message delivery to client systems 16a-c. Another embodiment of content server 12 is a computer system that hosts a blog or social networking site. In some embodiments, user-contributed content circulates over network 18 in the form of electronic documents (also referred to as target documents in the following description). Electronic documents include web pages (e.g., HTML documents) and electronic messages (e.g., emails and short message service (SMS) messages, etc.). Some of the user-contributed data received at server 12 may include unsolicited and/or fraudulent messages and documents.

In some embodiments, sender system 13 comprises a computer system that sends unsolicited communications (e.g., spam email messages) to client systems 16a-c. Such messages may be received at server 12 and subsequently sent to client systems 16a-c. Alternatively, messages received at server 12 may be made available (e.g., via a web interface) for retrieval by client systems 16a-c. In other embodiments, sender system 13 may send unsolicited communications (e.g., spam blog comments or spam emails posted to social networking sites) to content server 12. Client systems 16a-c may then retrieve such communications via a protocol (e.g., Hypertext Transfer Protocol (HTTP)).

The security server 14 may include one or more computer systems that perform classification of electronic documents (as shown in detail below). Performing such classification may include identifying unsolicited messages (spam) and/or fraudulent electronic documents (e.g., phishing messages and web pages). In some embodiments, performing such classification includes collaborative spam/fraud detection transactions between the security server 14 and the content server 12 and/or between the security server 14 and the client systems 16a-b.

Client systems 16a-c may comprise end-user computers, each having a processor, memory, and storage, and running an operating system (e.g., Windows XP or Linux). Some client computer systems 16a-c may be mobile computing and/or telecommunications devices, such as tablet PCs, mobile phones, personal digital assistants (PDAs), and home devices (e.g., televisions or music players, etc.). In some embodiments, client systems 16a-c may represent individual clients, or several client systems may belong to the same client. Client systems 16a-c may access electronic documents (e.g., email messages) by receiving them from sender system 13 and storing them in a local inbox or by retrieving such documents over network 18 (e.g., from a website served by content server 12).

FIG2-A shows an exemplary hardware configuration of client system 16 (e.g., systems 16a-c of FIG1 ). FIG2-A shows a computer system for illustrative purposes; the hardware configuration of other devices (e.g., mobile phones) may differ. In some embodiments, client system 16 includes a processor 20, a memory unit 22, a set of input devices 24, a set of output devices 26, a set of storage devices 28, and a communication interface controller 30, all of which are connected by a set of buses 34.

In some embodiments, processor 20 comprises a physical device (e.g., a multi-core integrated circuit) configured to perform computations and/or logical operations using signals and/or data sets. In some embodiments, such logical operations are delivered to processor 20 in the form of a sequence of processor instructions (e.g., machine code or other software type). Memory unit 22 may comprise volatile computer-readable media (e.g., RAM) that stores data/signals accessed or generated by processor 20 during the execution of instructions. Input devices 24 may include a computer keyboard, mouse, microphone, etc., including corresponding hardware interfaces and/or adapters that allow a user to enter data and/or instructions into system 16. Output devices 26 may include a display device (e.g., a monitor and speakers, etc.), as well as hardware interfaces/adapters (e.g., a graphics card) that allow system 16 to communicate data to the user. In some embodiments, input devices 24 and output devices 26 may share common hardware, as is the case in the case of a touchscreen device. Storage device 28 comprises computer-readable media that enables non-volatile storage, reading, and writing of software instructions and/or data. Exemplary storage devices 28 include magnetic and optical disks and flash memory devices, as well as removable media (e.g., CD and/or DVD disks and drives). Communication interface controller 30 enables system 16 to connect to network 18 and/or other devices/computer systems. Bus 34 collectively represents multiple system, peripheral, and chipset buses, and/or all other circuitry that enables internal communications between devices 20-30 of client system 16. For example, bus 34 may include a north bridge that connects processor 20 to memory 22, and/or a south bridge that connects processor 20 to devices 24-30, and so on.

FIG2-B shows an exemplary hardware configuration of a security server 14 according to some embodiments of the present invention. The security server 14 includes a processor 120 and a memory unit 122, and may further include a collection of storage devices 128 and at least one communication interface controller 130, all of which are interconnected via a collection of buses 134. In some embodiments, the operation of the processor 120, memory 122, and storage devices 128 may be similar to the operation of items 20, 22, and 28, respectively, as described above with respect to FIG2-A. The memory unit 122 stores data/signals accessed or generated by the processor 120 during the execution of instructions. The controller 130 enables the security server 14 to connect to the network 18 to transmit data to and/or receive data from other systems connected to the network 18.

FIG2-C shows an exemplary hardware configuration of content server 12 according to some embodiments of the present invention. Content server 12 includes a processor 220 and a memory unit 222, and may further include a collection of storage devices 228 and at least one communication interface controller 230, all of which are interconnected by a collection of buses 234. In some embodiments, the operation of processor 220, memory 222, and storage devices 228 may be similar to the operation of items 20, 22, and 28, respectively, as described above. Memory unit 222 stores data/signals accessed or generated by processor 220 during the execution of instructions. In some embodiments, interface controller 230 enables content server 12 to connect to network 18 and transmit data to and/or receive data from other systems connected to network 18.

FIG3-A shows an exemplary target document 36a comprising a junk email message according to some embodiments of the present invention. The target document 36a may include a header and a payload, wherein the header includes message routing data (e.g., an indicator of the sender and/or the recipient), and/or other data (e.g., a timestamp and an indicator of the content type (e.g., a Multipurpose Internet Mail Extensions (MIME) type)). The payload may include data displayed to the user as text and/or an image. Software executing on the content server 12 and/or the client systems 16a-c may process the payload to generate a target text block 38a of the target document 36a. In some embodiments, the target text block 38a includes a sequence of signs and/or symbols that are intended to be interpreted as text. The text block 38a may include special characters such as punctuation marks, as well as character sequences representing network addresses, uniform resource locators (URLs), email addresses, pseudonyms, and aliases, etc. The target text block 38a may be embedded directly into the target document 36a (e.g., as a plain text MIME part), or may comprise the result of processing a set of computer instructions embedded in the document 36a. For example, the target text block 38a may comprise the result of rendering a set of Hypertext Markup Language (HTML) instructions, or the result of executing a set of client-side script instructions or a set of server-side script instructions (e.g., PHP, Javascript) embedded in the target document 36a. In another embodiment, the target text block 38a may be embedded in an image, as is the case in image spam.

FIG3-B shows another exemplary target document 36b, which includes a comment posted on a webpage (e.g., a blog, an online news page, or a social networking page). In some embodiments, document 36b includes the contents of a collection of data fields (e.g., fields of a form embedded in an HTML document). For example, filling in such form fields can be performed remotely by a human operator and/or automatically by software executing on sender system 13. In some embodiments, the display of document 36b includes a text block 38b, which consists of a sequence of characters and/or symbols that are intended to be interpreted as text by a user accessing the corresponding website. Text block 38b can include hyperlinks, special characters, emoticons, images, and the like.

FIG3-C illustrates another exemplary target document 36c, which comprises a phishing webpage. Document 36c may be delivered as a set of HTML and/or server-side or client-side script instructions that, when executed, determine a document viewer (e.g., a web browser) to generate a set of images and/or a set of text blocks. FIG3-C illustrates two such exemplary text blocks 38c-d. Text blocks 38c-d may include hyperlinks and email addresses.

FIG4-A shows an exemplary spam/fraud detection transaction between an exemplary client system 16 (e.g., client systems 16a-c of FIG1 ) and a security server 14, according to some embodiments of the present invention. The exchange illustrated in FIG4-A occurs, for example, in an embodiment of system 10 configured to detect email spam. After receiving a target document 36 (e.g., an email message) from a content server 12, the client system 16 may determine a target indicator 40 for the target document 36 and may send the target indicator 40 to the security server 14. The target indicator 40 includes data that allows the security server 14 to perform classification of the target document 36 to determine, for example, whether the document 36 is spam. In response to receiving the target indicator 40, the security server 14 may send a target tag 50 to the respective client system 16 indicating whether the document 36 is spam.

Another embodiment of a spam detection transaction is illustrated in FIG4-B and occurs between a content server 12 and a security server 14. Such an exchange may occur, for example, to detect unsolicited communications posted to blogs and/or social networking websites, or to detect phishing web pages. The content server 12, which hosts and/or displays the corresponding website, may receive a target document 36 (e.g., a blog comment). The content server 12 may process the corresponding communication to generate a target indicator 40 for the corresponding document and may send the target indicator to the security server 14. In return, the server 14 may determine a target label 50 indicating whether the corresponding document is spam or fraudulent and send the label 50 to the content server 12.

FIG5 shows an exemplary target indicator 40 determined for an exemplary target document 36 (e.g., email message 36a in FIG3-A). In some embodiments, target indicator 40 is a data structure that includes a message identifier 41 (e.g., a hash index) uniquely associated with target document 36 and a text fingerprint 42 determined for a block of text in document 36 (e.g., block 38a in FIG3-A). Target indicator 40 may further include a sender indicator 44 indicating the sender of document 36, a routing indicator 46 indicating the network address (e.g., an IP address) from which document 36 was sent, and a timestamp 48 indicating the time at which document 36 was sent and/or received. In some embodiments, target indicator 40 may include other spam-indicating and/or fraud-indicating features of document 36, such as a flag indicating whether document 36 contains an image, a flag indicating whether document 36 contains a hyperlink, a document layout indicator determined for document 36, and the like.

FIG6 shows an exemplary set of components executed on a client system 16 according to some embodiments of the present invention. The configuration illustrated in FIG6 is suitable, for example, for detecting spam email messages received at a client system 16. System 16 includes a document digester 52 and a document display manager 54 connected to the document digester 52. The document digester 52 may further include a fingerprint calculator 56. In some embodiments, the document digester 52 receives a target document 36 (e.g., an email message) and processes the document 36 to generate a target indicator 40. Processing the document 36 may include parsing the document 36 to identify distinct data fields and/or types, distinguishing header data from payload data, and the like. When the document 36 is an email message, the exemplary parsing may generate distinct data objects for the sender, IP address, subject, timestamp, and content of the corresponding message, among others. When the content of the document 36 includes data of multiple MIME types, the parsing may generate distinct data objects for each MIME type (e.g., plain text, HTML, image, etc.). Document digester 52 may then formulate target indicator 40, such as by filling in the corresponding fields of target indicator 40 (eg, sender, routing address, timestamp, etc.) The software component of client system 16 may then transmit target indicator 40 to secure server 14 for analysis.

In some embodiments, the document display manager 54 receives the target document 36, converts it into a visual form, and displays it on an output device of the client system 16. Some embodiments of the display manager 54 may also allow the user of the client system 16 to interact with the displayed content. The display manager 54 may integrate with existing document display software (e.g., web browsers, email readers, e-book readers, media players, etc.). For example, such integration may be implemented as a software plug-in. The display manager 54 may be configured to assign target documents 36 (e.g., incoming emails) to document categories (e.g., spam, legitimate, and/or various other categories and subcategories of documents). Such categorization may be determined based on the target tags 50 received from the secure server 14. The display manager 54 may further be configured to group spam/scam messages into separate folders and/or display only legitimate messages to the user. The manager 54 may also tag documents 36 based on such categorization. For example, the document display manager 54 may display spam/scam messages in different colors or display a flag next to each spam/scam message indicating the message's category (e.g., spam, phishing, etc.). Similarly, when the document 36 is a fraudulent web page, the display manager 54 may prevent the user from accessing the corresponding page and/or display a warning to the user.

In an embodiment configured to detect spam/fraud posted as comments on blogs and social networking sites, the document digester 52 and display manager 54 can be executed on the content server 12 (instead of the client systems 16a-c shown in FIG6). Such software can be implemented on the content server 12 in the form of a server-side script, which can be further incorporated (e.g., as a plug-in) into a larger script package (e.g., as an anti-spam/anti-fraud plug-in for a WordPress or online publishing platform). Once a target document 36 is determined to be spam or fraudulent, the display manager 54 can be configured to block the corresponding message, thereby preventing it from being displayed on the corresponding website.

Fingerprint calculator 56 ( FIG. 6 ) is configured to determine a text fingerprint of target document 36 , which forms part of target indicator 40 (e.g., item 42 in FIG. 5 ). In some embodiments, the fingerprint determined for the target electronic document comprises a sequence of characters whose length is constrained between predetermined upper and lower limits (e.g., between 129 and 256 characters, inclusive). Keeping such fingerprints within the predetermined length range may be desirable, allowing for efficient comparison with a reference fingerprint set to identify blocks of text that include spam and/or scams, as described in greater detail below. In some embodiments, the characters forming the fingerprint may include alphanumeric characters, special characters, and symbols (e.g., *, /, $, etc.), among others. Other exemplary characters used to form the text fingerprint include digits or other symbols used to represent numbers in various encodings, such as binary, hexadecimal, and Base64, among others.

FIG7 illustrates an exemplary sequence of steps performed by the fingerprint calculator 56 to determine a text fingerprint. In step 402, the fingerprint calculator may select a target text block of the target document 36 for fingerprint calculation. In some embodiments, the target text block may consist of substantially the entire text content of the target document 36 (e.g., a plain text MIME portion of the document 36). In some embodiments, the target text block may consist of a single paragraph of the text portion of the document 36. In an embodiment configured to filter web-based spam, the target text block may consist of the content of a blog comment, or another type of message sent by a user and intended for publication on a corresponding website (e.g., a wall post, a tweet, etc.). In some embodiments, the target text block includes the contents of a section of an HTML document (e.g., a section indicated by a DIV or SPAN tag).

In step 404, the fingerprint calculator 56 may segment the target text block into text tokens. FIG8 shows an exemplary segmentation of the text block 38 into a plurality of text tokens 60a-c. In some embodiments, a text token is a sequence of characters/symbols separated from other text tokens by any set of delimiter characters/symbols. Exemplary delimiters for Western scripts include space, line break, tab, '\r', '\0', period, comma, colon, semicolon, parentheses and/or brackets, backward and/or forward slashes, double slashes, mathematical symbols (e.g., '+', '-', '*', '^'), punctuation (e.g., '!' and '?'), and special characters (e.g., '$' and '|', etc.). The exemplary tokens in FIG8 are individual words; other examples of text tokens may include multi-word sequences, email addresses, URLs, etc. To identify individual tokens of the text block 38, the fingerprint calculator may use any string tokenization algorithm known in the art. Some embodiments of the fingerprint calculator 56 may consider certain tokens, such as common words in the English language (e.g., 'a' and 'the'), to be ineligible for fingerprint calculation. In some embodiments, tokens exceeding a predetermined maximum length are further broken into shorter tokens.

In some embodiments, the length of the text fingerprint determined by calculator 56 is constrained to be within a predetermined range (e.g., between 129 and 256 characters, inclusive), regardless of the length or token count of the corresponding target text block. To calculate such a fingerprint, in step 406, fingerprint calculator 56 may first determine the count of text tokens of the target text block and compare the count to a predetermined upper threshold value, which is determined based on an upper limit on the fingerprint length. When the token count exceeds the upper threshold value (e.g., 256), calculator 56 may determine a reduced fingerprint in step 408, as shown in detail below.

When the tag count drops below the upper threshold, in step 410, the fingerprint calculator may calculate a hash for each text tag. Figure 8 shows exemplary hashes 62a to c, which are determined for text tags 60a to c, respectively. Hashes 62a to c are shown in hexadecimal notation. In some embodiments, such hashes are the result of applying a hash function to each tag 60a to c. Many such hash functions and algorithms are known in the art. Hash algorithms are fast, but typically produce a large number of collisions (cases where different tags have the same hash). More complex hashes (e.g., hashes calculated by message digest algorithms such as MD5) are said to be collision-free, but their computational cost is huge. Some embodiments of the present invention use a hash algorithm that provides a trade-off between computational speed and collision avoidance to calculate hashes 62a to c. An example of such an algorithm is attributed to Robert Sedgewick and is known in the art as RSHash. Pseudocode for RSHash is shown below:

Wherein a and b represent integers, for example, a=63,689, and b=378,551.

The size (number of bits) of hashes 62a-c can affect the likelihood of collisions and, therefore, the spam detection rate. Generally speaking, using small hashes increases the likelihood of collisions. Larger hashes are generally less prone to collisions but are more expensive in terms of computational speed and memory. Some embodiments of fingerprint calculator 56 calculate items 62a-c as 30-bit hashes.

Fingerprint calculator 56 can now determine the actual textual fingerprint of the target text block. FIG8 further illustrates an exemplary fingerprint 42 determined for target text block 38. Textual fingerprint 42 includes a character sequence determined based on hashes 62a-c determined in step 410. In some embodiments, for each token 60a-c, fingerprint calculator 56 determines a fingerprint segment, which is illustrated in FIG8 as items 64a-c. In some embodiments, such segments are then concatenated to generate fingerprint 42.

Each fingerprint segment 64a-c may include a sequence of characters determined from a hash 62a-c of the corresponding tag 60a-c. In some embodiments, all fingerprint segments 64a-c have the same length: in the example of FIG. 8 , each segment 64a-c consists of two characters. The length of the fingerprint segments is determined so that the corresponding fingerprint has a length within a desired range (e.g., between 129 and 256 characters). In some embodiments, the length of the fingerprint segment is referred to as the magnification factor k. For example, a segment of length 1 is an unscaled segment (magnification factor 1), thereby producing an unscaled fingerprint; a segment of length 2 is a 2x magnified segment (magnification factor 2), thereby producing a 2x magnified segment, and so on. FIG. 9 shows a plurality of text fingerprints 42a-c determined for a text block 38 at various magnification factors k.

In step 412 ( FIG. 7 ), the fingerprint calculator 56 determines a value for the magnification factor k that results in a fingerprint length within a desired predetermined range. For example, when the marker count is greater than a lower threshold (which is determined based on the lower limit of the desired fingerprint length), the fingerprint calculator may decide to calculate an unscaled fingerprint (k=1) because the unscaled fingerprint is already within the desired length range. For example, when the text block 38 has too few markers, the fingerprint calculator may calculate a 2x or 3x magnified fingerprint.

Then, in step 414, fingerprint calculator 56 calculates fingerprint segments for each token based on the corresponding hash of the token. To determine segments 64a-c, fingerprint calculator 56 may use any encoding scheme known in the art (e.g., binary or Base64 representations of hashes 62a-c). Such encoding schemes establish a one-to-one mapping between numbers and character sequences from a predetermined alphabet. For example, when using Base64 representation, each group of six consecutive bits of the hash can be mapped to a character.

In some embodiments, multiple fingerprint fragments can be determined for each hash by varying the number of characters used to represent the corresponding hash. To generate a fragment of length 1 (e.g., a magnification factor of 1), some embodiments use only the six least significant bits of the corresponding hash. An additional six bits of the corresponding hash can be used to generate a fragment of length 2 (e.g., a magnification factor of 2), and so on. In Base64 representation, a 30-bit hash can thus result in a fingerprint fragment up to 5 characters long, corresponding to five magnification factors. Table 1 shows exemplary fingerprint fragments (from exemplary text block 38 in FIG. 9 ) calculated at various magnification factors.

Table 1

In step 416 ( FIG. 7 ), fingerprint calculator 56 combines text fingerprints 42 , for example, by concatenating the segments calculated in step 414 .

Returning to step 406, when the token count is found to be greater than the upper threshold, fingerprint calculator 56 determines a reduced fingerprint for the corresponding text block. In some embodiments, reduction includes calculating fingerprint 42 from only a subset of the tokens of text block 38. Selecting the subset may include pruning the plurality of text tokens determined in step 404 according to a hash selection criterion. FIG10 illustrates an exemplary sequence of steps for performing such calculations. Step 422 selects a reduction factor for fingerprint calculation. In some embodiments, the reduction factor, denoted as k, indicates that, on average, only 1/k of the tokens of text block 38 are used for fingerprint calculation. Fingerprint calculator 56 may therefore select the reduction factor based on the token count determined in step 406 (FIG. 7). In some embodiments, the initial selection of the reduction factor may fail to produce a fingerprint within the desired length range (see below); in such cases, steps 422 through 430 may be performed in a trial-and-error loop until a fingerprint of the appropriate length is produced. For example, fingerprint calculator 56 may initially select a reduction factor k=2; when this value fails to produce a sufficiently short fingerprint, calculator 56 may select k=3, and so on.

The fingerprint calculator may then select a tag based on the hash selection criteria. When zooming out, the fingerprint calculator 56 may use the tags already determined in step 404 ( FIG. 7 ), or may calculate new tags from the text block 38 . In the example illustrated in FIG. 10 , in step 424 , the fingerprint calculator 56 determines an aggregate tag set for the text block 38 . In some embodiments, the aggregate tag is determined by concatenating consecutive individual tags (illustrated as item 60 d in FIG. 9 ). The count of tags used to form the aggregate tag may vary depending on the zoom factor.

In step 426, a hash is calculated for each aggregate tag (e.g., using the method described above). In step 428, calculator 56 selects a subset of aggregate tags based on a hash selection criterion. In some embodiments, for a reduction factor k, the selection criterion requires that all hashes determined for the members of the selected subset be equal modulo k. For example, to determine a 2x reduction fingerprint, calculator 56 may only consider aggregate tags whose hashes are equal modulo 2 (i.e., only odd hashes, or only even hashes). In some embodiments, the hash selection criterion includes selecting only tags whose hashes are divisible by the reduction factor k.

In step 430, fingerprint calculator 56 may check whether the count of the token selected in step 428 is within the desired fingerprint length range. If the count is not within the desired fingerprint length range, calculator 56 may return to step 422 and restart with another reduction factor k. When the count of the selected token is within the range, calculator 56 determines fingerprint segments from each hash of the selected token in step 432. In step 434, such segments are combined to generate fingerprint 42. FIG9 illustrates some reduced fingerprints 42d through h determined for text block 38. Table 2 shows exemplary fingerprint segments determined for the same text block 38 in FIG9 (at various reduction factors).

Table 2

FIG11 shows exemplary components executed on a secure server (see also FIG1 ) according to some embodiments of the present invention. The secure server 14 includes a document classifier 72 connected to a communication manager 74 and a fingerprint database 70. The communication manager 74 manages spam/scam detection transactions with the client systems 16 a-c, as shown above with respect to FIG4-A-B. In some embodiments, the document classifier 72 is configured to receive a target indicator 40 via the communication manager 74 and determine a target tag 50 indicating a classification of the target document 36.

In some embodiments, classifying a target document 36 includes assigning the document 36 to a document category based on a comparison between a text fingerprint determined for the document 36 and a set of reference fingerprints, each reference fingerprint indicating a document category. For example, classifying the document 36 may include determining whether the document 36 is spam and/or fraudulent, and determining that the document 36 belongs to a subcategory of spam/fraud (e.g., product offers, phishing, or Nigerian scams). To classify the document 36, the document classifier 72 may utilize any method known in the art in conjunction with fingerprint comparison. Such methods include blacklists and whitelists, pattern matching algorithms, and the like. For example, the document classifier 72 may calculate a plurality of individual scores, each score indicating membership of the document 36 in a particular document category (e.g., spam), each score determined by a different classification method (e.g., fingerprint comparison, blacklisting, and the like). The classifier 72 may then determine the classification of the document 36 based on a composite score determined as the individual scores.

The document classifier 72 may further include a fingerprint comparator 78 (as shown in FIG. 12 ) configured to classify the target document 36 by comparing the target document's fingerprint with a set of reference fingerprints stored in the database 70. In some embodiments, the fingerprint database 70 includes a repository of text fingerprints determined for a set of reference documents (e.g., email messages, web pages, website comments, etc.). The database 70 may include fingerprints for spam/fraud, but also fingerprints for legitimate documents. For each reference fingerprint, the database 70 may store an indicator of an association between the corresponding fingerprint and a document category (e.g., spam).

In some embodiments, all fingerprints in the reference fingerprint subset in the database 70 have lengths within a predetermined range (e.g., between 129 and 256 characters). Furthermore, the range is consistent with the length range of the target fingerprint determined by the fingerprint calculator 56 ( FIG. 6 ) for the target document. Such a configuration (where all reference fingerprints have approximately the same size and where the reference fingerprints have a length that is approximately equal to the length of the target fingerprint) can facilitate comparisons between target and reference fingerprints for document classification purposes.

For each reference fingerprint, some embodiments of the database 70 may store an indicator of the length of the text block for which the corresponding fingerprint was determined. Examples of such indicators include the string length of the corresponding text block, the segment length used in determining the corresponding fingerprint, and the magnification/reduction factor, etc. Storing an indicator of the length of the text block with each fingerprint may facilitate document comparison by enabling the fingerprint comparator 78 to selectively retrieve reference fingerprints representing text blocks that are similar in length to the text block from which the target fingerprint 42 was generated.

To classify a target document 36, the classifier 72 may receive the target indicator 40, extract a target fingerprint 42 from the indicator 40, and forward the fingerprint 42 to a fingerprint comparator 78. The comparator 78 may interface with the database 70 to selectively retrieve a reference fingerprint 82 for comparison with the target fingerprint 42. In some embodiments, the fingerprint comparator 78 may preferably retrieve a reference fingerprint calculated for a block of text having a length similar to that of the target block of text.

Document classifier 72 further determines the classification of target document 42 based on a comparison of target fingerprint 42 with reference fingerprints retrieved from database 70. In some embodiments, the comparison includes calculating a similarity score indicating the degree of similarity between fingerprints 42 and 82. For example, such a similarity score may be determined as:

Where f _T and f _R represent the target fingerprint and reference fingerprint, respectively, d(f _T , f _R ) represents the edit distance (e.g., Levenshtein distance) between the two fingerprints, and where |f _T | and |f _R | represent the lengths of the target fingerprint and reference fingerprint, respectively. Score S can take any value between 0 and 1, with values close to 1 indicating a high degree of similarity between the two fingerprints. In an exemplary embodiment, when score S exceeds a predetermined threshold T (e.g., 0.9), target fingerprint 42 is said to match reference fingerprint 82. When target fingerprint 42 matches at least one reference fingerprint from database 70, document classifier 72 may classify the target document according to the document category indicator of the corresponding reference fingerprint and may formulate target label 50 to reflect the classification. For example, when target fingerprint 42 matches a reference fingerprint determined for a spam message, target document 36 may be classified as spam, and target label 50 may indicate the spam classification.

The exemplary systems and methods described above allow for the detection of unsolicited communications (spam) in electronic messaging systems (e.g., email and user-contributed websites), as well as the detection of fraudulent electronic documents (e.g., phishing websites). In some embodiments, a text fingerprint is calculated for each target document, the fingerprint comprising a character sequence determined based on a plurality of text tokens for the corresponding document. The fingerprint is then compared to a reference fingerprint determined for a collection of documents (including spam/fraudulent and legitimate documents). When the target fingerprint matches a reference fingerprint determined for a spam/fraudulent message, the target communication can be labeled as spam/fraudulent.

When a target communication is positively identified as spam/scam, components of the anti-spam/anti-scam system may modify the display of the corresponding document. For example, some embodiments may prevent the display of the corresponding document (e.g., not allowing spam comments to be displayed on the website), may display the corresponding document in a separate location (e.g., a junk email folder, a separate browser window), and/or may display an alert.

In some embodiments, the text tokens may include individual words or sequences of words of the target text, as well as email addresses and/or network addresses (e.g., uniform resource locators (URLs) contained in the text portion of the target document). Some embodiments of the present invention recognize multiple such text tokens within the target document. A hash is calculated for each token, and a fingerprint segment is determined based on the corresponding hash. In some embodiments, the fingerprint segments are then combined, for example, by concatenation, to generate a text fingerprint for the corresponding document.

Some electronic documents (e.g., email messages) can vary greatly in length. In some conventional anti-spam/anti-fraud systems, the length of fingerprints determined for such documents varies accordingly. In contrast, in some embodiments of the present invention, the length of text fingerprints is constrained to be within a predetermined length range (e.g., between 129 and 256 characters), regardless of the length of the target text block or document. Keeping all text fingerprints within the predetermined length limit can substantially improve the efficiency of inter-message comparisons.

To determine fingerprints within a predetermined length range, some embodiments of the present invention use a magnification and reduction method. When the text block is relatively short, magnification is achieved by adjusting the length of the fingerprint segment to produce a fingerprint of the desired length. In an exemplary embodiment, every 6 bits of a 30-bit hash can be converted into a character (using, for example, Base64 representation), so that the corresponding hash can produce fingerprint segments between 1 and 5 characters in length.

For relatively long blocks of text, some embodiments of the present invention achieve reduction by computing fingerprints from a subset of tokens, the subset being selected according to a hash selection criterion. An exemplary hash selection criterion includes selecting only tokens whose hashes are divisible by an integer k (e.g., 2, 3, or 6). For a given example, such selection results in computing fingerprints from approximately 1/2, 1/3, or 1/6 of the available tokens, respectively. In some embodiments, reduction may further include applying such token selection to multiple aggregate tokens, where each aggregate token comprises a concatenation of several tokens (e.g., a sequence of words from a corresponding electronic document).

Various hash functions can be used to determine fingerprint fragments. In a computer experiment, various hash functions known in the art were applied to a set of 122,000 words extracted from email messages in various languages. The goal was to determine the number of hash collisions (different words producing the same hash) that would produce actual spam for each hash function. The results, illustrated in Table 3, show that the hash function known in the art, known as RSHash, produced the fewest collisions of all the hash functions tested.

Table 3

Hash function 32-bit hash collision 30-bit hash collision RSHash 0 4 BKDRHash 1 6 SDBMHash 2 7 OneAtATimeHash 2 6 APHash 4 6 FNVHash 7 10 FNV1aHash 7 10 JSHash 266 277 DJBHash 266 268 DEKHash 435 720 PJWHash 1687 1687 ELFHash 1687 1687 BPHash 61907 70909

In another computer experiment, some embodiments of the present invention were used to analyze a collection of email messages (consisting of the total amount of email received by an enterprise server during a week, and including both spam and legitimate messages). To determine text fingerprints between 129 and 256 characters in length, 20.8% of the messages required no scaling, 18.5% required 2x scaling, 8.1% required 3x scaling, and 8.7% required 6x scaling. Among the same message collection, 14.8% required 2x magnification, 9.7% required 4x magnification, and 11.7% required 8x magnification. The above results suggest that fingerprint lengths between 129 and 256 characters may be optimal for detecting email spam, because the aforementioned partitioning of the actual email stream into groups based on magnification and/or reduction factors results in relatively evenly populated groups; such a situation is advantageous for fingerprint comparison, because all groups can be searched at approximately the same time.

In another computer experiment, a continuous spam stream consisting of approximately 865,000 messages collected over 15 hours was divided into message sets, each set consisting of messages received during a distinct 10-minute interval. Each message set was analyzed using a document classifier constructed according to some embodiments of the present invention (e.g., see Figures 11-12). For each message set, a fingerprint database 70 consisted of fingerprints determined for spam messages belonging to the earlier time interval. The spam detection rate obtained using equation [1] and a threshold value T = 0.75 (solid line) is shown in Figure 13, compared to the spam detection rate obtained for the same message set using a conventional spam detection method (known in the art as fuzzy hashing) (dashed line).

It will be apparent to those skilled in the art that the above embodiments can be modified in many ways without departing from the scope of the present invention. Therefore, the scope of the present invention should be determined by the appended claims and their legal equivalents.

Claims (22)

1. A client computer system comprising at least one processor configured to determine a text fingerprint of a target electronic document, such that the length of the text fingerprint is constrained between a lower limit and an upper limit, wherein the lower limit and the upper limit are predetermined, and wherein determining the text fingerprint includes: Selecting multiple text tags in the target electronic document, wherein selecting the multiple text tags includes: Select a preliminary set of text tags for the target electronic document. Determine the count of the initial plurality of text tags, and In response, when the count of the initial plurality of text tags exceeds a predetermined threshold, the initial plurality of text tags are trimmed to form a selected plurality of text tags such that the count of the selected plurality of tags does not exceed the predetermined threshold; In response to selecting the plurality of text markers, the fingerprint fragment size is determined based on the upper and lower limits and the count of the selected plurality of text markers; Multiple fingerprint segments are determined, each of which is determined based on a hash of dissimilar text tags selected from a plurality of text tags. Each fingerprint segment consists of a character sequence whose length is selected to be equal to the size of the fingerprint segment. The multiple fingerprint fragments are cascaded to form the text fingerprint. 2. The client computer system of claim 1, wherein the at least one processor is further configured to: Send the text fingerprint to the server computer system; and The server computer system receives target tags determined for the target electronic document, the target tags indicating the document category to which the target electronic document belongs, wherein determining the target tags includes: A reference fingerprint is retrieved from a database of reference fingerprints, the reference fingerprint being determined for a reference electronic document belonging to the aforementioned category, and the reference fingerprint being selected based on its length such that the length of the reference fingerprint is between the upper and lower limits; and Whether the target electronic document belongs to the category is determined by comparing the text fingerprint with the reference fingerprint. 3. The client computer system according to claim 2, wherein the document category is the spam category. 4. The client computer system according to claim 2, wherein the document category is a fraudulent document category. 5. The client computer system of claim 1, wherein determining the text fingerprint further comprises: determining each character in the character sequence based on dissimilar groups of bits of the hash of the dissimilar text tokens. 6. The client computer system of claim 1, wherein trimming the initial plurality of text tags comprises: selecting the target text tag as a selected plurality of text tags based on a hash of the target text tag among the initial plurality of text tags. 7. The client computer system of claim 6, wherein trimming the initial plurality of text tags further comprises: Determine whether the hash of the target text tag is divisible by a shrinking factor; and In response, when the target text mark is divisible by the reduction factor, the target text mark is selected as one of the selected text marks. 8. The client computer system of claim 1, wherein selecting the plurality of text tags further comprises, when the count of the initial plurality of text tags exceeds the predetermined threshold: Determine a plurality of aggregated text tags, each of the plurality of aggregated text tags comprising a cascade of text tag sets of the initial plurality of text tags; and The aggregated text tags are selected as the selected multiple text tags based on the hash of the aggregated text tags. 9. The client computer system according to claim 1, wherein the target electronic document is selected from the group consisting of email messages and Hypertext Markup Language (HTML) documents. 10. The client computer system of claim 1, wherein the dissimilar text tag includes items selected from a group consisting of words of the target electronic communication, email addresses, and Uniform Resource Locators (URLs). 11. A server computer system comprising at least one processor configured to perform transactions with a plurality of client systems, wherein the transactions include: A text fingerprint is received from one of the client systems, the text fingerprint being determined for a target electronic document, such that the length of the text fingerprint is constrained between a lower limit and an upper limit, wherein the lower limit and the upper limit are predetermined; and Send a target tag indicating the document category to which the target electronic document belongs to the client system. Determining the text fingerprint includes: Selecting multiple text tags in the target electronic document, wherein selecting the multiple text tags includes: Select a preliminary set of text tags for the target electronic document. Determine the count of the initial plurality of text tags, and In response, when the count of the initial plurality of text tags exceeds a predetermined threshold, the initial plurality of text tags are trimmed to form a selected plurality of text tags such that the count of the selected plurality of tags does not exceed the predetermined threshold; In response to selecting the plurality of text markers, the fingerprint fragment size is determined based on the upper and lower limits and the count of the selected plurality of text markers; Multiple fingerprint segments are determined, each of which is determined based on a hash of dissimilar text tags selected from a plurality of text tags. Each fingerprint segment consists of a character sequence whose length is selected to be equal to the size of the fingerprint segment. The multiple fingerprint fragments are cascaded to form the text fingerprint. And the determination of the target label includes: A reference fingerprint is retrieved from a database of reference fingerprints, the reference fingerprint being determined for a reference electronic document belonging to the aforementioned category, and the reference fingerprint being selected based on its length such that the length of the reference fingerprint is between the upper and lower limits; and Whether the target electronic document belongs to the category is determined by comparing the text fingerprint with the reference fingerprint. 12. The server computer system of claim 11, wherein the document category is the spam category. 13. The server computer system of claim 11, wherein the document category is a fraudulent document category. 14. The server computer system of claim 11, wherein determining the text fingerprint further comprises: determining each character in the character sequence based on dissimilar groups of bits of the hash of the dissimilar text tokens. 15. The server computer system of claim 11, wherein pruning the initial plurality of text tags comprises: selecting the target text tags as a selected plurality of text tags based on a hash of the target text tags among the initial plurality of text tags. 16. The server computer system of claim 15, wherein trimming the initial plurality of text tags further comprises: Determine whether the hash of the target text tag is divisible by a shrinking factor; and In response, when the target text mark is divisible by the reduction factor, the target text mark is selected as one of the selected text marks. 17. The server computer system of claim 11, wherein selecting the plurality of text tags further comprises, when the count of the initial plurality of text tags exceeds the predetermined threshold: Determine a plurality of aggregated text tags, each of the plurality of aggregated text tags comprising a cascade of text tag sets of the initial plurality of text tags; and The aggregated text tags are selected as the selected multiple text tags based on the hash of the aggregated text tags. 18. The server computer system of claim 11, wherein the target electronic document is selected from the group consisting of email messages and Hypertext Markup Language (HTML) documents. 19. The server computer system of claim 11, wherein the dissimilar text tag includes items selected from the group consisting of words of target electronic communication, email addresses, and Uniform Resource Locators (URLs). 20. A method comprising using at least one processor of a client computer system to determine a text fingerprint of a target electronic document, such that the length of the text fingerprint is constrained between a lower limit and an upper limit, wherein the lower limit and the upper limit are predetermined, and wherein determining the text fingerprint includes: Selecting multiple text tags in the target electronic document, wherein selecting the multiple text tags includes: Select a preliminary set of text tags for the target electronic document. Determine the count of the initial plurality of text tags, and In response, when the count of the initial plurality of text tags exceeds a predetermined threshold, the initial plurality of text tags are trimmed to form a selected plurality of text tags such that the count of the selected plurality of tags does not exceed the predetermined threshold; In response to selecting the plurality of text markers, the fingerprint fragment size is determined based on the upper and lower limits and the count of the selected plurality of text markers; Multiple fingerprint segments are determined, each of which is determined based on a hash of dissimilar text tags selected from a plurality of text tags. Each fingerprint segment consists of a character sequence whose length is selected to be equal to the size of the fingerprint segment. The multiple fingerprint fragments are cascaded to form the text fingerprint. 21. The method of claim 20, further comprising using the at least one processor to determine the document category to which the target electronic document belongs based on the text fingerprint. 22. A method comprising using at least one processor of a server computer system configured to perform transactions with a plurality of client systems to: A text fingerprint is received from one of the client systems, the text fingerprint being determined for a target electronic document, such that the length of the text fingerprint is constrained between a lower limit and an upper limit, wherein the lower limit and the upper limit are predetermined; and The system sends a target tag, determined for the target electronic document, to the client system. The target tag indicates the document category to which the target electronic document belongs. Determining the text fingerprint includes: Selecting multiple text tags in the target electronic document, wherein selecting the multiple text tags includes: Select a preliminary set of text tags for the target electronic document. Determine the count of the initial plurality of text tags, and In response, when the count of the initial plurality of text tags exceeds a predetermined threshold, the initial plurality of text tags are trimmed to form a selected plurality of text tags such that the count of the selected plurality of tags does not exceed the predetermined threshold; In response to selecting the plurality of text markers, the fingerprint fragment size is determined based on the upper and lower limits and the count of the selected plurality of text markers; Multiple fingerprint segments are determined, each of which is determined based on a hash of dissimilar text tags selected from a plurality of text tags. Each fingerprint segment consists of a character sequence whose length is selected to be equal to the size of the fingerprint segment. The multiple fingerprint fragments are cascaded to form the text fingerprint. And the determination of the target label includes: A reference fingerprint is retrieved from a database of reference fingerprints, the reference fingerprint being determined for a reference electronic document belonging to the aforementioned category, and the reference fingerprint being selected based on its length such that the length of the reference fingerprint is between the upper and lower limits; and Whether the target electronic document belongs to the category is determined by comparing the text fingerprint with the reference fingerprint.