WO2023278885A1

WO2023278885A1 - Moderation of user content for a social messaging platform

Info

Publication number: WO2023278885A1
Application number: PCT/US2022/036070
Authority: WO
Inventors: Andrew COURTER; Christine SU
Original assignee: Twitter Inc
Current assignee: Twitter Inc
Priority date: 2021-07-02
Filing date: 2022-07-05
Publication date: 2023-01-05
Anticipated expiration: 2024-01-02

Abstract

A social messaging platform provides moderation features for user to include a user-defined content filter and one or more labels and/or guidelines to display with a base message posted by a content creator. The user-defined content filter includes a list of offensive expressions selected, in part, by the content creator, to moderate reply messages posted by content consumers in response to a creator's content base message. In this manner, the user-defined content filter provides users of the platform the ability to selectively choose offensive expressions especially phrases that can have different meanings in different contexts. The labels and/or guidelines further provide users a way to explicitly display injunctive social norms to users and, thus, establish rules for what content is socially acceptable in a reply message.

Description

MODERATION OF USER CONTENT FOR A SOCIAL MESSAGING PLATFORM

CROSS-REFERENCE TO RELATED APPLICATION(S)

[0001] The present application claims priority to U.S. Provisional Application No. 63/218,155, filed July 2, 2021, entitled “Author Norms and Language Moderation,” which is incorporated by reference herein in its entirety.

BACKGROUND

[0002] Social messaging platforms and network-connected personal computing devices allow users to create and share content across multiple devices in real-time. Sophisticated mobile computing devices, such as smartphones and tablets, further make it easy and convenient for people (in a personal and/or professional capacity), businesses, and other entities to use social networking messaging platforms and applications.

[0003] Popular social messaging platforms generally provide functionality for users to draft and post/send messages, including video and/or audio content, both synchronously and asynchronously to other users. Other common features include the ability to post messages that are visible to one or more identified other users of the platform, to other users by virtue of a connection to the authoring user on the platform, or even publicly to any user of the platform without specific designation by the authoring user. Examples of popular social messaging platforms include Facebook®, Instagram®, Pinterest®, and Twitter®. (“Facebook” and “Instagram” are trademarks of Facebook, Inc. “Pinterest” is a trademark of Pinterest, Inc. “Twitter” is a trademark of Twitter, Inc.).

[0004] The users of the social messaging platform are typically permitted to, and capable of, both authoring messages for others and receiving messages from others. Some users, however, are more adept at generating content/authoring messages and/or are famous, such that there is widespread interest in their messages. These users are sometimes referred to as “authoring users,” “content creators,” or “creators.” For example, content creators are often celebrities who are users of the social messaging platform. In turn, some users of the social messaging platform who are connected to the content creators predominantly consume the content generated by the content creators (e.g., by reading, rating, and/or sharing the messages authored by the content creators). Social messaging platforms also typically permit users to post one or more messages in response to the content creator’ s message, thus providing a way for users to directly engage with a content creator. In some instances, this allows users to participate in a conversation with the content creator and other users where the content creators and the users post one or messages in response to one another. These users are sometimes referred to as “content consumers,” “subscribers,” or “followers.” It should be appreciated that content creators can be followers of other users and each follower can themselves be a content creator.

[0005] Social messaging platforms also typically provide a user interface to display multiple messages for users to view and consume. In one example, the user interface can display messages in a stream (also referred to herein as a “timeline” or a “feed”) where the messages are arranged, for example, in chronological order or reverse chronological order based on the respective dates the messages were posted or according to a computationally predicted relevance. The stream can further facilitate display of messages in a conversation. For example, the stream can display a first message by one user and a second message posted by another user in response to the first message.

[0006] Herein, the first message is sometimes referred to or characterized as a base message and the second message can be referred to or characterized as a response message or a reply message, which is a message posted in direct response to the base message and sent to at least the authoring user of the base message. One base message and one or more reply messages posted in response to the base message constitute a message branch. It should be appreciated a base message can be characterized as a root message (i.e., a message that is not posted in response to another message) or a reply message (e.g., a message posted in response to the root message or another reply message). A message thread is initiated by a single root message and can further include one or more reply messages posted in response to the root message or another reply message. Thus, a message thread can include multiple message branches.

SUMMARY

[0007] The Inventors have recognized and appreciated that social messaging platforms are generally well-suited for users and, in particular, content creators to post messages and/or share other social media content with a large audience (e.g., the content creator’s followers). However, the Inventors have also recognized providing users access to a large audience can also give rise to unwanted attention or, worse, harassment by other users of the social messaging platform.

[0008] In particular, popular users and/or users with a large number of followers often receive a large volume of unwanted reply messages containing toxic and/or abusive content. Additionally, the rate at which reply messages are posted is typically greatest within the first few days after a user posts a base message. These factors contribute to make it difficult for users to manage and/or moderate unwanted reply messages particularly for popular or high visibility conversations where many reply messages may be posted (e.g., hundreds, thousands, or tens of thousands of reply messages). In some situations, the failure to moderate unwanted reply messages can further lead to a greater number of unwanted reply messages due, in part, to the normalization of toxic and/or abusive content amongst users participating in a conversation. Said in another way, users are more likely to post messages with toxic and/or abusive content when they see other users posting messages with toxic and/or abusive content without punishment.

[0009] Although social messaging platforms generally include a set of rules and, in some instances, content filters to deter users from including toxic and/or abusive language in their messages (e.g., profanity, slurs), users can readily circumvent these rules and/or content filters using creative language in their content. In particular, users can include language that is offensive in some situations based on the context of the conversation and/or the users involved, but benign in other situations, which is challenging to detect and moderate using a broad set of rules and/or content filters. Content-specific language can also vary based on several factors including, but not limited to, the geographic location of the users, the nationality of the users, and the native language of the users.

[0010] The detection of context-specific language that is offensive to only a subset of users of a social messaging platform also poses an immense technical challenge. In particular, some social messaging platforms process millions, tens of millions, or hundreds of millions of messages each day. Thus, it is not computationally feasible for the platform to generate and apply specific rules and/or content filters to each message that accounts for the various factors that affect context- specific language. It is also not computationally feasible to continuously update the rules and/or content filters to cover new toxic and/or abusive language over time.

[0011] In view of the foregoing, the present disclosure is directed to various inventive implementations of a social messaging platform that provides users a way to create user-defined content filters to moderate the language in reply messages posted in response to a user’s base message. The present disclosure is also directed to providing users a way to label messages and/or provide guidelines to users posting reply messages of the user’s preferred tone and/or injunctive social norms. In this manner, the social messaging platforms disclosed herein provide users greater control over the reply messages posted in response to their content while simultaneously improving the ease of managing reply messages. Lastly, the present disclosure is directed to various methods of implementing the above features using one or more platform servers and/or user devices of the social messaging platform.

[0012] The user-defined content filter disclosed herein (also sometimes referred to herein as a “smellcheck” or a “nudge”) allows users to selectively choose certain expressions (e.g., words or phrases) to moderate content posted by other users. This can be accomplished by each user (e.g., a content creator) defining one or more lists of offensive expressions for their user account that they want to discourage and/or do not want to see in reply messages posted by other users (e.g., content consumers) in response to that user’s base messages. The platform, in turn, can detect the presence of one or more offensive expressions as users draft a reply message in response to a base message by comparing the draft of the reply message against the list(s) of offensive expressions defined by the user who authored the base message. If an offensive expression is detected, the platform can highlight the offensive expression in the draft of the reply message to notify the user authoring the reply message their message includes the offensive expression and the actions (e.g., a warning, a penalty) taken by the platform for posting the reply message without removing the offensive expression. The platform can further execute the actions against the users who post reply messages with offensive expression(s).

[0013] By allowing each user to define a custom list of offensive expressions, the user-defined content filter provides a way to detect and moderate context-specific language that is offensive to certain users, but not other users especially if the offensive expression has different meanings in different situations. The offensive expressions can generally include words or phrases in textual form, one or more icons (e.g., emojis, emoticons), or any combinations of the foregoing. Additionally, the platform can provide users one or more lists of predetermined offensive expressions that are commonly considered to be toxic and/or abusive. In some implementations, the list(s) of predetermined offensive expressions can tailored to include offensive expressions that are more likely to be relevant and/or understood by a user based on one or more attributes of that user’s user account, such as a location of the user’s user device (e.g., the user device includes a position tracking sensor to monitor the location), a region/location associated with the user account, a nationality associated with the user account, and/or a default language associated with the user account.

[0014] The social messaging platforms disclosed herein further provide users a way to assign labels and/or guidelines (also sometimes referred to collectively as “house rules” or “author norms”) to their base message. The labels and/or guidelines can define one or more injunctive social norms users should follow when posting a reply message. By explicitly displaying the labels and/or guidelines to users viewing the branch of the message thread with the base message, users are less likely to use social cues within the message thread itself (e.g., reply messages posted by other users) as a way to determine what language and/or content is acceptable. This, in turn, increases the likelihood of users posting reply messages that conform with the preferences of the user who posted the base message.

[0015] For example, users can assign one or more labels to their base message so that other users who view and post a reply message can see the desired tone of the user who posted the base message. As an illustrative example, a content creator can post a base message with labels of “positive,” “curious,” and “thoughtful” as a way to encourage users who post reply messages to include content with these tones, tenor, mood, attitude, and/or intent. When viewing the content creator’s base message, the label(s) can further be displayed between the base message and the reply messages such that users see the content creator’s expected tone before viewing any reply message. In another example, users can post a base message with guidelines containing details on the content creator’ s expectations, values, and/or preferences for the content of the reply messages. The guidelines can be generated, in part, based on user input. The platform can also provide standard guidelines that can be modified by the user. The guidelines can also be generated automatically, for example, based on the label(s) selected by the user. In some implementations, the labels associated with a base message can be interactive such that users who select the labels are directed to the content creator’s guidelines.

[0016] It should be appreciated that all combinations of the foregoing concepts and additional concepts discussed in greater detail below (provided such concepts are not mutually inconsistent) are contemplated as being part of the inventive subject matter disclosed herein. In particular, all combinations of claimed subject matter appearing at the end of this disclosure are contemplated as being part of the inventive subject matter disclosed herein. It should also be appreciated that terminology explicitly employed herein that also may appear in any disclosure incorporated by reference should be accorded a meaning most consistent with the particular concepts disclosed herein.

BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The skilled artisan will understand that the drawings primarily are for illustrative purposes and are not intended to limit the scope of the inventive subject matter described herein. The drawings are not necessarily to scale; in some instances, various aspects of the inventive subject matter disclosed herein may be shown exaggerated or enlarged in the drawings to facilitate an understanding of different features. In the drawings, like reference characters generally refer to like features (e.g., functionally similar and/or structurally similar elements).

[0018] FIG. 1A shows an example social messaging platform that supports moderation of reply messages.

[0019] FIG. IB shows example components of a user device in the platform of FIG. 1 A.

[0020] FIG. 1C shows example components of a platform server in the platform of FIG. 1A.

[0021] FIG. 2A shows an example user interface of account settings associated with a user account of a user with an option to activate or deactivate moderation for reply messages posted in response to a message authored by the user associated with the user account.

[0022] FIG. 2B shows an example user interface of the moderation settings associated with the user account of FIG. 2 A.

[0023] FIG. 2C shows an example user interface to manage a list of offensive expressions associated with the user account of FIG. 2 A.

[0024] FIG. 2D shows the user interface of FIG. 2C when an offensive expression is queried and is determined to not be in the list of offensive expressions.

[0025] FIG. 2E shows the user interface of FIG. 2C when an offensive expression is queried and is determined to be in the list of offensive expressions.

[0026] FIG. 2F shows an example user interface to display the list of offensive expressions.

[0027] FIG. 3 shows another example user interface of moderation settings associated with a user account of a user. [0028] FIG. 4A shows an example user interface displayed on a first user device associated with a first user account of a first user to compose a base message. The user interface includes an interactive label to direct the first user to the moderation settings of FIG. 2B.

[0029] FIG. 4B shows an example user interface to manage muted words for a user account of a user.

[0030] FIG. 4C shows an example user interface with a message thread displayed on a user device associated with the user account of FIG. 4B. The user interface further includes a notification of a muted word selected by the user and a button to direct the user to the moderation settings of FIG. 2B.

[0031] FIG. 5A shows an example user interface displayed on a first user device associated with a first user account of a first user to compose a reply message in response to a base message posted by a second user account of a second user where a draft of the reply message includes an offensive expression. A prompt is further displayed to warn the first user the draft of the reply message includes an offensive expression.

[0032] FIG. 5B shows the user interface of FIG. 5 A with the prompt expanded to summarize the actions taken by the platform when the reply message is posted without removing the offensive expression. The prompt further includes a user interface element to direct the first user to additional information on the second user’s moderation settings on reply messages.

[0033] FIG. 5C shows the user interface of FIG. 5B after selecting the user interface element.

[0034] FIG. 6A shows a first portion of a flow chart of an example method for moderating a reply message posted by a second user in response to a base message posted by a first user.

[0035] FIG. 6B shows a second portion of the flow chart of FIG. 6 A.

[0036] FIG. 7A shows an example user interface displayed on a first user device associated with a first user account of a first user to compose a base message. The user interface further includes a prompt to manage reply message settings with a user interface element to direct the first user to labels and guidelines to associate with the base message.

[0037] FIG. 7B shows an example user interface of various settings associated with the labels and guidelines of FIG. 7 A. [0038] FIG. 8A shows an example user interface on a second user device associated with a second user account of a second user that includes a message thread with the base message authored by the first user account of FIG. 7 A.

[0039] FIG. 8B shows the user interface of FIG. 8 A after selecting the label(s) associated with the base message.

[0040] FIG. 9 shows a flow chart of an example method for assigning and displaying one or more labels with a message.

[0041] FIG. 10 shows a flow chart of an example method for assigning and displaying guidelines with a message.

DETAILED DESCRIPTION

[0042] Following below are more detailed descriptions of various concepts related to, and implementations of, a social messaging platform that provides user-controlled moderation features including a user-defined filter to moderate reply messages and features to label a base message and/or provide guidelines of the desired tone of the conversation. Various aspects of creating/editing a list of offensive expressions, creating/editing one or more label(s) and/or guidelines to associate with a message, defining penalties for users who post reply messages with one or more offensive expression(s), notifying users that a reply message includes one or more offensive expression(s) are also disclosed herein. It should be appreciated that various concepts introduced above and discussed in greater detail below may be implemented in multiple ways. Examples of specific implementations and applications are provided primarily for illustrative purposes so as to enable those skilled in the art to practice the implementations and alternatives apparent to those skilled in the art.

[0043] The figures and example implementations described below are not meant to limit the scope of the present implementations to a single embodiment. Other implementations are possible by way of interchange of some or all of the described or illustrated elements. Moreover, where certain elements of the disclosed example implementations may be partially or fully implemented using known components, in some instances only those portions of such known components that are necessary for an understanding of the present implementations are described, and detailed descriptions of other portions of such known components are omitted so as not to obscure the present implementations.

[0044] In the discussion below, various examples of inventive social messaging platforms are provided, wherein a given example or set of examples showcases one or more particular features or aspects related to the generation, management, and enforcement of a list of offensive expressions for moderation and the generation, management, and application of one or more label(s) and/or guidelines to a base message. It should be appreciated that one or more features discussed in connection with a given example of a social messaging platform may be employed in other examples of social messaging platforms according to the present disclosure, such that the various features disclosed herein may be readily combined in a given social messaging platform according to the present disclosure (provided that respective features are not mutually inconsistent).

1. An Example Social Messaging Platform with User-Controlled Moderation

[0045] FIG. 1A illustrates an example online social messaging platform 100 and example user devices 104a-104n configured to interact with the platform over one or more wired or wireless data communication networks 120. Users 102a-102n of the platform use their user devices 104a- 104n, on which client software 106a-106n is installed, to use the platform. A user device can be any Internet-connected computing device, e.g., a laptop or desktop computer, a smartphone, or an electronic tablet. The user device can be connected to the Internet through a mobile network, through an Internet service provider (ISP), or otherwise.

[0046] Each user device is configured with software, which will be referred to as a client or as client software 106a-106n, that in operation can access the platform 100 so that a user can post and receive messages, view, interact with, and create streams of messages and other content items, and otherwise use the service provided by the platform. Generally, the client software 106a-106n can be adapted for operation on different user devices and/or different operating systems. For example, the client software 106a-106n can run on various operating systems including, but not limited to, Google Android™, Apple iOS®, Google Chrome OS™, Apple MacOS®, Microsoft Windows®, and Linux®. The client software 106a-106n can further include web applications and cloud-based smartphone applications (e.g., the client software 106a isn’t installed directly onto the user’s device, but is rather accessible through a web browser on the user’s device).

[0047] Generally, a message posted on the platform 100 contains data representing content provided or selected by the author of the message. The message may be an instance of a container data type (also sometimes referred to as a ‘message object’) storing the content data. The types of data that may be stored in a message include text, graphics, images, video, audio content, and computer code, e.g., uniform resource locators (URLs), for example. Messages can also include key phrases or tags (e.g., a hashtag represented by “#”), that can aid in categorizing messages or in linking messages to topics. Messages can further include one or more fields for metadata that may or may not be editable by the message author or account holder, depending on what the platform permits. Examples of fields for message metadata can include, but is not limited to, a time and date of authorship, the user account of the authoring user, a geographical location of the user device when the client posted the message, an indication the message contains one or more offensive expression(s) and/or a list of the offensive expression(s) in the message (see Section 2), and one or more labels and/or guidelines associated with message (see Section 3). In some implementations, what metadata is provided to the platform by a client is determined by privacy settings controlled by the user or the account holder.

[0048] Messages composed by one account holder may include references to other accounts, other messages, or both. For example, a message may be composed in reply to another message posted by another account or by the user. Messages may also be re-publications of messages received from another account. Generally, an account referenced in a message may appear as visible content in the message, e.g., as the name of the account, and may also appear as metadata in the message. As a result, the referenced accounts can be interactive in the platform. For example, users may interact with account names that appear in their message stream to navigate to message streams of those accounts. The platform also allows users to designate particular messages as private; a private message will only appear in the message streams of the composing and recipient accounts. Generally, messages on the platform are microblog posts, which differ from email messages in a number of ways, for example, in that an author of the post does not necessarily need to specify, or even know, which accounts the platform will select to provide the message to.

[0049] The platform 100 is implemented on one or more servers l lOa-l lOm in one or more locations (also referred to more generally as a “platform server 110”). Each server is implemented on one or more computers, e.g., on a cluster of computers. In some implementations, the platform 100 further includes a database 117 to store, for example, various data on each user account, such as one or more lists of offensive expressions, moderation settings, and settings for applying labels and/or guidelines to a message. The platform, the user devices, or both are configured, as will be described, to implement or perform one or more of the innovative technologies described in this specification. Further information about user devices, clients, servers, and the platform is provided later in this specification (see Section 4).

[0050] Aspects disclosed herein are generally directed to the moderation in reply messages using, in part, user-defined content filter(s) and display of injunctive social norms to users by applying one or more labels and/or guidelines to a base message. Such aspects may be executable by any suitable components of the platform 100 such as, for example, by one or more of the platform servers l lOa-l lOm, and/or by any suitable components of the user devices 104a-104n. FIG. IB shows an expanded view of the user device 104a. The user device 104a includes one or more processors 101 and a memory 105. Unless indicated otherwise, all components of the user device 104a herein can be in communication with each other. FIG. 1C shows an expanded view of the platform server 110a. The platform server 110a includes one or more processors 111 and a memory 115. The platform server 110a can further be communicatively coupled to the database 117. Unless indicated otherwise, all components of the platform server 110a herein can be in communication with each other.

[0051] One or more of the servers 110 implement a moderation module 112, directed to the detection of offensive expressions in reply messages posted in response to a base message as well as the notification and enforcement of moderation settings of the user who authored the base message (see, for example, the moderation module 112 in the memory 115 of FIG. 1C). The moderation module 112 is also directed to managing the application of label(s) and/or guidelines to a base message. In some implementations, the client software 106a-106n also includes a moderation module 108 to facilitate user interaction and communication with the moderation module 112 on the server(s) 110 (see, for example, the moderation module 108 in the memory 105 of FIG. IB). For example, the functions provided by the moderation module 108 can include, but is not limited to, providing a user interface for users to manage moderation settings, create and/or edit a list of offensive expressions, and assign one or more label(s) and/or guidelines to a base message.

[0052] The one or more processors 101 and 111 can each (independently) be any suitable processing device configured to run and/or execute a set of instructions or code associated with its corresponding user device 104, platform server 110, and/or the platform 100. Each processor can be, for example, a general-purpose processor, a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), and/or the like. The one or more processors 101 and 111 can execute the moderation modules 108 and 112, respectively, as described in further detail below.

[0053] The memory 105, the memory 115, and the database 117 can encompass, for example, a random-access memory (RAM), a memory buffer, a hard drive, a database, an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), a read-only memory (ROM), Flash memory, and/or so forth. The memory 105, the memory 115, and the database 117 can store instructions to cause the one or more processors 101 and 111, respectively, to execute processes and/or functions associated with the moderation modules 108 and 112, the user device 104, the platform server 110, and/or the platform 100. The memory 105, the memory 115, and the database 117 can store any suitable content for use with, or generated by, the platform including, but not limited to, one or more connection graphs, a rule repository, and/or the like.

[0054] It should be appreciated users are not limited to posting only reply messages in response to a base message. Users can also post a share message, which is a message intended for other users that may or may not include the authoring user of the base message and includes the content of the base message without any additional content from the responding user. Users can also post a quote message, which is a message intended for other users that may or may not include the authoring user of the base message and includes the content of the base message along with additional content from the responding user. Thus, it should be understood that while often described herein with respect to a reply message for simplicity, the moderation modules 108 and 112 can also be executed on a share message and/or a quote message to the extent applicable.

[0055] It should also be understood that while often described herein with respect to a content creator (or a first user) and a content consumer (or a second user) for simplicity, the platform 100 can encompass a large number of users (e.g., thousands, hundreds of thousands, millions, hundreds of millions) each of whom can post a base message and/or a reply message, define one or more lists of offensive expressions, have moderation settings to control moderation of reply messages, assign label(s) and/or guidelines to any base message. Moreover, each user can be both a content creator and a content consumer.

2. A User-Defined Filter for Moderation

[0056] The moderation modules 108 and 112 disclosed herein are configured to moderate reply messages posted by content consumers in response to a base message posted by a content creator using list(s) of offensive expressions to identify language the content creator wants to discourage and/or does not want to see in the reply messages. The list(s) of offensive expressions can be stored in memory (e.g., the memory 105, the memory 115, the database 117) and associated with the content creator’ s user account. The moderation modules 108 and 112 can further be executed when content consumers are drafting a reply message to analyze the reply message as it is being drafted and to notify the content consumer when the content of the reply message includes an offensive expression from the content creator’s list(s) of offensive expressions. The notification can include, for example, visual indicators (e.g., highlights) displayed on a user interface of the content consumer’s user device to identify the offensive expression in the draft of the reply message and/or a prompt to warn the content consumer of actions that will be taken by the platform 100 for posting a reply message with the offensive expression(s). The moderation modules 108 and 112 can further apply one or more penalties to users who proceed to post reply messages with offensive expression(s).

[0057] In this manner, the moderation modules 108 and 112 provide users of the social messaging platform 100 greater control over the moderation of reply messages. By allowing content creators to define a custom list of offensive expressions, context-specific language that is offensive to the content creator can be detected and/or removed from view of the content creator automatically by the platform 100. This can include offensive expressions that normally do not violate the rules of the platform 100 and/or any existing content filters. Thus, the list(s) of offensive expressions associated with each user account of the users of the social messaging platform 100 can be different from other user accounts. Further, the moderation modules 108 and 112 also reduce the burden on each user to personally manage their reply messages since the platform 100 can automatically moderate the reply messages displayed on the content creator’s user device.

[0058] Herein, an offensive expression is made up of one or more characters. The characters can be arranged to form one or more words in textual form. The characters can also include icons, such as emojis (e.g., a pictogram) or emoticons (e.g., a combination of punctuation marks, letters, numbers, and/or the like arranged to resemble a face or an object). In some implementations, an offensive expression can be formed using a standardized set of characters, such as the Unicode Standard. In some implementations, an offensive expression can include a wildcard (e.g., an asterisk ‘*’, a question mark ‘?’), which can be used to represent expressions spelled in different ways or represent multiple expressions. Additionally, offensive expressions formed of multiple words and/or icons can also be divided into individual words and/or icons to facilitate proximity matching as described further below. For purposes of processing via one or more processors (e.g., the processor(s) 101, the processor(s) 111), the characters can be a char data type and the offensive expression can be a string data type.

[0059] In some implementations, the platform 100 can also permit a content consumer to post a reply message with an offensive expression. This can be accompanied by a warning in the form of a label, a prompt, or a notification when the content consumer drafts the reply message. The warning can notify the content consumer of actions that can be taken by the platform 100 when posting a reply message with an offensive expression, such as one or more penalties being applied to the content consumer’s user account and/or content. This approach can be used to encourage users to proactively change their behavior by giving the content consumer opportunities to amend their reply message to remove the offensive expression(s). However, it should be appreciated that, in some implementations, the platform 100 can provide users the option to outright prohibit posting of reply messages with offensive expression(s).

[0060] In the following, further details of the moderation modules 108 and 112 are described. Specifically, the creation and management of a user-defined content filter and various moderation settings are described in Section 2.1. The enforcement of the user-defined content filter is described in Section 2.2. An example method for moderating a reply message is described in Section 2.3.

2.1 Creating and Managing a User-Defined Filter for Moderation

[0061] The moderation modules 108 and 112 can be configured to generally provide each user account of the platform 100 with several moderation settings. The moderation settings can control various aspects of moderating reply messages including, but not limited to, the creation and management of one or more lists of offensive expressions associated with a content creator’s user account, the display of notifications on a content consumer’s user device when drafting a reply message with at least one offensive expression, and the management of one or more penalties applied to content consumers who post reply messages with at least one offensive expression.

[0062] The moderation settings associated with a content creator’s user account are generally applied to any reply message posted in direct response to a base message posted by the content creator. Said in another way, if a content creator posts a base message (e.g., a root message, a reply message), the content creator’s moderation settings dictate the moderation of any reply messages within the branch formed by that base message. In some implementations, a message thread with multiple branches can thus have different moderation settings at each branch according to the moderation settings of the user who posted the base message of that branch. In some implementations, the moderation settings of a content creator can also be applied to reply messages that do not directly respond to the content creator’s base message. For example, if a first reply message is posted in direct response to a content creator’s base message and a second reply message is posted in direct response to the first reply message, the content creator’s moderation settings can apply to both the first and second reply messages. To prevent conflicts between competing moderation settings, the moderation settings of the user account that posted the first reply message can be superseded by the content creator’s moderation settings.

[0063] The moderation settings associated with each user account can be stored in memory (e.g., the memory 105, the memory 115, the database 117). For example, a record of the content creator’s user account can include the moderation settings along with other data associated with the user account including, but not limited to, a username, a user account name, authentication information (e.g., an email, a password), and a profile picture. The moderation settings can thereafter be retrieved from memory, for example, when the processor 111 of the platform 100 executes the moderation modules 108 and 112 to moderate a reply message drafted by a content consumer. In some implementations, the moderation settings and the list(s) of offensive expressions can be stored together (e.g., locally in the memory of a user device) or separately (e.g., the moderation settings are stored in a database, the list(s) of offensive expressions are stored locally in the memory of a user device).

[0064] The moderation settings associated with a content creator’s user account can generally be applied to each message posted by the content creator. In some implementations, the moderation settings can also be modified on a per message basis. Said in another way, the content creator can apply different moderation settings to different base messages. This can be achieved, in part, by applying the moderation settings from the content creator’s user account when the content creator activates the moderation settings while drafting the base message. The content creator can then modify the moderation settings for that base message as desired.

[0065] The moderation settings can be modified by the content creator via one or more user interfaces displayed on their user device and managed by the moderation modules 108 and/or 112. For example, FIG. 2 A shows an example user interface of privacy and safety related settings associated with the user account of a content creator. The user interface is displayed on a user device of the content creator and is accessible, for example, via an account settings page or profile page associated with the content creator’ s user account. It should be appreciated the user interface of FIG. 2A is not limited only to content creators of the platform 100, but the same user interface can be displayed for any user of the platform 100. As shown, the user interface can include a section 250a for the moderation settings with a status indicator element 252 to indicate whether moderation settings are activated or deactivated. The section 250a further includes a user interface element 254, which when selected, displays another user interface where various moderation settings can be modified as shown in FIG. 2B.

[0066] It should be appreciated the privacy and safety related settings user interface of FIG. 2A is one non-limiting example to access the moderation settings of a user account. In another example, FIG. 4A shows a user interface to compose a base message displayed on the user device of the content creator. As shown, the user interface can include a space 210 to contain the content of the base message. The user interface can further include a submit button 212 to post the base message to the platform 100 and a cancel button 214 to cancel the base message (e.g., delete the draft of the base message). FIG. 4A shows the user interface can also include an interactive label 250b provided by the moderation modules 108 and/or 112, which when selected by the content creator, displays the user interface of FIG. 2B. In another example, a notification or a message can be displayed in a stream on the content creator’ s user device with a user interface element that displays the user interface of FIG. 2B.

[0067] The user interface of FIG. 2B includes a section 260a with a description of the moderation feature (sometimes referred to as “smellcheck”) and a toggle switch 262 to activate or deactivate moderation of reply messages. The user interface further includes a section 260b, which provides a summary of the offensive expressions selected by the content creator and a user interface element 264, which when selected, displays another user interface to facilitate the creation and/or management of one or more lists of offensive expressions (see, for example, FIG. 2C). As shown in FIG. 2B, the summary of offensive expressions in the section 260b can include the number of offensive expressions in each of the content creator’s lists of offensive expressions. The user interface further includes a section 260c with various settings on penalties that can be applied to content consumers who post reply messages with an offensive expression in response to a content creator’s base message.

[0068] FIG. 2C shows an example user interface to create a list of offensive expressions and/or manage one or more list(s) of offensive expressions, such as by adding or removing an offensive expression. As shown, the user interface includes a search box 270 that can be used to find an offensive expression in a list of offensive expressions and/or to add an offensive expression to a list of offensive expressions. Additionally, the user interface of FIG. 2C can display one or more lists of offensive expressions associated with the content creator’s user account. As an illustrative example, the content creator’s user account includes a list 272a containing offensive expressions with hateful, hurtful, or violent language, a list 272b containing offensive expressions with profanity, and a list 272c containing offensive expressions added manually by the content creator. It should be appreciated the lists 272a-272c are non-limiting examples and that each user account can generally include any number of lists according to the user’s preferences.

[0069] Each of the lists 272a, 272b, and 272c include a toggle switch 274 to activate or deactivate the list. Herein, the platform 100 only checks a reply message for offensive expressions in a list when that list is activated. In some implementations, the user interface of FIG. 2C is designed to avoid displaying any offensive expressions in the lists of offensive expressions to reduce the likelihood of traumatizing or re-traumatizing the content creator. Instead, the user interface of FIG. 2C includes a user interface element 276, which when selected, displays another user interface with the offensive expressions in each of the lists of offensive expressions to provide the content creator the option to activate or deactivate individual offensive expressions in each list (see, for example, FIG. 2F).

[0070] The lists of offensive expressions generally include (A) user-defined lists of offensive expressions (e.g., the list 272c) where the content creator manually creates the list and adds offensive expressions to the list thereafter and (B) lists of predetermined offensive expressions (e.g., the list 272a) that are generated by the platform 100. Each list further includes a list name chosen by the content creator or generated automatically by the platform 100, such as “hateful, hurtful, or violent language,” “profanity,” or “your custom list.”

[0071] The content creator can manually add an offensive expression using the search box 270. For example, FIG. 2D shows that when an offensive expression (e.g., an emoji 281a) is entered in the search box 270, a search result 280a is displayed indicating the offensive expression is not in any list (e.g., the lists 272a-272c). The search result 280a further includes an add button 282 to add the offensive expression to a list (e.g., the list 272c). The user interface of FIG. 2D further shows a user interface element 284, which when selected, allows the content creator to search and display additional offensive expressions to add to a list. In some implementations, upon selecting the add button 282, the user interface can display the lists of offensive expressions associated with the content creator’s user account for selection. Upon selecting one or more of the lists, the offensive expression is added to those lists. Alternatively, the user interface can display an option to create a new list of offensive expressions along with one or more prompts for the content creator to input, for example, a list name for the list. Generally, an offensive expression can be manually added to a user-defined list of offensive expressions or a list of predetermined offensive expressions.

[0072] It should be appreciated the user interface of FIG. 2D is a non-limiting example and that the platform 100 can support other ways of adding offensive expressions to a content creator’s list of offensive expressions for moderation. For example, users can add an offensive expression for moderation when muting that offensive expression. FIG. 4B shows an example user interface for users to add muted expressions. When a user mutes an expression, that expression is removed from any content displayed on the user interface of the content creator’s user device. This includes any message displayed in a stream, any notifications, and/or any direct messages. This can be accomplished, for example, by replacing the expression with several asterisks or disabling display of any message or notification containing that expression. As shown, the user interface of FIG. 4B includes a space 256 to enter the expression to be muted. FIG. 4C shows that when an expression is added, a notification 251 can be displayed, for example, over a message thread on the content creator’ s user device. The notification 251 notifies the content creator the expression is muted and can further include a button 258 provided by the moderation modules 108 and/or 112 to add the expression to a list of offensive expressions for moderation. [0073] The list(s) of predetermined offensive expressions generally includes offensive expressions that are commonly used by users on the social messaging platform 100, and especially users that engage with the content creator. These offensive expressions can be identified, for example, by agents of the social messaging platform 100, e.g., a human operator that reviews messages for toxic and/or abusive content. For example, agents can identify expressions commonly used in messages reported for toxic and/or abusive content that are likely to be offensive to most users. The offensive expressions can thereafter be collectively compiled into a list of predetermined offensive expressions and shared with one or more users of the platform 100.

[0074] It should be appreciated, however, some social messaging platforms include users from around the world. Thus, the conversations on the social messaging platform can vary appreciably due to cultural differences and/or the users’ native language. In some implementations, the list(s) of predetermined offensive expressions can be tailored, for example, to include offensive expressions that are more likely to be relevant and/or understood by the content creator via the moderation modules 108 and/or 112. This can be achieved, in part, by the moderation modules 108 and/or 112 generating list(s) of predetermined offensive expressions based on one or more attributes of the content creator’s user account. For example, the attributes associated with a user account can include, but is not limited to, a location of the user device associated with the user account (e.g., determined by a position tracking sensor in the user device), a region associated with the user account (e.g., the Southern region of the United States, the West Coast region of the United States), a nationality associated with the user account, and a default language associated with the user account (e.g., the native language of the content creator). In this manner, the list(s) of predetermined offensive expressions can provide some degree of personalization to each user account.

[0075] As described above, each offensive expression in a list of offensive expressions can be individually activated or deactivated in addition to activating or deactivating each list of offensive expression(s). Similar to a list of offensive expressions, the platform 100 only checks a reply message for an offensive expression when that offensive expression is activated. For example, FIG. 2E shows the user interface when an offensive expression 281b that is present in at least one list of offensive expressions is entered in the search box 270. As shown, multiple search results 280b, 280c, and 280d are displayed corresponding to offensive expressions that either match the offensive expression 281b or include the offensive expression 281b. Each of the search results 280b, 280c, and 280d indicate the corresponding offensive expression is present in at least one list of offensive expressions. In some implementations, the name of the list containing the offensive expression can be displayed. It should be appreciated the user interface can also display search results with offensive expressions that are not present in any list of offensive expressions (e.g., the search result 280a) together with the search results 280b, 280c, and 280d. For each of the search results 280b, 280c, and 280d, a toggle switch 274 is also provided for the content creator to activate or deactivate that offensive expression for moderation.

[0076] FIG. 2F shows another example user interface that lists the offensive expressions in the lists of offensive expressions associated with the content creator’ s user account. The user interface of FIG. 2F can be displayed, for example, by selecting the user interface element 276 on the user interface of FIG. 2C. As shown, the user interface can include offensive expressions 273a, 273b, 273c, 273d, 273e, and 273f with each offensive expression having a toggle switch 274 to activate or deactivate that offensive expression. The user interface can further include a display filter button 286, which when selected, provides several options to filter the offensive expressions displayed on the user interface according to different criteria. The criteria can include, but is not limited to, membership in a list of offensive expressions (e.g., the lists 272a-272c), the category of the offensive expression (e.g., profanity, hateful, violet), the date the offensive expression was added, and the most frequently detected offensive expressions in the reply messages posted in response to the content creator’ s base message. The user interface further includes a hide button 288 to close the list of offensive expressions and return to the user interface of FIG. 2C. Any changes made by the content creator are thereafter stored in memory (e.g., the memory 105, the memory 115, the database 117). As shown in FIG. 2F, each offensive expression can be partially hidden from view by replacing a portion of the offensive expression with asterisks to reduce the likelihood of traumatizing or re-traumatizing the content creator. In some implementations, each offensive expression can be interactive to provide users the option to display the original offensive expression.

[0077] By providing each user the option to tailor the list(s) of offensive expressions according to their personal preferences, each user account of the platform 100 can potentially have a unique combination of lists. Thus, in some implementations, the list(s) of offensive expressions for each user account can be stored locally on one or more user devices associated with that user account (e.g., a smartphone, a tablet). Each list can further include one or more fields for each offensive expression to indicate whether the offensive expression should be searched or not (e.g., whether the toggle switch 274 is activated or deactivated). To facilitate the moderation of a reply message especially as a reply message is being drafted by a content consumer, a copy of the content creator’s list(s) can be transmitted from the content creator’ s user device to the platform 100 (e.g., a platform server 110) for temporary storage. When the platform 100 detects a content consumer is drafting a reply message in response to the content creator’s base message, the platform 100 can transmit the content creator’s list(s) to the content consumer’ s user device and the processor of the user device can thereafter evaluate the draft of the reply message based on the content creator’s list(s). Once the content consumer either posts the reply message or cancels the reply message, the content creator’s list(s) can be removed from the content consumer’s device and/or the platform 100.

[0078] In some implementations, the content creator’s list(s) can remain in memory on the platform 100 for a limited period of time so that the content consumer’ s user device does not have to repeatedly transmit a copy of the content creator’s list(s) when multiple users draft a reply message. For example, the period of time can be chosen according to the time when reply messages are most likely to be posted responding to a base message (e.g., 1-3 days after the content creator’s base message is posted). The moderation module 112 can further be executed to monitor the duration that the content creator’s list(s) are stored in memory on the platform 100. It should be appreciated, however, that copies of the content creator’s list(s) can be stored indefinitely on the platform 100 (e.g., the memory 115, the database 117) and periodically updated or replaced whenever the content creator’s list(s) are changed on the content creator’s user device via the moderation modules 108 and/or 112.

[0079] The section 260c in the user interface of FIG. 2B shows several example penalties that can be applied by execution of the moderation modules 108 and/or 112 when a content consumer posts a reply message with one or more offensive expressions. For example, the section 260c includes a penalty 266a, which when activated, causes a reply message with an offensive expression to be downranked in the branch of the message thread such that the reply message has a lower priority for display compared to other reply messages within that branch. The user interface further includes a toggle switch 268 to activate or deactivate the penalty 266a.

[0080] The downranking process can be facilitated, in part, by each reply message in a branch of the message thread having a rank parameter to determine the order of the reply messages displayed on the user device of a user of the platform 100. The rank parameter represents the position or index to display a reply message. In some implementations, a rank parameter with a higher value can correspond to a higher position for display. The reply messages can be arranged such that the reply message having the highest rank parameter value is displayed first, the reply message with the next highest rank parameter value is displayed second, and so on. However, it should be appreciated that, in some implementations, a lower rank parameter value can correspond to a higher position for display where the reply message with the lowest rank parameter value is displayed first, the reply message with the next highest rank parameter value is displayed second, and so on.

[0081] As an illustrative example, a message thread can include a single branch with a first reply message, a second reply message, and a third reply message posted in response to a base message (e.g., a root message) of a content creator. The first reply message includes a first rank parameter, the second reply message includes a second rank parameter with a value lower than the first rank parameter, and the third reply message includes a third rank parameter with a value lower than the second rank parameter. The first reply message can be displayed first in a message thread followed by the second reply message and lastly the third reply message. However, if the platform 100, in executing the moderation module 108 and 112, determines the first reply message includes an offensive expression from the list of offensive expressions in the moderation settings of the content creator, the first reply message is downranked such that the first rank parameter has a value lower than the second and third rank parameters. This, in turn, causes the second reply message to be displayed first in the message thread followed by the third reply message and lastly the first reply message.

[0082] In another example, FIG. 2B shows a penalty 266b, which when activated, causes the user account associated with a reply message having an offensive expression to be blocked from the content creator’s user account. The penalty 266b also includes a toggle switch 268 to activate or deactivate the penalty 266b. Generally, when a first user (e.g., the content creator) blocks a second user (e.g., the content consumer), the second user is no longer allowed to view the first user’s account, view any content posted by the first user, follow the first user, or send any messages or content to the first user. Furthermore, any content posted by the second user is no longer visible to the first user. Thus, any reply messages posted by the content consumer in the message thread can be made not visible to the user device of the content creator.

[0083] It should be appreciated the penalties in FIG. 2B are non-limiting examples and that other penalties can be applied to content consumers that post reply messages with an offensive expression. For example, a penalty can be applied to disable display of a reply message with an offensive expression. In another example, FIG. 3 shows a similar user interface as FIG. 2B, but with a penalty 266c, which when activated, causes the user account associated with a reply message with an offensive expression to be muted from the content creator’s user account. Generally, when a first user (e.g., the content creator) mutes a second user (e.g., the content consumer), any content posted by the second user is not visible to the first user. However, the second user can still view the first user’s account, view any content posted by the first user, and/or follow the first user.

[0084] In some implementations, each penalty can also include additional conditions before the penalty is applied to the offending content consumer’s reply message and/or user account. For example, a penalty can be applied only when the number of reply messages with an offensive expression posted by a content consumer exceeds a predetermined threshold. The predetermined threshold can be chosen by the content creator and can generally range between one reply message to ten reply messages, or more, including all values and sub-ranges in between. As an illustrative example, FIG. 2B shows the penalty 266b is only applied when the content consumer posts two reply messages with an offensive expression. Thus, content consumers can post one reply message with an offensive expression without being blocked by the content creator, per the penalty 266b. However, in this example, the penalty 266a does not include any threshold condition and can thus be applied. In another example, a penalty can be applied only when the rate at which reply messages with an offensive expression are posted by a content consumer exceeds a predetermined threshold. As an illustrative example, the predetermined threshold can range between one reply messages having an offensive expression per hour to five reply messages having an offensive expression per hour, or more, including all values and sub-ranges in between.

[0085] For some penalties, the penalty can be applied for a limited period of time. For example, the penalties 266b and 266c to block and mute, respectively, a content consumer’s user account can be applied for a limited period of time, such as 1 hour, 1 day, 1 week, 1 month, or 1 year. The language modules 108 and/or 112 can further monitor the elapsed time for each penalty to determine whether the penalty should remain or be removed from a content consumer’s user account. In this manner, the content creator can provide an opportunity for the content consumer to post reply messages after the period of time elapses. In some implementations, if the content consumer again posts a reply message with an offensive expression, the content consumer’s account can be blocked and/or muted for a longer period of time (e.g., twice the period of time selected by the content creator) or indefinitely. It should be appreciated that for some penalties, such as the penalty 266a, the penalties can be applied indefinitely.

[0086] Additionally, the extent in which a penalty is applied can also vary depending on the moderation settings of the content creator and/or the structure of the message thread. In one example, a penalty is only applied within the branch that the reply message was posted (e.g., the content consumer can still post reply messages in other branches without restriction). In another example, a penalty can be applied to the entire message thread (e.g., the content consumer is blocked from posting a reply message to the message thread). In yet another example, a penalty can be applied to other message threads that include a base message posted by the content creator. For example, the penalty 266a to downrank a reply message can be applied within the branch that the reply message was posted (e.g., the reply message is downranked only within that branch) or the entire message thread (e.g., the reply message is downranked relative to all reply messages within the message thread). In another example, the penalties 266b and 266c can extend to any message thread in which the content creator posts a base message.

[0087] The messages in a message thread can be stored in memory (e.g., the memory 105, the memory 115, the database 117). In some implementations, each reply message can further include one or more fields for metadata, as described above, to facilitate moderation. For example, the metadata fields can include a field to identify the user account of the base message that the reply message is directly responding to. The user account can be used to retrieve, for example, the moderation settings associated with that user account to evaluate the application of any penalties to the reply message. In another example, the metadata fields can include a first field to indicate whether the reply message includes an offensive expression. The metadata fields can further include a second field with an indexed number representing the number of reply messages with offensive expressions posted by that user account for that branch or message thread. For example, if a content consumer posts two reply messages with offensive expressions, the second field of the first reply message can be ‘ G and the second field of the second reply message can be ‘2.’ In yet another example, the metadata fields can include a field to indicate the type of penalties that should be applied to that reply message and/or the user account of the content consumer who posted that reply message. In this manner, reply messages that include offensive expressions can be tracked and penalized by updating the messages stored in memory as new reply messages are posted to the branch and/or message thread.

2.2 Moderation of a Reply Message

[0088] The moderation modules 108 and 112 disclosed herein can evaluate a reply message as it is drafted, i.e., substantially in real-time, by a content consumer to identify whether the draft of the reply message includes an offensive expression. If the reply message is determined to include an offensive expression, a warning can be displayed to notify the content consumer that the reply message includes the offensive expression and actions that may be taken by the platform 100 (e.g., penalties) against the content consumer’s user account and/or content if they do not remove the offensive expression. Additionally, the platform 100 can permit content consumers to post a reply message with an offensive expression. In this manner, the moderation features disclosed herein can be configured to encourage the content consumer to proactively change their behavior, in part, by giving the content consumer a choice to either remove an offensive expression from a reply message or post the reply message with offensive expression at the expense of being penalized.

[0089] As an illustrative example, FIGS. 5A-5C show several user interfaces displayed on a user device associated with a content consumer managed by the language modules 108 and/or 112. In particular, FIG. 5 A shows a user interface to compose a reply message 201. The user interface includes the space 210 for the message 201, the submit button 212, and the cancel button 214. In this example, the message 201 includes one offensive expression 203, which can be detected, for example, by the processor of the content consumer’s user device (e.g., the processor(s) 101) as described in further detail below.

[0090] As shown, when an offensive expression is detected in the message 201, a visual indicator 205 can be displayed to visually distinguish the offensive expression from other portions of the message 201. The visual indicator 205 can be represented in various forms including, but not limited to, a highlight of the offensive expression as shown in FIG. 5A, an underline of the offensive expression, the offensive expression being in bolded font, the offensive expression being shown in an italicized font, and any combinations of the foregoing. The user interface can further display a prompt 220 with a message to notify the content consumer that the draft of the reply message includes the offensive expression 203. In some implementations, a notification 222 can also be displayed to notify the content consumer that the content creator activated the moderation features for this particular branch of the message thread.

[0091] In some implementations, the prompt 220 can be an interactive such that, when selected in FIG. 5A, the prompt 220 expands to display a message 224 explaining why the offensive expression is prohibited as shown in FIG. 5B. The message 224 can further include the penalties that the content consumer’s user account and/or reply message will receive if the offensive expression is not removed. The prompt 220 can further include a confirmation button 226, which when selected, closes the prompt 220. The prompt 220 can also include an ignore button 228, which when selected, adds the offensive expression 203 to a list of ignored offensive expressions associated with the content consumer’s user account. If an offensive expression is in the list of ignored offensive expressions, the content consumer’s user device will not display a prompt 220 thereafter when that offensive expression is included in a draft of a reply message. However, the visual indicator 205 can still be displayed whenever the offensive expression is detected in the reply message. Lastly, the prompt 220 in FIG. 5B includes an information button 230, which when selected, displays another prompt with information on the content creator’s moderation settings as shown in FIG. 5C.

[0092] As shown, the prompt of FIG. 5C includes a section 232 with a message summarizing the purpose of the moderation features. The prompt also includes a section 234 summarizing some of the types of offensive expressions the content creator included in their list(s). The section 234 can include, for example, the list names of each list associated with the content creator’s user account. The prompt further includes a section 236 summarizing the penalties incurred if the content consumer posts the message 201 without removing the offensive expression 203. The prompt further includes a section 238 covering the list of ignored offensive expressions associated with the content consumer’s user account. In some implementations, the section 238 includes a user interface element 242, which when selected, displays the list of ignored offensive expressions. The content creator can further add or remove offensive expressions from the list of ignored offensive expressions. Lastly, the prompt of FIG. 5C includes a confirmation button 240, which when selected, closes the prompt to return to the user interfaces of FIGS. 5 A or 5B. Any changes made by the content creator, for example, to the list of ignored offensive expressions are thereafter stored in memory (e.g., the memory 105, the memory 115, the database 117).

[0093] The detection of an offensive expression in a draft of the reply message can be accomplished, in part, by first transferring a copy of the list(s) of offensive expressions associated with the content creator’s user account to the content consumer’ s user device (e.g., via the platform server 110). Thereafter, the processor(s) of the content consumer’s user device can evaluate the offensive expressions in the list(s) against the draft of the reply message to determine whether the reply message includes an offensive expression. Alternatively, a copy of the draft of the reply message can be transmitted to a platform server and the processor(s) of the platform server can evaluate the draft of the reply message for any offensive expressions. This evaluation process can be accomplished in several ways in real-time or substantially in real-time as the content consumer is drafting the reply message.

[0094] In some implementations, the processor(s) of the user device or the platform server can determine whether the draft of the reply message includes an expression that exactly matches an offensive expression in the content creator’s list(s) of offensive expressions. In other words, the processor(s) evaluate whether the sequence of characters forming the offensive expression appear identically in the draft of the reply message. For example, each offensive expression in the list(s) of offensive expressions can be represented as a string data type. The draft of the reply message can also be represented as a string data type. The processor(s) of the content consumer’s user device can execute a loop over the list(s) of offensive expressions such that each offensive expression is compared against the draft of the reply message using, for example, a string compare function. If the string compare function returns a ‘True’ value, the offensive expression is contained within the draft of the reply message as a substring. Otherwise, if a ‘False’ value is returned, the draft of the reply message does not include the offensive expression.

[0095] It should be appreciated the above approach using a loop to evaluate the presence of each offensive expression in the draft of the reply message is a non-limiting example. In another example, the draft of the reply message can be represented as a list of substrings where each substring corresponds to a single word (e.g., a series of characters that begins and ends with a space) or an icon in the reply message. The list of substrings representing the reply message can then be compared directly against the list(s) of offensive expressions to determine whether the respective lists include any matches. If the offensive expressions include phrases formed from two or more words and/or icons, the draft of the reply message can be divided into individual words and/or icons as well as combinations of consecutive words and/or icons. For example, the reply message “This is an offensive expression” can be represented in a list as [“This”; “is”; “an”; “offensive”; “expression”; “This is”; “is an”; “an offensive”; “offensive expression”] to cover offensive expressions formed of one word or two words.

[0096] In some implementations, the processor(s) of the user device or the platform server can determine whether the draft of the reply message includes an expression that approximately matches an offensive expression in the content creator’s list(s) of offensive expressions. This approach can account for reply messages with offensive expressions that are misspelled and/or altered to avoid detection. For this approach, an approximate string-matching algorithm (also referred to in the art as a “fuzzy string searching algorithm”) can be used to evaluate the closeness between an offensive expression in the list(s) of offensive expressions and the draft of the reply message.

[0097] For example, the Levenshtein algorithm can be implemented in the moderation modules 108 and 112 to evaluate the closeness between two strings using an edit distance (also referred to in the art as a “Levenshtein distance”). The edit distance represents the number of primitive operations applied to one string (e.g., a portion of the reply message) for that string to exactly match another string (e.g., an offensive expression). The primitive operations include modifications to individual characters of the string, such as insertion where a single character is added to the string, deletion where a single character is removed from the string, substitution where one character in the string is replaced by another character, and/or transposition where two characters are swapped in the string.

[0098] The edit distance can be computed by comparing each offensive expression in the list(s) of offensive expressions against different portions of the draft of the reply message using the Levenshtein algorithm. For example, the draft of the reply message can be divided into a list of substrings as described above. The list of substrings can include individual words, individual icons, and/or combinations of words and/or icons. Each substring in the draft of the reply message can then be compared against each offensive expression to determine an edit distance.

[0099] To determine whether the draft of the reply message includes a substring that approximately matches an offensive expression, the edit distance can be compared against a predetermined threshold. Generally, a lower edit distance value between two strings indicates the two strings are more closely matched (i.e., fewer primitive operations are applied to modify one string to exactly match the other string). Thus, if the portion of the draft of the reply message that is compared against an offensive expression results in an edit distance below the predetermined threshold, that portion can be considered to include that offensive expression. For example, the predetermined threshold can range between 1 to 3.

[0100] In some implementations, the processor(s) of the user device or the platform server can also use a proximity string matching approach where the draft of the reply message is determined to include an offensive expression with two or more words and/or icons if the words if the words and/or icons are in sufficient proximity to one another in the draft of the reply message. For example, an offensive expression with multiple words can be divided into a list of substrings where each substring represents one of the words. Each substring can then be compared against the draft of the reply message to determine whether the substrings are present using, for example, the exact string-matching approach or approximate string -matching approach described above. If the substrings are present in the draft of the reply message, but not positioned next to one another, the number of words and/or icons separating the two words can then be computed for each pair of substrings.

[0101] The number of words and/or icons separating each pair of substrings and/or all respective pairs of substrings can then be compared against a predetermined threshold to determine whether the substrings are sufficiently close to one another to match the offensive expression. As an illustrative example, the offensive expression “huge jerk” can be divided into the substrings “huge” and “jerk.” If the draft of the reply message includes the expression “huge insufferable jerk,” the substrings “huge” and “jerk” are separated by one word. If the predetermined threshold is two words, then the expression “huge insufferable jerk” is determined to be a match with the offensive expression “huge jerk.” More generally, the predetermined threshold can range between one word/icon to five words/icons, or more, including all values and sub-ranges in between.

[0102] It should be appreciated the above approaches can also accommodate offensive expressions with wildcards. A wildcard is generally a character used to represent zero or more characters. Common wildcards include, but are not limited to, an asterisk ‘ *’ and a question mark ‘?’ . In some implementations, wildcards can be used to cover offensive expressions with different spelling or multiple offensive expressions. For example, the offensive expression “stupid*” can cover different forms of the same expression in the reply message, such as “stupidity,” “stupidly,” and “stupidest.” In another example, the expression “*stupid*” can cover different expressions that include the word “stupid” in the reply message, such as “stupidhead,” “stupid-head”, and “super stupidhead.”

[0103] It should also be appreciated the above approaches to evaluate the draft of the reply message for any offensive expressions can also be applied to the evaluation of a posted reply message for any offensive expressions. For example, when a content consumer posts a reply message, the processor(s) of the platform server (e.g., the processor(s) 111) can evaluate the reply message in the same manner as described above to determine whether the reply message includes an offensive expression. If it is determined the reply message includes an offensive expression, the processor(s) of the platform server can determine whether a penalty should be applied to the reply message and/or the content consumer’s user account. This can be facilitated, in part, by retrieving from memory the moderation settings associated with the content creator’s user account to determine which penalties and their conditions should be applied and the messages in the message thread to assess whether the content consumer previously posted one or more reply messages with an offensive expression. If it is determined a penalty should be applied, the processor(s) of the platform server can apply the penalty, e.g., by blocking or muting the content consumer’s user account from the content creator’ s user account, downranking the reply message within the branch or message thread, or disabling display of the reply message on other user devices including the content creator’s user device.

2.3 An Example Method for Moderating a Reply Message

[0104] FIGS. 6A and 6B show an example method 300 for moderating a reply message posted by a second user account (e.g., a content consumer) in response to a base message posted by a first user account (e.g., a content creator). The method 300 can generally be executed via a combination of processor(s) of the platform server, the processor(s) of the content consumer’s user device, and/or the processor(s) of the content creator’s user device. At step 302, the platform server receives a base message, moderation settings, and a list of offensive expressions via transmission from a first user device associated with a first user account. The base message, moderation settings, and a list of offensive expression can be generated at the content creator’ s user device. At step 304, the base message, the moderation settings, and the list of offensive expressions are then stored in memory on the platform (e.g., the memory 115, the database 117). In some implementations, the moderation settings and the list of offensive expressions can be stored in memory for a limited period of time to facilitate transmission to one or more user devices with user accounts drafting a reply message in response to the first user account’s base message. At step 306, the base message is transmitted from the platform server to a plurality of user devices associated with a plurality of user accounts for display, e.g., in a stream on the user device.

[0105] Thereafter, when the second user starts to draft a reply message in response to the base message of the first user account, an indication is received from a second user device associated with the second user account that a reply message is being drafted in step 308. This indication can be detected by the second user device and transmitted to the platform server. In one example, the indication can correspond to the second user selecting a user interface element (e.g., a reply button) to post a reply message. In another example, the indication can represent the user interface to compose a reply message being opened on the second user device (e.g., the user interface of FIG. 5A). As the second user drafts the reply message, the moderation settings and the list of offensive expressions are transmitted from the platform server to the second user device in step 310.

[0106] The draft of the reply message is then evaluated to determine whether the second user is drafting the reply message in step 312 via, for example, at the second user device. This can be accomplished, for example, by evaluating whether the second user has selected the submit button 212 to post the reply message or the cancel button 214 to cancel the reply message. If it is determined that the reply message is being drafted, the draft of the reply message can then be evaluated to determine whether the draft includes an offensive expression in the list of offensive expressions in step 314. If it is determined the draft does not include an offensive expression, the method 300 returns to step 312. Otherwise, if it is determined the draft does include an offensive expression, the offensive expression is highlighted in the user interface on the second user device in step 316. A warning is also displayed showing the penalties that will be applied if the second user account posts the reply message without removing the offensive expression in step 318. The method then returns to step 312.

[0107] The steps 312-318 can then be repeated by the second user device until the reply message is no longer being drafted (i.e., it is either posted or cancelled). In one example, the steps 312-318 can repeat periodically over predetermined time interval (e.g., 1 second, 5 seconds). In another example, the steps 312-318 can repeat once a change is made to the draft of the reply message. This can be accomplished, for example, by detecting user input from the second user via the second user device (e.g., when the second user touches a touchscreen on a smartphone). That way, if the second user does not provide any input for an extended period of time (e.g., due to a sudden interruption while drafting the reply message), the steps 312-318 are not being executed continuously, thus reducing the computational load when executing the method 300.

[0108] When it is determined the reply message is not being drafted, the moderation settings and the list of offensive expressions are removed from the second user device in step 320. However, it should be appreciated that, in some implementations, step 320 can be omitted (i.e., the moderation settings and the list of offensive expressions remain on the second user device). Thereafter, the second user device determines whether the reply message was posted to the social messaging platform in step 322. If it is determined the reply message was not posted (i.e., it was cancelled), the method is terminated in step 324. Otherwise, if the reply message is posted (e.g., transmitted to the platform server), the reply message is stored in memory (e.g., the memory 115, the database 117) in step 326. Thereafter, the reply message is evaluated by the platform server to determine whether the reply message includes an offensive expression from the list of offensive expressions in step 328. If it is determined the reply message does not include an offensive expression, the reply message is then transmitted to the plurality of user devices including the first user device for display thereon (e.g., in a stream of the user device) in step 330.

[0109] Otherwise, if it is determined the reply message does include an offensive expression, the moderation settings associated with the first user account are evaluated to determine whether they include a relationship penalty, such as the penalties 266b and 266c. If the moderation settings do not include a relationship penalty, the method proceeds to step 336. If the moderation settings do include a relationship penalty, the penalty can thereafter be applied to the second user account in step 334. Additional conditions can also be evaluated beforehand to determine whether a penalty should be applied (e.g., a threshold number of reply messages with an offensive expression posted by the second user account). Thereafter, the method proceeds to step 336. At step 336, the moderation settings associated with the first user account are evaluated by the platform server to determine whether they include a display penalty, such as the penalty 266a or a penalty to disable display of the reply message. If the moderation settings do not include a display penalty, the method proceeds to step 340. If the moderation settings do include a display penalty, the penalty can thereafter be applied to the second user account in step 338. Again, additional conditions can be evaluated beforehand to determine whether a penalty should be applied. At step 340, the reply message can be transmitted to the plurality of user devices including the first user device for display thereon (e.g., in a stream of the user device) barring any penalties that prohibit the display of the reply message. The penalties can include, for example, the second user account being blocked or muted from the first user account, thus disabling display of the reply message on the first user device. In another example, the penalty can disable display of the reply message, in which case the reply message may not be transmitted to the plurality of user devices.

3. Label(s) and Guidelines for Message Threads

[0110] The moderation modules 108 and 112 disclosed herein provide features for content creators to assign one or more labels and/or guidelines to their base message. The label(s) and/or guidelines are thereafter displayed together with the base message on each content consumer’s user device. In this manner, the content creator can explicitly communicate one or more social injunctive norms to content consumers so that the content consumers are less likely to rely upon other social cues, such as the reply messages posted by other users, to determine what language and/or content is acceptable in the branch of the message thread.

[0111] In the following, further details of the moderation modules 108 and 112 are described. Specifically, the creation and assignment of one or more labels and/or guidelines to a base message are described in Section 3.1. Example methods for assigning label(s) and/or guidelines with a base message is described in Section 3.2.

3.1 Creating and Applying Label(s) and/or Guidelines to a Base Message

[0112] FIG. 7A shows an example user interface to compose a base message on a content creator’s user device. As shown, the user interface includes the space 210 to contain the base message, the submit button 212, and the cancel button 214 described above. Additionally, the user interface includes a reply message settings element 410, which when selected, displays a prompt 411 with various reply message settings. Herein, the platform 100 can provide content creators different reply message settings to control which users can post a reply message to the content creator’s base messages and/or the order of the reply messages displayed in the branch and/or message thread. The reply message settings can generally be accessed and/or modified via the prompt 411. As shown, the prompt 411 can further include a confirmation button 416 to close the prompt 411. Any changes made by the content creator, for example, to the reply message settings and the assignment of labels and/or guidelines are thereafter stored in memory (e.g., the memory 105, the memory 115, the database 117).

[0113] For example, FIG. 7A shows a reply message setting 412a to control which group of users can reply to the content creator’s content. As shown, the group of users can include, but is not limited to, all the users of the platform 100, users that are being followed by the content creator, and uses that are referenced, for example, in the content creator’s base message (e.g., by typing “@” followed by the username associated with the user account being mentioned).

[0114] FIG. 7A further shows a reply messaging setting 412b to control the order of the reply messages displayed when users view the branch and/or the message thread. As shown, in one example, the reply messages can be displayed according to a “regular” or default arrangement. This can include displaying the reply messages in a chronological or reverse chronological order. In another example, the reply messages with a Graphics Interchange Format (GIF) can be displayed first, e.g., before reply message with only textual content. If there are multiple reply messages with GIFs, these reply messages an be displayed according to chronological or reverse chronological order. In yet another example, the reply messages posted by the content creator’s friends can be displayed first. Herein, a friend is a user that has an established relationship with the content creator as indicated by a connection graph of the user and/or the content creator. Specifically, the connection graph includes an edge connecting a node representing the user’s user account and another node representing the content creator’s user account.

[0115] The user interface of FIG. 7A further includes a user interface element 414, which when selected, displays a user interface to manage label(s) and/or guidelines associated with the base message as shown in FIG. 7B. As shown, the user interface of FIG. 7B can include a toggle switch 420 to activate or deactivate assignment of label(s) and/or guidelines with the base message. The label(s) and/or guidelines are only displayed with the base message when the switch 420 is activated.

[0116] The user interface further includes a section 421 with one or more labels 422 available for selection to associate with the base message. The labels 422 are intended to convey the content creator’s desired tone(s) for the branch and/or the message thread. In some implementations, the labels 422 can be standardized such that all the users of the platform 100 select from the same set of labels. Standardizing the labels available for selection can help the users of the platform 100 better understand the desired tone(s) for the branch and/or the message thread as well as expectations on type of content to include in a reply message. However, it should be appreciated that, in some implementations, users can also generate labels for their base message with a user- defined tone.

[0117] As shown in the user interface of FIG. 7B, multiple labels 422 can be selected. In some implementations, the moderation modules 108 and 112 can limit the number of labels the content creator can select. For example, the user interface of FIG. 7B shows up to three labels can be selected. In another example, the content creator may only be allowed to select one label for their base message. More generally, the number of labels a user can select can range between one label to five labels.

[0118] It should also be appreciated the labels 422 shown in the user interface of FIG. 7B are not necessarily all the labels available for selection. Rather, in some implementations, the section 421 can display labels that have recently been used by the content creator in other base messages. As shown in FIG. 7B, the user interface can further include a user interface element 423 to display additional labels for selection.

[0119] The user interface of FIG. 7B further includes a toggle switch 424 to activate or deactivate the display of guidelines with the base message. The guidelines are only displayed and/or accessible when the toggle switch 424 is activated. Generally, the toggle switches 420 and 424 can be independently activated or deactivated. Thus, the label(s) and the guidelines can be displayed independent of one another. The user interface of FIG. 7B further includes a user interface element 426, which when selected, displays a user interface (not shown) to draft and/or edit a set of guidelines. The guidelines generally include more details on the content creator’s preferences for the content of the reply messages posted in the branch and/or message thread. For example, the guidelines can elaborate on the type of advice to provide in a reply message, rules when content consumers disagree with the content of the base message and/or the reply messages, and/or other expectations from the content creator.

[0120] Generally, the content creator can draft the guidelines themselves. In some implementations, the platform 100 can provide standard guidelines to each user account, which can thereafter be edited by the content creator as desired. In some implementations, the guidelines can be automatically generated by the processor(s) on the content creator’s user device based on the labels selected. For example, each label can be associated with a statement that describes the purpose of the label in more detail. When the label is selected, the statement can be automatically added to the guidelines.

[0121] In some implementations, the label(s) and/or guidelines can be stored as metadata of the base message. The base message, in turn, can be stored in memory (e.g., the memory 105, the memory 115, the database 117). For example, the base message can include a first field to indicate whether labels should be displayed and a second field containing the label(s) to display with the base message. In another example, the base message can include a third field to indicate whether guidelines should be displayed and a fourth field containing the guidelines to display with the base message. When the base message is transmitted to a user device for display, these metadata fields can be utilized to determine whether label(s) and/or guidelines should be displayed with the base message.

[0122] FIG. 8A shows an example user interface with a message thread displayed on a user device associated with a user different from the content creator. As shown, the message thread includes a base message 430 posted by the content creator of FIGS. 7A and 7B. In this example, the base message 430 is also a root message of the thread. The user interface further includes a user interface element 432 that displays the labels 422 selected in FIG. 7B. In some implementations, the user interface element 432 can be positioned directly between the base message 430 and the reply messages so that the labels 422 are seen by users first before viewing any reply messages. In this manner, users are more likely to be exposed to the content creator’s expected tones for the conversation, which function as injunctive social norms, before being influenced by social cues from the reply messages.

[0123] In some implementations, the user interface element 432 can further include a confirmation button (not shown) that requires content consumers to select before being able to view the reply messages and/or post their own reply messages. The confirmation button can further reinforce a content consumer’s commitment to post reply messages with content that conforms to the expected tones and/or guidelines of the message thread.

[0124] The user interface element 432 can further be interactive, such that when selected, a prompt 434 is displayed as shown as shown in FIG. 8B. As shown, the prompt 434 can include the content creator’s guidelines. Additionally, the prompt 434 can include a confirmation button 436, which when selected, closes the prompt 434.

3.2 An Example Method for Assigning Label(s) and/or Guidelines to a Base Message

[0125] FIG. 9 shows an example method 500 for assigning and displaying one or more label(s) with a base message. The method 500 can generally be executed via a combination of processor(s) of the platform server, the processor(s) of the content consumer’s user device, and/or the processor(s) of the content creator’s user device. At step 502, the message, an indication to display label(s) with the message, and the label(s) selected are received via transmission from a first user device associated with a first user account. The message, the indication, and/or the label(s) can be generated at the content creator’s user device. The indication can represent the toggle switch 420 being activated. As described above, the indication and/or the label(s) can be stored in metadata fields of the message. Thereafter, the message, the indication, and the label(s) are stored in memory on the platform (e.g., the memory 115, the database 117) in step 504. The message, the indication, and the label(s) can then be transmitted by a platform server to a plurality of user devices associated with a plurality of user accounts in step 506. The message can then be displayed together with the label(s) by each user device of the plurality of user devices in a stream on that user device in step 508. For example, the label(s) can be displayed in a user interface element, such as the user interface element 432.

[0126] FIG. 10 shows an example method 600 for assigning and displaying guidelines with a base message. The steps of method 600 can be similar to the method 500. Moreover, the method 600 can generally be executed via a combination of processor(s) of the platform server, the processor(s) of the content consumer’s user device, and/or the processor(s) of the content creator’ s user device. For example, at step 602, the message, an indication to display guidelines with the message, and the guidelines are received via transmission from a first user device associated with a first user account. Here, the indication can represent the toggle switch 424 being activated. The message, the indication, and/or the guidelines can be generated at the content creator’s user device. Additionally, the indication and/or the guidelines can be stored in metadata fields of the message. Thereafter, the message, the indication, and the guidelines are stored in memory on the platform (e.g., the memory 115, the database 117) in step 604. The message, the indication, and the guidelines can then be transmitted to a plurality of user devices associated with a plurality of user accounts in step 606. The message can then be displayed with the guidelines in a stream on each user device of the plurality of user devices in step 608. For example, the stream can include a user interface element (e.g., the user interface element 432), which when selected, displays a prompt with the guidelines (e.g., the prompt 434).

4. Additional Information

[0127] Referring back to the components shown in FIG. 1A, the following paragraphs provide further information about the platform and its clients.

4.1 User Devices and Clients

[0128] On any particular user device, the client may be a web browser and an HTML (hypertext markup language) document rendered by the web browser. The client may be or include JavaScript code or Java code. Or the client may be dedicated software, e.g., an installed app or installed application, that is designed to work specifically with the platform. The client may include, for example, a Short Messaging Service (SMS) interface, an instant messaging interface, an email - based interface, and HTML-based interface, or an API function-based interface for interacting with the platform.

[0129] A user device can include a camera, microphone, or both, and the client can include or be coupled to software to record pictures, audio, and video. The user device can include both a front- facing, i.e., a user-facing, camera, and a rear-facing camera.

4.2 Platform

[0130] The platform may have many millions of accounts, and anywhere from hundreds of thousands to millions of connections may be established or in use between clients and the platform at any given moment. The accounts may be accounts of individuals, businesses, or other entities, including, e.g., pseudonym accounts, novelty accounts, and so on.

[0131] The platform and client software are configured to enable users to draft messages and to use the platform, over data communication networks, to post messages to the platform and to receive messages posted by other users. The platform and client software are configured to enable users to post other kinds of content, e.g., image, video, or audio content, or a combination of kinds of content, either separately or combined with text messages. [0132] Optionally, the platform is configured to enable users to define immediate or scheduled sessions with individual or groups of users for audio or audio and video interactions. The platform enables users to specify participation in such sessions using the relationships defined, i.a., in the connection graphs maintained by the platform.

[0133] The platform is configured to deliver content, generally messages, to users in their home feed stream. The messages will generally include messages from accounts the user is following, meaning that the recipient account has registered to receive messages posted by the followed account. The platform generally also includes in the stream messages that the platform determines are likely to be of interest to the recipient, e.g., messages on topics of particular current interest, as represented by the number of messages on the topics posted by platform users, or messages posted on topics of apparent interest to the recipient, as represented by messages the recipient has posted or engaged with, or messages on topics the recipient has expressly identified to the platform as being of interest to the recipient, as well as selected advertisements, public service announcements, promoted content, or the like.

[0134] The platform enables users to send messages directly to one or more other users of the platform, allowing the sender and recipients to have a private exchange of messages. The platform is configured with interfaces through which a client can post messages directed to other users, both synchronously and asynchronously. Thus, users are able to exchange messages in real-time, i.e., with a minimal delay, creating what are essentially live conversations, or to respond to messages posted earlier, on the order of hours or days or even longer.

[0135] The platform also indexes content items and access data that characterizes users’ access to content. The platform provides interfaces that enable users to use their clients to search for users, content items, and other entities on the platform.

4.3 Relationships

[0136] Accounts will generally have relationships with other accounts on the platform. Relationships between accounts of the platform are represented by connection data maintained by the platform, e.g., in the form of data representing one or more connection graphs. The connection data can be maintained in a connection repository. Data repositories of the platform are generally stored in distributed replicas for high throughput and reliability. A connection graph includes nodes representing accounts of the platform and edges connecting the nodes according to the respective relationships between the entities represented by the nodes. A relationship may be any kind of association between accounts, e.g., a following, friending, subscribing, tracking, liking, tagging, or other relationship. The edges of the connection graph may be directed or undirected based on the type of relationship.

[0137] The platform can also represent relationships between accounts and entities other than accounts. For example, when an account belongs to a company, a team, a government, or other organized group, a relationship with that account can also be, for example, a relationship of being a member of the group, having a particular role in the group, or being an expert about the group. The platform can also represent abstract entities, e.g., topics, activities, or philosophies, as entities that can have relationships with accounts and, in some implementations, other entities. Such relationships can also be represented in a common connection graph or in one or more separate connection graphs, as described above.

4.4 Engagements

[0138] The platform records user engagements with messages and maintains, in a message repository, data that describes and represents at least a collection of recent messages as well as the engagements with the messages.

[0139] Engagement data relative to messages includes data representing user activity with respect to messages. Examples of engagement by a user with a message include reposting the message, marking the message to indicate it is a favorite of, liked by, or endorsed by the user, responding to the message, responding to a message with a response having a sentiment determined by the platform to be positive or negative, quoting the message with further comments, and mentioning or referencing the message.

[0140] Engagement data relative to accounts includes data representing connections between accounts. Examples of engagements by a user with respect to an account include aggregated measures of engagement with messages authored by the account. Other examples include how many followers and followees the account has, i.e., how many other accounts are following the account and how many other accounts the account is following. Other examples include measures of similarity between the groups of followers, the groups of followees, or both, of two accounts, including non-account followees. [0141] Data about engagements can be represented on the platform as graphs with connections between accounts and messages, and stored in a graph repository.

4.5 Services provided by platform servers

[0142] The servers of the platform perform a number of different services that are implemented by software installed and running on the servers. The services will be described as being performed by software modules. In some cases, particular servers may be dedicated to performing one or a few particular services and only have installed those components of the software modules needed for the particular services. Some modules will generally be installed on most or all of the non special-purpose servers of the platform. In some cases, multiple instances of a module may operate in parallel so as to complete a request for service within a short period of time, so that the platform can respond to users with low latency. The software of each module may be implemented in any convenient form, and parts of a module may be distributed across multiple computers in one or more locations so that the operations of the module are performed by multiple computers running software performing the operations in cooperation with each other. In some implementations, some of the operations of a module are performed by special-purpose hardware.

4.5.1 Front end services

[0143] In some implementations, the platform includes numerous different but functionally equivalent front end servers, which are dedicated to managing network connections with remote clients.

[0144] The front end servers provide a variety of interfaces for interacting with different types of clients. For example, when a web browser accesses the platform, a web interface module in the front end module provides the client access. Similarly, when a client calls an API made available by the platform for such a purpose, an API interface provides the client access.

[0145] The front end servers are configured to communicate with other servers of the platform, which carry out the bulk of the computational processing performed by the platform as a whole.

4.5.2 Routing services

[0146] A routing module stores newly composed messages in a message repository. The routing module also stores an identifier for each message. The identifier is used to identify a message that is to be included in a stream. This allows the message to be stored only once and accessed for a variety of different streams without needing to store more than one copy of the message.

4.5.3 Relationship graph services

[0147] A graph module manages connections between accounts, between accounts and entities, and between entities. Connections determine which streams include messages from which accounts. In some implementations, the platform uses unidirectional connections between accounts and streams to allow account holders to subscribe to the message streams of other accounts. A unidirectional connection does not imply any sort of reciprocal relationship. An account holder who establishes a unidirectional connection to receive another account’s message stream may be referred to as a “follower,” and the act of creating the unidirectional connection is referred to as “following” another account.

[0148] The graph module receives client requests to create and delete unidirectional connections between accounts and updates the connection graph or graphs accordingly. Similarly, for entities that are represented by the platform as entities with which accounts can have relationships, the graph module can also receive client requests to create and delete connections representing account-to-entity relationships.

4.5.4 Recommendation services

[0149] A recommendation module of the platform can recommend content, accounts, topics, or entities to a user The recommendation module specifically tailors recommendations to the user.

[0150] A user or a client can generate a request for a recommendation, or another module of the platform can generate a request for a recommendation on its own, e.g., in order to include a recommendation in a stream being generated for a user. A recommendation can be a call to action, i.e., a suggestion that the user take a particular action, or the recommendation can be the recommended content itself, e.g., a message to include in the stream. The recommendation module can also provide a recommendation in response to a user action that does not explicitly request a recommendation, e.g., interacting with content on the platform in a way that indicates interest.

[0151] The recommendation module makes recommendations using, for example, information users provided about themselves and other data found in the users’ profiles, and data about the users’ engagements and relationships stored in graph data and otherwise in the platform’s repositories. [0152] To make a recommendation for a user, that user’s behavior and other users’ behaviors are taken into account. Thus, the relationships and interactions between (i) a user, on the one hand, and (ii) content or users or other entities, on the other hand, are used to make personalized content recommendations for the user. In addition to being presented to the user by a client, recommendations can be provided without going through the client, e.g., in an email, text message, or push notification. Recommendations can also identify, in a personalized way, content popular near a certain geographic location, or real-time trending topics.

[0153] The platform maintains data, especially about live events, with a high degree of currency and for quick access, so that the platform can provide recommendations of current interest, especially during live events.

[0154] In some implementations, the platform presents with a recommendation user-related reasons for the recommendation, e.g., because a message relates to a topic followed by the user or to a trending topic or to a topic trending in the user’s location, or because a message had strong engagement among the user’s followees, or because the message was endorsed by other users sharing common interests or sharing common followed topics with the user. In some implementations, the platform ranks recommendations according to the reasons for the recommendations, giving preference to recommendations based on endorsements from followees, experts, or celebrities.

4.5.5 Delivery services

[0155] A delivery module constructs message streams and provides them to requesting clients, for example, through a front end server. Responding to a request for a stream, the delivery module either generates the stream in real time, or accesses from a stream repository some or all of a stream that has already been generated. The delivery module stores generated streams in the stream repository. An account holder may request any of their own streams, or the streams of any other account that they are permitted to access based on privacy and security settings. If a stream includes a large number of messages, the delivery module generally identifies a subset of the messages to send to a requesting client, in which case the remaining messages are maintained in a stream repository from which more messages are sent upon client request. 4.5.6 Health and Safety

[0156] The platform includes modules that enable users to filter the content they receive from the platform. For example, users may select settings that cause the platform to filter out sensitive content. The platform also enables a user to control how the user is visible on the platform. For example, the platform enables a user to prevent particular users from following the user, from viewing the user’s messages on the platform, from sending messages directed to the user, or from tagging the user in a photo. The platform also enables a user to mute particular users to prevent messages from particular users from being included in any incoming streams, or to block incoming push or SMS notifications from particular users. The platform enables a user to mute another user while continuing to be a follower of the other user.

[0157] In addition, the platform itself can filter out content that is identified by the platform as toxic or abusive, or that originates from accounts identified by the platform as toxic or abusive, with or without a user request to do so.

4.5.7 Account services

[0158] An account module enables account holders to manage their platform accounts. The account module allows an account holder to manage privacy and security settings, and their connections to other account holders. In particular, a user can choose to be anonymous on the platform. Data about each account is stored in an account repository.

4.5.8 Engagement services

[0159] Client software allows account holders receiving a stream to engage, e.g., interact with, comment on, or repost, the messages in the stream. An engagement module receives these engagements and stores them in an engagement repository. Types of engagement include selecting a message for more information regarding the message, selecting a URI (universal resource identifier) or hashtag in a message, reposting the message, or making a message a favorite. Other example engagement types include opening a card included in a message, which presents additional content, e.g., an image, that represents a target of a link in the message, or that links to an application installed on the user device. Account holders may engage further with the additional content, e.g., by playing a video or audio file or by voting in a poll.

[0160] In addition to recording active interactions with messages through explicitly received user input, the engagement module may also record passive interactions with messages. An impression occurs when a client presents the content of a message on a user device. Impression engagements include the mere fact that an impression occurred, as well as other information, e.g., whether a message in a stream appeared on a display of the user device, and how long the message appeared on the display.

[0161] Any engagement stored in the engagement repository may reference the messages, accounts, or streams involved in the engagement.

[0162] Engagements may also be categorized beyond their type. Example categories include engagements expressing a positive sentiment about a message (“positive engagements”), engagements expressing a negative sentiment about a message (“negative engagements”), engagements that allow an account to receive monetary compensation (“monetizable engagements”), engagements that are expected to result in additional future engagements (“performance engagements”), or engagements that are likely to result in one account holder following another account (“connection engagements”). The negative engagements category includes, for example, engagements dismissing a message or reporting a message as offensive, while the positive engagements category typically includes engagements not in the negative engagements category. Example performance engagements include selecting a URL in a message or expanding a card. Example monetizable engagements include, for example, engagements that result in an eventual purchase or a software application installation on a user device. Generally, categories and types are not coextensive, and a given type of engagement may fall into more than one category and vice versa.

5. Conclusion

[0163] While various inventive embodiments have been described and illustrated herein, those of ordinary skill in the art will readily envision a variety of other means and/or structures for performing the function and/or obtaining the results and/or one or more of the advantages described herein, and each of such variations and/or modifications is deemed to be within the scope of the inventive embodiments described herein. More generally, those skilled in the art will readily appreciate that all parameters, dimensions, materials, and configurations described herein are meant to be exemplary and that the actual parameters, dimensions, materials, and/or configurations will depend upon the specific application or applications for which the inventive teachings is/are used. Those skilled in the art will recognize or be able to ascertain, using no more than routine experimentation, many equivalents to the specific inventive embodiments described herein. It is, therefore, to be understood that the foregoing embodiments are presented by way of example only and that, within the scope of the appended claims and equivalents thereto, inventive embodiments may be practiced otherwise than as specifically described and claimed. Inventive embodiments of the present disclosure are directed to each individual feature, system, article, material, kit, and/or method described herein. In addition, any combination of two or more such features, systems, articles, materials, kits, and/or methods, if such features, systems, articles, materials, kits, and/or methods are not mutually inconsistent, is included within the inventive scope of the present disclosure.

[0164] Also, various inventive concepts may be embodied as one or more methods, of which an example has been provided. The acts performed as part of the method may be ordered in any suitable way. Accordingly, embodiments may be constructed in which acts are performed in an order different than illustrated, which may include performing some acts simultaneously, even though shown as sequential acts in illustrative embodiments.

[0165] All definitions, as defined and used herein, should be understood to control over dictionary definitions, definitions in documents incorporated by reference, and/or ordinary meanings of the defined terms.

[0166] The indefinite articles “a” and “an,” as used herein in the specification and in the claims, unless clearly indicated to the contrary, should be understood to mean “at least one.”

[0167] The phrase “and/or,” as used herein in the specification and in the claims, should be understood to mean “either or both” of the elements so conjoined, i.e., elements that are conjunctively present in some cases and disjunctively present in other cases. Multiple elements listed with “and/or” should be construed in the same fashion, i.e., “one or more” of the elements so conjoined. Other elements may optionally be present other than the elements specifically identified by the “and/or” clause, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, a reference to “A and/or B”, when used in conjunction with open-ended language such as “comprising” can refer, in one embodiment, to A only (optionally including elements other than B); in another embodiment, to B only (optionally including elements other than A); in yet another embodiment, to both A and B (optionally including other elements); etc.

[0168] As used herein in the specification and in the claims, “or” should be understood to have the same meaning as “and/or” as defined above. For example, when separating items in a list, “or” or “and/or” shall be interpreted as being inclusive, i.e., the inclusion of at least one, but also including more than one, of a number or list of elements, and, optionally, additional unlisted items. Only terms clearly indicated to the contrary, such as “only one of’ or “exactly one of,” or, when used in the claims, “consisting of,” will refer to the inclusion of exactly one element of a number or list of elements. In general, the term “or” as used herein shall only be interpreted as indicating exclusive alternatives (i.e. “one or the other but not both”) when preceded by terms of exclusivity, such as “either,” “one of,” “only one of,” or “exactly one of.” “Consisting essentially of,” when used in the claims, shall have its ordinary meaning as used in the field of patent law.

[0169] As used herein in the specification and in the claims, the phrase “at least one,” in reference to a list of one or more elements, should be understood to mean at least one element selected from any one or more of the elements in the list of elements, but not necessarily including at least one of each and every element specifically listed within the list of elements and not excluding any combinations of elements in the list of elements. This definition also allows that elements may optionally be present other than the elements specifically identified within the list of elements to which the phrase “at least one” refers, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, “at least one of A and B” (or, equivalently, “at least one of A or B,” or, equivalently “at least one of A and/or B”) can refer, in one embodiment, to at least one, optionally including more than one, A, with no B present (and optionally including elements other than B); in another embodiment, to at least one, optionally including more than one, B, with no A present (and optionally including elements other than A); in yet another embodiment, to at least one, optionally including more than one, A, and at least one, optionally including more than one, B (and optionally including other elements); etc.

[0170] This specification uses the term “configured to” in connection with systems, apparatus, and computer program components. That a system of one or more computers is configured to perform particular operations or actions means that the system has installed on it software, firmware, hardware, or a combination of them that in operation cause the system to perform those operations or actions. That one or more computer programs is configured to perform particular operations or actions means that the one or more programs include instructions that, when executed by data processing apparatus, cause the apparatus to perform those operations or actions. That special- purpose logic circuitry is configured to perform particular operations or actions means that the circuitry has electronic logic that performs those operations or actions.

[0171] In any claims, as well as in the specification above, all transitional phrases such as “com prising,” “including,” “carrying,” “having,” “containing,” “involving,” “holding,” “composed of,” and the like are to be understood to be open-ended, i.e., to mean including but not limited to. Only the transitional phrases “consisting of’ and “consisting essentially of’ shall be closed or semi- closed transitional phrases, respectively, as set forth in the United States Patent Office Manual of Patent Examining Procedures, Section 2111.03.

Claims

1. A method for facilitating moderation of a reply message submitted in response to a base message, the base message being from a first user account of a plurality of user accounts and the reply message being from a second user account of the plurality of user accounts, the method comprising: receiving, from a first user device associated with the first user account: the base message; moderation settings; and a list of offensive expressions from the first user account; transmitting, to a second user device associated with the second user account, the base message for display at the second user device; receiving, from the second user device, an indication of the second user account preparing a draft of the reply message in response to the base message; transmitting, to the second user device, the moderation settings and the list of offensive expressions; while the reply message is being drafted, determining whether the reply message includes an offensive expression from the list of offensive expressions; in response to determining the reply message includes a first offensive expression from the list of offensive expressions, determining at least one penalty for posting the reply message with the first offensive expression based on the moderation settings; and displaying, at the second user device while the reply message is being drafted, a warning notification to notify the second user account of the at least one penalty for posting the reply message with the first offensive expression.

2. The method of claim 1, further comprising: displaying, at the second user device while the reply message is being drafted, a visual indicator onto the first offensive expression in the reply message.

3. The method of claim 1, wherein at least one offensive expression of the list of offensive expressions includes a word in textual form.

4. The method of claims 1-3, wherein at least one offensive expression of the list of offensive expressions includes an emoji or an emoticon.

5. The method of claim 1, wherein at least one offensive expression of the list of offensive expressions is based on user input from the first user account.

6. The method of claim 1, wherein the list of offensive expressions includes a list of predetermined offensive expressions.

7. The method of claim 6, further comprising, before receiving, from the first user device, the list of offensive expressions: determining, via a position tracking sensor on the first user device, a location of the first user device; and selecting the list of predetermined offensive expressions based on the location of the first user device.

8. The method of claim 6, further comprising, before receiving, from the first user device, the list of offensive expressions: selecting the list of predetermined offensive expressions based on a location associated with the first user account.

9. The method of claim 6, further comprising, before receiving, from the first user device, the list of offensive expressions: selecting the list of predetermined offensive expressions based on a nationality associated with the first user account.

10. The method of claim 6, further comprising, before receiving, from the first user device, the list of offensive expressions: selecting the list of predetermined offensive expressions based on a default language associated with the first user account.

11. The method of claim 1, further comprising, before receiving the indication of the second user account preparing the draft of the reply message: displaying, on the second user device, the base message with one or more user interface elements to facilitate interaction with the base message; and generating the indication when a user interface element of the one or more user interface elements is selected to compose a reply message in response to the base message.

12. The method of claim 1, further comprising, when determining whether the reply message includes an offensive expression from the list of offensive expressions: for each offensive expression of the list of offensive expressions, comparing that offensive expression to different portions of the reply message, wherein the reply message is determined to include the at least one offensive expression when a portion of the reply message exactly matches at the least one offensive expression.

13. The method of claim 1, further comprising, when determining whether the reply message includes an offensive expression from the list of offensive expressions: for each offensive expression of the list of offensive expressions, determining an edit distance between that offensive expression and different portions of the reply message, wherein the reply message is determined to include the at least one offensive expression when the edit distance between the at least one offensive expression and a corresponding portion of the reply message is less than a predetermined threshold.

14. The method of claim 1, further comprising: receiving, from the second user device, the reply message; determining whether the reply message includes an offensive expression from the list of offensive expressions; and in response to determining the reply message includes at least one offensive expression from the list of offensive expressions, applying the at least one penalty to the second user account.

15. The method of claim 14, further comprising, when determining whether the reply message includes an offensive expression from the list of offensive expressions: for each offensive expression of the list of offensive expressions, comparing that offensive expression to different portions of the reply message, wherein the reply message is determined to include the at least one offensive expression when a portion of the reply message exactly matches at the least one offensive expression.

16. The method of claim 14, further comprising, when determining whether the reply message includes an offensive expression from the list of offensive expressions: for each offensive expression of the list of offensive expressions, determining an edit distance between that offensive expression and different portions of the reply message, wherein the reply message is determined to include the at least one offensive expression when the edit distance between the at least one offensive expression and a corresponding portion of the reply message is less than a predetermined threshold.

17. The method of claim 14, further comprising, when applying the at least one penalty: disabling display of the second user account and transmission of messages from the second user account to the first user account; and disabling display of the first user account and transmission of messages from the first user account to the second user account.

18. The method of claim 14, further comprising, when applying the at least one penalty: disabling display of the second user account and transmission of messages from the second user account to the first user account.

19. The method of claim 14, further comprising, when applying the at least one penalty: decreasing a rank parameter associated with the reply message such that the reply message has a rank parameter lower than respective rank parameters of other reply messages submitted in response to the base message, the rank parameter determining an order of reply messages for display on respective user devices of the plurality of users.

20. The method of claim 14, further comprising, when applying the at least one penalty: disabling display of the reply message on respective user devices associated with the plurality of user accounts including the first user device associated with the first user account.

21. The method as in any of claims 14-20, wherein applying the at least one penalty only occurs when the reply message causes a number of reply messages to be equal to or greater than a predetermined threshold, the number of reply messages including only reply messages having at least one offensive expression from the list of offensive expressions submitted by the second user account in response to the base message from the first user account.

22. The method of claim 21, wherein the predetermined threshold is equal to two.

23. A non-transitory computer-readable storage medium storing instructions for facilitating moderation of a reply message submitted in response to a base message, the base message being from a first user account of a plurality of user accounts and the reply message being from a second user account of the plurality of user accounts, the instructions, when executed, cause at least one processor to: receive, from a first user device associated with the first user account: the base message; moderation settings; and a list of offensive expressions from the first user account; transmit, to a second user device associated with the second user account, the base message for display at the second user device; receive, from the second user device, an indication of the second user account preparing a draft of the reply message in response to the base message; transmit, to the second user device, the moderation settings and the list of offensive expressions; while the reply message is being drafted, determine whether the reply message includes an offensive expression from the list of offensive expressions; in response to determining the reply message includes a first offensive expression from the list of offensive expressions, determine at least one penalty for posting the reply message with the first offensive expression based on the moderation settings; and display, at the second user device while the reply message is being drafted, a warning notification to notify the second user account of the at least one penalty for posting the reply message with the first offensive expression.

24. The non-transitory computer-readable storage medium of claim 23, wherein the instructions, when executed, further cause the at least one processor to: display, at the second user devices while the reply message is being drafted, a visual indicator onto the first offensive expression in the reply message.

25. The non-transitory computer-readable storage medium of claim 23, wherein at least one offensive expression of the list of offensive expressions includes a word in textual form.

26. The non-transitory computer-readable storage medium as in any of claims 23-25, wherein at least one offensive expression of the list of offensive expressions includes an emoji or an emoticon.

27. The non-transitory computer-readable storage medium of claim 23, wherein at least one offensive expression of the list of offensive expressions is based on user input from the first user account.

28. The non-transitory computer-readable storage medium of claim 23, wherein the list of offensive expressions includes a list of predetermined offensive expressions.

29. The non-transitory computer-readable storage medium of claim 28, wherein the instructions, when executed, further cause the at least one processor to, before receiving, from the first user device, the list of offensive expressions: monitor a location of the first user device using a position tracking sensor on the first user device; and select the list of predetermined offensive expressions based on the location of the first user device.

30. The non-transitory computer-readable storage medium of claim 28, wherein the instructions, when executed, further cause the at least one processor to, before receiving, from the first user device, the list of offensive expressions: select the list of predetermined offensive expressions based on a location associated with the first user account.

31. The non-transitory computer-readable storage medium of claim 28, wherein the instructions, when executed, further cause the at least one processor to, before receiving, from the first user device, the list of offensive expressions: select the list of predetermined offensive expressions based on a nationality associated with the first user account.

32. The non-transitory computer-readable storage medium of claim 28, wherein the instructions, when executed, further cause the at least one processor to, before receiving, from the first user device, the list of offensive expressions: select the list of predetermined offensive expressions based on a default language associated with the first user account.

33. The non-transitory computer-readable storage medium of claim 23, wherein the instructions, when executed, further cause the at least one processor to, before receiving the indication of the second user account preparing the draft of the reply message: display, on the second user device, the base message with one or more user interface elements to facilitate interaction with the base message; and generate the indication when a user interface element of the one or more user interface elements is selected to compose a reply message in response to the base message.

34. The non-transitory computer-readable storage medium of claim 23, wherein the instructions, when executed, further cause the at least one processor to, when determining whether the reply message includes an offensive expression from the list of offensive expressions: for each offensive expression of the list of offensive expressions, compare that offensive expression to different portions of the reply message, wherein the reply message is determined to include the at least one offensive expression when a portion of the reply message exactly matches at the least one offensive expression.

35. The non-transitory computer-readable storage medium of claim 23, wherein the instructions, when executed, further cause the at least one processor to, when determining whether the reply message includes an offensive expression from the list of offensive expressions: for each offensive expression of the list of offensive expressions, determine an edit distance between that offensive expression and different portions of the reply message, wherein the reply message is determined to include the at least one offensive expression when the edit distance between the at least one offensive expression and a corresponding portion of the reply message is less than a predetermined threshold.

36. The non-transitory computer-readable storage medium of claim 23, wherein the instructions, when executed, further cause the at least one processor to: receive, from the second user device, the reply message; determine whether the reply message includes an offensive expression from the list of offensive expressions; and in response to determining the reply message includes at least one offensive expression from the list of offensive expressions, apply the at least one penalty to the second user account.

37. The non-transitory computer-readable storage medium of claim 36, wherein the instructions, when executed, further cause the at least one processor to, when determining whether the reply message includes an offensive expression from the list of offensive expressions: for each offensive expression of the list of offensive expressions, compare that offensive expression to different portions of the reply message, wherein the reply message is determined to include the at least one offensive expression when a portion of the reply message exactly matches at the least one offensive expression.

38. The non-transitory computer-readable storage medium of claim 36, wherein the instructions, when executed, further cause the at least one processor to, when determining whether the reply message includes an offensive expression from the list of offensive expressions: for each offensive expression of the list of offensive expressions, determine an edit distance between that offensive expression and different portions of the reply message, wherein the reply message is determined to include the at least one offensive expression when the edit distance between the at least one offensive expression and a corresponding portion of the reply message is less than a predetermined threshold.

39. The non-transitory computer-readable storage medium of claim 36, wherein the instructions, when executed, further cause the at least one processor to, when applying the at least one penalty: disable display of the second user account and transmission of messages from the second user account to the first user account; and disable display of the first user account and transmission of messages from the first user account to the second user account.

40. The non-transitory computer-readable storage medium of claim 36, wherein the instructions, when executed, further cause the at least one processor to, when applying the at least one penalty: disable display of the second user account and transmission of messages from the second user account to the first user account.

41. The non-transitory computer-readable storage medium of claim 36, wherein the instructions, when executed, further cause the at least one processor to, when applying the at least one penalty: decrease a rank parameter associated with the reply message such that the reply message has a rank parameter lower than respective rank parameters of other reply messages submitted in response to the base message, the rank parameter determining an order of reply messages for display on respective user devices of the plurality of users.

42. The non-transitory computer-readable storage medium of claim 36, wherein the instructions, when executed, further cause the at least one processor to, when applying the at least one penalty: disable display of the reply message on respective user devices associated with the plurality of user accounts including the first user device associated with the first user account.

43. The non-transitory computer-readable storage medium as in any of claims 36-42, wherein the instructions, when executed, further cause the at least one processor to: apply the at least one penalty to the second user account only when the reply message causes a number of reply messages to be equal to or greater than a predetermined threshold, the number of reply messages including only reply messages having at least one offensive expression from the list of offensive expressions submitted by the second user account in response to the base message from the first user account.

44. The non-transitory computer-readable storage medium of claim 43, wherein the predetermined threshold is equal to two.

45. A social messaging platform (100), associated with a plurality of user accounts including a first user account and a second user account, to facilitate moderation of a reply message from the second user account submitted in response to a base message from the first user account, the social messaging platform comprising: a platform server (110) comprising at least one first processor (111) and memory (115); a first user device (104) associated with the first user account; a second user device (104) associated with the second user account having at least one second processor (101); and a computer-readable storage medium, stored in the memory, storing instructions that, when executed, cause the at least one first processor to: receive, from the first user device: the base message; moderation settings; and a list of offensive expressions from the first user account; transmit, to the second user device, the base message for display at the second user device; receive, from the second user device, an indication of the second user account preparing a draft of the reply message in response to the base message; and transmit, to the second user device, the moderation settings and the list of offensive expressions, the instructions, when executed, further cause the at least second processor to: while the reply message is being drafted, analyze the reply message to determine whether the reply message includes an offensive expression from the list of offensive expressions; in response to determining the reply message includes a first offensive expression from the list of offensive expressions, determine at least one penalty for posting the reply message with the first offensive expression based on the moderation settings; and display, at the second user device while the reply message is being drafted, a warning notification to notify the second user account of the at least one penalty for posting the reply message with the first offensive expression.

46. The platform of claim 45, wherein the instructions, when executed, further cause the at least one second processor to: display, at the second user device while the reply message is being drafted, a visual indicator onto the first offensive expression in the reply message.

47. The platform of claim 45, wherein at least one offensive expression of the list of offensive expressions includes a word in textual form.

48. The platform as in any of claims 45-47, wherein at least one offensive expression of the list of offensive expressions includes an emoji or an emoticon.

49. The platform of claim 45, wherein at least one offensive expression of the list of offensive expressions is based on user input from the first user account.

50. The platform of claim 45, wherein the list of offensive expressions includes a list of predetermined offensive expressions.

51. The platform of claim 50, wherein: the first user device includes a position tracking sensor to monitor a location of the first user device; and the list of predetermined offensive expressions is selected based on the location of the first user device.

52. The platform of claim 50, wherein the list of predetermined offensive expressions is selected based on a location associated with the first user account.

53. The platform of claim 50, wherein the list of predetermined offensive expressions is selected based on a nationality associated with the first user account.

54. The platform of claim 50, wherein the list of predetermined offensive expressions is selected based on a default language associated with the first user account.

55. The platform of claim 45, wherein the instructions, when executed, further cause at least one third processor of the first user device to, before receiving the indication of the second user account preparing the draft of the reply message: display, on the second user device, the base message with one or more user interface elements to facilitate interaction with the base message; and generate the indication when a user interface element of the one or more user interface elements is selected to compose a reply message in response to the base message.

56. The platform of claim 45, wherein the instructions, when executed, further cause the at least one second processor to, when determining whether the reply message includes an offensive expression from the list of offensive expressions: for each offensive expression of the list of offensive expressions, compare that offensive expression to different portions of the reply message, wherein the reply message is determined to include the at least one offensive expression when a portion of the reply message exactly matches at the least one offensive expression.

57. The platform of claim 45, wherein the instructions, when executed, further cause the at least one second processor to, when determining whether the reply message includes an offensive expression from the list of offensive expressions: for each offensive expression of the list of offensive expressions, determine an edit distance between that offensive expression and different portions of the reply message, wherein the reply message is determined to include the at least one offensive expression when the edit distance between the at least one offensive expression and a corresponding portion of the reply message is less than a predetermined threshold.

58. The platform of claim 45, wherein the instructions, when executed, further cause the at least one first processor to: receive, from the second user device, the reply message; determine whether the reply message includes an offensive expression from the list of offensive expressions; and in response to determining the reply message includes at least one offensive expression from the list of offensive expressions, apply the at least one penalty to the second user account.

59. The platform of claim 58, wherein the instructions, when executed, further cause the at least one first processor to, when determining whether the reply message includes an offensive expression from the list of offensive expressions: for each offensive expression of the list of offensive expressions, compare that offensive expression to different portions of the reply message, wherein the reply message is determined to include the at least one offensive expression when a portion of the reply message exactly matches at the least one offensive expression.

60. The platform of claim 58, wherein the instructions, when executed, further cause the at least one first processor to, when determining whether the reply message includes an offensive expression from the list of offensive expressions: for each offensive expression of the list of offensive expressions, determine an edit distance between that offensive expression and different portions of the reply message, wherein the reply message is determined to include the at least one offensive expression when the edit distance between the at least one offensive expression and a corresponding portion of the reply message is less than a predetermined threshold.

61. The platform of claim 58, wherein the instructions, when executed, further cause the at least one first processor to, when applying the at least one penalty: disable display of the second user account and transmission of messages from the second user account to the first user account; and disable display of the first user account and transmission of messages from the first user account to the second user account.

62. The platform of claim 58, wherein the instructions, when executed, further cause the at least one first processor to, when applying the at least one penalty: disable display of the second user account and transmission of messages from the second user account to the first user account.

63. The platform of claim 58, wherein the instructions, when executed, further cause the at least one first processor to, when applying the at least one penalty: decrease a rank parameter associated with the reply message such that the reply message has a rank parameter lower than respective rank parameters of other reply messages submitted in response to the base message, the rank parameter determining an order of reply messages for display on respective user devices of the plurality of users.

64. The platform of claim 58, wherein the instructions, when executed, further cause the at least one first processor to, when applying the at least one penalty: disable display of the reply message on respective user devices associated with the plurality of user accounts including the first user device associated with the first user account.

65. The platform as in any of claims 58-64, wherein the instructions, when executed, further cause the at least one first processor to: apply the at least one penalty to the second user account only when the reply message causes a number of reply messages to be equal to or greater than a predetermined threshold, the number of reply messages including only reply messages having at least one offensive expression from the list of offensive expressions submitted by the second user account in response to the base message from the first user account.

66. The platform of claim 65, wherein the predetermined threshold is equal to two.