WO2018035703A1

WO2018035703A1 - A hybrid approach to advanced quality of service (qos)

Info

Publication number: WO2018035703A1
Application number: PCT/CN2016/096338
Authority: WO
Inventors: Benjamin MENCHACA; Sakthi Vignesh RADHAKRISHNAN; Xuyang Zhou; Harsimran GREWAL; Sol Kavy
Original assignee: Qualcomm Inc
Current assignee: Qualcomm Inc
Priority date: 2016-08-23
Filing date: 2016-08-23
Publication date: 2018-03-01
Anticipated expiration: 2019-02-23

Abstract

This disclosure provides systems, methods, and devices for quality of service (QoS) management in networks. A network forwarding device may queue packets in a quality of service (QoS) tree, wherein the QoS tree has a root node implemented by a first network processing layer, a first set of nodes implemented by at least the first network processing layer, and a second set of nodes implemented by a second network processing layer. A portion of the packets may be forwarded from the second processing layer to the first processing layer. At least a portion of the first set of nodes may be bypassed at the first processing layer for the portion of the packets forwarded to the first processing layer from the second processing layer.

Description

A HYBRID APPROACH TO ADVANCED QUALITY OF SERVICE (QOS)

TECHNICAL FIELD

This disclosure generally relates to network communication, and more specifically to a hybrid approach to advanced Quality of Service (QoS) .

DESCRIPTION OF THE RELATED TECHNOLOGY

In networks such as home or enterprise networks, many data packets are processed. A certain level of Quality of Service (QoS) may be necessary to achieve overall network throughput goals. Typically, a hierarchy of QoS algorithms is constructed on an egress interface to provide a desired QoS. This hierarchy is called a QoS tree and QoS algorithms are usually constructed into trees. All QoS trees have a set of leaf nodes or “queues” at the bottom which buffer packets. These packets then move up the tree to intermediate nodes which run various algorithms. The packets are then provided to a “root” node.

In general, QoS trees execute after a forwarding engine of a router and just before an interface driver. A QoS tree is implemented by one or more network processing layers. As used herein, a network processing layer refers to a scalable set of modular logic that performs routing/forwarding and QoS functionality, and may be implemented as either hardware or software. A network processing layer in this context is defined by scalability and redundancy, and is not to be confused with a protocol layer of a network stack (although one or more redundant network processing layers implement the network layer of a protocol stack for a router) .

If there is only one network processing layer, all traffic to this particular interface will always flow through a single tree. An example of this is an Advanced RISC Machines (ARM) processor running Linux. In another example where two network processing layers of data plane are available, a QoS tree for egress interface of a router may be implemented and executes in both layers. This results in distributed accounting causing QoS algorithms, such as basic rate limiting, to fail.

Traditional solutions implement the QoS tree in just one layer (either hardware or software) , and route traffic from other layers through it. This ensures a single place for accounting, implying all QoS algorithms will work as intended, but constrains the system to the capabilities of a single layer. For example, the layer executing the QoS tree might be a high performance hardware data plane, with limitations in the number of queues or complexity of the algorithms it supports. Trying to improve hardware to support larger number of queues or complex algorithms may result in a very expensive piece of hardware that no longer fits into a targeted market segment. On the other hand, if the QoS tree is executed on a highly flexible software data plane, then overall performance of the system may be reduced. The reason that overall performance may be reduced is because every packet going out through this interface (irrespective of which layer processed it) will have to touch software for QoS before it can go out.

SUMMARY

The systems, methods and devices of this disclosure each have several innovative aspects, no single one of which is solely responsible for the desirable attributes disclosed herein.

One innovative aspect of the subject matter described in this disclosure can be implemented in a method of network communication. In some implementations, the method can include queuing packets in a quality of service (QoS) tree at a network device, wherein the QoS tree has a root node implemented by a first network processing layer, a first set of nodes implemented by at least the first network processing layer, and a second set of nodes implemented by a second network processing layer, forwarding a portion of the packets from the second processing layer to the first processing layer, and bypassing at least a portion of the first set of nodes at the first processing layer for the portion of the packets forwarded to the first processing layer from the second processing layer.

Another innovative aspect of the subject matter described in this disclosure can be implemented in an apparatus for network communication is described. The apparatus can include means for queuing packets in a QoS tree at a network device, wherein the QoS tree has a root node implemented by a first network processing layer, a first set of nodes implemented by at least the first network processing layer, and a second set of nodes implemented by a second network processing layer, means for forwarding a portion of the packets from the second processing layer to the first processing layer, and means for bypassing at least a portion of the first set of nodes at the first processing layer for the portion of the packets forwarded to the first processing layer from the second processing layer.

Another innovative aspect of the subject matter described in this disclosure can be implemented in an apparatus for network communication is described. The apparatus can include a processor, memory in electronic communication with the processor, and instructions stored in the memory. The instructions can be operable to cause the processor to queue packets in a QoS tree at a network device, wherein the QoS tree has a root node implemented by a first network processing layer, a first set of nodes implemented by at least the first network processing layer, and a second set of nodes implemented by a second network processing layer, forward a portion of the packets from the second processing layer to the first processing layer, and bypass at least a portion of the first set of nodes at the first processing layer for the portion of the packets forwarded to the first processing layer from the second processing layer.

Some examples of the method and apparatuses described above can further include processes, features, or means for forwarding a portion of packets from the first processing layer to the second processing layer. Some examples of the method and apparatuses described above can further include processes, features, or means for sending classification information for QoS handling in the second processing layer from the first processing layer to the second processing layer.

In additional examples of the method and apparatuses described above, forwarding the portion of the packets from the second processing layer to the first processing layer comprises: marking the portion of the packets for processing by the first processing layer, wherein the bypassing may be based at least in part on the marking of the packet. Some examples of the method and apparatuses described above can further include processes, features, or means for forwarding classification information for QoS handling to the first processing layer.

Further examples of the method and apparatuses described above can further include processes, features, or means for transferring a packet from the first network processing layer to the second network processing layer prior to forwarding the packet from the second processing layer to the first processing layer. Some examples of the method and apparatuses described above can further include processes, features, or means for providing dedicated direct memory access (DMA) to a main memory of the networking device for each of the network processing layers.

In some examples of the method and apparatuses described above, the first network processing layer can be implemented using special-purpose hardware and the second network processing layer may be implemented by a processor executing special-purpose software. In some examples of the method and apparatuses described above, the first set of nodes can perform rate limiting QoS functions. In some examples of the method and apparatuses described above, the second set of nodes performs at least non-rate limiting QoS function.

In some examples of the method and apparatuses described above, the first and second sets of nodes may be disjoint and the first set of nodes may be smaller than the second set of nodes. Some examples of the method and apparatuses described above can further include processes, features, or means for processing the packets such that all packets exit the QoS tree at the root node. In some examples of the method and apparatuses described above, the first and second network processing layers may be implemented in a single system-on-a-chip (SOC) .

Details of one or more implementations of the subject matter described in this disclosure are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, the drawings and the claims. Note that the relative dimensions of the following figures may not be drawn to scale.

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 illustrates an example of a system for network communication that supports a hybrid approach to advanced QoS in accordance with aspects of the present disclosure.

Figure 2 illustrates an example of a QoS node tree that supports the hybrid approach to advanced QoS in accordance with aspects of the present disclosure.

Figure 3 illustrates a block diagram of network processing layers that support the hybrid approach to advanced QoS in accordance with aspects of the present disclosure.

Figure 4 illustrates a conceptual diagram of packet flow in a hardware layer that supports the hybrid approach to advanced QoS in accordance with aspects of the present disclosure.

Figure 5 illustrates a conceptual diagram of packet flow reinjected into a hardware layer from a software layer in accordance with aspects of the present disclosure.

Figures 6–8 show block diagrams of a network forwarding device that supports a hybrid approach to advanced QoS in accordance with aspects of the present disclosure.

Figure 9 illustrates a block diagram of a system including a network forwarding device that supports a hybrid approach to advanced QoS in accordance with aspects of the present disclosure.

Figures 10–11 illustrate a method for a hybrid approach to advanced QoS in accordance with aspects of the present disclosure.

DETAILED DESCRIPTION

Systems, methods, and devices are provided herein for improved QoS processing of data packets in wireless communications. A QoS tree may be implemented in different network processing layers of a network forwarding device. A network processing layer (referred to herein as “layer” ) refers to one of several layered processing units which might operate at different speeds. A layer points to separate packet processing engines each implemented with some combination of software and hardware, or pure hardware. Packets may be exchanged among the layers according to the QoS tree. In one example, packets sent from one layer to another layer may be tagged with classification information that informs how the packets are to be processed. In some examples, if a packet re-enters a layer, it may bypass a number of processing steps or nodes.

Some solutions result in the over-or under-utilization of bandwidth when flows are unevenly distributed between the different layers. An example of an interface limited to 100 Megabits per second (Mbps) is discussed here in the context of two network processing layers. In this example, two methods may be implemented that do not result in the full use of the available bandwidth. First, the rate limiter in the QoS trees could be configured to 100 Mbps each. However, this solution results in an incorrect aggregate of 200 Mbps flowing out of the interface. In a second solution, the rate limiter in the QoS trees could be configured to 50 Mbps each, but this results in underutilization of bandwidth when the flows of data packets are not evenly distributed among these layers. Thus, a distributed implementation may not be very efficient, even with the most fundamental algorithms in QoS, let alone more complex ones.

In one example implementation, the QoS branch in a first layer 1 may connect up to the appropriate point in the QoS tree implemented in a second layer 2. Such a distribution uses the best of both layers without compromising on the integrity of the QoS tree. For example, layer 1 may be a high performance hardware data plane and layer 2 may be a flexible software data plane. In such a setup, a small number of queues and nodes can be implemented that execute basic rate limiting in the hardware data plane (layer 1) , and move hundreds or more other queues and complex algorithms into software (layer 2) . A single QoS tree is maintained across the system. One problem with having to rate limit the interface to 100 Mbps can be solved by setting the rate of the root node to 100 Mbps. The system may maintain a very high throughput for hardware offloaded flows going through that portion of the tree, while selectively providing flexibility for the ones going through another branch.

Code can be shared between the layers. For example, code can be shared between hardware and software layers to mark packets that are sent to software for QoS purposes. When packets processed in the software layer are reinjected into the hardware, the packets may skip select stages in the hardware packet processing pipeline. Dedicated direct memory access (DMA) rings can support a given number of QoS branches offloaded into software. This may reduce head-of-line blocking between different QoS branches. Hardware flow control can be applied on a subset of DMA rings. This may reduce drops in specific DMA channels that are processing QoS packets (post dequeuer from software) . The software layer can direct packets into the correct portion of the QoS tree in hardware (post dequeuer from software) .

Techniques described herein accelerate packet processing through a HW/SW hybrid approach in regards to QoS. The techniques allow the HW to indicate to the SW how to process certain packets and for the HW pipeline to be modified even when items are in queue for processing. The techniques also allow for steps in the SW processing to be skipped (for example, where HW is more efficient) . The techniques and structure also support dedicated DMA rings so that a QoS branch which is heavily used does not prevent operations of another QoS branch.

Aspects of the disclosure are initially described in the context of a wireless communications system. An example QoS tree is shown and aspects of the disclosure are discussed in terms of the QoS tree implemented in a specific layer example. Processing and forwarding of packets through the layers is shown in the context of conceptual diagrams and a swim chart. Aspects of the disclosure are further illustrated by and described with reference to apparatus diagrams, system diagrams, and flowcharts that relate to a hybrid approach to advanced QoS.

Particular implementations of the subject matter described in this disclosure can be implemented to realize one or more of the following potential advantages. The QoS tree distribution techniques described herein may make good use of all layers without compromising the integrity of the QoS tree. A single QoS tree may be used for a complicated system. Processing of data packets throughout the system may become more efficient and faster. Example architecture of the system may allow the system to maintain very high throughput for hardware offloaded flows while selectively providing flexibility for packets going through the software layers. These techniques may be used regardless of the types of layers in the system or what types of constraints the layers have.

Figure 1 illustrates a network 100 configured in accordance with various aspects of the present disclosure. The network 100 may include a network forwarding device 105 and multiple network devices 115 in communication with the network forwarding device. The network forwarding device 105 may be, for example, a router. The network devices 115 may represent wired or wireless devices such as smartphones, tablets, personal computers, servers, workstations, internet of things (IoT) devices or appliances, display devices (such as TVs, computer monitors, etc. ) , printers, or other types of network devices. In this example, the network devices 115 may communicate with the network forwarding device 105 through wired connections 125 (such as Ethernet connections) or wireless connections 120 (such as WiFi connections) . The various network devices 115 in the network 100 are able to communicate with one another through the network forwarding device 105. The network forwarding device 105 may also be a component of or communicatively coupled with an external network 130 (such as the Internet, an intranet, or other packet-switched network) , and network devices 115 may communicate with other devices (not shown) over the external network 130 by way of network forwarding device 105.

The network forwarding device 105 may implement a Quality of Service (QoS) scheme to prioritize certain types of traffic over other types of traffic. QoS is a descriptor of the level of performance provided in trans-mission of data. Typical QoS metrics include bit rate, delay, bit/block error rate, maximum blocking probability and outage probability, among others. QoS guarantees may be important for real-time streaming applications, such as Voice over Internet Protocol (VoIP) , since these applications are often delay-sensitive. QoS guarantees are often implemented using a number of logical nodes arranged in a tree hierarchy. In the past, QoS trees have been typically implemented either completely in hardware or completely in software. The network forwarding device 105 of Figure 1 may include a QoS manager 140 to use a hybrid approach (such as a combined hardware and software approach) to implement a QoS tree. Techniques for implementing QoS in this way are described below.

Figure 2 illustrates an example of a QoS node tree 200 for a hybrid approach to advanced QoS. Figure 2 shows an example of a simple tree. The QoS node tree 200 includes a root node 205 (Node) , two intermediate nodes 210 (Node0 and Node1) , and four leaf nodes 215 (Node2, Node3, Node4, and Node5) . As used herein, leaf nodes may also be referred to as queues. All QoS trees have a set of leaf nodes or “queues” at the bottom which buffer packets. These packets then move up the tree to intermediate nodes which run various algorithms. The packets are then provided to a “root” node.

The nodes are grouped into two groups. A first group 220 may be implemented in a first network processing layer and a second group 225 may be implemented in a second network processing layer. Packets from one group may transfer to the other group. Packets may re-enter the original group and bypass stages associated with the network processing layer of that group. In an example, the first group 220 may be a set of algorithms and queues that can be supported by the first layer and the second group 225 may be more efficiently implemented in the second layer because the first layer cannot support those algorithms and queues.

Figure 3 illustrates a block diagram 300 of network processing layers 305 and 310 that support the hybrid approach to advanced QoS in accordance with aspects of the present disclosure. The advanced QoS discussed in Figure 3 may be implemented by a network forwarding device 105 as described with reference to Figure 1. The QoS manager 140 of Figure 1 may implement the structure described in Figure 3.

The first network processing layer 305 may include a first interface driver 315, a first forwarding engine 320, a first portion of a QoS tree 325, and a second interface driver 330. Packets may enter the first interface driver 315 and be sent either to the first forwarding engine 320 or to the second forwarding engine 340. The second network processing layer 310 may include a second forwarding engine 340 and a second portion of the QoS tree 345. In some examples, the first portion of the QoS tree 325 implements the nodes in the first group 220 and the second portion of the QoS tree 345 implements the nodes in the second group 225 of Figure 2. In other example, other tree structures and groupings may be used.

In one example, the first group 220 may be in the first network processing layer 305, which may be a high performance hardware data plane. The second group 225 may be associated with the second network processing layer 310, which may be a flexible software data plane. In such a setup, a small number of queues and nodes that execute basic rate limiting can be implemented in the portion of the QoS tree 325 of the hardware data plane of the first network processing layer 305. Other queues and complex algorithms may be moved into the portion of the QoS tree 345 of the software data plane of the second network processing layer 310. All this can be done while still maintaining a single QoS tree across the system for this interface.

Using this example distribution, having to rate limit the interface to 100 Mbps may be achieved by setting the rate of the root node to 100 Mbps in the first group 220. This architecture also allows the system to maintain very high throughput for hardware offloaded flows going through the first group 220 part of the QoS node tree 200, while selectively providing flexibility for packets going through the second group 225. Codes may be shared between the network processing layers 305 and 310 to mark packets for processing, such as classification information. The QoS tree 345 may re-inject packets into the hardware layer (such as the first network processing layer 305) without having to perform certain stages in the packet processing pipeline of the hardware layer. A packet forwarding component may use classification information associated with the packets to determine how to process the packets upon re-injection into the hardware layer.

Rate limiting is used herein as a simple example for ease of understanding. However, these techniques may extend to a whole variety of shaping and scheduling algorithms (such as min/max shaping, weighted round robin, strict priority, etc. ) .

In some examples, the network processing layers 305 and 310 may be implemented onto a single system-on-a-chip (SoC) . Such an implementation may provide good system level integration. A SoC may have multiple layers of data plane that can process and forward packets. For example, a SoC may have three such layers. The SoC may implement, for example, a specialized piece of software that is written for high speed forwarding of packets. Another layer may be special purpose hardware designed for packet forwarding which does not involve execution of any kind of software. All of the data plane layers may have the ability to perform egress QoS on packets.

Note that two example layers implementing the hybrid QoS techniques described herein do not have to be a hardware and a software layer. The two layers could be both software layers, or hardware layers as well. The techniques described herein are generic and can be applied as long as the two or more layers implement the necessary features required for hybrid QoS. The layers may be one of several “layered” processing units which might operate at different speeds (for example, Linux running on an ARM processor, software network acceleration running on an embedded processor, and a hardware packet processing engine) .

Furthermore, any number of layers may be used. The layering can be nested to more than two layers, with all layers collectively representing a single hierarchy. In some examples with N layers, wherein N is a positive integer, all layers may agree on a particular hierarchy among the layers. In some implementations, the hierarchy may be programmed into the layers. Each layer may be able to process its own portion of the QoS tree. Each layer may also be capable of receiving packets to be scheduled by that layer in addition to transmitting or forwarding packets that have been scheduled to the lowest layer. The lowest layer may receive post-scheduled packets and transmit without looping back through the scheduling. All layers may be able to move packets to the next layer as either “primary” (not yet scheduled) or secondary (already scheduled) and out.

Figure 4 illustrates a conceptual diagram 400 of packet flow in a hardware (HW) layer 405 that supports the hybrid approach to advanced QoS in accordance with aspects of the present disclosure. The hardware layer 405 may be an example of aspects of the first network processing layer 305 as described with reference to Figure 3. The advanced QoS discussed in Figure 4 may be implemented in a network forwarding device 105 as described with reference to Figure 1.

In one example, the hardware layer 405 has a three level QoS hierarchy. In this example, level 0 is used to round robin between the different hardware ports. Level 1 and level 2 help with the construction of the QoS hierarchies for each of the individual ports. These two levels have a scheduling nodes such as strict priority (SP) and deficit round robin (DRR) . The scheduling nodes in level 1 and level 2 can be attached to one another. The scheduling nodes also may attach to hardware queues provided in the hardware layer 405 to form a complete QoS tree. In one example, there are a total of 256 hardware queues. In other examples, other numbers of hardware queues may be used.

One or more data packets may enter the flow table 420. For example, this may be a 5 tuple based flow entry lookup. A match in the flow table 420 may provide a packet with a service code and QoS tag values. This is then indexed by service code by the service code to queue mapping table 425. Next, the packet may be processed by an enhanced DMA (EDMA) queue 430 which provides service code to queue mapping using a table, for example.

Sometimes flows offloaded in the hardware layer 405 require advanced QoS processing that are beyond capability of the hardware layer 405 to perform egress QoS. As an example, flows that need to share bandwidth with other flows using algorithms such as hierarchical token bucket (HTB) will have to be handed over to a different layer, such as a software layer, for queueing because the hardware layer 405 does not support such algorithms.

In some examples, to redirect packets to CPUs, the hardware layer 405 provides service codes. Each flow entry in the hardware layer 405 can be programmed with a service code which can then be used to redirect packets to a specific queue. In case a flow requires advanced QoS processing, the flow entry will be programmed with a service code that redirects the packet to the EDMA queue 430 which provides the packet to an EDMA engine, which may be a hardware engine that is responsible for moving packets between the hardware layer 405 and CPUs of a software (SW) layer 410. Once the EDMA engine receives the packet, the EDMA engine copies the packet to DDR memory and presents software running on the CPUs with a descriptor and a pre-header containing metadata information for processing the data packet.

Tables 1 and 2 provide some examples of relevant fields from the Rx descriptor and Rx pre-header.

TABLE 1: QoS related fields in Rx-descriptor

TABLE 2: QoS related fields in Rx pre-header

In Table 1, the field QDISC (Rx descriptor) may specify if a pre-header contains a timestamp or QoS information. For QoS this will be set to type QoS.

In Table 2, the field DST_INFO (Rx pre-header) contains the actual destination where this packet will egress after QoS processing is over. The field SERVICE_CODE (Rx pre-header) may contain the service code information useful for cross verifying if the packets are being received for QoS processing. The field TREE+QoS TAG (Rx pre-header) may contain the QoS Tag information that software will use to identify the leaf node to which the packet needs to be enqueued.

Figure 5 illustrates a conceptual diagram 500 of packet flow reinjected into a hardware layer 405-a from a software layer 410-a in accordance with aspects of the present disclosure. The hardware layer 405-a may be an example of aspects of the first network processing layer 305 as described with reference to Figure 3 and the hardware layer 405 as described with respect to Figure 4. The advanced QoS discussed in Figure 5 may be implemented in a network forwarding device 105 as described with reference to Figure 1.

In some examples, the hardware layer 405-a may be a Packet Processing Engine (PPE) . As described herein, the PPE is a hardware implementation consisting of input and output ports with programmable hardware logic that allows one to process, modify, and perform QoS on network packets. The PPE may be a hardware layer in a SoC. The PPE exposes certain configuration registers and tables that SW can configure to make it operate in a specific fashion. The PPE may be capable of processing (but not limited to) basic IPv4/IPv6, TCP/UDP flows, and supports QoS algorithms such as, for example, Strict Priority, Deficit Round Robin, Rate limiting, Random Early Drop, and First in First Out.

Continuing the example of Figure 4, when the software layer 410-a reads the descriptors associated with a packet that includes QoS information, the software layer 410-a uses the service code to verify that the packet has been sent for advanced QoS processing. It may then use the QoS tag information to identify a leaf node to which the packet needs to get enqueued. This tag may match with one of the leaf nodes that resides in the software portion of the tree. For example, assuming the second group 225 of Figure 2 is running in a software layer, the QoS tag received from the hardware layer of the first group 220 will match either Node4 or Node5. Using this information, the software layer 410-a will be able to enqueue the packet to the appropriate leaf node. Once this is done, the QoS dequeue process in the software layer 410-a may dequeue this packet and send it back to the hardware layer 405-a.

For example, the software layer 410-a may modify two parameters in an EDMA Tx-preheader. The parameter SERVICE_CODE may be modified. That is, the software layer 410-a may provide a new service code value through the Tx pre-header which helps select the appropriate bypass bitmap to skip hardware stages in the hardware layer 405-a. In Figure 5, tables IN_SERVICE_TBL 505, IN_L2_SERVICE_TBL 510, and IPO_ACTION_TBL 515 are index by the service code value sent by the software layer. Each of these look-up tables provides a bypass bitmap (programmed at boot time by software) that helps skip particular stages/tasks in the hardware layer 405-a.

The software layer 410-a also may modify the INT_PRI parameter. This parameter is an internal priority value that may be modified such that a DST_INFO +INT_PRI based queue_id look-up may result in the packet getting enqueued to the correct QoS node in the hardware layer 405-a. This is the look-up happening in IPO action table 515 that sends packets over to a Queue ID table 520 and then to one of the queues mapped to a port (port 2 in the example of Figure 5) . The Queue ID table 520 provides DST-INFO +INT_PRI to QUEUE_ID mapping.

Tables 3-5 provide examples stages that can be bypassed by each of the tables 505, 510, and 515. In other examples, other stages may be bypassed.

TABLE 3: Stages bypassable by Table 505 of Figure 5

TABLE 4: Stages bypassable by Table 510 of Figure 5

TABLE 5: Stages bypassable by Table 515 of Figure 5

Figure 6 shows a block diagram 600 of a network forwarding device 605 that supports a hybrid approach to advanced QoS in accordance with various aspects of the present disclosure. Network forwarding device 605 may be an example of aspects of the network forwarding device 105 as described with reference to Figure 1. Network forwarding device 605 may include receiver 610, QoS manager 615, and transmitter 620. Network forwarding device 605 also may include a processor. Each of these components may be in communication with one another (such as via one or more buses) .

Receiver 610 may receive information such as packets, user data, or control information associated with various information channels (such as control channels, data channels, and information related to a hybrid approach to advanced QoS, etc. ) . Information may be passed on to other components of the device. The receiver 610 may be an example of aspects of the transceiver 935 described with reference to Figure 9.

QoS manager 140-a may be an example of aspects of the QoS manager 140 described with reference to Figure 1. QoS manager 615 may queue packets in a QoS tree at a network forwarding device, where the QoS tree has a root node implemented by a first network processing layer, a first set of nodes implemented by at least the first network processing layer, and a second set of nodes implemented by a second network processing layer, forward a portion of the packets from the second processing layer to the first processing layer, and bypass at least a portion of the first set of nodes at the first processing layer for the portion of the packets forwarded to the first processing layer from the second processing layer.

Transmitter 620 may transmit signals generated by other components of the device. In some examples, the transmitter 620 may be collocated with a receiver 610 in a transceiver module. For example, the transmitter 620 may be an example of aspects of the transceiver 935 described with reference to Figure 9. The transmitter 620 may include a single antenna, or it may include a set of antennas.

Figure 7 shows a block diagram 700 of a network forwarding device 705 that supports a hybrid approach to advanced QoS in accordance with various aspects of the present disclosure. Network forwarding device 705 may be an example of aspects of a

network forwarding device

105, 605 as described with reference to Figures 1 and 6. Network forwarding device 705 may include receiver 710, QoS manager 140-b, and transmitter 720. Network forwarding device 705 also may include a processor. Each of these components may be in communication with one another (such as via one or more buses) .

Receiver 710 may receive information such as packets, user data, or control information associated with various information channels (such as control channels, data channels, and information related to a hybrid approach to advanced QoS, etc. ) . Information may be passed on to other components of the device. The receiver 710 may be an example of aspects of the transceiver 935 described with reference to Figure 9.

QoS manager 140-b may be an example of aspects of the QoS manager 140 described with reference to Figures 1 and 6. QoS manager 715 also may include QoS tree component 725, packet forwarding component 730, and node bypass component 735.

QoS tree component 725 may queue packets in a QoS tree at a network forwarding device, where the QoS tree has a root node implemented by a first network processing layer, a first set of nodes implemented by at least the first network processing layer, and a second set of nodes implemented by a second network processing layer. The first network processing layer may be implemented using special-purpose hardware and the second network processing layer is implemented by a processor executing special-purpose software. In some cases, the first set of nodes performs rate limiting QoS functions. In some examples, the second set of nodes performs at least non-rate limiting QoS function. In other examples, the first and second sets of nodes are disjoint and the first set of nodes is smaller than the second set of nodes. The first and second network processing layers may be implemented in a single SoC.

Packet forwarding component 730 may forward a portion of the packets from the second processing layer to the first processing layer and forward a portion of packets from the first processing layer to the second processing layer. In some cases, forwarding the portion of the packets from the second processing layer to the first processing layer includes: marking the portion of the packets for processing by the first processing layer, where the bypassing is based on the marking of the packet.

Node bypass component 735 may bypass at least a portion of the first set of nodes at the first processing layer for the portion of the packets forwarded to the first processing layer from the second processing layer.

Transmitter 720 may transmit signals generated by other components of the device. In some examples, the transmitter 720 may be collocated with a receiver 710 in a transceiver module. For example, the transmitter 720 may be an example of aspects of the transceiver 935 described with reference to Figure 9. The transmitter 720 may include a single antenna, or it may include a set of antennas.

Figure 8 shows a block diagram 800 of a QoS manager 140-c that supports a hybrid approach to advanced QoS in accordance with various aspects of the present disclosure. The QoS manager 140-c may be an example of aspects of a QoS manager 140 described with reference to Figures 1, 6, 7, and 9. The QoS manager 140-c may include QoS tree component 820, packet forwarding component 825, node bypass component 830, classification information component 835, packet transfer component 840, memory access component 845, and packet processing component 850. Each of these modules may communicate, directly or indirectly, with one another (such as via one or more buses) .

QoS tree component 820 may queue packets in a QoS tree at a network forwarding device, where the QoS tree has a root node implemented by a first network processing layer, a first set of nodes implemented by at least the first network processing layer, and a second set of nodes implemented by a second network processing layer.

Packet forwarding component 825 may forward a portion of the packets from the second processing layer to the first processing layer and forward a portion of packets from the first processing layer to the second processing layer. In some cases, forwarding the portion of the packets from the second processing layer to the first processing layer includes: marking the portion of the packets for processing by the first processing layer, where the bypassing is based on the marking of the packet.

Node bypass component 830 may bypass at least a portion of the first set of nodes at the first processing layer for the portion of the packets forwarded to the first processing layer from the second processing layer.

Classification information component 835 may send classification information for QoS handling in the second processing layer from the first processing layer to the second processing layer and forward classification information for QoS handling to the first processing layer.

Packet transfer component 840 may transfer a packet from the first network processing layer to the second network processing layer prior to forwarding the packet from the second processing layer to the first processing layer. Memory access component 845 may provide dedicated direct memory access (DMA) to a main memory of the networking device for each of the network processing layers. Packet processing component 850 may process the packets such that all packets exit the QoS tree at the root node.

Figure 9 shows a diagram of a system 900 including a network forwarding device 905 that supports a hybrid approach to advanced QoS in accordance with various aspects of the present disclosure. Network forwarding device 905 may be an example of or include the components of a

network forwarding device

105, 605, 705, as described above, such as with reference to Figures 1, 6 and 7. Network forwarding device 905 may include components for bi-directional voice and data communications including components for transmitting and receiving communications, including QoS manager 915, processor 920, memory 925, software 930, transceiver 935, and I/O controller 940. These components may be in electronic communication via one or more busses (such as bus 910) .

Processor 920 may include an intelligent hardware device, (such as a general-purpose processor, a digital signal processor (DSP) , a central processing unit (CPU) , a microcontroller, an application-specific integrated circuit (ASIC) , an field-programmable gate array (FPGA) , a programmable logic device, a discrete gate or transistor logic component, a discrete hardware component, or any combination thereof) . In some cases, processor 920 may be configured to operate a memory array using a memory controller. In other cases, a memory controller may be integrated into processor 920. Processor 920 may be configured to execute computer-readable instructions stored in a memory to perform various functions (such as functions or tasks supporting a hybrid approach to advanced QoS) .

Memory 925 may include random access memory (RAM) and read only memory (ROM) . The memory 925 may store computer-readable, computer-executable software 930 including instructions that, when executed, cause the processor to perform various functions described herein. In some cases, the memory 925 may contain, among other things, a basic input/output system (BIOS) which may control basic hardware and/or software operation such as the interaction with peripheral components or devices.

Software 930 may include code to implement aspects of the present disclosure, including code to support a hybrid approach to advanced QoS. Software 930 may be stored in a non-transitory computer-readable medium such as system memory or other memory. In some cases, the software 930 may not be directly executable by the processor but may cause a computer (such as when compiled and executed) to perform functions described herein.

Transceiver 935 may enable bi-directional communication with one or more network devices 115, via one or more wired or wireless links as described above. For example, the transceiver 935 may represent a wireless transceiver and/or an Ethernet interface, and may communicate bi-directionally with a transceiver of a network device 115. The transceiver 935 also may include a modem to modulate the packets and provide the modulated packets to the antennas for transmission, and to demodulate packets received from the antennas.

I/O controller 940 may manage input and output signals for network forwarding device 905. I/O controller 940 also may manage peripherals not integrated into network forwarding device 905. In some cases, I/O controller 940 may represent a physical connection or port to an external peripheral. In some cases, I/O controller 940 may utilize an operating system such as

or another known operating system.

Figure 10 shows a flowchart illustrating a method 1000 for a hybrid approach to advanced QoS in accordance with various aspects of the present disclosure. The operations of method 1000 may be implemented by a network forwarding device 105 or its components as described herein. For example, the operations of method 1000 may be performed by a QoS manager 140 as described with reference to Figures 1 and 6–9. In some examples, a network forwarding device 105 may execute a set of codes to control the functional elements of the device to perform the functions described below. Additionally or alternatively, the network forwarding device 105 may perform aspects the functions described below using special-purpose hardware.

At block 1005 the network forwarding device 105 may queue packets in a QoS tree at a network device, wherein the QoS tree has a root node implemented by a first network processing layer, a first set of nodes implemented by at least the first network processing layer, and a second set of nodes implemented by a second network processing layer. The operations of block 1005 may be performed according to the methods and structures described with reference to Figures 1–5. In certain examples, aspects of the operations of block 1005 may be performed by a QoS tree component as described with reference to Figures 6–9.

At block 1010 the network forwarding device 105 may forward a portion of the packets from the second processing layer to the first processing layer. The operations of block 1010 may be performed according to the methods described with reference to Figures 1 and 2. In certain examples, aspects of the operations of block 1010 may be performed by a packet forwarding component as described with reference to Figures 6–9.

At block 1015 the network forwarding device 105 may bypass at least a portion of the first set of nodes at the first processing layer for the portion of the packets forwarded to the first processing layer from the second processing layer. The operations of block 1015 may be performed according to the methods described with reference to Figures 1 and 2. In certain examples, aspects of the operations of block 1015 may be performed by a node bypass component as described with reference to Figures 6–9.

Figure 11 shows a flowchart illustrating a method 1100 for a hybrid approach to advanced QoS in accordance with various aspects of the present disclosure. The operations of method 1100 may be implemented by a network forwarding device 105 or its components as described herein. For example, the operations of method 1100 may be performed by a QoS manager 140 as described with reference to Figures 1 and 6–9. In some examples, a network forwarding device 105 may execute a set of codes to control the functional elements of the device to perform the functions described below. Additionally or alternatively, the network forwarding device 105 may perform aspects the functions described below using special-purpose hardware.

At block 1105, packets are queued in a QoS tree in a first processing layer of a network device. At block 1110, some of those packets from the first processing layer are forwarded to a second processing layer. These packets are sent with classifying information from the first processing layer to the second processing layer at block 1115.

At block 1120, the method 1100 determines whether there are any packets in the second processing layer to be processed further by the first processing layer. If not, the method 1100 follows path 1130 to block 1145. If so, the method 1100 follows path 1125 to block 1135 where the packets are marked for processing by the first processing layer. The method 1100 then includes reinjecting the packets into the first processing layer, bypassing at least some of the nodes in the first processing layer. At block 1145, the method 1100 processes the packets such that all of the packets exit the QoS tree at the root node.

It should be noted that the methods described above describe possible implementations, and that the operations and the steps may be rearranged or otherwise modified and that other implementations are possible. Furthermore, aspects from two or more of the methods may be combined.

The description set forth herein, in connection with the appended drawings, describes example configurations and does not represent all the examples that may be implemented or that are within the scope of the claims. The term “exemplary” used herein means “serving as an example, instance, or illustration, ” and not “preferred” or “advantageous over other examples. ” The detailed description includes specific details for the purpose of providing an understanding of the described techniques. These techniques, however, may be practiced without these specific details. In some instances, well-known structures and devices are shown in block diagram form in order to avoid obscuring the concepts of the described examples.

In the appended figures, similar components or features may have the same reference label. Further, various components of the same type may be distinguished by following the reference label by a dash and a second label that distinguishes among the similar components. If just the first reference label is used in the specification, the description is applicable to any one of the similar components having the same first reference label irrespective of the second reference label.

Information and signals described herein may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

The various illustrative blocks and modules described in connection with the disclosure herein may be implemented or performed with a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor also may be implemented as a combination of computing devices (such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration) .

The functions described herein may be implemented in hardware, software executed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Other examples and implementations are within the scope of the disclosure and appended claims. For example, due to the nature of software, functions described above may be implemented using software executed by a processor, hardware, firmware, hardwiring, or combinations of any of these. Features implementing functions also may be physically located at various positions, including being distributed such that portions of functions are implemented at different physical locations. Also, as used herein, including in the claims, “or” as used in a list of items (for example, a list of items prefaced by a phrase such as “at least one of” or “one or more of” ) indicates an inclusive list such that, for example, a list of at least one of A, B, or C means A or B or C or AB or AC or BC or ABC (that is, A and B and C) .

Computer-readable media includes both non-transitory computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A non-transitory storage medium may be any available medium that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, non-transitory computer-readable media can comprise RAM, ROM, electrically erasable programmable read only memory (EEPROM) , compact disk (CD) ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to carry or store desired program code means in the form of instructions or data structures and that can be accessed by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) , or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) , or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, include CD, laser disc, optical disc, digital versatile disc (DVD) , floppy disk and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above are also included within the scope of computer-readable media.

The description herein is provided to enable a person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other variations without departing from the scope of the disclosure. Thus, the disclosure is not limited to the examples and designs described herein, but is to be accorded the broadest scope consistent with the principles and novel features disclosed herein.

Claims

An apparatus for network communication, in a system comprising:

a processor；

memory in electronic communication with the processor； and

instructions stored in the memory and operable, when executed by the processor, to cause the apparatus to:

queue packets in a quality of service (QoS) tree at a network forwarding device, wherein the QoS tree has a root node implemented by a first network processing layer, a first set of nodes implemented by at least the first network processing layer, and a second set of nodes implemented by a second network processing layer；

forward a portion of the packets from the second processing layer to the first processing layer； and

bypass at least a portion of the first set of nodes at the first processing layer for the portion of the packets forwarded to the first processing layer from the second processing layer.
The apparatus of claim 1, wherein the instructions are further executable by the processor to:

forward a portion of packets from the first processing layer to the second processing layer； and

send classification information for QoS handling in the second processing layer from the first processing layer to the second processing layer.
The apparatus of claim 1, wherein:

forward the portion of the packets from the second processing layer to the first processing layer comprises mark the portion of the packets for processing by the first processing layer, wherein the bypassing is based at least in part on the marking of the packet； and

the instructions are further executable to forward classification information for QoS handling to the first processing layer.
The apparatus of claim 1, wherein the instructions are further executable by the processor to:

transfer a packet from the first network processing layer to the second network processing layer prior to forwarding the packet from the second processing layer to the first processing layer.
The apparatus of claim 1, wherein the instructions are further executable by the processor to:

provide dedicated direct memory access (DMA) to a main memory of the networking device for each of the network processing layers.
The apparatus of claim 1, wherein:

the first network processing layer is implemented using special-purpose hardware and the second network processing layer is implemented by a processor executing special-purpose software.
The apparatus of claim 6, wherein:

the first set of nodes performs rate limiting QoS functions； and

the second set of nodes performs at least non-rate limiting QoS function.
The apparatus of claim 7, wherein:

the first and second sets of nodes are disjoint and the first set of nodes is smaller than the second set of nodes.
The apparatus of claim 1, wherein the instructions are further executable by the processor to:

process the packets such that all packets exit the QoS tree at the root node.
The apparatus of claim 1, wherein:

the first and second network processing layers are implemented in a single system-on-a-chip (SOC) .
A method for network communication, comprising:

queuing packets in a quality of service (QoS) tree at a network forwarding device, wherein the QoS tree has a root node implemented by a first network processing layer, a first set of nodes implemented by at least the first network processing layer, and a second set of nodes implemented by a second network processing layer；

forwarding a portion of the packets from the second processing layer to the first processing layer； and

bypassing at least a portion of the first set of nodes at the first processing layer for the portion of the packets forwarded to the first processing layer from the second processing layer.
The method of claim 11, further comprising:

forwarding a portion of packets from the first processing layer to the second processing layer； and

sending classification information for QoS handling in the second processing layer from the first processing layer to the second processing layer.
The method of claim 11, wherein:

forwarding the portion of the packets from the second processing layer to the first processing layer comprises: marking the portion of the packets for processing by the first processing layer, wherein the bypassing is based at least in part on the marking of the packet； and

the method further comprising forwarding classification information for QoS handling to the first processing layer.
The method of claim 11, further comprising:

transferring a packet from the first network processing layer to the second network processing layer prior to forwarding the packet from the second processing layer to the first processing layer.
The method of claim 11, further comprising:

providing dedicated direct memory access (DMA) to a main memory of the networking device for each of the network processing layers.
The method of claim 11, wherein:

the first network processing layer is implemented using special-purpose hardware and the second network processing layer is implemented by a processor executing special-purpose software.
The method of claim 16, wherein:

the first set of nodes performs rate limiting QoS functions； and

the second set of nodes performs at least non-rate limiting QoS function.
The method of claim 17, wherein:

the first and second sets of nodes are disjoint and the first set of nodes is smaller than the second set of nodes.
The method of claim 11, further comprising:

processing the packets such that all packets exit the QoS tree at the root node.
The method of claim 11, wherein:

the first and second network processing layers are implemented in a single system-on-a-chip (SOC) .
An apparatus for network communication, comprising:

means for queuing packets in a quality of service (QoS) tree at a network forwarding device, wherein the QoS tree has a root node implemented by a first network processing layer, a first set of nodes implemented by at least the first network processing layer, and a second set of nodes implemented by a second network processing layer；

means for forwarding a portion of the packets from the second processing layer to the first processing layer； and

means for bypassing at least a portion of the first set of nodes at the first processing layer for the portion of the packets forwarded to the first processing layer from the second processing layer.
The apparatus of claim 21, further comprising:

means for forwarding a portion of packets from the first processing layer to the second processing layer； and

means for sending classification information for QoS handling in the second processing layer from the first processing layer to the second processing layer.
The apparatus of claim 21, wherein

forwarding the portion of the packets from the second processing layer to the first processing layer comprises: marking the portion of the packets for processing by the first processing layer, wherein the bypassing is based at least in part on the marking of the packet； and

the apparatus further comprising means for forwarding classification information for QoS handling to the first processing layer.
The apparatus of claim 21, further comprising:

means for transferring a packet from the first network processing layer to the second network processing layer prior to forwarding the packet from the second processing layer to the first processing layer.
The apparatus of claim 21, wherein

the first network processing layer is implemented using special-purpose hardware and the second network processing layer is implemented by a processor executing special-purpose software.
A non-transitory computer readable medium storing code for network communication, the code comprising instructions executable by a processor to:

queue packets in a quality of service (QoS) tree at a network forwarding device, wherein the QoS tree has a root node implemented by a first network processing layer, a first set of nodes implemented by at least the first network processing layer, and a second set of nodes implemented by a second network processing layer；

forward a portion of the packets from the second processing layer to the first processing layer； and

bypass at least a portion of the first set of nodes at the first processing layer for the portion of the packets forwarded to the first processing layer from the second processing layer.
The non-transitory computer-readable medium of claim 26, wherein the instructions are further executable by the processor to:

forward a portion of packets from the first processing layer to the second processing layer； and

send classification information for QoS handling in the second processing layer from the first processing layer to the second processing layer.
The non-transitory computer-readable medium of claim 26, wherein

forwarding the portion of the packets from the second processing layer to the first processing layer comprises: marking the portion of the packets for processing by the first processing layer, wherein the bypassing is based at least in part on the marking of the packet； and

the instructions are further executable to forward classification information for QoS handling to the first processing layer.
The non-transitory computer-readable medium of claim 26, wherein the instructions are further executable by the processor to:

transfer a packet from the first network processing layer to the second network processing layer prior to forwarding the packet from the second processing layer to the first processing layer.
The non-transitory computer-readable medium of claim 26, wherein

the first network processing layer is implemented using special-purpose hardware and the second network processing layer is implemented by a processor executing special-purpose software.