p2pnet news P2P:- | Freedom:- Yesterday was D-for-Disclosure-Day for Bell Canada —- the day it had to comply with the CRTC order to publicly reveal levels of congestion it says forced it to launch its heavily criticised bandwidth throttling practices.

But isn’t there something missing?

There doesn’t seem to be any mention of Bell’s Ethernet Network segments.

Just about all information centres on ATM only.

But here it is, as is, in full, as submitted by Mirko Mr 5% Bibic, Bell Canada’s chief of regulatory affairs, together with a supplemental version “filed in confidence with the CRTC” as per “CRTC letter dated 2008 06 19″.

And by way of a PS —

“Just my thoughts (not a network guy myself),” posts Shinda on dslreports:

“The graphs on page 12 (Network Traffic Growth) and Network Capacity Growth (while on different scales) show that Bell (since 2003) has had sufficient network in place to manage even today’s usage.

“And unless I’m mis-interrupting this paragraph:

As described further in Bell Canada(CRTC)15May08-1 CAIP Part VII, Bell Canada has limited records of traffic make-up (i.e., P2P vs. non-P2P traffic) prior to exercising its traffic shaping using DPI devices. Therefore, the available data is insufficient to show a trend due to the short time period since DPI was introduced.

“They are conceding that P2P isn’t the problem, they really don’t know what the problem is (apart from increased growth in usage), which would be better correlated against the amount of new rich media content that has sprung up on the net, in the selected time frame, (YouTube for instance).”

Jon Newton - p2pnet

INFORMATION REQUESTED BY CANADIAN RADIO-TELEVISION AND TELECOMMUNICATIONS COMMISSION

Q. Bell Canada is to disclose the following information related to the chart on page 5 of 15 of its response to Bell Canada(CRTC)15May08-2b) on the public record, using the numerical information, including the total number of links (i.e. congested and non-congested) in its network provided in response to Bell Canada(CRTC)10Jun08-1:

For each link type (i.e. central office DSLAM, aggregation network, BAS and backbone network), for each month for the period March 2007 to April 2008, the percentage (in numerical form) that the number of congested links is of the number of total links, aggregated for the Ontario and Quebec regions.

A.Pursuant to the Commission staff letter of 19 June 2008, Bell Canada hereby files on the public record its response to the additional information requested. As Commission staff requested information in a format that was not included as part of the original Bell Canada(CRTC)10Jun08-1 CAIP Part VII as filed by Bell Canada (specifically expressing the requested information as a numerical percentage rather than absolute numbers and aggregating the data for Ontario and Québec), Bell Canada has elected to file the information requested on the public record in the form of this new interrogatory.

Percentage of number of congested links, aggregated for the Ontario and Québec Regions

As can be observed in the table above, the total percentage of all four types of congested network links during a given month in the period in question has varied between 2.6% and 5.2%. While these numbers may seem low to the average lay person, they are significant to network traffic engineers such that it is important to consider the number of congested links in the proper context.

It is important to understand that end-users’ high-speed traffic travels across multiple links in Bell Canada’s network. Therefore if a single link along the way is congested, an end-user will experience negative impacts of congestion such as a slow connection or inability to connect. Once congestion has been observed, this means that latency has already been occurring in the network thereby producing negative impacts on customers. This phenomenon is analogous to a road system. When some of the major arterial roads are congested (analogous to Backbone links), cars travelling from the suburbs through downtown are impacted by traffic regardless of the state of congestion on the roads in the suburbs (similar to DSLAM and Aggregation links). Just like a single traffic roadblock can hinder drivers going to multiple destinations that pass through the road that is blocked, a very small amount of congested links can seriously affect a large number of high-speed end-users’ traffic. When traffic levels increase, the potential for drivers to be impacted by more than one roadblock also increases. The impact of that congestion is clearly perceptible to the end-user, though to varying degrees. The effects are even more obvious in the case of time sensitive applications such as voice communications when the content can become severely degraded to the point of being unrecognizable.

One needs to be cautious about simply looking at absolute number of congested links over time. For example, as noted in Bell Canada(CRTC)15May08-2 CAIP Part VII, non-P2P traffic that is not being managed as part of the Company’s traffic management solution has been able to use up and flow more freely using the bandwidth previously occupied by P2P file sharing traffic during peak periods. Similarly, other non-P2P types of traffic such as video streaming are growing at a faster pace than in prior years. Furthermore, one must consider the impacts of seasonal fluctuations in traffic patterns generally. Finally, the number of congested links over the period in the table above is also affected by the rate of deployment of the Company’s traffic management solution.

As noted in Bell Canada(CRTC)10Jun08?1 CAIP Part VII, depending on the type of link, the potential number of impacted customers will vary. The geographic location of the link in the network is also a factor to consider. For instance, a congested OC-48 which is a Backbone link will impact more than twenty times as many customers as an OC-3 link which is a large DSLAM link or a small Aggregation link. The Company had also noted that it had projected that the number of retail and wholesale customers that could be affected by congested links could reach as high as 790,000 by the end of the first quarter of 2009 had Bell Canada failed to apply its network management solution. When congestion reaches this level in a network, the impacts can become much more severe and protracted in nature. For example, multiple users on congested links could experience impacts ranging from a degraded browsing experience to potentially not being able to connect to the network. The Company determined that at this level, the congestion would have surpassed the limits of deployment projects that it could address within the appropriate relief timeframes given such factors as available manpower, release windows and volume of capital projects. As the Company noted in Bell Canada(CRTC)15May08-04 CAIP Part VII, it spent millions in CAPEX on its high-speed Internet service in 2007 and $110M of that CAPEX was additional unplanned capital spending solely to relieve congestion in its network resulting from growth in Internet traffic. The Company plans to spend close to half a billion dollars in 2008. Even with these investments, given the year-over-year growth trends, Bell Canada estimates that congestion would increase to a point where the projected investment would not be sufficient to support the growth, hence the necessity for additional measures.

SUPPLEMENTAL VERSION AS PER CRTC LETTER DATED 2008 06 19
FILED IN CONFIDENCE WITH THE CRTC

INFORMATION REQUESTED BY CANADIAN RADIO-TELEVISION AND TELECOMMUNICATIONS COMMISSION

Q. In paragraphs 20 and 21 of Bell Canada’s Answer, the Company submitted that it had implemented its Internet traffic management solution to ease network congestion.

a) Provide the criteria that Bell Canada uses to conclude that there is congestion in any network elements and links of its network used to provision high-speed access (GAS and retail Internet access).

b) Describe where congestion is occurring in Bell Canada’s network used to provision high-speed access. The response should discuss which parts of the network (e.g. central office DSLAM, aggregation network, BAS, backbone network) and which network elements or links within the different parts of the network are the sources of congestion. The response should address any differences of the sources of congestion over different geographic regions of the network.

c) Describe all other approaches, if any, considered by Bell Canada as an alternative to shaping P2P traffic to address the network congestion it described, and explain why each approach was rejected. Include a discussion of the conditions under which Bell Canada would augment its network capacity to address congestion.

d) Refer to paragraph 20 of Bell Canada’s Answer. During peak periods before deployment of its traffic management solution, 60% of total traffic corresponded to 33% of available bandwidth. Commission staff notes that 100% of the total traffic would correspond to 55% (100/60 x 33%) of the available bandwidth. Provide a detailed explanation of why utilization of 55% of available bandwidth would require the use of traffic management to ease congestion.

e) Provide data on the growth of traffic on Bell Canada’s network for supporting Internet access from 2004 to 2008, indicating the mix of P2P and Non-P2P traffic over time. Further, describe to what extent Bell Canada has increased its network capacity to support the traffic growth.

A.Pursuant to the Commission’s ruling dated 19 June 2008, Bell Canada is providing this supplemental response to disclose on the public record 1) the link utilization thresholds on page 3 of 15, and 2) the link congestion criterion on page 4 of 15.

Pursuant to section 39 of the Telecommunications Act, certain information in this response is being provided in confidence to the Commission. Release of this information on the public record would allow existing and potential competitors to formulate more effective business plans and marketing strategies, thereby prejudicing Bell Canada’s competitive position and causing specific direct harm to Bell Canada. An abridged version is provided for the public record.

a) For an overview of the problems associated with the congestion on the Bell Canada network, as well as a summary of its proposed solutions to address this problem, please see Bell Canada(CRTC)15May08-4 CAIP Part VII.

It is well accepted that the Internet consists of multiple interconnected networks and that applications that run over the Internet can experience congestion in locations that may be external to Bell Canada’s network. This interrogatory response will specifically address congestion in Bell Canada’s network as it pertains to the delivery of high-speed Internet services (retail and wholesale GAS).

Most applications that run over the Internet use protocols with inherent congestion detection and control mechanisms that are able to automatically adapt the delivery speed of packets when congestion is detected (see Appendix 1 to Bell Canada’s Answer). However, these mechanisms are effective only to the extent that traffic sent across the network does not exceed the capacity that each of the network elements are capable of processing at any given moment. When capacity is exceeded, packets are delayed before being delivered to their destination (also called latency) or dropped, in which case the application may decide to retransmit the packets. Therefore, latency and dropped packets are considered symptoms of congestion in a network. The impact of congestion on Internet end-users will vary from slower traffic to not being able to connect to the Internet at all.

Bell Canada measures latency and dropped packets (referred to as “cells” in the case of an ATM network) as well as the level of utilization of links1 in the network. Although both measurements are important, it is the level of utilization of links that Bell Canada uses as the primary criteria to determine where the congestion is occurring. Such an approach is more efficient since utilization level measurements are more consistently and readily available than latency measurements. The approach is also reliable given the observed close relationship between the measured utilization levels and the latency (the higher the utilization in a link, the higher the latency).

A common practice in the Industry and network management is to develop thresholds at which the utilization level in a link has a very high probability of producing negative impacts on end-users. Therefore, it is not feasible to ever utilize a link to its full capacity. The actual thresholds used by Bell Canada were developed by analyzing the plotted latency data for each type of link in its network. Using this method, Bell Canada was able to determine the specific level of utilization of a link at which the number of congestion events (latency and dropped packets) increases dramatically and the events become longer in duration. This is highlighted by the fact that latency and dropped packets occur well below 100% link utilization. Bell Canada has developed thresholds for each type of link using the same methodology:

Link utilization measurements are performed at 15 minute intervals. The figures, however, hide the fact that instantaneous traffic peaks can be higher than actual measured utilization.

The fact the thresholds for DS-3 links are lower than they are for OC-48 links is explained in part due to the lower processing capacity of the DS-3 equipment and the longer periods of time traffic is able to remain in queues waiting to be processed, increasing the potential for higher latency.

Bell Canada is able to actively monitor congested links in order to take appropriate action to resolve the problem. Utilization measurements in the network occur every 15 minutes. For a link to be considered congested, the threshold must have been exceeded at least once on 5 or more different days of a 14 consecutive day period. Bell Canada believes that this approach to measuring and identifying congestion in its network is consistent with the methodology used by many other service providers throughout North America and the UK.

b) In this question, the Commission has asked Bell Canada to identify the parts of the network where congestion is occurring. Following the Commission’s suggestion, Bell Canada has broken down its network into the four suggested categories, namely DSLAM, Aggregation Network, BAS and Backbone Network. The location of those parts of the network, and the links associated with those parts, can be seen in the following diagram.

The chart below illustrates the number of congested links per month for the period of March 2007 to April 2008 for each location in the network (i.e. central office DSLAM, aggregation network, BAS and backbone network).

As can be observed from this chart, even though congestion is occurring in all the link types of the network, congestion is most severe on DSLAM and aggregation links. As discussed in Bell Canada(CRTC)15May08 4 CAIP Part VII, GAS and retail traffic transit through the same path/equipment from the DSLAM up to the BAS (and the associated DPI), the point in the network where retail and wholesale traffic is aggregated from multiple central offices. The GAS and retail traffic also transits through some of the same backbone network links.

Depending on the type of link, the potential number of impacted customers will vary. As an example, a congested OC-3 link can impact as many as # customers whereas for an OC-48, this number increases to # customers. The Company has projected that the number of customers that could be affected by congested links could reach as high as 790,000 by the end of the first quarter of 2009 were Bell Canada to fail to apply its network management solution.2 As discussed in detail in Bell Canada(CRTC)15May08-4 CAIP Part VII, Bell Canada has invested heavily in expanding capacity of its DSL network to address congestion issues. Even with these investments, given the year over year growth trends, Bell Canada estimates that congestion would increase to a point where the projected investment would not be sufficient to support the growth, hence the need for additional measures.

In an effort to address the Commission’s request for geographic regions, the following chart illustrates the same information as the previous one, but separating Ontario and Québec regions. Bell Canada notes that the data includes the portions of Bell Aliant’s network for the provinces of Québec and Ontario. In addition, a small number of backbone congested links that are located south of the border are captured in the Ontario figures.

The map below illustrates the central offices with congested links demonstrating that while congestion occurs throughout the Bell Canada network, unsurprisingly it is more concentrated in the main population centres of Ottawa, Montréal and the Greater Toronto Area.

c) As described in Bell Canada(CRTC)15May08-4 CAIP Part VII, Bell Canada will invest close to $500 million to expand and upgrade the infrastructure of its DSL network in 2008. This being said, augmenting capacity in its network has been and continues to be the first approach Bell Canada uses to address congestion. Bell Canada monitors the utilization of links and, at specific utilization thresholds, network provisioning groups are automatically notified to initiate the necessary projects to augment network capacity on a specific link. The utilization thresholds used to trigger the network relief are designed to be lower than the thresholds where the link is considered congested. The network relief threshold needs to be low enough to provide sufficient time for the additional capacity to be operational and available before it becomes congested. Ideally, the network relief threshold would always be low enough so that congestion is avoided altogether on the links. Bell Canada uses the following network relief thresholds:

DS-3: #;
OC-3: #; and
OC-12 and OC-48: #.

Bell Canada also performs rerouting of traffic on less congested links. However, this mechanism to manage congestion is limited since too much rerouting increases the delay in delivery of traffic and therefore latency is not improved.

But as indicated in Bell Canada(CRTC)15May08-4 CAIP Part VII, the exceptional growth and the economic and market realities clearly show that a pure capital spend solution is not viable.

Thus, in addition to expanding capacity, Bell Canada uses pricing-based mechanisms to manage network traffic such as bandwidth caps and usage-based billing. For further information see Bell Canada(CRTC)15May08-5 CAIP Part VII.

The implementation of DPI in Bell Canada’s network as a traffic management tool did not affect the methodology used to detect and address congestion and augmentation of capacity in the network. Even though the DPI equipment was originally intended to introduce customer usage data collection functionality for Bell Canada’s usage billing, it was subsequently determined that DPI should also be used for traffic shaping during peak periods as an additional measure to address congestion. Prior to implementing traffic shaping during peak periods, high usage resulted in many links being identified as congested for very short periods of time, triggering network capacity projects. However, outside the peak period, the same links were not congested and performing satisfactorily. This situation did not promote efficient capital spending to augment capacity in the network.

It is somewhat misleading to discuss a single DSL network. Initially, Bell Canada constructed an ADSL network that uses ATM as the technology underpinning its backbone network.  In 2004, Bell Canada started building its next generation DSL network using Fibre to the Node technology (FTTN) and Optical Ethernet (OE) in its aggregation network. Bell Canada is heavily investing in its Ethernet aggregation network as well as FTTN, but the vast majority ( #) of its retail and wholesale GAS customers currently reside on the ADSL-ATM network. When relief is required to address congested links on the ADSL ATM network, Bell Canada is required to invest in legacy technology to support the ATM network. While ATM is both expensive and limited in its capacity when compared to OE, nonetheless, as noted above, Bell Canada continues to make those investments on behalf of its customers. Bell Canada is increasingly investing in its Ethernet aggregation network as well as its FTTN network but it will take time until it is expanded and available to the majority of Internet end-users. Therefore transferring customers from the ADSL ATM network to the Ethernet aggregation network and the FTTN network to relieve network pressure was not possible for the majority of locations where congestion was occurring.

The introduction of traffic shaping was a necessary measure to manage investments in the network to address the congestion at peak periods. Since traffic shaping has been introduced, Bell Canada has observed that non-P2P traffic has significantly increased during peak periods. This increase is likely a result of two main events: 1) other non-P2P types of traffic such as streaming are growing at a faster pace than in prior years; and 2) non-P2P traffic is now able to use up and flow more freely using the bandwidth previously occupied by P2P traffic.

See Bell Canada(CRTC)15May08-5 CAIP Part VII for a description of the Company’s Internet traffic management solution.

d) As noted in Bell Canada(CRTC)15May08-1 CAIP Part VII part c),

the reference to “available bandwidth” in paragraph 20 is the total retail traffic used during the peak period measured by the DPI device during the sampling period. To clarify, the sentence should read: “During peak periods, that same 5% of users were generating 33% of the total retail high-speed Internet traffic that transits through the DPI devices”. The statistics in paragraph 20 of Bell Canada’s submission are not absolute figures of total usage in the network.

As such, one cannot compare the percentage of traffic to the total bandwidth available. Further, the 60% figure cited in paragraph 20 refers to overall traffic, but the 33% figure refers to the traffic during peak periods. Therefore, the calculation proposed in this interrogatory is based on an incorrect comparison.

The quoted statistics were meant to illustrate that a very small proportion of Internet users on Bell Canada’s network were generating a disproportionate amount of traffic and hence using a disproportionate amount of bandwidth. A simple calculation to derive total bandwidth usage in the network is not representative of the state of all the links that make up the network. The Commission’s question appears designed to try to understand the overall congestion in the DSL network. For a description of congestion problems please see above.

e) The first chart below illustrates the growth of traffic (in Gigabits per second) in various parts of Bell Canada’s network for the years 2004 to 2008 while the second chart illustrates the growth of capacity in the network for the same period. In both charts, the capacity and traffic data for the Backbone and Aggregation portions of the network only includes data for links between central offices. The data excludes links which connect routers in the same central office location. However, the data presented is representative of overall network trends.

Although capacity is shown to be at all times above the total traffic demand, these figures can be misleading since they are simply averages of traffic measured at specific moments in time and are not able to reflect the peak traffic usage every day. To highlight that looking at total capacity without looking at peak can be misleading, the following graph shows a significantly increasing number of dropped cells on the ATM network over a very recent period compared to previous years even though total average capacity was at all times above the total average usage in the network.

A sharp increase in cell loss directly translates to high packet loss in the network.3 Overflowing traffic on a link is first queued (delayed) until it can be delivered. If the delay of delivery is too long or the quantity of traffic surpasses the queuing capacity, then cells (packets in the case of an IP network) are dropped. Therefore, an indication of dropped cells means that latency is likely at its highest level.

The compounded effects of delay and dropped cells negatively affect Internet end-users to varying degrees. The effects are clearly worse in the case of time sensitive applications where the content such as a voice communication can become severely deteriorated to the point of being unrecognizable.

As described further in Bell Canada(CRTC)15May08-1 CAIP Part VII, Bell Canada has limited records of traffic make-up (i.e., P2P vs. non-P2P traffic) prior to exercising its traffic shaping using DPI devices. Therefore, the available data is insufficient to show a trend due to the short time period since DPI was introduced.

Stay tuned.

. .Stumble It!

---

Use free p2pnet newsfeeds for your site. It’s really easy!

Subscribe to p2pnet.net | | rss feed: http://p2pnet.net/p2p.rss | | Mobile - http://p2pnet.net/index-wml.php

---

Net access blocked by government restrictions? Use Psiphon from the Citizen Lab at the University of Toronto. Go here for details. Download here.

This entry was posted on Tuesday, June 24th, 2008 at 10:14 am and is filed under Freedom, P2P. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

HOME

Viewed 4345 times