20/08/2024 | Press release | Distributed by Public on 20/08/2024 13:12
This post was co-authored by Timur Snoke.
Border Gateway Protocol (BGP) routing is a core part of the mechanism by which packets are routed on the Internet. BGP routing gets email to its destination, enables DNS to work, and web pages to load. An important aspect of routing is that packets cross the boundaries of the many autonomously controlled networks that, together, comprise the Internet. This allows us to access, for example, the Amazon website from a phone on the Verizon network. It allows military commanders to see pictures of troop transports in one location and pictures of tanks in another. The BGP protocol, though less well known than low-level protocols such as IP, TCP, and UDP, has a critical role in facilitating - and negotiating - the flows of packets among the many autonomous networks that comprise the Internet.
Consequently, vulnerabilities in the BGP protocol are a very big deal. We have a long-standing expectation that the Internet is robust, particularly with regard to the actions of the many organizations that operate portions of the network - and therefore that a system designed to keep the traffic flowing could only be disrupted by a very large event. In this post, we will examine how a small issue, a deprecated path attribute, can cause a major interruption to traffic.
BGP: A path vector running protocol
BGP is a path vector routing protocol that was defined by the Internet Engineering Task Force (IETF) in RFC 1654. As with many Internet protocols, there are many other Requests for Comments (RFC) associated with BGP methods and processes. For example, RFC 4271 covers BGP path attributes, which can be used when making path selections and building routing tables to assist in routing decisions.
According to RFC 4271, there are well-known mandatory attributes that must be supported on all BGP implementations. However, these attributes are extensible and allow for customized announcements as RFC 4271 explains: "Well-known mandatory attributes must be included with every prefix advertisement, while well-known discretionary attributes may or may not be included". The custom attributes can be used internally by your organization or externally (to communicate important information to other organizations). They also may remain unused but available. Attributes can contain information about updates and network origin or weight an Autonomous System Number (ASN) to prioritize it in routing.
The menace of multiple BGP implementations
The CERT/CC recently dealt with a relevant case, Vulnerability Note VU#347067 (Multiple BGP implementations are vulnerable to improperly formatted BGP updates. In this case, a researcher saw a significant outage stemming from an improperly formatted path attribute BGP UPDATE that caused vulnerable routers, when they received an update, to de-peer (terminate a peering relationship that allows packets to flow from one network to another)).
Unaffected routers might also pass the crafted updates across the network, potentially leading to the update arriving at an affected router from multiple sources, causing multiple links to fail. The flaw was that, instead of ignoring the improperly formatted attributes, the receiving router dropped the routing update and lost the information being provided about other routes. This situation resulted in a real-world de-peering of routers and loss of traffic passed between them.
In short, this vulnerability disrupted the flow of information that BGP routing was designed to ensure.
Routers that are designed for resiliency should still function if they ignore a deprecated attribute. They are not expected to use the attribute as it was originally designed since it is no longer part of the official protocol. Adding to the problem is inconsistent updating - there are many older versions of the BGP protocol specification deployed on the Internet because not everyone can upgrade to the latest and best version every time one is released. However, all implementations of any Internet protocol should be able to function if new attributes show up because the protocols are always changing. Remember, the Internet was designed for survivability - a few errors in attributes shouldn't interrupt it. Unfortunately, the response to unspecified attributes is unexpected - one organization's router might handle the attribute without a problem while another one might not.
Awareness of the deprecated attributes being used is the first step to determining whether a particular installation is vulnerable. The inconsistency of updates means that the organizations announcing routes don't know what software other organizations are using. They assume all core routers can handle the traffic. In this circumstance, verifying that your software isn't affected is a good way to stay connected on the Internet. This suggests that organizations contact their router vendors to learn how they handle deprecated attributes and whether responses are robust. They should be able to tell you if the response can be modified and what steps to take.
An analysis of BGP data
The Software Engineering Institute (SEI) CERT Division collects BGP data, so we examined the last two years of data to find out what deprecated attributes are still announced. To do so, we used the list of deprecated attributes published by the Internet Assigned Numbers Authority (IANA).
The three attributes we found are listed in Table 1.
The deprecated attribute that caused the problem in VU#347067 was ENTROPY_LEVEL. This is the attribute particular to this instance, but other deprecated attributes might cause issues later. To reduce this risk and avoid further vulnerabilities it is essential to understand more about the conditions that could cause it. In this case, it's how the router handles deprecated attributes (or doesn't).
We looked at the number of BGP announcements each day that used these deprecated attributes (Figure 1).
The greatest number of announcements (3,620) occurred on 7 July 2022. That doesn't seem like very many in the context of millions and millions of route announcements a day, but a small attribute, mishandled by the wrong router, can cause havoc, as we saw previously.
An important feature of this situation is that routers that announce the deprecated attribute cannot sense that they are causing a problem. The configuration or software still uses these deprecated attributes, and consequently, the router will freely share it with the Internet as they are designed to do. The real troublemakers are the routers that receive the routes.
Deprecated attribute use over time
In routing, as with many things on the Internet, we know who did it, when they did it, how they did it, and even most likely where they did it. We just don't know why these organizations are using these deprecated attributes. It's their internal decision to use them and usually, it isn't relevant.
With regard to the who, we examined the time series of this data, which shows, on the vertical axis, the number of ASNs announcing a deprecated attribute each day over a span of nine days (Figure 2).
Figure 2 illustrates a time series with a dip followed by a peak, which was followed by another dip. That is, the general number of ASNs announcing deprecated attributes dropped, then increased. Rather than guessing when that happened, we used an algorithm to find these change points (Figure 3).
In this case, change point analysis looked for shifts in the average value across time. Starting at 20221020, the average number of announcers dips then recovers briefly at 20230108. It dips again at 20230324 and then, finally, starting at 20230618, the average number of announcers goes down again. The number of Autonomous Systems that used these deprecated attributes, on average, decreased over time. The change in the announcements over time tells us that at certain points, abrupt changes were made in the number of organizations that used the attributes. The good news is that fewer are using them. The bad news is we don't know why those who use them continue to do so.
Avoiding the havoc of deprecated attributes
Now we have a better idea of when something happened. We do not know what caused organizations to start or stop announcing the deprecated attributes. We do know, however, that organizations receiving routes on the live Internet should be aware of the potential problem and ensure that they aren't vulnerable to announcements using deprecated attributes. Because of the significance of BGP in managing 'cross-border' flows in the Internet, the potential consequences could be large.
In conclusion, we advise that organizations work with vendors to verify and understand their process for handling deprecated attributes.
Read the CERT Vulnerability Note 'Multiple BGP implementations are vulnerable to improperly formatted BGP updates ' (VU#347067).
Leigh Metcalf is a Senior Network Security Research Analyst at the Software Engineering Institute's CERT Division specializing in systems architecture and administration, research and development, and using mathematical techniques for cybersecurity. She is the co-author of Cybersecurity Myths and Misconceptions: Avoiding the Hazards and Pitfalls that Derail Us
Timur Snoke is a senior member of technical staff at Software Engineering Institute's CERT Division with interests in network design, networking, and security controls
Metcalf, L., and Snoke, T., 2024: The Threat of Deprecated BGP Attributes. Carnegie Mellon University, Software Engineering Institute's Insights (blog), Accessed August 9, 2024, .
Copyright © 2024 Carnegie Mellon University
This material is based upon work funded and supported by Department of Homeland Security under Contract No. FA8721-05-C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center sponsored by the United States Department of Defense.
Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of Department of Homeland Security or the United States Department of Defense.
NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN "AS-IS" BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.