Does My Internet Service Provider Intercept Packets

Introduction

In 2015, the Federal Communications Committee (FCC) reclassified broadband Cyberspace service providers (ISPs) as common carriers under Title II of the Communications Act. This shift triggered a statutory mandate for the FCC to protect the privacy of broadband Cyberspace subscribers' information. The FCC is now considering how to arts and crafts new rules to clarify the privacy obligations of broadband providers.

Last week, the Plant for Information Security & Privacy at Georgia Tech released a working paper whose senior author is Professor Peter Swire, entitled "Online Privacy and ISPs."Throughout this written report, we refer to the February 29, 2016 version of the Swire paper, which may modify in the time to come. The paper describes itself as a "factual and descriptive foundation" for the FCC as the Commission considers how to approach broadband privacy. The newspaper suggests that certain technical factors limit ISPs' visibility into their subscribers' online activities. It as well highlights the information collection practices of other (not-Internet service provider) players in the Net ecosystem.

We believe that the Swire paper, although technically accurate in most of its particulars, could leave readers with some mistaken impressions about what broadband ISPs can see. We offer this report as a complement to the Swire newspaper, and an culling, technically expert cess of the present and potential future monitoring capabilities available to ISPs.

We notice that:

i. Truly pervasive encryption on the Cyberspace is nonetheless a long way off. The fraction of total Cyberspace traffic that's encrypted is a poor proxy for the privacy interests of a typical user. Many sites even so don't encrypt: for instance, in each of three key categories that we examined (wellness, news, and shopping), more than than 85% of the tiptop l sites still fail to encrypt browsing by default. This long tail of unencrypted web traffic allows ISPs to see when their users research medical conditions, seek advice near debt, or shop for any of a wide gamut of consumer products.

two. Even with HTTPS, ISPs can nonetheless see the domains that their subscribers visit. This type of metadata can be very revealing, especially over time. And ISPs are already known to await at this data — for example, some ISPs analyze DNS query data for justified network management purposes, including identifying which of their users are accessing domain names indicative of malware infection.

3. Encrypted Internet traffic itself can be surprisingly revealing. In contempo years, computer science researchers accept demonstrated that network operators can learn a surprising corporeality about the contents of encrypted traffic without breaking or weakening encryption. Past examining the features of network traffic — like the size, timing and destination of the encrypted packets — it is possible to uniquely identify certain spider web page visits or otherwise obtain information about what the traffic contains.

4. VPNs are poorly adopted, and tin provide incomplete protection. VPNs have been commercially available for years, but they are used sparsely in the United States, for a range of reasons we describe beneath.

We agree that public policy needs to be congenital on an accurate technical foundation, and we believe that thoughtful policies, particularly those related to Cyberspace technologies, should be reasonably robust to foreseeable technical developments.

We intend for this study to assist policymakers, advocates, and the general public as they consider the technical capabilities of broadband ISPs, and the broader technical context inside which this policy debate is happening. This paper does not, still, take a position on whatever question of public policy.

Four Key Technical Clarifications

1. Truly pervasive encryption on the Internet is still a long style off.

Today, a significant portion of Net action remains unencrypted. When a spider web site uses the unencrypted Hypertext Transfer Protocol (HTTP), an ISP can run across the full Uniform Resources Locator (URL) and the content for whatsoever web page requested by the user. Although many pop, high-traffic web sites take adopted encryption by default, a "long tail" of web sites have not.

The fraction of full traffic that is encrypted on the Internet is a poor guide to the privacy interests of a typical user. The Swire paper argues that "the norm has become that deep links and content are encrypted on the Internet," basing its claim on the true observation that "an estimated seventy percentage of traffic will be encrypted by the end of 2016." However, this number includes traffic from sites like Netflix, which itself accounts for about 35% of all downstream Internet traffic in Northward America.

Sensitivity doesn't depend on volume. For instance, watching the total Ultra HD stream of The Amazing Spider-Man could generate more than 40GB of traffic, while retrieving the WebMD page for "pancreatic cancer" generates less than 2MB. The page is xx,000 times less data by volume, but likely far more sensitive than the picture show. (WebMD has yet to offer users the selection of secure HTTPS connections, much less to make that option the sole or default choice.)

We conducted a cursory survey of the 50 nearly popular web sites in the each of three categories — health, news and shopping — as ranked past Alexa.

The Long Tail of Unencrypted Web Traffic: Alexa Top l Sites, by Category

Category	Per centum of Sites that Do Not Encrypt Browsing by Default	Example URLs for Unencrypted Web Sites
Health	86%	`http://`webmd.com/hiv-aids/guide… `http://`mayclinic.org/…cancer… `http://`medicinenet.org/…eczema… `http://`health.com/sexual-health `http://`who.int/…childhood-hearing-loss…
News	90%	`http://`nytimes.com/…revenue enhancement-tips… `http://`huffingtonpost.com/divorce `http://`video.foxnews.com/…sexual practice-after-50… `http://`time.com/…gay-rights… `http://`bankrate.com/debt-management…
Shopping	86%	`http://`ikea.com/…bath… `http://`target.com/…study-bible… `http://`macys.com/…motherhood-apparel… `http://`bedbathandbeyond.com/…acne-wash… `http://`toysrus.com/…Toddler-Toys…

We plant that the vast majority of these web sites — more than than 85% of sites in each of the three areas — still do not fully support encrypted browsing by default. These sites included references on a full range of medical conditions, advice about debt direction, and production listings for hundreds of millions of consumer products. For these unencrypted pages, ISPs can run into both the full web site URLs and the specific content on each web page. Many sites are pocket-sized in data book, only high in privacy sensitivity. They can paint a revealing motion-picture show of the user's online and offline life, even inside a short period of time.

Sites struggle to adopt encryption. From the perspective of i of these unencrypted web sites, it tin can be very challenging to drift to HTTPS, especially when the site relies on a wide range of third-party partners for services including advertising, analytics, tracking, or embedded videos. In order for a site to migrate to HTTPS without triggering warnings in its users' browsers, each 1 of the third-party partners that site uses on its pages must back up HTTPS.

Ad tracker HTTPS support rates on the Alexa top 100 news sites (via Citizenlab)

Advertizement tracker HTTPS back up rates on the Alexa top 100 news sites (via Citizenlab)

Getting third-party partners to support HTTPS is a serious hurdle, fifty-fifty for sites that desire to make the switch. For case, in a 2015 survey of 2,156 online advertising services, more than 85% did not support HTTPS. Moreover, every bit of early 2015, but 38% of the 123 services in the Digital Advertising Alliance's own database supported HTTPS. In the figure to a higher place, describing the superlative 100 news sites, each unit of red or burgundy indicates a third-party partner that does non support HTTPS. In order for whatsoever one of these news sites to provide its content to users securely (without creating alarm or fault letters) the publisher must either wait for all of its red and burgundy partners to plough green, or else abandon those partners on any secure parts of its site. The online ad industry is working to improve its security posture, only clearly there remains a long road ahead.

Internet of Things devices oftentimes transmit data without encryption. Information technology'south non merely spider web sites that fail to encrypt traffic transmitted over broadband connections. Many Internet of Things (IoT) devices, such as smart thermostats, home voice integration systems, and other appliances, neglect to encrypt at least some of the traffic that they send and receive. For case, researchers at the Middle for Information technology Policy at Princeton recently establish a range of popular devices — from the Nest thermostat to the Ubi vocalism organization, to the PixStar photo frame — transmitting unencrypted data across the network. "Investigating the traffic to and from these devices turned out to be much easier than expected," observed Professor Nick Feamster.

As more than users adopt mobile devices, they communicate with a greater number of ISPs. Use of mobile devices is growing rapidly as a portion of users' overall Internet activity. The Swire paper observes that today's ISPs face a more "fractured world" in which they accept a "less comprehensive view of a user'southward Internet activity." It is truthful that today, many consumers' personal Internet activities are spread out over several connections: a home provider, a workplace provider, and a mobile provider. However, a user often has repeated, ongoing, long-term interactions with both her mobile and her wireline provider. Over fourth dimension, each ISP can see a substantial amount of that user'southward Net traffic. There's plenty of activity to get around: The amount of time U.South. consumers spend on continued devices has increased every year since 2008.

2. Even with HTTPS, ISPs can all the same meet the domains that their subscribers visit.

The increased use of encryption on the Web is a substantial privacy comeback for users. When a spider web site does use HTTPS, an Isp cannot come across URLs and content in unencrypted grade. Nonetheless, ISPs can nonetheless almost always see the domain names that their subscribers visit.

DNS queries are about never encrypted. ISPs tin see the visited domains for each subscriber by monitoring requests to the Domain Proper noun System (DNS). DNS is a public directory that translates a domain proper noun (like bankofamerica.com) into a corresponding IP addresses (like 171.161.148.150). Earlier the user visits bankofamerica.com for the start time, the user'south estimator must first learn the site's IP address, and then the computer automatically sends a background DNS query almost bankofamerica.com.

Even if connections to bankofamerica.com are encrypted, DNS queries about bankofamerica.com are not. In fact, DNS queries are near never encrypted. ISPs could simply monitor what queries its users are making over the network.

Collection and utilise of DNS queries by ISPs is practical, is cost effective, and happens today on Isp networks. Because the user'south estimator is assigned by default to use the ISP's DNS server, the ISP is generally capable of retaining and analyzing records of the queries, which the users themselves transport to the ISP in the normal course of their browsing. The Swire paper asserts that it "appears to be impractical and cost-prohibitive" to collect and utilise DNS queries, but cites no technical or other say-so for that assessment. Our technical experience indicates that logging is both feasible and relatively inexpensive to practice: Modern networking equipment can easily log these requests for later analysis. Moreover, even if the user's computer is specially configured to use an external DNS server (non operated by the user's ISP), the DNS queries must nevertheless attain that external server unencrypted, and those queries must all the same travel over the ISP's network, creating the opportunity to inspect them.

In fact, ISPs already do monitor user DNS queries for valid network management purposes, including to find potential infections of malicious software on user devices. Certain domain names are used solely past malicious software tools, and existent user traffic can be analyzed to place and cake such domains. Moreover, when an individual user visits a compromised domain, this is a stiff sign that 1 or more of that user'southward devices is infected, and commercially available tools let ISPs to notify the user about the potential infections. Co-ordinate to literature from a network equipment vendor, Comcast currently deploys this security-focused, per-subscriber DNS monitoring functionality on its network.

Researchers in 2011 also found that several small ISPs were already leveraging their role as DNS providers to non only monitor, but actively interfere with, DNS resolution for their users. To be clear, we are not aware of any evidence that large ISPs have however begun to use DNS queries in privacy-invasive ways, much less to interfere with subscribers' queries forth the lines detected in 2011. Nosotros detect here simply that it is technologically feasible today for ISPs both to monitor and to interfere with DNS queries.

Although network security is not substantially impacted by a modest to moderate amount of VPN usage, there are meaningful technology downsides to a future in which most or all DNS queries are cryptographically concealed from the end user's ISP. (Such a future could, for example, make it more difficult for ISPs to provide early and detection and swift response for some kinds of malware attacks.) At the same time, as long every bit the user'southward DNS queries are visible to the Isp for network direction purposes, the ISP will too have a technologically feasible option to clarify those queries in ways that would compromise user privacy.

Even a brusk series of visited domains from ane subscriber can be sensitive. A pivotal moment in a user's life, for case, could generate the following log at the user'due south Isp (assuming the user hasn't invested in special privacy tools):

Over a longer period of time, metadata can paint a revealing motion picture almost a subscriber's habits and interests. As other policy discussions have made clear in recent years, metadata is very revealing over time. For example, in the context of telephony metadata, the President'south Review Group on Intelligence and Communications Technologies found that "the record of every telephone call an individual makes or receives over the course of several years can reveal an enormous amount about that individual'southward private life." The Grouping went on to note that "[i]n a world of always more circuitous engineering, it is increasingly unclear whether the stardom between 'meta-data' and other information carries much weight."

This reasoning applies with equal strength to domain names, which nosotros believe are likely to be fifty-fifty more than revealing than telephone records. Such a list of domains could also indicate the presence of various "smart" devices in the subscriber'south habitation, based on the known domains that these devices automatically connect to.

3. Encrypted Internet traffic itself can exist surprisingly revealing.

Encryption stops ISPs from simply reading content and URL data straight off the wire. However, information technology is important to understand that encryption nonetheless leaves open up a wide variety of other, less direct methods for ISPs to monitor their users if they chose.

A growing body of computer science enquiry demonstrates that a network operator tin can learn a surprising amount nearly the contents of encrypted traffic without breaking or weakening encryption. By examining the features of the traffic — like the size, timing and destination of the encrypted packets — information technology is possible to uniquely place sure web page visits or otherwise reveal information about what those packets likely incorporate. In the technical literature, inferences reached in this way are called "side channel" information.

Some of these methods are already in use in the field today: in countries that conscience the Net, regime regime are able to identify and disrupt targeted data access based on its secondary traits even when admission is encrypted. Concerningly, such nations often rely on Western technology vendors, whose advanced products allow censors increasingly to analyze and human action on traffic at "line speed" (that is, in real fourth dimension, as the information passes through a network).

The side channel methods that we describe below are likely not used (or at to the lowest degree not widely used) by ISPs today. Just equally encryption spreads, these techniques might go much more compelling. Policymakers should have a clear understanding of what's possible for ISPs to larn, both now and in the futurity.

Identifying specific sites and pages. Web site fingerprinting is a well-known technique that allows an ISP to potentially identify the specific encrypted web folio that a user is visiting. This technique leverages the fact that different spider web sites take unlike features: they transport differing amounts of content, and they load unlike 3rd-political party resources, from different locations, in dissimilar orders. Past examining these features, it'southward oftentimes possible to uniquely identify the specific web page that the user is accessing, despite the utilise of strong encryption when the web site is in transit.

Researchers have published numerous studies on the topic of spider web site fingerprinting. In one early study using a relatively basic technique, researchers plant that approximately 60% of the spider web pages they studied were uniquely identifiable based on such unconcealed features. Later on studies take introduced more avant-garde techniques, also as possible countermeasures. Only fifty-fifty with various defenses in place, researchers were yet able to distinguish precisely which out of a hundred different sites a user was visiting, more than 50% of the time.

This body of research illustrates that decrypting a communication isn't necessarily the only way to "see" information technology. The Swire newspaper asserts that "[w]ith encrypted content, ISPs cannot see detailed URLs and content fifty-fifty if they attempt." To be fully accurate, however, that claim requires qualification: ISPs generally cannot decrypt detailed URLs and content. But, this grade of inquiry demonstrates that with some amount of effort, it would indeed be feasible for ISPs to learn detailed URLs (and through those URLs, in some instances, the bodily content of web pages) in a range of real-earth situations.

The autocomplete feature on Google's search engine.

Deriving search queries. Popular search engines — like Google, Yahoo and Bing —provide a convenient feature chosen auto-advise: afterward the user enters each graphic symbol, the search engine suggests a list of popular search queries that match the current prefix, in an attempt to guess what the user is searching for. By analyzing the distinctive size of these encrypted suggestion lists that are transmitted after each primal press, researchers were able to deduce the private characters that the user typed into the search box, which together reveal the user'south unabridged search query.

Inferring other "subconscious" content. Researchers have practical similar methods to infer the medical condition of users of a personal wellness web site, and the annual family income and investment choices of users of a leading financial web site — even though both of those sites are simply reachable via encrypted, HTTPS connections. (Again, the researchers obtained these results without decrypting the encrypted traffic.) Other researchers of side-channel methods have been able to reconstruct portions of encrypted VoIP conversations, and user actions from within encrypted Android apps.

Such examples take led researchers to conclude that side-channel information leaks on the web are "a realistic and serious threat to user privacy." These types of leaks are often difficult or expensive to forestall. At that place has been pregnant computer science research into applied defenses to defeat these side-channel methods. But equally one grouping of researchers concluded, "in the context of website identification, it is unlikely that bandwidth-efficient, general-purpose [traffic analysis] countermeasures can ever provide the type of security targeted in prior piece of work."

These methods are in the lab today — not still in the field, as far as we know. But the path from estimator science research to widespread deployment of a new applied science tin exist short.

4. VPNs are poorly adapted, and tin provide incomplete protection.

1 manner that subscribers tin protect their Internet traffic in transit is to employ a virtual private network (VPN). VPNs are ofttimes constitute in business concern settings, enabling employees who are away from the function to connect securely over the Internet to their company'south internal network (often with setup help from the employer's Information technology department). When using a VPN, the user's computer establishes an encrypted tunnel to the VPN server (say, the ane operated by the employee'southward visitor) so, depending on the VPN configuration, sends some or all of the user's Net traffic through the encrypted tunnel.

The Swire paper presents VPNs (and other encrypted proxy services) as an upwards-and-coming source of protection for subscribers. Withal, there are reasons to question whether VPNs will in fact have a pregnant impact on personal Internet employ in the United States.

U.S. subscribers rarely brand personal utilize of VPNs. VPNs have been commercially bachelor for years, but they are used sparsely in the The states. According to a 2014 survey cited by the Swire paper, only 16% of N American users have used a VPN (or a proxy service) to connect to the Internet. This figure describes the percent of users who have ever used a VPN or a proxy before — not those who use such services on a consistent or daily ground, which is what protection from persistent Isp monitoring would actually require. Moreover, many of the sixteen% of users who accept used a VPN are likely business organisation users, rather than personal users looking to protect their privacy. It is off-white to conclude that only a very modest number of U.Southward. users actually utilize a VPN or proxy service on a consistent basis for personal privacy purposes.

Moreover, several adoption hurdles are likely to deter unsophisticated users. Reliable VPNs can be costly, requiring an additional paid monthly subscription on top of the user's Internet service. They also ho-hum down the user'due south Net speeds, since they road traffic through an intermediate server. (There are free VPN services available, but subscribers more often than not get what they pay for.)

Relative to other countries, the charge per unit of VPN use in the U.South. is among the lowest in the world. VPN use is much more than pronounced in other countries like Republic of indonesia, Thailand and China, where Net users turn to VPNs a manner to circumvent online censorship, and to actively proceeds access to restricted content.

VPNs are not a privacy silver bullet. The use of VPNs and encrypted proxies simply shifts user trust from one intermediary (the ISP) to another (the VPN or proxy operator). In gild to more thoroughly protect their traffic from their ISP, a subscriber must entrust that same traffic to some other network operator.

Furthermore, VPNs may non protect users equally well every bit the Swire paper might lead readers to believe. The newspaper states that "Where VPNs are in place, the ISPs are blocked from seeing . . . the domain proper name the user visits." Only this is not e'er truthful: whether ISPs can see the domain names that users visit depends entirely on the user'southward VPN configuration — and information technology would be quite hard for non-experts to tell whether their configuration is properly tunneling their DNS queries, let lone to know that this is a question that needs to be asked. This is particularly mutual for Windows users.

Decision

Today, ISPs tin can see a significant corporeality of their subscribers' Internet activity, and have the ability to infer substantial amounts of sensitive information from information technology. This is especially true when that traffic is unencrypted. However, even when Net traffic is encrypted using HTTPS, ISPs mostly retain visibility into their subscribers' DNS queries. Detailed analysis of DNS query information on a per-subscriber basis is not only technically feasible and cost-constructive, just actually takes place in the field today. Moreover, ISPs and the vendors that serve them have articulate opportunities to develop methods of inferring of import data fifty-fifty from encrypted data flows. VPNs are ane tool that subscribers can employ to protect their online activities, just VPNs are poorly adopted, can be difficult to use, and ofttimes provide incomplete protections.

We hope that this report will contribute to a more consummate understanding of the technical capabilities of broadband ISPs, and the broader technical context within which the broadband privacy debate is happening.

About This Written report

This report is designed to provide technical grounding for policymakers and other interested parties, regarding the extent of Internet access provider visibility into the activities of their subscribers.

The report aims to provide technical information but, and is not intended to have a position on whatever thing of public policy.

Readers who identify whatsoever factual errors in this study, or who accept other feedback regarding its contents, are warmly invited to contact us at hello@teamupturn.com. This written report was supported past the Media Democracy Fund.