This article was originally published by Pro Publica in 2013. Although much additional information has since surfaced it still offers an overview helpful in understanding the mission and work of the National Security Agency.

What information does the NSA collect and how?

A record of most calls made in the U.S., including the telephone number of the phones making and receiving the call, and how long the call lasted. This information is known as “metadata” and doesn’t include a recording of the actual call (but see below). This program was revealed through a leaked secret court order instructing Verizon to turn over all such information on a daily basis. Other phone companies, including AT&T and Sprint, also reportedly give their records to the NSA on a continual basis. All together, this is several billion calls per day.

Email, Facebook posts and instant messages for an unknown number of people, via PRISM, which involves the cooperation of at least nine different technology companies. Google, Facebook, Yahoo and others have denied that the NSA has “direct access” to their servers, saying they only release user information in response to a court order. Facebook has revealed that, in the last six months of 2012, they handed over the private data of between 18,000 and 19,000 users to law enforcement of all types -- including local police and federal agencies, such as the FBI, Federal Marshals and the NSA.

Massive amounts of raw Internet traffic The NSA intercepts huge amounts of raw data, and stores billions of communication records per day in its databases. Using the NSA’s XKEYSCORE software, analysts can see “nearly everything a user does on the Internet” including emails, social media posts, web sites you visit, addresses typed into Google Maps, files sent, and more. Currently the NSA is only authorized to intercept Internet communications with at least one end outside the U.S., though the domestic collection program used to be broader. But because there is no fully reliable automatic way to separate domestic from international communications, this program also captures some amount of U.S. citizens’ purely domestic Internet activity, such as emails, social media posts, instant messages, the sites you visit and online purchases you make.

The contents of an unknown number of phone calls There have been several reports that the NSA records the audio contents of some phone calls and a leaked document confirms this. This reportedly happens “on a much smaller scale” than the programs above, after analysts select specific people as “targets.” Calls to or from U.S. phone numbers can be recorded, as long as the other end is outside the U.S. or one of the callers is involved in "international terrorism". There does not seem to be any public information about the collection of text messages, which would be much more practical to collect in bulk because of their smaller size.

The NSA has been prohibited from recording domestic communications since the passage of the Foreign Intelligence Surveillance Act but at least two of these programs -- phone records collection and Internet cable taps -- involve huge volumes of Americans’ data.

Does the NSA record everything about everyone, all the time?

The NSA records as much information as it can, subject to technical limitations (there’s a lot of data) and legal constraints. This currently includes the metadata for nearly all telephone calls made in the U.S. (but not their content) and massive amounts of Internet traffic with at least one end outside the U.S. It’s not clear exactly how many cables have been tapped, though we know of at least one inside the U.S., a secret report about the program by the NSA’s Inspector General mentions multiple cables, and the volume of intercepted information is so large that it was processed at 150 sites around the world as of 2008. We also know that Britain’s GCHQ, which shares some intelligence with the NSA, had tapped over 200 cables as of 2012, belonging to seven different telecommunications companies.           

Until 2011 the NSA also operated a domestic Internet metadata program which collected mass records of who emailed who even if both parties were inside the U.S.

Because it is not always possible to separate domestic from foreign communications by automatic means, the NSA still captures some amount of purely domestic information, and it is allowed to do so by the Foreign Intelligence Surveillance Court.

The collected information covers “nearly everything a user does on the Internet,” according to a presentation on the XKEYSCORE system. The slides specifically mention emails, Facebook chats, websites visited, Google Maps searches, transmitted files, photographs, and documents of different kinds. It’s also possible to search for people based on where they are connecting from, the language they use, or their use of privacy technologies such as VPNs and encryption, according to the slides.

This is a massive amount of data. The full contents of intercepted Internet traffic can only be stored for up to a few days, depending on the collection site, while the associated “metadata” (who communicated with whom online) is stored up to 30 days. Telephone metadata is smaller and is stored for five years. NSA analysts can move specific data to more permanent databases when they become relevant to an investigation.

The NSA also collects narrower and more detailed information on specific people, such as the actual audio of phone calls and the entire content of email accounts. NSA analysts can submit a request to obtain these types of more detailed information about specific people.

Watching a specific person like this is called “targeting” by the Foreign Intelligence Surveillance Act, the law which authorizes this type of individual surveillance. The NSA is allowed to record the conversations of non-Americans without a specific warrant for each person monitored, if at least one end of the conversation is outside of the U.S. It is also allowed to record the communications of Americans if they are outside the U.S. and the NSA first gets a warrant for each case. It’s not known exactly how many people the NSA is currently targeting, but according to a leaked report the NSA intercepted content from 37,664 telephone numbers and email addresses from October 2001 to January 2007. Of these, 8% were domestic: 2,612 U.S. phone numbers and 406 U.S. email addresses.

How the NSA actually gets the data depends on the type of information requested. If the analyst wants someone's private emails or social media posts, the NSA must request that specific data from companies such as Google and Facebook. Some technology companies (we don't know which ones) have FBI monitoring equipment installed "on the premises" and the NSA gets the information via the FBI's Data Intercept Technology Unit. The NSA also has the capability to monitor calls made over the Internet (such as Skype calls) and instant messaging chats as they happen.

For information that is already flowing through Internet cables that the NSA is monitoring, or the audio of phone calls, a targeting request instructs automatic systems to watch for the communications of a specific person and save them.

It’s important to note that the NSA probably has information about you even if you aren’t on this target list. If you have previously communicated with someone who has been targeted, then the NSA already has the content of any emails, instant messages, phone calls, etc. you exchanged with the targeted person. Also, your data is likely in bulk records such as phone metadata and Internet traffic recordings. This is what makes these programs “mass surveillance,” as opposed to traditional wiretaps, which are authorized by individual, specific court orders.

What does phone call metadata information reveal, if it doesn’t include the content of the calls?

Even without the content of all your conversations and text messages, so-called “metadata” can reveal a tremendous amount about you. If they have your metadata, the NSA would have a record of your entire address book, or at least every person you’ve called in the last several years. They can guess who you are close to by how often you call someone, and when. By correlating the information from multiple people, they can do sophisticated “network analysis” of communities of many different kinds, personal or professional -- or criminal.

Phone company call records reveal where you were at the time that a call was made, because they include the identifier of the radio tower that transmitted the call to you. The government has repeatedly denied that it collects this information, but former NSA employee Thomas Drake said they do. For a sense of just how powerful location data can be, see this visualization following a German politician everywhere he goes for months, based on his cellphone’s location information.

Even without location data, records of who communicated with whom can be used to discover the structure of groups planning terrorism. Starting from a known "target" (see above), analysts typically reconstruct the social network "two or three hops" out, examining all friends-of-friends, or even friends-of-friends-of-friends, in the search for new targets. This means potentially thousands or millions of people might be examined when investigating a single target.

Metadata is a sensitive topic because there is great potential for abuse. While no one has claimed the NSA is doing this, it would be possible to use metadata to algorithmically identify, with some accuracy, members of other types of groups like the Tea Party or Occupy Wall Street, gun owners, undocumented immigrants, etc. An expert in network analysis could start with all of the calls made from the time and place of a protest, and trace the networks of associations out from there.

Phone metadata is also not “anonymous” in any real sense. The NSA already maintains a database of the phone numbers of all Americans for use in determining whether someone is a “U.S. person” (see below), and there are several commercial number-to-name services in any case. Phone records become even more powerful when they are correlated with other types of data, such as social media posts, local police records and credit card purchase information, a process known as intelligence fusion.

Does the NSA need an individualized warrant to listen to my calls or look at my emails?

It’s complicated, but not in all cases. Leaked court orders set out the "minimization" procedures that govern what the NSA can do with the domestic information it has intercepted. The NSA is allowed to store this domestic information because of the technical difficulties in separating foreign from domestic communications when large amounts of data are being captured.

Another document shows that individual intelligence analysts make the decision to look at previously collected bulk information. They must document their request, but only need approval from their "shift coordinator." If the analyst later discovers that they are looking at the communications of a U.S. person, they must destroy the data.

However, if the intercepted information is “reasonably believed to contain evidence of a crime” then the NSA is allowed to turn it over to federal law enforcement. Unless there are other (still secret) restrictions on how the NSA can use this data this means the police might end up with your private communications without ever having to get approval from a judge, effectively circumventing the whole notion of probable cause.

This is significant because thousands or millions of people might fall into the extended social network of a single known target, but it is not always possible to determine whether someone is a U.S. person before looking at their data. For example, it’s not usually possible to tell just from someone’s email address, which is why the NSA maintains a database of known U.S. email addresses and phone numbers. Internal documents state that analysts need only “51% confidence” that someone is a non-U.S. person before looking at their data, and if the NSA does not have “specific information” about someone, that person is “presumed to be a non-United States person.”

Also, the NSA is allowed to provide any of its recorded information to the FBI, if the FBI specifically asks for it.

Is all of this legal?

Yes, assuming the NSA adheres to the restrictions set out in recently leaked court orders. By definition, the Foreign Intelligence Surveillance Court decides what it is legal for the NSA to do. But this level of domestic surveillance wasn’t always legal, and the NSA's domestic surveillance program has been found to violate legal standards on more than one occasion.

The NSA was gradually granted the authority to collect domestic information on a massive scale through a series of legislative changes and court decisions over the decade following September 11, 2001. See this timeline of loosening laws. The Director of National Intelligence says that authority for PRISM programs comes from section 702 of the Foreign Intelligence Surveillance Act and the Verizon metadata collection order cites section 215 of the Patriot Act. The author of the Patriot Act disagrees that the act justifies the Verizon metadata collection program.

The NSA's broad data collection programs were originally authorized by President Bush on October 4, 2001. The program operated that way for several years, but in March 2004 a Justice Department review declared the bulk Internet metadata program was illegal. President Bush signed an order re-authorizing it anyway. In response, several top Justice Department officials threatened to resign, including acting Attorney General James Comey and FBI director Robert Mueller. Bush backed down, and the Internet metadata program was suspended for several months. By 2007, all aspects of the program were re-authorized by court orders from the Foreign Intelligence Surveillance Court.

In 2009, the Justice Department acknowledged that the NSA had collected emails and phone calls of Americans in a way that exceeded legal limitations.

In October 2011, the Foreign Intelligence Surveillance Court ruled that the NSA violated the Fourth Amendment at least once. The Justice Department has said that this ruling must remain secret, but we know it concerned some aspect of the "minimization" rules the govern what the NSA can do with domestic communications. The Foreign Intelligence Surveillance Court recently decided that this ruling can be released, but Justice Department has not yet done so.

Civil liberties groups including the EFF and the ACLU dispute the constitutionality of these programs and have filed lawsuits to challenge them.

How long can the NSA keep information on Americans?

The NSA can generally keep intercepted domestic communications for up to five years. It can keep them indefinitely under certain circumstances, such as when the communication contains evidence of a crime or when it’s “foreign intelligence information,” a broad legal term that includes anything relevant to “the conduct of the foreign affairs of the United States.”

The NSA can also keep encrypted communications indefinitely. That includes any information sent to or from a secure web site, that is, a site with a URL starting with "https".

Does the NSA do anything to protect Americans’ privacy?

Yes. First, the NSA is only allowed to intercept communications if at least one end of the conversation is outside of the U.S. -- though it doesn't have to distinguish domestic from foreign communication until the "earliest practicable point" which allows the NSA to record bulk information from Internet cables and sort it out later. When the NSA discovers that previously intercepted information belongs to an American, it must usually destroy that information. Because this determination cannot always be made by computer, this sometimes happens only after a human analyst has already looked at it.

The NSA also must apply certain safeguards. For example, the NSA must withhold the names of U.S. persons who are not relevant to ongoing investigations when they distribute information -- unless that person’s communications contain evidence of a crime or are relevant to a range of national security and foreign intelligence concerns.

Also, analysts must document why they believe someone is outside of the U.S. when they ask for addition information to be collected on that person. An unknown number of these cases are audited internally. If the NSA makes a mistake and discovers that it has targeted someone inside the U.S., it has five days to submit a report to the Department of Justice and other authorities.

What if I’m not an American?

All bets are off. There do not appear to be any legal restrictions on what the NSA can do with the communications of non-U.S. persons. Since a substantial fraction of the world’s Internet data passes through the United States, or its allies, the U.S. has the ability to observe and record the communications of much of the world’s population. The European Union has already complained to the U.S. Attorney General.

The U.S. is hardly the only country doing mass surveillance, though its program is very large. GCHQ, which is the British counterpart to the NSA, has a similar surveillance program and shares data with the NSA. Many countries now have some sort of mass Internet surveillance now in place. Although passive surveillance is often hard to detect, more aggressive governments use intercepted information to intimidate or control their citizens, including Syria, Iran, Egypt, Bahrain and China. Much of the required equipment is sold to these governments by American companies.