Hello friends, FYI I've put my blood sweat and tears into this diary ... it occurred to me that non-technical people would not intuitively understand the enormous risks to national security that Hillary took by using a personal email account hosted on a private server. I'm part technologist and part teacher - so I tried to create something that non-techies could be able to read. Can I challenge you to forward the URL for this essay to five people that you can think of who might find it useful? And then challenge them to do the same? National secrets are not something to be trifled with. But most people don't really understand exactly what Hillary even did. Used a personal email account? That doesn't sound so horrific ...

FYI, the focus of the ongoing FBI investigation is on security; when a sample of 40 of her emails were examined, four were found to contain classified information.

In March of 2015, a controversy began when the public technically savvy members of the public learned that

1. Hillary Clinton had used a personal email account for official business while she served as Secretary of State, and
2. the personal email account had been hosted on a private server that she owned, and was located in her residence.

What about members of the public who are not so technically savvy? Were they troubled by this news? Probably not. I think I know why. Do the words “hosted on a private server” sound especially menacing to you? No? If anything, they sound more than a tad boring.

Here’s the thing. At it’s heart, the controversy has to do with a number of issues, including national security. National security! Let’s talk strictly in hypotheticals for a moment. Suppose those at high levels in the government were — intentionally or unintentionally — habitually sloppy about keeping sensitive, classified material safe from our enemies … is that boring? Now imagine that those very same folks were using their clout to intimidate and silence underlings who tried to “plug up some of the security holes”, so to speak … is that boring? In Tinker Tailor Soldier Spy, and other spy movies, the plot often includes treasonous government officials selling secrets to the “other side”. National secrets. Secrets that affect national security. Are those secrets boring? Are you kidding me? This stuff is not boring. Anyone who thinks that this topic IS boring clearly must not quite understand what everyone else is talking about.

Some people are geeks. Some are not. The latter aren’t bad people or anything, they just lack certain knowledge. I want to take some time and explain IMPORTANT BUT GEEKY concepts, so that well-intentioned laymen can understand what all of the fuss is about. My goal is to explain the concepts, not to force anyone to agree with my views. I want folks to be able to understand the facts; I think the facts rather speak for themselves. But I do have opinions, and those will undoubtedly leak out in my writing below.

THE OIG REPORT (MAY 2016)

A watchdog group within the State Department launched a project one month after the controversy began (April 2015) to gather information, including

• the practices of other Secretaries of State, and
• the relevant laws and/or government policies that were in effect during various administrations

The group, a.k.a. the Office of the Inspector General (OIG), recently completed their work and published a set of reports to document their findings. The fourth and final report, Office of the Secretary: Evaluation of Email Records Management and Cybersecurity Requirements, was issued on May 26 to address

“efforts undertaken by the Department of State (Department) to preserve and secure electronic records and communications involving the Office of the Secretary” (p.4)

A close reading of the title reveals there are not one but two different areas of concern regarding Clinton’s email practices:

1. Email records management (that is, preserve electronic records), and
2. Cybersecurity (that is, secure electronic records)

Yikes, we are not even past the title and mucho geekiness has already emerged. Records management? Cybersecurity? WTF? Let me provide some quick translations. The TWO primary concerns about Clinton’s email practices that were researched and summarized in the OIG report are:

1. Did Clinton fail to take steps to ensure that the public can review the work she was doing as Secretary of State?

   The public has a right to request access to “records” that document the work of any federal agency. In addition, the head of every federal agency (such as the State Department) has the responsibility to create and preserve those records.

   The infrastructure that Clinton used appears to diminish the ability of the public to review her work, since it was essentially “off the grid”.

   Furthermore, the OIG report shows that Bryan Pagliano, the former State Department IT specialist who provided technical support to Clinton (and who was hired in 2009 in an unusual manner as a political appointee, after having worked as an IT director for Clinton's 2008 presidential campaign) was providing that support without the knowledge of his direct supervisors. (p. 39)

2. Were any risks being taken with classified material?

   In other words, was State Department information stored on her server safe and secure at all times (from the prying eyes of hackers, enemies of the state, and anyone else who was not authorized to view it)?

While both of these concerns are important, in my view the second one is the most critical. The second one also happens to be

• the one that many laymen seem not to grasp
• the focus of the investigation being conducted by the FBI

The basic idea: when American intelligence falls into the wrong hands, American national security is compromised. The position of Secretary of State by its very nature is one that involves large amounts of sensitive data. If any of it happened to creep into Clinton’s email (we now know that it did), and that material was not “secure and protected from threats” (p. 26), then … the bad guys might have seen it. This is bad, very bad, for it means that any national secrets that were discussed on Clinton’s emails were rather poorly kept secrets.

Note: Both former Defense Secretary Robert Gates and former acting CIA Director Michael Morell have stated publicly that they believe a number of foreign countries (possibly including Iran, China, or Russia) may have hacked into and seen the contents of Clinton’s private server.

EMAIL ACCOUNT USAGE BY SECRETARIES OF STATE FOR THE PAST TWENTY YEARS

When reviewing Clinton’s email practices, it is helpful to compare and contrast them to those of other recent Secretaries of State. The chart below summarizes various findings from the OIG report.

NOTES:

1. Two other Secretaries have had private email accounts and used them to conduct official business: Colin Powell and John Kerry. Neither of these email accounts was hosted on a private server that they owned and maintained, however. According to the OIG report, Powell was not diligent about preserving his emails (p. 21), but Kerry is meeting current requirements to preserve email records. (p.25)
2. Other than Clinton, no other Secretaries have used a private email server that they owned and maintained for official business of the State Department.
3. Other than Clinton, no other Secretaries worked in an environment that lacked a permanent Inspector General in the State Department.

WSJ- State Department Lacked Top Watchdog During Hillary Clinton Tenure

The State Department had no permanent inspector general—the lead watchdog charged with uncovering misconduct and waste—during Hillary Clinton’s entire tenure as secretary, leaving in place an acting inspector who had close ties to State Department leadership. [...]

Five months after Mrs. Clinton left office, Mr. Obama nominated a permanent inspector general, who was confirmed by the Senate three months later.

The lack of a confirmed inspector general raises questions about oversight of the department under Mr. Obama and Mrs. Clinton. The department has been criticized for its failure to gather and archive the email records of Mrs. Clinton and other officials and for responses to public-record requests that lawmakers and advocacy groups say were insufficient, including its response to requests for information from a congressional panel investigating the 2012 terror attack in Benghazi, Libya. [...]

The White House declined to elaborate on reason for the lack of an appointment.

AREA OF CONCERN — CYBERSECURITY (SECURE ELECTRONIC RECORDS)

One area that the OIG report focused on was “the use of non-Departmental systems to conduct official business.” The big question: was the information related to said business safe when it was on Clinton’s private server?

“In addition to complying with records management and preservation requirements, Department employees, including those in the Office of the Secretary, must comply with cybersecurity policies. Department information must be secure and protected from threats.” (p. 26)

Since 1996, State Department policies have contained provisions enacted in the interest of cybersecurity (also known as computer security). These policies have evolved over time.

Cybersecurity is the set of activities that are required to keep the hardware, software, and information on all components of a computer system safe from unauthorized persons. It includes safeguards to control both physical access and also remote access via the Internet. Protection includes preventing intruders from reading and/or modifying any portion of the system.

To understand potential security issues regarding Clinton’s email practices, one really needs to first understand “how email works”. Below I provide a highly simplified explanation which provides a very limited set of details.

In the images below, the top half shows a real world, snail-mail example; the bottom half shows the equivalent situation for email.

In the real world, a person might compose a letter at a desk in their house. In order to mail it, the letter needs to be taken to the Post Office. The mail carrier usually performs this service.

With email, a person will compose a message using an Email Client. In order to “mail” it, the message needs to be sent to an Email Server, which is a (usually dedicated) machine that hosts processes that are equivalent to those of a Post Office. The job of the processes running on this machine is to accept incoming messages and then take the appropriate steps to send them out on the Internet to where they need to go. An Email Client is usually connected to an Email Server via a Local Area Network (LAN) or the Internet; when the user presses the “send” button, the message is then transported from the Email Client (a program on a machine) to the Email Server (a program on a (usually different) machine).

The next step in the process is invisible to most of us. In the real world, the Post Office that has the letter now takes the appropriate action to send the letter to a (usually different) Post Office that is closer to where the recipient lives.

With email, a program running on the first Email Server reads the distribution list, makes a copy of the message for each address on the list, makes a call to a DNS server to convert each user-friendly address on the email to an IP address, and then sends a copy of the message to each IP address. Note: each IP address is actually the network location of a computer that is configured as an Email Server. If someone sent a message to hdr22@clintonemail.com (the address that Clinton used for official business as Secretary of State), the call to DNS would return the IP address of the computer that was located in Clinton’s basement. When a personal email account is “hosted on a private server,” it means that messages sent to that email account are sent to a computer that is (usually) owned and maintained by the owner of the email account.

In the final step, the destination Post Office receives the letter and takes action to deliver the letter to a mailbox associated with that particular person (either a PO box at the post office or a mailbox where the recipient resides). The recipient must go to the mailbox to retrieve the letter, and then they can read it.

With email, when a program running on the destination Email Server receives the message, it puts that message into a mailbox that resides on the Email Server. The next time that the user runs their email program on the Email Client, that program automatically checks the mailbox on the Email Server and downloads any new messages that are found for the user to their Email Client. The Email Client now lists the new message, and the user is able to read it using their email program.

Congratulations! You have now completed Email 101! If you are interested in setting up your own personal email account hosted on your own private server, just follow the easy directions below!

How to Set Up a Clinton-Style Home Email Server

Setting up a server is no simple task. “It’s a pretty big job to maintain a server like that and make sure it’s properly configured,” says Peter Firstbrook, an Internet security researcher at Gartner. Firstbrook says such an endeavor is “highly unusual.” He has not heard of any companies whose executives had set up personal servers for work emails, let alone government officials.

To set a personal email server, someone would need to:

• Buy a server, which is about the size of a desktop computer.
• Buy an operating system to run the server, most likely a version of Microsoft Windows or Linux.
• Buy an exchange program to manage the flow of emails (Microsoft Exchange Server is the most common).
• Buy a digital certificate to certify that the server has been encrypted.
• Buy a domain name (in this case, clintonemail.com).
• Install the software.
• Install virus and spam filters.
• Set up firewalls, including a message-transfer agent, an email-specific firewall.
• Get a business-class Internet connection—a regular consumer connection likely isn’t reliable enough.
• Configure the devices using the server, such as Clinton’s BlackBerry.

For any email account, the hosting provider has the responsibility for maintaining the machines that act as Email Servers; they must provide

• sufficient security to protect the machine and the information stored on it.
• day to day technical support and also upgrades as necessary.

For email accounts that are hosted on a *.gov server, an IT organization with the federal government is responsible for security and technical support.

For email accounts that are hosted by a provider such as AOL or Google, that hosting provider is responsible for security and technical support.

For personal email accounts hosted on a private server, the owner of the server is responsible for security and technical support.

Now that you are an expert on how email works, let go back and review:

• An Email Server is a computer that has a special relationship to one or more Email Clients.
• Setting up a private Email Server is no simple task. It is much more difficult than going to google or yahoo and requesting an email account from them.
• Hillary Clinton was the only Secretary of State who used a personal email account hosted on a private email server that she owned and maintained.

You also know that If someone sent a message to hdr22@clintonemail.com,

• the Email Server for the sender would translate that email address to an IP address (like 208.91.197.27), and then send the message to the IP address — which happened to point to a computer that lived in Clinton’s basement
• the message would be stored at least temporarily on Clinton’s private Email Server (i.e. 208.91.197.27)

Here’s the thing ... persons who are called “hackers,” “spys”, “cyber terrorists,” etc., are also experts on how email works. They know much, much more than has been shown here, including how to “hack into” or “attack” any computer that is accessible on the Internet. That is why most everyone installs firewalls and anti-virus software on their home computers hooked up to the Internet: some hackers enjoy setting traps that will damage the computers of those who run into them. If one doesn’t take steps to safeguard their system, it is vulnerable and most likely will be attacked.

Think about it: if so many Email Servers are able to translate hdr22@clintonemail.com into an IP address, the “bad guys” can translate it too. They know which machine Clinton is using. They also know that as a Secretary of State, she is routinely involved in situations that involve “state secrets” that might be extremely valuable to know. It’s almost as if the bad guys had been handed a map to buried treasure, with a big red “X” marking the spot.

The government and global corporations understand the importance of cybersecurity and hire experts to review their computer systems and take appropriate action to keep those systems safe. The experts monitor for possible cyber attacks; and if/when those occur, they move quickly to understand the vulnerabilities and eliminate them.

In contrast, Clinton’s personal email account was a late addition to a home-grown solution that had already existed for years and had not been designed with security as a top priority. The OIG report indicates that she did not ask for a review to ensure that it was adequate for her official business, but she had an obligation to do so with Diplomatic Security.

OIG found no evidence that the Secretary requested or obtained guidance or approval to conduct official business via a personal email account on her private server. According to the current CIO and Assistant Secretary for Diplomatic Security, Secretary Clinton had an obligation to discuss using her personal email account to conduct official business with their offices, who in turn would have attempted to provide her with approved and secured means that met her business needs. However, according to these officials, DS and IRM did not—and would not—approve her exclusive reliance on a personal email account to conduct Department business, because of the restrictions in the FAM and the security risks in doing so.

During Secretary Clinton’s tenure, the FAM also instructed employees that they were expected to use approved, secure methods to transmit SBU information and that, if they needed to transmit SBU information outside the Department’s OpenNet network on a regular basis to non- Departmental addresses, they should request a solution from IRM. However, OIG found no evidence that Secretary Clinton ever contacted IRM to request such a solution, despite the fact that emails exchanged on her personal account regularly contained information marked as SBU. (p. 37)

*DS - Bureau of Diplomatic Security
*IRM - Bureau of Information Resource Management
*FAM - Foreign Affairs Manual
*SBU - sensitive but unclassified

Technical support for Clinton’s private Email Server was provided by two persons with questionable expertise in security:

• “an individual based in New York who provided technical support for Secretary Clinton’s personal email system but who was never employed by the Department” (p.2)

  Justin Cooper is a longtime adviser to former President Bill Clinton, and registered the clintonemail.com domain name on January 13, 2009.

• “a Special Advisor to the Deputy Chief Information Officer (2009-13) who provided technical support for Secretary Clinton’s personal email system” (p.2)

  Bryan Pagliano worked as an IT director for Clinton's 2008 presidential campaign, and then was hired in 2009 by the state department as a political appointee in what the HR manager described as “not a traditional supervisor/employee relationship.” Turns out that Pagliano was providing technical support to Clinton without the knowledge of his direct supervisors, who

  • “believed that [his] job functions were limited to supporting mobile computing issues across the entire Department” (p. 39)
  • “did not know he was providing ongoing support to the Secretary’s email system during working hours” (p. 39)

Several hacking incidents are described in the OIG report. Instead of inviting IT experts in to help understand the vulnerabilities and eliminate them, the incidents were apparently not reported to anyone else in the State Department.

Department policy requires employees to report cybersecurity incidents to IRM security officials when any improper cyber-security practice comes to their attention. 12 FAM 592.4 (January 10, 2007). Notification is required when a user suspects compromise of, among other things, a personally owned device containing personally identifiable information. 12 FAM 682.2-6 (August 4, 2008). However, OIG found no evidence that the Secretary or her staff reported these incidents to computer security personnel or anyone else within the Department. (p.40)

Finally … you know the FBI investigation we keep hearing about? As it turns out, the focus of the FBI investigation is on security:

WP - FBI looking into the security of Hillary Clinton’s private e-mail setup

The FBI has begun looking into the security of Hillary Rodham Clinton’s private e-mail setup, contacting in the past week a Denver-based technology firm that helped manage the unusual system, according to two government officials.

Turns out the investigation was initiated after the Inspector General from the intelligence community, I. Charles McCullough III,

found information that should have been designated as classified in four e-mails out of a “limited sample” of 40 that his agency reviewed. As a result, he said, he made the “security referral,” acting under a federal law that requires alerting the FBI to any potential compromises of national security information.

“The main purpose of the referral was to notify security officials that classified information may exist on at least one private server and thumb drive that are not in the government’s possession,” McCullough said in a statement, which was also signed by the State Department’s inspector general, Steve A. Linick. — link

Now let’s move on to the other area of concern in the OIG report ...

AREA OF CONCERN — EMAIL RECORDS MANAGEMENT (PRESERVE ELECTRONIC RECORDS)

The OIG report also focused on “records preservation requirements” and “Freedom of Information Act (FOIA) compliance.” The big question: by using for official business an “off the grid” personal email account which was hosted on a private server that she owned and maintained, did Clinton fail to create and preserve sufficient documentation about said business as proscribed by relevant laws and policies? The most important of these are

• the Federal Records Act and
• the Freedom of Information Act (FOIA)

The head of each federal agency is required to make and preserve records that document the work of the agency. One reason for this requirement is to satisfy requests from the public made under FOIA. When records are not properly created and preserved, citizens lose the ability to review the activities of their government. “As Congress, the President, and the Supreme Court have all recognized, the FOIA is a vital part of our democracy.”
When President Obama was sworn in, he made a beautiful speech about the importance of FOIA and transparency in government, and made a pledge that “Transparency and the rule of law will be the touchstones of this presidency.”

The way to make government responsible is to hold it accountable. And the way to make government accountable is to make it transparent so that the American people can know exactly what decisions are being made, how they're being made, and whether their interests are being well served.

The directives I am giving my administration today on how to interpret the Freedom of Information Act will do just that. For a long time now there's been too much secrecy in this city. The old rules said that if there was a defensible argument for not disclosing some thing to the American people, then it should not be disclosed. That era is now over. Starting today, every agency and department should know that this administration stands on the side not of those who seek to withhold information, but those who seek to make it known.

To be sure, issues like personal privacy and national security must be treated with the care they demand. But the mere fact that you have the legal power to keep something secret does mean you should always use it. The Freedom of Information Act is perhaps the most powerful instrument we have for making our government honest and transparent, and of holding it accountable. And I expect members of my administration not simply to live up to the letter but also the spirit of this law.

The Federal Records Act provides “the legal framework for federal records management, including record creation, maintenance, and disposition.”

The Federal Records Act requires the head of each agency to “make and preserve records containing adequate and proper documentation of the organization, functions, policies, decisions, procedures, and essential transactions of the agency and designed to furnish the information necessary to protect the legal and financial rights of the Government and of persons directly affected by the agency’s activities.” Effective records management is critical for ensuring that sufficient documentation of an agency’s business is created, that an agency can efficiently locate and retrieve records needed in the daily performance of its mission, and that records of historical significance are identified, preserved, and made available to the public. (p. 2-3)

When the act was passed back in 1950, the personal computer had not yet been invented and email did not exist; complying with the Federal Records Act primarily involved making and storing copies of relevant papers. Email communications are a different kind of beast, nevertheless, the Federal Records Act still applies. Records must be created and preserved for all events that meet the definitions specified in the Act, even those that originated on a medium other than paper.

“Although emails were not explicitly mentioned in the Federal Records Act or FAM until the mid-1990s, the law has stated since 1943 that a document can constitute a record ‘regardless of physical form or characteristics.’ “ (p. 4)

Key point: Records management requirements have always applied to emails exchanged on personal email accounts, provided their content meets the definition of a record. In addition, for the last two decades, both Department of State policy and Federal regulations have explicitly stated that emails may qualify as Federal records.

Key point: When official business is conducted on a private email account, it is highly unlikely that appropriate records are either made or preserved for that business. Thus the requirements of the Federal Records Act and FOIA are not met, so citizens rights under FOIA are violated.

FINAL THOUGHTS

My goal was to explain technical and other concepts so that well-intentioned laymen can understand the concerns that are in the news these days regarding Hillary’s use of a personal email account that was hosted on a private server that she owned. Hopefully certain buzzwords are now more meaningful and the OIG report doesn’t sound quite as “cryptic”.

To review: OIG published a report on May 26 called, Office of the Secretary: Evaluation of Email Records Management and Cybersecurity Requirements. It was the fourth and final report in a set of reports that were published. FYI, the names of the other reports can be found on page 1 of the recent document.

A close reading of the title reveals there are not one but two different areas of concern regarding Clinton’s email practices:

1. Email records management

   In other words: Did Clinton fail to take steps to ensure that the public can review the work she was doing as Secretary of State?

   The public has a right to request access to “records” that document the work of any federal agency. In addition, the head of every federal agency (such as the State Department) has the responsibility to create and preserve those records.

2. Cybersecurity

   In other words: Were any risks being taken with classified material?
   
   Was classified State Department information stored on Clinton’s server safe and secure at all times?

   Note: the focus of the ongoing FBI investigation is on security; when a sample of 40 of her emails were examined, four were found to contain classified information.

The title of the OIG report also indicates it is an evaluation. OIG also conducts audits, inspections, and investigations regarding programs and operations of the State Department. The objective of an OIG evaluation is to provide a set of recommendations that will improve the efficiency and effectiveness of the State Department.

• Note: the objective of an evaluation does not include rendering legal judgement about the guilt or innocence guilt of any party, therefore the lack of such a verdict in this report is not surprising. The OIG report does not make any explicit remarks about Clinton being “innocent” or “guilty” (that I have found); the lack of such remarks does not imply any verdict.

Also note that as part of this project, OIG interviewed dozens of former and current Department employees, including Madeleine Albright, Colin Powell, and Condoleezza Rice, and John Kerry. OIG also requested an interview with Clinton; however, that request was declined.

In conjunction with the interviews, OIG reviewed paper and electronic records and documents associated with these offices. OIG also consulted with NARA officials. Finally, OIG interviewed Secretary Kerry and former Secretaries Albright, Powell, and Rice. Through her counsel, Secretary Clinton declined OIG’s request for an interview.

In addition to Secretary Clinton, eight former Department employees declined OIG requests for interviews: (p. 2)

1. “the Chief of Staff to Secretary Powell (2002-05);”
   Lawrence Wilkerson
2. “the Counselor and Chief of Staff to Secretary Clinton (2009-13);”
   Cheryl Mills
3. “the Deputy Chief of Staff for Policy to Secretary Clinton (2009-11) and the Director of Policy Planning (2011-13);”
   Jake Sullivan
4. “the Deputy Chief of Staff for Operations to Secretary Clinton (2009-13);
   Huma Abedin
5. “the Deputy Assistant Secretary for Strategic Communication (2009-13);
   Philippe Reines
6. “the Director of the S/ES Office of Information Resources Management (2008-13);
   John Bentel
7. “a Special Advisor to the Deputy Chief Information Officer (2009-13) who provided technical support for Secretary Clinton’s personal email system;
   Bryan Pagliano
8. “a Senior Advisor to the Department, who supervised responses to Congressional inquiries (2014-15).
   Heather Samuelson (determined which of Clinton's emails to delete in late 2014)

Two additional individuals did not respond to OIG interview requests: (p. 2)

• "the Deputy Secretary of State for Management and Resources (2011-13)"
  Thomas Nides
• "an individual based in New York who provided technical support for Secretary Clinton’s personal email system but who was never employed by the Department."
  Justin Cooper (a longtime adviser to former President Bill Clinton. Sec. Clinton, who registered the clintonemail.com domain name on January 13, 2009)