Email Archiving Myths
To understand email archiving basics, it’s best to understand its history. The first generation archiving systems provided the ability to manage Exchange email box sizes by automatically moving an email message and/or attachment from the user’s mailbox to the email archive and replacing it with a pointer. To the end-user, the archived email still appeared to be in their mailbox because the pointer (or stub) mimicked the email in Outlook and when clicked on, the email and attachment appeared to the end-user.
Emails and attachments make up the huge majority of data stored on the Exchange server, so stubbing and moving email and attachments to the archive reduces the size of the email store. Many believed this produced a higher performance, more reliable email system.
Microsoft discourages email stubbing
After email stubbing gained in popularity, Microsoft began to discourage its use because of the unforeseen issues it produced. The apparent problem stubbing was supposed to alleviate was the performance hit on the Exchange server if a given mailbox was too large (over 2 GB). In fact, the actual problem was with the number of items in a given folder, so stubbing actually compounded the server performance problem.
Microsoft’s view of stubbing was clearly stated by Perry Clarke, GM for Exchange Mailbox Server, when in 2010 he said “I consider the stubbing approach to be one of those kluges that occur in the software industry regularly that are done out of necessity.”
To be clear, it is the number of emails (or objects) that has a greater impact on server performance, not the combined object size of all of the emails. Microsoft’s 2011 white paper entitled “Planning for Large Mailboxes with Exchange 2007” pointed out that email archiving solutions that utilize stubbing and reduce the object size of emails do not gain the desired reduction in storage workload or performance improvement. “Removing the message bodies and attachments from Exchange reduces the mailbox size, but it does not significantly change the server performance for users accessing Exchange via Outlook in online mode and Outlook Web Access. Item counts are the primary performance driver for the Exchange store, and not aggregate size. For example, server performance with a folder containing 100 KB of full e-mail message items is similar to a folder containing 100 KB of stub files.“
The Microsoft white paper also pointed out that with the deployment of large mailboxes in the newer releases of Exchange and the previous issues related to high mailbox item counts, stub files should not be necessary and should be avoided.
Stubbing is no longer needed
In older versions of Exchange, the best practices for limiting the number of items in any one Exchange folder was to keep the count to less than 5000 items, otherwise the Exchange server performance would suffer. To many organizations, this translated to the size of the folder being the problem. But as has already been pointed out, the size of the files were not the problem, the number of items in a particular folder was. Many legacy email archiving solution providers mistakenly tried to fix the problem by enabling the stubbing feature, where in reality, it caused additional problems. With the newer versions of Exchange (on-premise), the size and numbers of items in a particular mailbox is no longer an issue and in fact can be unlimited.
Stubbing and Migration don’t mix
Because the email archive stubbing feature was made to be transparent, the end-user cannot tell which mailbox items are stubs and which ones are “whole” messages. This may not seem like a problem but when it comes time to migrate your legacy on-premise email archive to a new system, the stubbing feature creates several problems for both IT and the end-user:
- If the archive is migrated to any place other than the Exchange server containing the message stubs, those millions of stubs won’t work anymore and will end up generating error messages to the end-user – dramatically impacting end-user productivity, and creating a help desk nightmare.
- Exchange does not have an easy way for Administrators to find and delete those orphaned message stubs so the possibility of orphaned message stubs will persist for long periods of time.
- Message stubs can include additional metadata generated from their movement from folder to folder within Exchange. This additional metadata could be relevant in legal discovery so message stub metadata should not be arbitrarily deleted which could trigger charges of spoliation.
- Systematically syncing individual message stubs with the archived messages in a forensically defensible process is extremely difficult and time consuming.
So now, the accepted guidance when moving to a newer email archiving system is to stay away from email stubbing and allow the individual mailboxes to be controlled (limited) via the placement of mailbox limits.
Email Archiving Basics #2 – The myth: All Companies have regulatory retention requirements
Some organizations have fought the need to archive their email/communications because they are not in a heavily regulated industry, for example an automobile parts manufacturer. This could not be further from the truth. All publicly held companies have regulatory retention requirements that highlight the need for archiving. For example, the Sarbanes-Oxley Act (SOX), spans all industries and imposes severe penalties on publicly held companies who deliberately alters or deletes documents with the intent to defraud third parties. SOX requires company auditors to retain audit records for a minimum of five years from the end of the fiscal year. Even though it is a US law, SOX is also applicable to European companies with US listings as well as to companies which do business with the US.
Employment law compliance
Corporate Human relations Departments receive, generate and retain huge volumes of documents such as employment applications, resumes, reference check reports, drug testing data, personnel records, payroll records, reviews, and disciplinary files. Combine this content with a myriad of federal, state, and local laws and regulations that require that certain personnel records, whether stored electronically or on paper, be kept for a specified period of time and even well after the termination of an employee’s employment. There are many additional relevant federal and state record retention laws that all organizations that employ anyone should be aware of.
For example, under The Americans with Disabilities Act (ADA) and Title VII of the Civil Rights Act of 1964, covered employers must retain personnel or employment records they made or keep them for one year from the date the record was made or from when an action was taken (e.g., termination), whichever is later.
The Age Discrimination in Employment Act (ADEA), also has separate retention standards for records containing particular employee information. Complicating things further, separate rules also apply for different categories of employers. Employment agencies for example, must keep records on placements, referrals, job orders by employers, applications, test papers completed by applicants as part of the selection process, and advertisements or notices relating to job openings.
The Fair Labor Standards Act (FLSA) and the Family and Medical Leave Act (FMLA) also have separate, strict requirements for retention of specific documents relating to an individual’s employment, such as payroll slips, time sheets or others documents on which wage computations are based, any records relating to any leave time the employee has taken, and documentation of employee benefits.
The Lilly Ledbetter Fair Pay Act, which changed the statute of limitations on when an employee can file a pay discrimination claim, does retain some limits on employer liability by restricting back-pay awards to two years, but employer questions and concerns can still arise, particularly regarding record retention requirements since claims can be filed based on decisions made years earlier.
There are additional federal employment laws that employers should look at to make sure they are keeping documents long enough to meet the laws’ requirements or to cover the period of time in which an employee can file suit under the laws. These include:
- Health Insurance Portability and Accountability Act (HIPAA)
- The Consolidated Omnibus Budget Reconciliation Act (COBRA)
- Uniformed Services Employment and Reemployment Rights Act (USERRA)
- Occupational Health and Safety Act (OSH Act)
- National Labor Relations Act (NLRA)
- Employee Retirement Income Security Act (ERISA)
- Immigration Reform and Control Act (IRCA)
- Equal Pay Act (EPA)
- The Fair Credit Reporting Act (FCRA)
Email Archiving Basics #3 – The myth: Litigation preparedness
Regulatory compliance is not the only driver for organizations to consider social communications archiving. All organizations face the possibility of a civil litigation and eDiscovery. When litigation (or a government investigation) is pending or anticipated, an organization can greatly affect/reduce the probable costs (and risks) of responding to an eDiscovery request or regulatory information request by managing their information in a manner that makes it easier to manage and query. Especially for those organizations that are regularly involved in eDiscovery, proactive preparation can greatly reduce eDiscovery costs. There are three rules of thumb in eDiscovery (and regulatory) requests:
- Huge amounts of information will be requested
- An organization’s communications, including email, will be a major target
- How an organization secures its information and responds to the request can mean the difference between winning and losing the case
An important first step in litigation preparedness is stopping deletions of all potentially relevant information as soon as litigation is anticipated. In other words an organization should be able to apply a legal hold quickly, no matter where potentially relevant data is stored. Email systems are notorious for letting end-users inadvertently (or on purpose) delete emails without restriction. Because of this, email archiving solutions are used to ensure security/litigation hold requirements.
Another aspect of litigation preparedness that archives can enable is Early Case Assessment (ECA). the ECA process is comprised of two different parts: estimating the cost and time needed to defend (or prosecute) a legal case and, more importantly, quickly searching for and analyzing potential evidence from all locations for case strategy development to create an eDiscovery and litigation plan to best achieve a successful resolution. In other words, searching for as much relevant information as quickly as possible to determine early on if the case has merit and therefore should be settled or continue to trial. Obviously the more relevant evidence an organization can collect will provide the basis of a more successful ECA strategy. Because communications (email) is almost always a major target of eDiscovery, the ability to access and search email stores is extremely necessary for successful ECA.
The actual eDiscovery process encompasses several distinct but dependent processes. At its most basic, eDiscovery is reliant on the ability to find (search), access, and review potentially relevant (case-specific) data. The costs (and risks) during the eDiscovery process skyrocket when an organization is unable to find, secure, and review all potentially relevant information in the time allotted by the court. The downside for not finding all responsive information or taking too much time can lead the court to take actions that put a case defense at risk, such as disallowing evidence or issuing and adverse inference instruction.
Over-collection of content can add millions of dollars to the cost of defending a single case however an under-collection situation (where all relevant content is not found and produced) can cause the organization to lose the case before it goes to trial because of an inadequate eDiscovery or spoliation ruling from the court. Proactively managing an organization’s information with effective archiving practices is the key to reducing both the cost and risk of eDiscovery.
Email Archiving Basics #4 – The myth: Archiving is only for Key Employees
ne of the strategies some companies employ when preparing for the inevitability of eDiscovery is to selectively archive only what they consider possible target or key employees, for example members of the executive staff. The reasoning behind this strategy is that these key employees are almost always the target of civil litigation and eDiscovery requests so why spend additional money archiving content from employees that will never(?) be included in a eDiscovery request. This reasoning is flawed and could be potentially dangerous.
Opposing Counsel will target what you haven’t planned for
The opposing counsel’s job is to win their client’s case as quickly as possible. One of the strategies they employ is to direct eDiscovery requests at those areas that will cause the defendant the most pain and cost with the hope that the case will be settled before it ever makes it to trial. A favorite target of discovery in the past was to ask for content that is difficult and or costly for the defendant to produce. By archiving only select employees (which will be disclosed at the meet and confer meeting) the responding party is inviting the opposing counsel to tailor their discovery request to include as many non-archived employees as they can justify. This opposing counsel’s strategy will cause time consuming and costly manual processes and can even include recovering backup tapes – an expensive and timely process. A recent example of backup tape eDiscovery can be found in the case of United States ex rel Guardiola v. Renown Health, No. 3:12-cv-00295-LRH-VPC, 2015 WL 5056726 (D. Nev. Aug. 25, 2015). In its finding, the court reasoned that Defendants must bear some responsibility for the consequences of the decision to use an “archival/backup solution that did not maintain ESI in an indexed or otherwise searchable manner.”
eDiscovery review often leads to additional custodians
In many cases, the collection and review of targeted custodian email leads to other (at the time unknown) custodians that should also be included in the eDiscovery. This is usually a result of trying to build email conversation threads based on a single suspicious email. For example, during the Enron scandal, emails were sent by key custodians to others referencing the term “Raptor”, which was later determined to refer to special purpose entities within Enron that were used to hedge investments. By building these conversation threads, many additional custodians were later targeted during eDiscovery. Additional examples where non-executive employees can be targeted for eDiscovery include IP or confidential information theft, HR issues such as sexual harassment, and discrimination investigations.
Email Archiving Basics #5 – The myth: Cloud Archiving is more costly
One of the biggest challenges organizations now face is the compound average growth rate of information (doubling every two years) followed by the tightening restrictions around regulatory compliance and eDiscovery. Many organizations are now archiving this information or are actively considering archiving their information to better address regulatory, eDiscovery, and corporate governance requirements. The question now is what is the best archiving platform to utilize; on premise or cloud-based…
An effective archiving solution must be able to current address storage, accessibility, security, and search capabilities as well as adapt to new requirements as they arise. Many technology analysts are now suggesting that cloud-based archiving solutions represent the most highly effective, cost efficient, and robust approach to archiving the growing mountains of enterprise data. In fact, a cloud-based archiving solution can better meet the regulatory, legal and business requirements of long-term information storage and retrieval over that of on premise archiving solutions due to its centralized nature and savings from the economies of scale – the cost advantages that cloud vendors obtain due to size, output, or scale of operation, with cost per unit of resources such as storage generally decreasing with increasing scale as fixed costs are spread out over more customers.
Years ago it may have been true that cloud-based archiving solutions were more expensive than on premise archiving solutions, but over the last several years, the cost benefit and increased functionality of cloud-based archiving platforms have far outpaced the aging on premise archiving applications.
The true cost of on premise storage
On premise archiving vendors argue that the cost of enterprise storage is constantly falling and therefore the cost of on premise archiving storage remains a good deal (of course you can say the same thing about enterprise class cloud storage). But this theory ignores the additional costs of storing huge amounts of information in an on premise repository. These additional costs include:
- Annual storage purchases
- Servers and software
- Nightly backups
- Disaster recovery planning and resources
- Annual hardware/software support agreements
- Floor space
- Cooling and electricity
- Additional employees to manage storage resources
- Cyber-liability insurance
As a matter of fact, on premise archive infrastructure is a costly and management-intensive undertaking. Due to the long-term nature of archiving, multiple hardware refreshes are required throughout the lifecycle of the archived information, annual support contracts must be renewed, and staff hiring and turnover taken into account. With cloud archiving, all of these costs are already included in the contractual pricing.
In fact, the additional costs associated with on premise storage can drive the fully loaded cost of enterprise class storage to well over $20 per GB. So the right cloud archiving platform can reduce your organization’s overall information management costs and risks while raising information accessibility, security, and value.
We hope this Email Archiving Basics blog will help you better identify your next archiving platform. For more information on email archiving, click here.