Disaster Recovery Tabletop Exercises

Being able to recover from a major system failure is essential for most businesses today. The key to recovery, beyond implementing technology that supports disaster recovery, is to practice for disasters. That means periodically meeting with members of the organization to review the process an IT department will go through in order to, step by step, bring critical systems back from an uncontrolled virus attack, hurricane, or major hardware failure. The table top exercise described below is a relatively inexpensive way for an organization to discuss scenarios for systems recovery and identify issues in the current disaster recovery plan.

Pre-Meeting Setup
The exercise will be most effective if ahead of time a network diagram is created that provides the following details for the existing information system:

  • Local network geographic locations
  • Workstations, network switches, routers, servers, phone systems, and related equipment
  • Local and Wide area network connections, speeds, and carriers
  • Existing contracts for business continuity services
  • Matrix of existing IT staff and areas of expertise

Where possible, the network diagram should identify the high level systems in place in the organization, and degree of criticality of each system to continued business operations in the event of a disaster. The criticality of a system determines its priority in a recovery effort.

In addition, if the organization has a policy on systems availability or the operations of the organization during particular disasters, this policy should be available to staff in the meeting. This policy may identify disasters where the organization will continue to operate, the chain of command to respond to system disasters, and organizational expectations for recovery time objectives.

Meeting Content
With the network diagram, the IT/IS department should meet with its leadership and senior staff, and some representatives from the business units served by the information systems for approximately two hours. A member of the meeting should be designated to take notes on the course of the meeting, and a different member should be designated as moderator. Notes taken should be organized by disaster, and should identify which staff resources were available during each scenario, potential issues with recovery that were identified during the discussion, and open questions about configuration or recoverability.

Meeting Process
During the meeting, the following activities should take place:

1. A disaster scenario for discussion should be identified which will impact business operations.
2. Members of the team should identify what systems will be unavailable, the estimated length of the interruption to service, what IT resources will be available to respond to the disaster, and what impact the unavailable systems will have on continued business operations.
3. Members of the team should then work through the stated policy on Disaster Recovery for the organization, and identify the operational and technical steps required to recover the affected system or systems.
4. The group should come prepared to discuss experience with similar recoveries, and identify potential issues with performing a timely recovery of data. Relevant to the discussion is the fact that such a recovery has not been previously tested, or other known resource limitations on performing a recovery based on the scenario.
5. A member of the team should be identified as the note keeper for the discussion, and will be responsible for distributing notes of the meeting to all participants.

The process above should be repeated so that the group can address two to three disaster scenarios during the meeting. The total length of the meeting should be limited to two hours.

Post-Meeting Review
Following the meeting, a list of issues to be addressed should be created which should form the basis for a project or work plan. Where necessary, a budget for capital and/or ongoing expenses and resources should be developed based on the work plan. Research may need to be conducted on potential technical solutions or workarounds to identified issues during the exercise. In addition, the policies of the organization may need to be reviewed or modified in order to reflect actual practice in responding to disasters, or based on feedback from the team meeting. A technical testing plan may also be required based on the findings or proposed technical solutions to a systems failure.

A follow-up table top exercise should then be scheduled, based on the estimated time required to be able to appropriately address the issues identified in the previous table top exercise. Results from successive table top exercises can be used to demonstrate progress in preparedness for disasters.

For more details or information on how to conduct a table top exercise, see The National Institutes of Standards and Technology, special publication 800-84, section 4-1. Need help organizing or facilitating a table top exercise? Contact us for more information.

Second Life: Virtual World Meets Real World Copyright

Second Life, (click here for their main web site) an online virtual worlds system, has become the center of a recent copyright controversy involving, yes, virtual sex toys.  Apparently, there are not enough legitimate or inexpensive sex toys for all the denizens of Second Life, so some residents have elected to make knock-offs.  Just like the real-world controversies surrounding real-world goods made by companies like Tiffany and Louis Vuitton, Eros LLC has filed suit against the alleged infringers and Linden Labs, for housing and supporting the alleged knock-offs.  (See Wired Article)

Vicarious and contributory liability for copyright infringement are recognized by the courts as a cause of action under federal copyright law.  This kind of liability has been raised in recent years against the various music file sharing services that came and went, such as Napster (originally a file sharing service without any copyright licensing from the music companies that owned the music being shared), Gnutella, and Limewire.  Each of these services were held to be liable for the file sharing of their users, in part based on the notion of vicarious liability.  Cases prior to Napster et al. that addressed this kind of liability have developed along two lines: landlord-tenants where the landlord exercised no control over the leased premises, and dance-hall cases where the operator of the hall controlled the premises and obtained a direct financial benefit from the infringing performances.  Fonovisa, Inc. v. Cherry Auction, Inc., 76 F.3d 259 (9th Cir. 1996).  Under common law, landlords have not been held to have copyright liability where dance-hall operators have infringed the copyrights of others.

In Fonovisa, the defendant Cherry Auction operated a swap meet where it rented stalls to individuals who were selling unlicensed copies of bootlegged music owned by the plaintiff.    For the swap meet operator to be liable, the plaintiff had to prove that the operator controlled the marketplace and obtained a direct financial benefit from the sales of infringing works.  The Court sided with the plaintiff in Fonovisa, even though the defendant Cherry Auction did not receive a commission from the sales of the infringing materials.

Unlike the auction house in Fonovisa, Second Life does allow users access to their information system without making a payment.  Anyone can download a copy of the Second Life client, establish a username, and log in to the system.  Users start to rack up fees when they purchase virtual real estate within the system.  In addition, Linden Labs provides a virtual currency of Linden Dollars that allow for the exchange of virtual goods within the system.  Linden Dollars can be exchanged for U.S. dollars using the credit card or paypal account associated with your Second Life account.  As a result of this connection with the physical world, there are a number of users that make an actual living in Second Life producing virtual goods for their fellow Second Life denizens.  The last time I visited, about 17 million worth of linden dollars were exchanged into real dollars on the Linden Labs exchange system in a day.  Linden Labs is generating a significant amount of commerce in spite of the national recession.

According to Eros Products LLC, his SexGen products line has sold about 1 million (that’s U.S. dollars) of product within Second Life over the past five years.  (A copy of the Complaint is here).  Competition being fierce in the digital world, others have been making sex toys that look a lot like Eros’, with some likely being copied straight from the source and resold.  This is possible in Second Life because Second Life provides “builder” tools to its users.  Included in the toolkit are functions to allow for the upload of image files.  In addition, there are apparently tools available from other software makers that allow a Second Life user to copy images within the system.

Assuming that Eros Products LLC (and other plaintiffs that may join the suit should the court certify this as a class action) can prove that they are the valid owner of the copyrighted works, the question for the court is whether Linden Labs can meet the standard for contributory liability.  Linden Labs is a virtual landlord in the sense that users of Second Life pay an annual subscription in order to own virtual real estate within the virtual world.  The right to own this virtual property is limited by payment of the subscription.  You will note, however, that there are plenty of users that do not acquire any virtual real estate in Second Life – and for them, there is no fee to participate.

However, Linden Labs also charges fees for the conversion of Linden Dollars into U.S. Dollars through the Linden Exchange.  For infringers seeking to sell pirated works in the virtual world, the real benefit to them is the ability to take the proceeds of those sales and convert them back into hard currency for use in the real world.  Approximately 250 Linden Dollars are worth a U.S. Dollar (the trading in this currency fluctuates).  In order to convert Linden Dollars back to U.S. Dollars, Linden Labs charges a fee of 3.5% of the value of the transaction.  So, indirectly, Linden Labs benefits from the sale of infringing goods every time that the infringer converts his Linden Dollar proceeds to hard currency.

There is a question, however, of whether Linden Labs is merely a landlord who relinquished control to his infringing tenant.  Eros Products LLC claims that Linden Labs did exercise control over the activities of its users because all of the virtual worlds within Second Life are ultimately housed on servers controlled by Linden Labs.  Pl.’s Complaint at ¶ 127-128.  And furthermore, Linden Labs has ultimate control over its software that operates Second Life, and I suppose that Linden Labs could alter its software to prevent copyright infringement if it wished to do so (how, exactly, is another story).  Factually, however, I think this is going to be tough to prove.  Unlike Grokster, who marketed itself as the successor to Napster for those looking to willfully infringe on the copyrights of others, Linden Labs has not marketed itself as a safe haven for willful copyright infringers.  On the contrary, Linden Labs gave some thought to copyright in its license agreement, granting its users rights in the works they create in-world.  (See Terms of Service here at section 3.2)

Update on Meaningful Use and Contracting

What constitutes “meaningful use” of an electronic health record system has been updated by the Centers for Medicare and Medicaid (CMS) and the Department of Health and Human Services (HHS) in volume 75, number 144 of the Federal Register, page 44565, as of July 28, 2010.  These new definitions for stage 1 meaningful use are to go into effect as of September 27, 2010.  Under the original rule that was published at the end of 2009, there were a total of 25 standards that needed to be met by an EHR user to qualify as a meaningful user.  Under the final rule published July 28, the list has ben somewhat shortened, so that there are 15 “core” standards that must be met, under § 495.6(d), and an additional 5 standards from the list of 10 found in § 495.6(e) for those who qualify as eligible professionals under the regulation.  In parallel, eligible hospitals and critical access hospitals must meet the core standards in § 495.6(f), and 5 additional standards in § 495.6(g).

Under § 495.6(d), the following are the fifteen mandatory standards for eligible professionals:

(d)(1) use computerized provider order entry for medication orders for at least 30% of patients seen

(d)(2) implement drug-drug and drug-allergy interaction checking

(d)(3) main an up to date problem list

(d)(4) e-prescribing for at least 40% of all permissible prescriptions

(d)(5) maintain active medication list

(d)(6) maintain active allergy list

(d)(7) record particular demographics

(d)(8) record specific vital signs, including BMI and capability for growth charts

(d)(9) record smoking status

(d)(10) report ambulatory clinical quality measures to CMS or your State Medicaid EP (though the specifics are not provided here)

(d)(11) implement a clinical decision support rule

(d)(12) provide an electronic copy of health information to patient on request

(d)(13) provide patient clinical summaries at each patient visit

(d)(14) prove capability to exchange key clinical information electronically by performing at least one test of this capability

(d)(15) protect protected health information by complying with the risk assessment and risk reduction guidance in the HIPAA security rule, at 45 CFR § 164.308.

Eligible professionals also must pick and implement five of the following:

(e)(1) implement drug-formulary checking

(e)(2) incorporate lab test results into EHR

(e)(3) generate patient lists by specific conditions for quality improvement, research, outreach

(e)(4) patient preventive care reminders

(e)(5) provide patients with timely access to their health information

(e)(6) provide patients with patient-specific education resources

(e)(7) perform medication reconciliation when a patient is received from another setting of care

(e)(8) provide summary of care for patients referred or transitioned to another care setting

(e)(9) prove ability to exchange immunization data with an electronic registry

(e)(1) prove ability to exchange surveillance data with an electronic registry.

Some of the above are new, and some of the previous draft standards were dropped (such as a requirement to use electronic insurance eligibility or submit claims electroncially, which are typically under the purview of a practice management system, but not typical of an EHR).

Prospective EHR vendors should be able to show you how they will aid you in complying with these requirements, and a reputable EHR vendor should be willing to put its compliance efforts into its contract for sale and implementation.  If you already have an EHR vendor that you use today, now is the time to start the conversation on compliance with these requirements.  Expect more news on this very important regulatory topic soon.