Skip to main content

Handling Emergency Situations

Handling Emergency Situations

Emergency situations involving encrypted data can result permanently losing access to the encrypted data. If an emergency situation arises, you must act immediately to minimize the risk of the data being lost forever.

This topic describes the steps to take if an emergency situation with encrypted data arises. To take preventative steps against an emergency situation, see Protecting Against Data Loss.

Handle Emergency Situations When Using Key Files

This topic describes what to do under certain circumstances when you are in danger of losing data. These situations include:

If the File Containing an Activated Key is Damaged or Missing

In this situation, the following circumstances have occurred:

  • A database encryption key has been activated for the InterSystems IRIS® data platform instance.

  • InterSystems IRIS is using encrypted data.

  • The key file containing the database encryption key becomes damaged.

If There Is a Backup Copy of the Key File with a Known Administrator Username and Password

Caution:

This procedure is for an emergency situation, where encrypted data in InterSystems IRIS databases is in danger of being lost.

If the file containing an activated key becomes inaccessible or damaged, immediately perform the following procedure:

  1. Get the backup copy of the key file. This is the copy that you stored as described in Protection from Accidental Loss of Access to Encrypted Data.

  2. Make a new backup copy of the key file and store it in a safe place.

  3. Set up InterSystems IRIS to use the new copy of the key:

    • If you are using interactive startup, incorporate the new copy of the key into your startup procedures.

    • If you are using unattended startup, then reconfigure your startup options using the new copy of the key file — even if you are setting it up for the same options as before.

If There Is No Backup Copy of the Key File or the Key has No Known Administrator Username and Password

WARNING:

THIS PROCEDURE IS FOR AN EMERGENCY SITUATION, WHERE ENCRYPTED DATA IN INTERSYSTEMS IRIS DATABASES IS IN DANGER OF BEING LOST.

If the file containing the activated key becomes inaccessible or damaged while InterSystems IRIS is running, immediately perform the following procedure for each database encrypted with that key:

  1. WARNING:

    Shutting down InterSystems IRIS or deactivating the active key will cause the permanent loss of your data.

    Do not shut down InterSystems IRIS.

    Do not deactivate the currently active key.

  2. Contact the InterSystems Worldwide Response Center. Engineers there can help guide you through the following procedure and answer any questions that may arise.

  3. Dismount the database. This prevents all users from making any changes to the database with encrypted content while copying its data to an unencrypted database:

    1. From the Management Portal home page, go to the Databases page (System Operation > Databases).

    2. On the Databases page, if the encrypted database is mounted, select the Dismount option in the next-to-last column in that database’s row. Then select OK in the confirmation dialog.

    3. When the Databases page appears again, select the Mount option in the last column in the database’s row.

    4. On the Mount database confirmation screen, check the Read Only box and select OK.

    It is critical that no one makes any changes to the database during this procedure. Mounting the database read-only prevents any user from changing any data.

  4. Copy all data in unencrypted form to another database. The procedure for copying the data is:

    1. In the Terminal, go to the %SYS namespace:

      REGULARNAMESPACE>set $namespace="%SYS"
      
    2. From that namespace, run the ^GBLOCKCOPY command:

      %SYS>d ^GBLOCKCOPY
      
      This routine will do a fast global copy from a database to another database or
      to a namespace. If a namespace is the destination, the global will follow any
      mappings set up for the namespace.
       
      1) Interactive copy
      2) Batch copy
      3) Exit
       
      Option?1
      
    3. At the ^GBLOCKCOPY prompt, specify 1, for an interactive copy:

      Option? 1
       
      1) Copy from Database to Database
      2) Copy from Database to Namespace
      3) Exit
       
      Option?
      
    4. When ^GBLOCKCOPY prompts for a copy type, select 1, for copying from database to database

      Option? 1
      Source Directory for Copy (? for List)?
      

      Here, either specify the name of the encrypted database or enter ? to see a numbered list of databases, which includes the encrypted database. If you enter ?, ^GBLOCKCOPY displays a list such as this one:

      Source Directory for Copy (? for List)? ?
       
      1) C:\InterSystems\MyIRIS\mgr\
      2) C:\InterSystems\MyIRIS\mgr\irislocaldata\
      3) C:\InterSystems\MyIRIS\mgr\irisaudit\
      4) C:\InterSystems\MyIRIS\mgr\irislib\
      5) C:\InterSystems\MyIRIS\mgr\iristemp\
      6) C:\InterSystems\MyIRIS\mgr\encrypted1\
      7) C:\InterSystems\MyIRIS\mgr\encrypted2\
      8) C:\InterSystems\MyIRIS\mgr\unencrypted\
       
      Source Directory for Copy (? for List)?
      

      Enter the number of the encrypted database, such as 7 here.

    5. When ^GBLOCKCOPY prompts for a destination directory for copying the data, enter the name of an unencrypted database or ? for a list similar to the one for the source directory.

    6. When ^GBLOCKCOPY asks if you wish to copy all globals, enter Yes (can be Yes, Y, y, and so on):

      All Globals? No => y
      
    7. If there is an empty global in the database, ^GBLOCKCOPY will now ask if you wish to copy it. This will appear something like the following:

      All Globals? No => y
       
      ^oddBIND     contains no data
      Include it anyway? No =>
      

      Enter No (can be No, N, n, and so on), which is the default.

    8. ^GBLOCKCOPY then asks if you wish to skip all the other empty globals. Enter Yes (can be Yes, Y, y, and so on), which is the default:

      Exclude any other similar globals without asking again? Yes =>
      

      There then appears a list of all the empty globals that are not being copied:

      Exclude any other similar globals without asking again? Yes => Yes
      ^oddCOM      contains no data --  not included
      ^oddDEP      contains no data --  not included
      ^oddEXT      contains no data --  not included
      ^oddEXTR     contains no data --  not included
      ^oddMAP      contains no data --  not included
      ^oddPKG      contains no data --  not included
      ^oddPROC     contains no data --  not included
      ^oddPROJECT  contains no data --  not included
      ^oddSQL      contains no data --  not included
      ^oddStudioDocument contains no data --  not included
      ^oddStudioMenu contains no data --  not included
      ^oddTSQL     contains no data --  not included
      ^oddXML      contains no data --  not included
      ^rBACKUP     contains no data --  not included
      ^rINC        contains no data --  not included
      ^rINCSAVE    contains no data --  not included
      ^rINDEXEXT   contains no data --  not included
      ^rINDEXSQL   contains no data --  not included
      ^rMACSAVE    contains no data --  not included
      9 items selected from
      29 available globals
      
    9. ^GBLOCKCOPY then asks if you wish to disable journaling for this operation:

      Turn journaling off for this copy? Yes =>
      

      Enter Yes (can be Yes, Y, y, and so on), which is the default.

    10. ^GBLOCKCOPY then asks if to confirm that you wish to copy the data:

      Confirm copy? Yes =>
      

      Enter Yes (can be Yes, Y, y, and so on), which is the default. Depending on the size of the database and the speed of the processor, you may see the status of the copy as it progresses. When it completes, ^GBLOCKCOPY displays a message such as:

      Copy of data has completed
      
    11. ^GBLOCKCOPY then asks if you wish to save statistics associated with the copy. Enter No (can be No, N, n, and so on), which is the default:

      Do you want to save statistics for later review? No =>
      

      Control then returns to the main prompt.

  5. Test that the copied data is valid. You can do this by examining the classes, tables, or globals in the Management Portal’s System Explorer for the database into which ^GBLOCKCOPY has copied the data.

  6. If the data is valid, perform steps 3 and 4 of this procedure for each database encrypted with the inaccessible or damaged key.

  7. Once you have made copies of every encrypted database into an unencrypted database, make a second copy of each database, preferably to a different machine than that which holds the first copy of each.

  8. Now — and only now — you can dismount all encrypted databases and deactivate the active key (that is, the key for which the key file is missing or damaged). InterSystems IRIS requires that you dismount all encrypted databases prior to deactivating their key.

You now have your data in one or more unencrypted databases and there is no activated key.

To re-encrypt the formerly encrypted databases, the procedure is:

  1. Create a new database encryption key according to the procedure described in Creating a Key.

  2. Create a new backup copy of the key file as described in Protection from Accidental Loss of Access to Encrypted Data.

    Caution:

    Make sure you take the precautions described in Protection from Accidental Loss of Access to Encrypted Data; failure to follow these procedures can result in the permanent loss of data.

  3. Create one or more new encrypted databases, using the new key.

  4. Import the data exported in the previous procedure into the new encrypted database(s).

Your data is now stored in encrypted databases for which you have a valid key and a backup copy of the key file containing that key.

If the Database Encryption Key File Is Required at Startup and Is Not Present

Under certain conditions related to the required use of a database encryption key file at startup, the system starts in single-user mode. These conditions are:

  • InterSystems IRIS is configured for either interactive or unattended startup.

  • Startup specifies that journal files and/or the IRISTEMP and IRISLOCALDATA databases are encrypted, or an encrypted database is specified as required at startup.

  • The database encryption key file is not present.

If You Can Make the Key File Available

This situation may have been caused simply by the appropriate key file not being present at InterSystems IRIS startup time — such as if the media holding it is not currently available.

To correct the condition, after InterSystems IRIS starts running in single-user mode, then the procedure is:

  1. Shut down InterSystems IRIS. For example, if the instance of InterSystems IRIS is called “MyIRIS”, the command to do this would be:

    iris force MyIRIS
    
  2. If you know the location where InterSystems IRIS is expecting to find the database encryption key file, then place the key file there. (Otherwise, you need to run ^STURECOV as specified in the next section.)

  3. Start InterSystems IRIS again.

InterSystems IRIS should start in its typical mode (multi-user mode) and operate as expected.

If a Backup Key File Is Available

If the appropriate key file is not present at InterSystems IRIS startup time and is not available, you may have a backup key file available. If so, then to correct the condition, after InterSystems IRIS starts running in single-user mode, then the procedure is:

  1. Contact InterSystems Worldwide Response Center. Engineers there can help guide you through the following procedure and answer any questions that may arise.

  2. Start a Administrator Terminal Session according to the instructions in the most recent entry in the messages.log file. Typically, this specifies starting a Terminal session with the -B flag..

    For example, at a Windows command line, for an instance of InterSystems IRIS called “MyIRIS” that is installed in the default location, the command would be:

    c:\InterSystems\MyIRIS\bin\irisdb -sc:\InterSystems\MyIRIS\mgr -B
    

    This connects you with InterSystems IRIS in the operating system terminal window; the prompt in that window changes from the operating system prompt to the InterSystems IRIS %SYS prompt.

  3. If you have or can obtain a copy of the database encryption key file (such as a backup), then place a copy of the key file in a location accessible to InterSystems IRIS.

  4. Run the ^STURECOV (startup recovery) routine at the Terminal prompt. In that routine, activate the encryption key using an administrator username and password in that file. (You do not need to exit ^STURECOV when you have completed this process.)

  5. When you are satisfied that InterSystems IRIS is ready for use, use ^STURECOV to complete the startup procedure. InterSystems IRIS then starts in multi-user mode.

InterSystems IRIS should now operate as expected.

If No Key File Is Available

If you do not have any copy of the database encryption key file, then the procedure is:

  1. Contact InterSystems Worldwide Response Center (WRC). Engineers there can help guide you through the following procedure and answer any questions that may arise.

  2. Start a Administrator Terminal Session according to the instructions in the most recent entry in the messages.log file. Typically, this specifies starting a Terminal session with the -B flag..

    For example, at a Windows command line, for an instance of InterSystems IRIS called “MyIRIS” that is installed in the default location, the command would be:

    c:\InterSystems\MyIRIS\bin\irisdb -sc:\InterSystems\MyIRIS\mgr -B
    

    This connects you with InterSystems IRIS in the operating system terminal window; the prompt in that window changes from the operating system prompt to the InterSystems IRIS %SYS prompt.

  3. If any encrypted databases require mounting at startup, disable this feature for them:

    1. From the Management Portal Home page, go to the Local Databases page (System Administration > Configuration > System Configuration > Local Databases).

    2. Click the name of the database in the table of databases. This displays the Edit: page for the database.

    3. On the Edit: page, clear the Mount Required at Startup check box.

    4. Click Save.

  4. Run the ^STURECOV routine at the Terminal prompt. In that routine, configure InterSystems IRIS database startup options not to require a database encryption key. This means that the IRISTEMP and IRISLOCALDATA databases as well as journal files should now operate as expected; it also means that any encrypted databases cannot be mounted.

  5. When you are satisfied that InterSystems IRIS is ready for use, use ^STURECOV to complete the startup procedure. InterSystems IRIS then starts in multi-user mode.

As you perform this procedure, you may need to perform other actions, according to the instructions of the representative from the WRC. Follow these instructions.

Caution:

If you have not performed the actions described in Protection from Accidental Loss of Access to Encrypted Data, then your data may no longer be available in any form. This is a very serious problem, but if you do not have a key, there is no way to retrieve the lost data.

Handle Emergency Situations When Using a KMIP Server

This topic describes what to do under certain circumstances when you are using a KMIP server and are in danger of losing data. These situations include:

If the KMIP Server Holding an Activated Key is Damaged or Missing

In this situation, the following circumstances have occurred:

  • A database encryption key has been activated for the InterSystems IRIS instance

  • InterSystems IRIS is using encrypted data

  • The KMIP server the database encryption key becomes damaged

If There Is a Backup Copy of the Key on the KMIP Server

If KMIP server holding an activated key becomes inaccessible or damaged, immediately perform restore procedures for the KMIP server according to your vendor’s instructions.

If There Is No Backup Copy of the Key on the KMIP Server

WARNING:

THIS PROCEDURE IS FOR AN EMERGENCY SITUATION, WHERE ENCRYPTED DATA IN INTERSYSTEMS IRIS DATABASES IS IN DANGER OF BEING LOST.

If there is no way to restore the KMIP server holding the activated key from backup, immediately perform the following procedure for each database encrypted with that key:

  1. WARNING:

    Shutting down InterSystems IRIS or deactivating the active key will cause the permanent loss of your data.

    Do not shut down InterSystems IRIS.

    Do not deactivate the currently active key.

  2. Contact the InterSystems Worldwide Response Center. Engineers there can help guide you through the following procedure and answer any questions that may arise.

  3. Dismount the database. This prevents all users from making any changes to the database with encrypted content while copying its data to an unencrypted database:

    1. From the Management Portal home page, go to the Databases page (System Operation > Databases).

    2. On the Databases page, if the encrypted database is mounted, select the Dismount option in the next-to-last column in that database’s row. Then select OK in the confirmation dialog.

    3. When the Databases page appears again, select the Mount option in the last column in the database’s row.

    4. On the Mount database confirmation screen, check the Read Only box and select OK.

    It is critical that no one makes any changes to the database during this procedure. Mounting the database read-only prevents any user from changing any data.

  4. Copy all data in unencrypted form to another database. The procedure for copying the data is:

    1. In the Terminal, go to the %SYS namespace:

      REGULARNAMESPACE>set $namespace="%SYS"
      
    2. From that namespace, run the ^GBLOCKCOPY command:

      %SYS>do ^GBLOCKCOPY
      
      This routine will do a fast global copy from a database to another database or
      to a namespace. If a namespace is the destination, the global will follow any
      mappings set up for the namespace.
       
      1) Interactive copy
      2) Batch copy
      3) Exit
       
      Option?1
      
    3. At the ^GBLOCKCOPY prompt, specify 1, for an interactive copy:

      Option? 1
       
      1) Copy from Database to Database
      2) Copy from Database to Namespace
      3) Exit
       
      Option?
      
    4. When ^GBLOCKCOPY prompts for a copy type, select 1, for copying from database to database

      Option? 1
      Source Directory for Copy (? for List)?
      

      Here, either specify the name of the encrypted database or enter ? to see a numbered list of databases, which includes the encrypted database. If you enter ?, ^GBLOCKCOPY displays a list such as this one:

      Source Directory for Copy (? for List)? ?
       
      1) C:\InterSystems\MyIRIS\mgr\
      2) C:\InterSystems\MyIRIS\mgr\irislocaldata\
      3) C:\InterSystems\MyIRIS\mgr\irisaudit\
      4) C:\InterSystems\MyIRIS\mgr\irislib\
      5) C:\InterSystems\MyIRIS\mgr\iristemp\
      6) C:\InterSystems\MyIRIS\mgr\encrypted1\
      7) C:\InterSystems\MyIRIS\mgr\encrypted2\
      8) C:\InterSystems\MyIRIS\mgr\unencrypted\
       
      Source Directory for Copy (? for List)?
      

      Enter the number of the encrypted database, such as 7 here.

    5. When ^GBLOCKCOPY prompts for a destination directory for copying the data, enter the name of an unencrypted database or ? for a list similar to the one for the source directory.

    6. When ^GBLOCKCOPY asks if you wish to copy all globals, enter Yes (can be Yes, Y, y, and so on):

      All Globals? No => y
      
    7. If there is an empty global in the database, ^GBLOCKCOPY will now ask if you wish to copy it. This will appear something like the following:

      All Globals? No => y
       
      ^oddBIND     contains no data
      Include it anyway? No =>
      

      Enter No (can be No, N, n, and so on), which is the default.

    8. ^GBLOCKCOPY then asks if you wish to skip all the other empty globals. Enter Yes (can be Yes, Y, y, and so on), which is the default:

      Exclude any other similar globals without asking again? Yes =>
      

      There then appears a list of all the empty globals that are not being copied:

      Exclude any other similar globals without asking again? Yes => Yes
      ^oddCOM      contains no data --  not included
      ^oddDEP      contains no data --  not included
      ^oddEXT      contains no data --  not included
      ^oddEXTR     contains no data --  not included
      ^oddMAP      contains no data --  not included
      ^oddPKG      contains no data --  not included
      ^oddPROC     contains no data --  not included
      ^oddPROJECT  contains no data --  not included
      ^oddSQL      contains no data --  not included
      ^oddStudioDocument contains no data --  not included
      ^oddStudioMenu contains no data --  not included
      ^oddTSQL     contains no data --  not included
      ^oddXML      contains no data --  not included
      ^rBACKUP     contains no data --  not included
      ^rINC        contains no data --  not included
      ^rINCSAVE    contains no data --  not included
      ^rINDEXEXT   contains no data --  not included
      ^rINDEXSQL   contains no data --  not included
      ^rMACSAVE    contains no data --  not included
      9 items selected from
      29 available globals
      
    9. ^GBLOCKCOPY then asks if you wish to disable journaling for this operation:

      Turn journaling off for this copy? Yes =>
      

      Enter Yes (can be Yes, Y, y, and so on), which is the default.

    10. ^GBLOCKCOPY then asks if to confirm that you wish to copy the data:

      Confirm copy? Yes =>
      

      Enter Yes (can be Yes, Y, y, and so on), which is the default. Depending on the size of the database and the speed of the processor, you may see the status of the copy as it progresses. When it completes, ^GBLOCKCOPY displays a message such as:

      Copy of data has completed
      
    11. ^GBLOCKCOPY then asks if you wish to save statistics associated with the copy. Enter No (can be No, N, n, and so on), which is the default:

      Do you want to save statistics for later review? No =>
      

      Control then returns to the main prompt.

  5. Test that the copied data is valid. You can do this by examining the classes, tables, or globals in the Management Portal’s System Explorer for the database into which ^GBLOCKCOPY has copied the data.

  6. If the data is valid, perform steps 3 and 4 of this procedure for each database encrypted with the inaccessible or damaged key.

  7. Once you have made copies of every encrypted database into an unencrypted database, make a second copy of each database, preferably to a different machine than that which holds the first copy of each.

  8. Now — and only now — you can dismount all encrypted databases and deactivate the active key (that is, the key for which the key file is missing or damaged). InterSystems IRIS requires that you dismount all encrypted databases prior to deactivating their key.

You now have your data in one or more unencrypted databases and there is no activated key.

To re-encrypt the formerly encrypted databases, the procedure is:

  1. Create a new database encryption key according to the procedure described in Create a Key on the KMIP Server.

  2. Create a new backup copy of the key file as described in Protection from Accidental Loss of Access to Encrypted Data.

    Caution:

    Make sure you take the precautions described in Protection from Accidental Loss of Access to Encrypted Data; failure to follow these procedures can result in the permanent loss of data.

  3. Create one or more new encrypted databases, using the new key.

  4. Import the data exported in the previous procedure into the new encrypted database(s).

Your data is now stored in encrypted databases for which you have a valid key and a backup copy of the key file containing that key.

If the KMIP Server Is Required at Startup and Is Not Accessible

Under certain conditions related to the required use of one or more database encryption keys at startup, the system starts in single-user mode. These conditions are:

  • InterSystems IRIS is configured for either interactive or unattended startup.

  • Startup specifies that journal files and/or the IRISTEMP and IRISLOCALDATA databases are encrypted, or an encrypted database is specified as required at startup.

  • The KMIP server that holds the required database encryption key is not accessible.

If the Connection to the KMIP Server is Briefly Unavailable

The simplest solution to this case is when there has been a problem such as a network outage or the KMIP server is otherwise temporarily not running; in these cases, address networking or server problem and, if required, restart InterSystems IRIS.

If the KMIP Server Suffers a Longer-Term Outage

If it is not possible to connect to the KMIP server longer-term:

  1. Start a Administrator Terminal Session according to the instructions in the most recent entry in the messages.log file. Typically, this specifies starting a Terminal session with the -B flag..

    For example, at a Windows command line, for an instance of InterSystems IRIS called “MyIRIS” that is installed in the default location, the command would be:

    c:\InterSystems\MyIRIS\bin\irisdb -sc:\InterSystems\MyIRIS\mgr -B
    

    This connects you with InterSystems IRIS in the operating system terminal window; the prompt in that window changes from the operating system prompt to the InterSystems IRIS %SYS prompt.

  2. If any encrypted databases require mounting at startup, disable this feature for them:

    1. From the Management Portal Home page, go to the Local Databases page (System Administration > Configuration > System Configuration > Local Databases).

    2. Click the name of the database in the table of databases. This displays the Edit: page for the database.

    3. On the Edit: page, clear the Mount Required at Startup check box.

    4. Click Save.

  3. Run the ^STURECOV routine at the Terminal prompt. In that routine, configure InterSystems IRIS database startup options not to require a database encryption key. This means that the IRISTEMP and IRISLOCALDATA databases as well as journal files should now operate as expected; it also means that any encrypted databases cannot be mounted.

  4. When you are satisfied that InterSystems IRIS is ready for use, use ^STURECOV to complete the startup procedure. InterSystems IRIS then starts in multi-user mode.

Caution:

If you have not performed the actions described in Protection from Accidental Loss of Access to Encrypted Data, then your data may no longer be available in any form. This is a very serious problem, but, if you do not have a key, there is no way to retrieve the lost data.

FeedbackOpens in a new tab