Skip to main content

SYS.Shadowing.Shadow

persistent class SYS.Shadowing.Shadow extends %Library.Persistent, %SYSTEM.Help

SQL Table Name: SYS_Shadowing.Shadow

API for the shadow (aka "Destination of Shadowing", the opposite of the data source)

Example 1: creating a shadow, the short version, assuming identical Cache configurations (down to the directory level) on the shadow and the source

    s MyShadow=##class(SYS.Shadowing.Shadow).%New()
    s MyShadow.Name="123.45.67.89"  ;source IP address
    s rc=MyShadow.%Save()
  
Example 2: creating a shadow, the longer version
    s MyShadow=##class(SYS.Shadowing.Shadow).%New()
    s MyShadow.Name="Shadow of a local Cache system"
    s MyShadow.SourceName="localhost"
    s MyShadow.SourcePort=50019
    s MyShadow.JournalStore="d:\shadow\"
    d MyShadow.SetDatabaseToShadow("c:\cachesys\mgr\user\","d:\shadow\user\")
    s rc=MyShadow.%Save()
  
Example 3: controlling and monitoring a shadow
    s MyShadow=##class(SYS.Shadowing.Shadow).%OpenId("Shadow of a local Cache system")
    w MyShadow.SourceName," ",MyShadow.SourcePort
    s rc=MyShadow.Start($lb("","",0,"0,"_$zu(78,3))) i 'rc d $System.Status.DisplayError(rc)
    w ##class(%ShadowState).LogicalToDisplay(MyShadow.Status)
    s rc=MyShadow.Suspend() i 'rc d $System.Status.DisplayError(rc)
    w ##class(%ShadowState).LogicalToDisplay(MyShadow.Status)
    s rc=##class(SYS.Shadowing.Shadow).CheckPointInfo(MyShadow.Name,.cpinfo) i rc zw cpinfo
    s rc=MyShadow.Resume() i 'rc d $System.Status.DisplayError(rc)
    s rc=MyShadow.Stop() i 'rc d $System.Status.DisplayError(rc) 
    s rc=MyShadow.Restart() i 'rc d $System.Status.DisplayError(rc) 
  

Property Inventory

Method Inventory

Parameters

parameter DOMAIN = %Utility;
Default Localization Domain

Properties

property DatabaseRedirection as %SysPath [ MultiDimensional ];
Database redirections for dejournaling (i.e., applying SETs and KILLs)
This is an array with keys being the source database directories and values, the corresponding shadow database directories. An entry for a cluster-mounted source database also contains a secondary key indicating the cluster mount state
Default = all user databases mounted on the shadow, on the assumption that the shadow is identical to the source, down to the database level.
Property methods: DatabaseRedirectionDisplayToLogical(), DatabaseRedirectionGet(), DatabaseRedirectionIsValid(), DatabaseRedirectionLogicalToDisplay(), DatabaseRedirectionLogicalToOdbc(), DatabaseRedirectionNormalize(), DatabaseRedirectionSet()
property DisableJournalUpdates as %Boolean [ InitialExpression = 0 ];
If true, updates that the shadow applies to the shadow databases are NOT journaled, regardless of the journal settings on the databases. By default, this is false, that is, shadow updates are journaled.
Property methods: DisableJournalUpdatesDisplayToLogical(), DisableJournalUpdatesGet(), DisableJournalUpdatesIsValid(), DisableJournalUpdatesLogicalToDisplay(), DisableJournalUpdatesNormalize(), DisableJournalUpdatesSet()
property ErrorMax as %Integer [ InitialExpression = 10 ];
Max number of unique error messages to keep
Default = 10
Property methods: ErrorMaxDisplayToLogical(), ErrorMaxGet(), ErrorMaxIsValid(), ErrorMaxLogicalToDisplay(), ErrorMaxNormalize(), ErrorMaxSet()
property Filter as %VarString;
Name of a user routine that decides whether a journal record should be dejournaled. (Optional)
Property methods: FilterDisplayToLogical(), FilterGet(), FilterIsValid(), FilterLogicalToDisplay(), FilterLogicalToOdbc(), FilterNormalize(), FilterSet()
property JournalStore as %SysPath;
The directory for storing temporary journal files copied from the source
For cluster shadow, this is the parent directory; actual journal files are stored in its subdirectories named numerically (1, 2, 3, etc.), one for each node of the source cluster.
Default = "shadow" subdirectory of the manager directory
Property methods: JournalStoreDisplayToLogical(), JournalStoreGet(), JournalStoreIsValid(), JournalStoreLogicalToDisplay(), JournalStoreLogicalToOdbc(), JournalStoreNormalize(), JournalStoreSet()
property KeepOld as %Integer [ InitialExpression = 0 ];
How many days of old shadow copies of source journal files to keep
By default, the shadow copies are purged as soon as they are no longer in use, i.e., have been completely dejournaled and contain no existing open transactions, etc.
Property methods: KeepOldDisplayToLogical(), KeepOldGet(), KeepOldIsValid(), KeepOldLogicalToDisplay(), KeepOldNormalize(), KeepOldSet()
property Latency as %Integer [ Calculated ];
Latency of shadowing (dejournaling)
The latency is the time (in seconds) needed to apply the journal data copied from the source to shadow databases, based on current applying rate. -1 if latency is (temporarily) unavailable.
Property methods: LatencyDisplayToLogical(), LatencyIsValid(), LatencyLogicalToDisplay(), LatencyNormalize()
property Name as %String [ Required ];
Name ("ID") of the shadow
The character "~" (tilde) is NOT allowed in the name as it is used as delimiter in cache.cpf and for cluster shadow
A name in the form of HostName:Port (e.g., "localhost:1972") also defines SourceName and SourcePort, if they are not specified explicitly
Property methods: NameDisplayToLogical(), NameGet(), NameIsValid(), NameLogicalToDisplay(), NameLogicalToOdbc(), NameNormalize()
property ProbeInterval as %Integer [ InitialExpression = 1 ];
when caught up, how long (in seconds) does shadow wait before polling the source for any updates
Property methods: ProbeIntervalDisplayToLogical(), ProbeIntervalGet(), ProbeIntervalIsValid(), ProbeIntervalLogicalToDisplay(), ProbeIntervalNormalize(), ProbeIntervalSet()
property SSLConfig as %String;
[OPTIONAL] Name of SSL Configuration to use for connection
Set this to the name of a SSL Configuration if SSL/TLS communications are to be used.
Property methods: SSLConfigDisplayToLogical(), SSLConfigGet(), SSLConfigIsValid(), SSLConfigLogicalToDisplay(), SSLConfigLogicalToOdbc(), SSLConfigNormalize(), SSLConfigSet()
property ShareJournalDirectory as %Boolean [ InitialExpression = 0 ];
Whether it is OK to use either primary or alternate journal directory to store shadow journal files
Default: NO (recommended). You may change it to YES only if you can be sure that the local journal files don't conflict the shadow ones because of, for example, different prefixes.
Property methods: ShareJournalDirectoryDisplayToLogical(), ShareJournalDirectoryGet(), ShareJournalDirectoryIsValid(), ShareJournalDirectoryLogicalToDisplay(), ShareJournalDirectoryNormalize(), ShareJournalDirectorySet()
property SourceDatabaseMountState as %String [ MultiDimensional ];
Transient property (not saved to disk)
SourceDatabaseMountState(SourceDatabaseDirectory)=MountState, where MountState can be obtained from SourceDatabase() query
Modified by SetDatabaseToShadow() and SetDatabaseNotToShadow() methods
Retrieved by GetSourceDatabaseMountState() classmethod
Property methods: SourceDatabaseMountStateDisplayToLogical(), SourceDatabaseMountStateGet(), SourceDatabaseMountStateIsValid(), SourceDatabaseMountStateLogicalToDisplay(), SourceDatabaseMountStateLogicalToOdbc(), SourceDatabaseMountStateNormalize(), SourceDatabaseMountStateSet()
property SourceName as Config.Host;
IP address or DNS name of the source (aka "database server")
The default value is either derived from shadow ID (the 1st ":"-delimited field, see Name property) or "localhost", if shadow ID doesn't contain a valid IP address or DNS name.
Property methods: SourceNameDisplayToLogical(), SourceNameGet(), SourceNameIsValid(), SourceNameLogicalToDisplay(), SourceNameLogicalToOdbc(), SourceNameNormalize(), SourceNameSet()
property SourcePort as %Integer;
DefaultPort of the source (aka "database server")
The default value is derived from shadow ID (the 2nd ":"-delimited field, see Name property) or DefaultPort of the shadow, if shadow ID doesn't contain one.
Property methods: SourcePortDisplayToLogical(), SourcePortGet(), SourcePortIsValid(), SourcePortLogicalToDisplay(), SourcePortNormalize(), SourcePortSet()
property StartPoint as %String [ MultiDimensional ];
The property has been deprecated. It is ignored when you start shadowing via ^SHADOW or System Management Portal. You should always specify StartPoint in the Start() method.
Property methods: StartPointDisplayToLogical(), StartPointGet(), StartPointIsValid(), StartPointLogicalToDisplay(), StartPointLogicalToOdbc(), StartPointNormalize(), StartPointSet()
property Status as %ShadowState [ Calculated ];
Status of the shadow (active, inactive, etc.)
Property methods: StatusCanResume(), StatusCanStart(), StatusCanStop(), StatusCanSuspend(), StatusDisplayToLogical(), StatusIsValid(), StatusLogicalToDisplay(), StatusNormalize()
property Type as %ShadowState [ Calculated ];
Type of the shadow (cluster or not), determined dynamically
Property methods: TypeCanResume(), TypeCanStart(), TypeCanStop(), TypeCanSuspend(), TypeDisplayToLogical(), TypeIsValid(), TypeLogicalToDisplay(), TypeNormalize()

Methods

classmethod CheckPointInfo(ShadowID As %String, ByRef cpinfo As %String) as %Status
CheckPointInfo:
  Return information (journal location, open transaction, etc.) about
  the latest checkpoint of a shadow as given by 'ShadowID'.
Return success (1) or failure (0).
Output (by reference):
  cpinfo = CheckPoint# (of the latest checkpoint)
    Below, csn is always 0 for non-cluster shadow. For cluster shadow,
    csn = 0, 1, 2, ..., corresponding to each node of the source cluster.
  cpinfo(csn,"JournalLocation") = Offset,FilePath
  cpinfo(csn,"JournalLocation","Offset") = Offset  [*]
  cpinfo(csn,"JournalLocation","File") = FilePath
    [*] For non-cluster shadow, Offset is always the last one processed;
    for cluster shadow, Offset can be either the last one processed or the
    next one to process (queued), distinguished by a sign -- in the former
    case, the (last processed) offset is prefixed by a negative sign (-).

    If there is any open transaction, the following gives the journal
    location of the earliest open transaction (TSTART):
  cpinfo(csn,"OldestOpenTranLoc") = Offset,FilePath
  cpinfo(csn,"OldestOpenTranLoc","Offset") = Offset
  cpinfo(csn,"OldestOpenTranLoc","File") = FilePath

    -- for Cluster Shadow only --
  cpinfo(csn,"ClusterSequence") = ClusterSessionID,ClusterSequence#
  cpinfo(csn,"ClusterSequence","SessionID") = ClusterSessionID
  cpinfo(csn,"ClusterSequence","Sequence#") = ClusterSequence#
method Clear() as %Status
Erase traces of prior shadowing session, including checkpoints, journal files, etc., resetting the shadow to a clean state in preparation for a new START or failover.
The shadow must be in stopped state.
method ClearErrors() as %Status
errors is not part of the shadow configuration; it is generated at run-time thus, we don't need to go through the Property/%Save() route to clear it since errors is not a property, we expose it via query
classmethod Delete(ShadowID As %String, GlobalOnly As %Boolean = 0) as %Status
Delete a shadow configuration (incl. the working copy in ^SYS("shdwcli"))
classmethod GetGapFileSize(ShadowID As %String, CluShadowID As %String) as %Integer
classmethod GetLatency(ShadowID As %String) as %Integer
classmethod GetSource(ShadowID As %String, ByRef Host As %String, ByRef Port As %Integer) as %Status
method GetSourceDatabaseMountState(SourceDatabase As %SysPath = "") as %String
method GetStartPoint(ByRef Time As %TimeStamp, ByRef Event As %String, ByRef location As %String, ByRef LocationTranslated As %String) as %Status
Deprecated. See property StartPoint.
classmethod GetStatus(ShadowID) as %ShadowState
Status of a shadow (running or not running)
classmethod IsValid(ShadowID As %CacheString) as %Status
Is it a valid shadow configuration (i.e., ready to start)
method LatencyGet() as %Integer
method NameSet(Name As %String) as %Status
Set the name of the shadow, checking for nonpermissible characters "~"
method Restart() as %Status
Restart the shadow, if it is in stopped state
method Resume() as %Status
Resume the shadow, if it is in suspended state
method SetDatabaseNotToShadow(SourceDatabase As %String) as %Status
Preventing updates to the SourceDatabase on the data server from being applied to the shadow
method SetDatabaseToShadow(SourceDatabase As %String, ShadowDatabase As %String, MountState As %String = "") as %Status
Allowing journaled updates to the SourceDatabase on the data server to be applied to a ShadowDatabase on the shadow
method SetStartPoint(Time As %TimeStamp = "", Event As %String = "", Location As %String = "") as %Status
Deprecated. See property StartPoint.
classmethod SourceClusterEventsClose(ByRef qHandle As %Binary) as %Status
classmethod SourceClusterEventsExecute(ByRef qHandle As %Binary, ShadowID As %String, Host As %String, Port As %Integer) as %Status
classmethod SourceClusterEventsFetch(ByRef qHandle As %Binary, ByRef Row As %List, ByRef AtEnd As %Integer = 0) as %Status
classmethod SourceDatabasesClose(ByRef qHandle As %Binary) as %Status
classmethod SourceDatabasesExecute(ByRef qHandle As %Binary, ShadowID As %String, Host As %String, Port As %Integer) as %Status
classmethod SourceDatabasesFetch(ByRef qHandle As %Binary, ByRef Row As %List, ByRef AtEnd As %Integer = 0) as %Status
classmethod SourceEventsClose(ByRef qHandle As %Binary) as %Status
classmethod SourceEventsExecute(ByRef qHandle As %Binary, ShadowID As %String, Host As %String, Port As %Integer) as %Status
classmethod SourceEventsFetch(ByRef qHandle As %Binary, ByRef Row As %List, ByRef AtEnd As %Integer = 0) as %Status
method Start(ByRef StartPoint As %String) as %Status
Start the shadow
StartPoint should have one of the following nodes defined:
  • StartPoint("jrnloc") = JournalOffset,JournalFilePath OR
  • StartPoint("cluseq") = ClusterSessionID,ClusterSequence#
  • Alternatively, StartPoint can be a $LIST string with the following elements:
    • Time - Timestamp in the form of YYYY-MM-DD HH:MM:SS or ""
    • EventName - Name of the corresponding source event (e.g., journal switch, cluster start, etc.) or ""
    • IsClusterSequence - 1 if the next element is a cluster sequence; 0 otherwise (that is, a journal location)
    • JournalOffset,JournalFilePath or ClusterSessionID,ClusterSequence#, depending on IsClusterSequence value
    Note:
    1. The journal location or cluster sequence in StartPoint is of the source of the shadowing.
    2. The Time and EventName elements are for mnemonic purpose and can be given as "".
classmethod StartGroup(ByRef GroupOperation As %String) as %Status
Start or resume shadows as a group for better memory allocation
Generally, GroupOperation is an array with keys being the IDs of the shadows to resume or start and values, the respective operations. Possible operation values are:
  • "RESUME" - to resume the suspended shadow from last checkpoint;
  • "RESTART" - to restart the stopped shadow from last checkpoint;
  • "" - to start the stopped shadow from default startpoint;
  • startpoint - to start the stopped shadow from the given startpoint.
See Start() method for startpoint format.
Note that subnodes of GroupOperation are not necessarily preserved.
Alternatively, GroupOperation can be assigned one of the values described below (for backward compatibility purpose):
  • "SUSPENDED" - to resume all suspended shadows;
  • "ALL" - to resume all suspended shadows and start all stopped shadows.
The shared memory allocated for shadowing starts at half of the available gmheap (for 1 shadow) and gradually increases as the number of shadows in the group, but never exceeds two thirds of the available gmheap.
On return, GroupOperation is an array with keys being the IDs of the shadows and values, the status of the specified operations on individual shadows, i.e., GroupOperation(ShadowID)=Status.
Return:
  • 0, if there is no shadow in the specified group;
  • >0 (= the number of shadows in the group), if every shadow in the group started successfully;
  • error(s), if at least one shadow failed to start.
method StatusGet() as %ShadowState
Status of a shadow (running or not running)
method Stop(Forced As %Boolean = 0, RollBack As %Boolean = 0) as %Status
Stop the shadow
method Suspend() as %Status
Suspend the shadow
method Test(WaitTime As %Integer = 10) as %Status
Test connection to the source without actually shadowing it
WaitTime: how long to wait (in seconds) for test to complete (either fail or succeed)
classmethod TranslateLatency(latency As %String) as %String
Translate comma-delimited Latency value to text
method TypeGet() as %ShadowType
Type of a running shadow
classmethod Upgrade() as %Status
Upgrade shadows from 5.0 or earlier
classmethod UpgradeCheckPoint() as %Status
Upgrade shadows to new format of checkpoints and open transaction entries (in 2008.1?)
classmethod UpgradeOne(ShadowID As %String) as %Status
Upgrade a shadow from 5.0 or earlier

Queries

query DatabaseRedirections(ShadowID As %String)
Selects Source Database As %String, Shadow Database As %String
Returns info about database redirections of a shadow
query Errors(ShadowID As %String, PriorToError As %Integer = "")
Selects Time As %TimeStamp, PID As %Integer, $ZERROR As %String, Text As %String, Reoccurrences As %Integer, First Occurred At As %TimeStamp
Returns error info (if any) about a shadow configuration, in reverse order of time (i.e., latest error first). Examples:
  s rs=##class(%ResultSet).%New("SYS.Shadowing.Shadow:Error")
  w rs.Execute("MyShadow")
  
query IncompleteTransactions(ShadowID As %String)
Selects Process ID As %Integer, Remote System ID As %Integer, Journal File As %String, Journal Offset As %Integer
Returns info about transactions that are not yet committed in a shadow
query JournalRedirections(ShadowID As %String)
Selects Source Journal Directory As %String, Shadow Journal Directory As %String
Returns info about journal redirections of a shadow
query List()
Selects Name As %String, Status As %ShadowState, Source Name As %String, Source Port As %Integer, SSL Config As %String, Start Point As %String, Filter As %String, LogicalStatus As %ShadowState
Returns a list of shadows with some snapshot info, most of which are unlikely to change on the fly
query Progress(ShadowID As %String = "")
Selects Name As %String, Status As %ShadowState, Checkpoint As %String, Errors As %Integer, Open Transactions As %Boolean, Latency As %String, LogicalStatus As %Integer, Applying Journal As %String, Transfering Journal As %String
Returns details about shadows that tend to change on the fly
  • Name: Name of the shadow
  • Status: see %ShadowState
  • CheckPoint: Checkpoint number
  • Errors: Number of errors
  • Open Transactions: [yes/no] Any transaction open?
  • Latency: Estimated delay in applying updates
  • LogicalStatus: Logical value of Status
  • Applying Journal: Path of the shadow journal file currently being applied to databases along with the percentage of completion; on a cluster shadow, it is a comma-delimited list of such (one for each source cluster node)
  • Transfering Journal: Path of the shadow journal file currently being copied to from the source; on a cluster shadows, it is a comma-delimited list of such (one for each source cluster node)
query SourceClusterEvents(ShadowID As %String, Host As %String, Port As %Integer)
Selects Time As %TimeStamp, Event As %String, Location As %String
(Obsolete)
query SourceDatabases(ShadowID As %String, Host As %String, Port As %Integer)
Selects Directory As %String, Mount State As %String
Returns info about source databases
query SourceEvents(ShadowID As %String, Host As %String, Port As %Integer)
Selects Time As %TimeStamp, Event As %String, Location As %String
Returns a list of source events (journal switches, markers, etc.)

Indexes

index (IDKEY on ) [IdKey, Type = key];
Index methods: IDKEYCheck(), IDKEYDelete(), IDKEYExists(), IDKEYOpen(), IDKEYSQLCheckUnique(), IDKEYSQLExists(), IDKEYSQLFindPKeyByConstraint(), IDKEYSQLFindRowIDByConstraint()

Inherited Members

Inherited Methods

FeedbackOpens in a new tab