Using Caché Objects
Using the Object Synchronization Feature

This appendix describes the object synchronization feature, which you can use to synchronize specific tables in databases that are on “occasionally connected” systems. This appendix includes the following sections:

When viewing this book online, use the preface of this book to quickly find other topics.
Introduction to Object Synchronization
Object synchronization is a set of tools available with Caché objects that allows application developers to set up a mechanism to synchronize databases on “occasionally connected” systems. By this process, each database updates its objects. Object synchronization offers complementary functionality to Caché system tools that provide high availability and shadowing. Object synchronization is not designed to provide support for real-time updates; rather, it is most useful for a system that needs updates at discrete intervals.
For example, a typical object synchronization application would be in an environment where there is a master copy of a database on a central server and secondary copies on client machines. Consider the case of a sales database, where each sales representative has a copy of the database on a laptop computer. When Mary, a sales representative, is off site, she makes updates to her copy of the database. When she connects her machine to the network, the central and remote copies of the database are synchronized. This can occur hourly, daily, or at any interval.
Object synchronization between two databases involves updating each of them with data from the other. However, Caché does not support bidirectional synchronization as such. Rather, updates from one database are posted to the other; then updates are posted in the opposite direction. For a typical application, if there is a main database and one or more local databases (as in the previous sales database example), it is recommended that updates are from the local to the main database first, and then from the main database to the local one.
For object synchronization, the idea of client and server is by convention only. For any two databases, you can perform bidirectional updates; if there are more than two databases, you can choose what scheme you use to update all of them (such as local databases synchronizing with a main database independently).
This section addresses the following topics:
To ensure that updates work properly, each object in a database should be uniquely distinguishable. To provide this functionality, Caché gives each individual object instance a GUID — a globally unique ID. The GUID makes each object universally unique.
The GUID is optionally created, based on the value of the GUIDENABLED parameter. If GUIDENABLED has a value of 1, then a GUID is assigned to each new object instance.
Consider the following example. Two databases are synchronized and each has the same set of objects in it. After synchronization, each database has a new object added to it. If the two objects share a common GUID, object synchronization considers them the same object in two different states; if each has its own GUID, object synchronization considers them to be different objects.
How Updates Work
Each update from one database to another is sent as a set of transactions. This ensures that all interdependent objects are updated together. The content of each transaction depends on the contents of the journal for the “source” database. The update can include one or more transactions, up to all transactions that have occurred since the last synchronization.
Resolution of the following conditions is the responsibility of the application:
The SyncSet and SyncTime Objects
When two databases are to be synchronized, each has transactions in it that the other lacks. This is illustrated in the following diagram:
Two Unsynchronized Databases
Here, database A and database B have been synchronized at transaction 536 for database A and transaction 112 for database B. The subsequent transactions for each database need to be updated from each to the other. To do this, Caché uses what is called a SyncSet object. This object contains a list of transactions that are used to update a database. For example, when synchronizing database B with database A, the default contents of the SyncSet object are transactions 547, 555, 562, and 569. Analogously, when synchronizing database A with database B, the default contents of the SyncSet object are transactions 117, 124, 130, and 136. (The transactions do not use a continuous set of numbers, because each transaction encapsulates multiple inserts, updates, and deletes — which themselves use the intermediate numbers.)
Each database holds a record of its synchronization history with the other. This record is called a SyncTime table. For database, its contents are of the form:
Database     Namespace     Last Transaction Sent     Last Transaction Received
B            User          536                       112
The numbers associated with each transaction do not provide any form of a time stamp. Rather, they indicate the sequence of filing for transactions within an individual database.
Once database B has been synchronized with database A, the two databases might appear as follows:
Two Databases, Where One Has Been Synchronized with the Other
Because the transactions are being added to database B, they result in new transaction numbers in that database.
Analogously, the synchronization of database B with database A results in 117, 124, 130, and 136 being added to database A (and receiving new transaction numbers there):
Two Synchronized Databases
Note that the transactions from database B that have come from database A (140 through 162) are not updated back to database A. This is because the update from B to A uses a special feature that is part of the synchronization functionality. It works as follows:
  1. Each transaction in a database is labeled with what can be called “a database of origin.” In this example, transaction 140 in database B would be marked as originating in database A, while its transaction 136 would be marked as originating in itself (database B).
  2. The SyncSet.AddTransactions() method, which bundles a set of transactions for synchronization, allows you to exclude transactions that originate in a particular database. Hence, when updating from B to A, AddTransactions() excludes all transactions that originate in database A — because those have already been added to the transaction list for database B.
This functionality prevents creating infinite loops in which two databases continually update each other with the same set of transactions.
Modifying the Classes to Support Synchronization
Object synchronization requires that the sites have data with matching sets of GUIDs. If you are starting with an already existing database that does not yet have GUIDs assigned for its records, you need to assign a GUID to each instance (record) in the database, and then make sure there are matching copies of the database on each site. In detail, the process is:
  1. For each class being synchronized, set the value of the OBJJOURNAL parameter to 1.
    Parameter OBJJOURNAL = 1;
    This activates the logging of filing operations (that is, insert, update, or delete) within each transaction; this information is stored in the ^OBJ.JournalT global. An OBJJOURNAL value of 1 specifies that the property values that are changed in filing operations are stored in the system journal file; during synchronization, data that needs to be synchronized is retrieved from that file.
    OBJJOURNAL can also have a value of 2, though the possible use of this value is restricted to special cases. It is never for classes using the default storage mechanism (%CacheStorage). A value of 2 specifies that property values that are changed in filing operations are stored in the ^OBJ.Journal global; during synchronization, data that needs to be synchronized is retrieved from that global. Also, storing information in the global increases the size of the database very quickly.
  2. For each class being synchronized, set the value of its GUIDENABLED parameter to 1; this tells Caché to allow the class to be stored with GUIDs.
    Parameter GUIDENABLED = 1;
    Note that if this value is not set, the synchronization does not work properly. Also, you must set GUIDENABLED for serial classes, but not for embedded objects.
  3. Recompile the class.
  4. For each class being synchronized, give each object instance its own GUID by running the AssignGUID() method:
     Set Status = ##class(%Library.GUID).AssignGUID(className,displayOutput)
    The method returns a %Status value, which you should check.
  5. Put a copy of the database on each site.
Performing the Synchronization
This section describes how to perform the synchronization. The database providing the updates is known as the source database; and the database receiving the updates is the target database. To perform the actual synchronization, the process is:
  1. Each time you wish to synchronize the two databases, go to the instance with the source database. On the source database, create a new SyncSet using the %New() method of the %SYNC.SyncSet class:
     Set SrcSyncSet = ##class(%SYNC.SyncSet).%New("unique_value")
    The integer argument to %New(), unique_value, should be an easily identified, unique value. This ensures that each addition to the transaction log on each site can be differentiated from the others.
  2. Call the AddTransactions() method of the SyncSet instance:
     Do SrcSyncSet.AddTransactions(FirstTransaction,LastTransaction,ExcludedDB)
    This method collects the synchronization data and puts it in a global, ready for export.
    Or, to synchronize all transactions since the last synchronization, omit the first and second arguments:
     Do SrcSyncSet.AddTransactions(,,ExcludedDB)
    This gets all transactions, beginning with the first unsynchronized transaction to the most recent transaction. The method uses information in the SyncTime table to determine the values.
    ExcludedDB is a $LIST created as follows:
     Set ExcludedDB = $ListBuild(GUID,namespace)
  3. Call the ErrCount() method to determine how many errors were encountered. If there have been errors, the SyncSet.Errors query provides more detailed information.
  4. Export the data to a local file using the ExportFile() method:
     Do SrcSyncSet.ExportFile(file,displaymode,bUpdate)
  5. Move the exported file from the source machine to the target machine.
  6. Create a SyncSet object on the target machine using the SyncSet.%New() method. Use the same value for the argument of %New() as on the source machine — this is what identifies the source of the synchronized transactions.
  7. Read the SyncSet object into the Caché instance on the target machine using the Import() method:
     Set Status = TargetSyncSet.Import(file,lastSync,maxTS,displaymode,errorlog,diag)
    This method puts data into the target database. It behaves as follows:
    1. If the method detects that the object has been modified on both the source and target databases since the last synchronization, it invokes the %ResolveConcurrencyConflict() callback method; like other callback methods, the content of %ResolveConcurrencyConflict() is user-supplied. (Note that this can occur if either the two changes both modified a common property or the two changes each modified non-intersecting sets of properties.) If the %ResolveConcurrencyConflict() method is not implemented, then the conflict remains unresolved.
    2. If, after the Import() method executes, there are unsuccessfully resolved conflicts, these remain in the SyncSet object as unresolved items. Be sure to take the appropriate action regarding the remaining conflicts; this may involve resolution, leaving the items in an unresolved state, and so on.
    If this process completes, the Import() method returns success.
  8. Once the first database updates the second database, perform the same process in the other direction so that the second database can update the first one.
Object synchronization does not support synchronization of file streams.
Translating Between GUIDs and OIDs
To determine the OID of an object from its GUID or vice versa, there are two methods available:
Manually Updating a SyncTime Table
To perform a manual update on the SyncTime table for a database, invoke the SetlTrn() method, which sets the last transaction number:
 Set Status=##class(%SYNC.SyncTime).SetlTrn(syncSYSID, syncNSID, ltrn)
The SetlTrn() method sets the highest transaction number synced in on the target system instead of the default behavior (which is to set the highest transaction number exported from the source system). Either approach is fine and is a choice available during application development.