Caché Specialized System Tools and Utilities
Migration and Conversion Utilities
[Back] 
   
Server:docs2
Instance:LATEST
User:UnknownUser
 
-
Go to:
Search:    

This chapter discusses some specialized tools and techniques that can be used when migrating a Caché database to a new system.

Using cvendian to Convert Between Big-Endian and Little-Endian Systems
Caché provides a utility to convert the byte order of a Caché database from Big-endian (that is, most-significant byte first) to Little-endian (that is, least-significant byte first), and vice versa. It is called cvendian, for convert endian. This is useful when moving a database among platforms of the two types. It also provides an option to report on the byte order of a given database.
Important:
Never use this utility on a mounted database. Always take the database offline before using this utility on it.
Platforms and Location of Utility
On Windows and UNIX® systems, the cvendian utility is the file <cache-install-dir>\Bin\cvendian.exe. On OpenVMS systems, this utility is not provided. For information about the Endianness of supported platforms, see “Platform Endianness” in InterSystems Supported Platforms.
Conversion Process
You can run cvendian on either the system that has the files to be converted or the system that will be using the converted files (unless one of those systems is running OpenVMS; see the preceding section).
For example, to convert a database from a Little-endian to a Big-endian system, you can perform the conversion on the Little-endian system and then transfer the database to the Big-endian system, or you can transfer the file first, and then convert it.
Note:
This utility does not work for backup and journal files. You must restore databases on a platform of the same endian, move the restored databases to the different endian platform, and then use the cvendian utility to convert the databases.
To convert a database, the process is:
  1. Make a copy of your database files, because the utility replaces the source files with the converted files.
  2. Run cvendian using the syntax described in the Utility Syntax section.
If you are using legacy multivolume databases, after converting a multivolume database file, use the ^LABEL utility to rename the directory in each volume. Using the Caché Terminal, from the %SYS namespace, call ^LABEL with the Do command; it prompts you for the proper input:
USER>zn "%SYS"
 
%SYS>Do ^LABEL
 
 
Enter the name of the directory in which the database is
stored. For a multi-volume database, enter the name of the
primary volume's directory, even if you want to relabel a
secondary volume. For a multi-volume legacy 2K database, you
should enter the name of the secondary volume directory if
you need to relabel it.
 
Directory: 
Utility Syntax
With the cvendian endian utility, you can specify the desired byte order, or you can report the current byte order without conversion. Use the following syntax:
cvendian [-option] file1 [file2 ... file8]
The option argument is one of the following:
You can shorten the options to their initial letter. If this is a conversion request (-big or -little), and the database already has the specified byte order, the utility displays a warning message and stops processing.
If you do not provide the option argument, the utility converts the database from the existing byte order to the other byte order. It is recommended, however, that you use the option argument.
The file1 through file8 arguments are the files to convert; each file can include a complete pathname. The file2 through file8 arguments are for a multivolume database; if you are converting a multivolume database, you must specify all the volumes on the command line, in order.
The utility performs the following actions:
For multivolume databases, if the files are out of order or the list is not complete, the utility does not perform any conversions and leaves the files as they are.
Example
For example, suppose you are converting a database for use on Solaris SPARC from Windows XP. Because SPARC and Intel have incompatible data representations, you must convert from Little-endian (for Intel) to Big-endian (for SPARC). The output from running cvendian on the Windows system before moving the file to the Solaris system looks similar to this:
C:\CacheSys\Bin>cvendian -big c:\temp\solarisdb\cache.dat

This database is little-endian.
This database has a block size of 8192 bytes.

This database has 1 volume and 1 map.
The last block in the primary volume is 18176.

Original manager directory is c:\temp\solarisdb\

No extension volumes.

Done converting c:\temp\solarisdb\cache.dat to big-endian

C:\CacheSys\Bin>
You can now move the converted database file to the Solaris system.
Using cdbmerge To Consolidate Database Extents Into One File
Historically, Caché allowed databases to span multiple files called extents. This was due, in part, because of the disk hardware and filesystems were unable to support files large enough to contain the entirety of a database.
Advances in storage technology and in filesystem capabilities have removed that impediment. As a result, the support for extents is no longer needed. In Caché version 10.1, support for extents was deprecated; in version 14.1, it was removed entirely.
Occasionally, however, it is necessary to mount and read extent-style databases. To address this issue, beginning with version 2012.2.4, InterSystems provides a stand-alone utility to consolidate a directory containing a database and its extents into a single cache.dat database file. That utility is called cdbmerge. It is located in the Bin subdirectory of <install_dir>.
To effect this conversion, do the following:
  1. Restore the database and extents to be consolidated to its own directory.
  2. Run the cdbmerge utility on that directory.
  3. Configure and mount the converted database on Caché.
The cdbmerge command has two forms:
cdbmerge <Original_Dir>

cdbmerge -srcdir <Original_Dir> -destdir <Final_Dir>
where
Note:
InterSystems recommends having a copy of the <Original_Dir> available before doing the merge as a safeguard against unexpected errors.
Converting FileMan Files into Caché Classes
FileMan is a set of utilities that provide metadata storage, access, and manipulation for MUMPS applications. This section describes how to map FileMan files to Caché classes. The FileMan source code is in the public domain and is not provided by InterSystems. However, Caché provides the FileMan Mapping Utility, which reads files created by FileMan and generates Caché classes that map them.
This section discusses the following topics:
This section assumes that the FileMan globals (^DD and ^DIC) are already loaded into the desired namespace in your system.
Overview of the FileMan Mapping Utility
The FileMan wizard enables you to quickly and easily create custom class mappings for your FileMan files. The classes created by the mapping utility enable you to access the file data via Caché ObjectScript and via SQL. You can map just one, many, or all FileMan files in a namespace. For each file, you can control details such as the following:
Accessing the FileMan Wizard
To access the FileMan wizard:
  1. Open the Management Portal.
  2. Click [System Explorer] > [SQL].
  3. Check the namespace in the header. Click Switch to change to the correct namespace, if necessary.
On this page, you can do the following:
Specifying the Default Settings
The FileMan wizard uses many settings that control how it generates classes. It is worthwhile to review these settings, modify them to values that you use most of the time, and save these as the default settings.
To modify the settings:
  1. Open the Management Portal.
  2. Make changes as needed and click Save.
Or access the wizard as described in Accessing the FileMan Wizard, modify the values on the first page, and then click Save As Default.
The available settings are as follows:
Mapping FileMan Files
To map FileMan files to Caché classes:
  1. Access the FileMan wizard (see Accessing the FileMan Wizard).
  2. Optionally change settings on the first page of the wizard. For information on the settings, see the previous section, Specifying the Default Settings.
  3. Click Next.
    You use the next page of the wizard to specify which files to map.
  4. Enter a complete or partial file name and click Search.
    The wizard displays all the FileMan files in this namespace that start with the given string.
  5. Do one of the following:
  6. Review the selected files.
    The wizard displays your selected files in a tree, with fields for each file. If you want to remove any of them, click a file name then click the "Remove" link above the tree. If you click a field then click "Remove" then the file name for that field is removed too. If you want to empty the Cart (remove all of them), just click the "Clear Cart" link above the tree.
  7. Optionally do any of the following:
  8. Click Finish.
    The wizard then starts the background job that map the files.
Programmatic Access
The FileMan wizard uses the %fm2class routine, which is installed as part of Caché and Ensemble 2009.1.
You can also obtain this routine as an XML file and install it in Caché or Ensemble version 5.0.* or higher.
This section describes how to install this routine, if needed, and how to use it. Rather than invoking the routine directly, you use the methods in $SYSTEM.OBJ.FM2Class.
System Requirements
You must be running a standard Caché or Ensemble installation, version 5.0.* or higher.
You need a working FileMan installation.
Installing the Routine on Earlier Product Versions
The %fm2class routine is delivered as an XML file that can be loaded into the system. To install on Caché or Ensemble version 5.1 or higher:
  1. First make sure the CACHELIB database is not mounted read only.
  2. Then load the XML file into the %SYS namespace using the following command:
    Do $SYSTEM.OBJ.Load("C:\Kits\FM2Class_v101.xml","psc") 
    Replace "C:\Kits\" with your actual path name. You are now ready to run the utility.
To install on Caché or Ensemble version 5.0, contact InterSystems Support for assistance.
Specifying Configuration Settings
The %fm2class routine uses the same settings that you specify in the FileMan wizard, as shown in the following table. For any setting that is displayed as a drop-down list, the following table indicates the corresponding values to use. For all yes/no settings, 1 means yes, and 0 means no. For other settings, use the same value as documented for the wizard.
Setting Name in FileMan Wizard Internal Name of Setting; Notes
Owner of the Classes Created owner
Package Name to Create the Classes in package
Super Classes superClasses
Table Name Format Based on the File Name and Number tableNameFormat
Child Table Name Format Based on the File Name and Number childTableNameFormat
Maximum length of Property, Trigger, and Foreign Key names nameLength
Field Name Format Based on the Fileman Field Name fieldNameFormat
Datatype to use for FileMan Date fields dateType
Datatype to use for FileMan DateTime fields datetimeType
Define STRICTDATA=1 for %FilemanData* datatypes strictData
Expand Pointers? expandPointers
Expand Set Of Codes fields xpandSetOfCodes
Create Value for Variable Pointer fields? variablePointerValueField
Define Set Of Codes fields as type? setOfCodesEnum
Define Required Properties for Fields that are requiredType
Extended Mapping extendedMapping
Name of the IEN Field ienFieldName
Retain Class? retainClass
Recursion recursion
0 means no recursion, 1 means partial recursion, and 2 means full recursion.
Word-Processing Fields Conversion wpIsList
Read Only? readonly
Log File logFile
Compile Classes? compile
Compile Flags compileQSpec
Delete Flags deleteQSpec
Display Result display
0 means no display, 1 means minimal display, and 2 means full display.
To modify the default settings, use a command like the following:
Set ^%SYS("sql","fm2class",setting) = value
Where setting is an internal setting name as shown in the previous table, and value is the value to assign to it.
When you use the %fm2class routine, you can pass in an array of the following format:
arrayName(setting) = value
For example:
%fm("display")=1 
%fm("package")="VISTA" 
%fm("superClasses")="%XML.Adaptor" 
%fm("wpIsList")= 1  
Mapping FileMan Files Programmatically
As noted earlier, rather than invoking the %fm2class routine directly, you use methods in %System.OBJ.FM2Class, as follows:
The arguments are as follows:
The class %System.OBJ.FM2Class also provides the methods Version() and GetVersion().
Next Steps
After using the conversion utility, you should verify that you have SQL access to the generated classes. To do so, use Management Portal. At a minimum, check that you can do the following:
It may also be necessary to create global and routine mappings in the namespace. Check the routines used in your generated classes to be sure that all of them are available in this namespace.
Notes
This section contains notes about how the %fm2class routine generates Caché class definitions.
Basics
To orient yourself, you may find it helpful to compare a simple FileMan file to the resulting Caché class definitions.
Consider the simple example of a file called PostalCode. If we look at the file attributes for this file (via the FileMan Data Dictionary Utility), we see the cross-references and indices for this file. For example:
The same utility also shows us the definitions of the fields in this file. In this case, the titles of the fields are as follows (when listed alphabetically)
The FileMan Data Dictionary Utility also shows that the PostalCode file includes two pointers:
If we map this file, five classes are created. This includes a class to represent PostalCode, as well as classes to which this class refers. In Caché Studio, we would see the following, if we had specified TEST as the package name:
The class definition for TEST.POSTALCODE is too long to show, but the details are summarized here:
Mapping of Variable Pointer Fields
Each variable pointer field in the file that is mapped to a Caché class defines a property in the class that is marked with the SqlComputed keyword. For each file to which this variable pointer field refers, an additional property is created in the class, and this property is marked with the SqlComputed and Transient keywords. If VariablePointerFieldName is the name of the variable pointer field, and PointerFileName is the name of the file to which it points, the added property is VariablePointerFieldNamePointerFileName; the SQL field name for this property is VariablePointerFieldName_PointerFileName.
For example, suppose the file ABC includes a variable pointer field called VP. The VP field can point to the Red, White, or Blue files. This creates the following properties in the class definition:
Property SqlFieldName Details
VP   SQL computed upon INSERT and UPDATE. Stored in the map definition. Contains the internal value for the variable pointer field. Usually something like "41;DIC(40.7,", which means it points to row 41 in the 40.7 file.
VPRed VP_Red A transient computed field that is null unless the VP field for this row points to the Red file. To update the value of VP via SQL INSERT or UPDATE to point to a row in the Red table/file, set the value of VP_Red to the ID of that row.
VPWhite VP_White A transient computed field that is null unless the VP field for this row points to the White file. To update the value of VP via SQL INSERT or UPDATE to point to a row in the White table/file, set the value of VP_White to the ID of that row.
VPBlue VP_Blue A transient computed field that is null that is NULL unless the VP field for this row points to the Blue file. To update the value of VP via SQL INSERT or UPDATE to point to a row in the Blue table/file, set the value of VP_Blue to the ID of that row.
In all cases, the VP field is computed and stored automatically.
In addition to defining the three reference fields VP_Red, VP_White, and VP_Blue, the utility also creates three foreign key constraints in the ABC class for these references. This is done to ensure referential integrity is maintained between the files.
Mapping of New-Style Cross-References
This utility converts new-style cross-references to index maps if the cross-reference is defined by simple set/kill logic. This allows the Caché query optimizer to choose a new-style cross-reference index (if one exists) and increase query performance in some cases.
Advanced Queries
You can use the classes %FileMan.File and %FileMan.Field to query the FileMan data dictionary.
Also, you can use the classes %FileMan.MappedFile and %FileMan.MappedField to view metadata regarding the mapping between FileMan and the class definition.
For information on the %FileMan classes, see the Intersystems Class Reference.
WebLink Developer Tags for Conversion to CSP
The tags listed in this section have been superseded by CSP tags. They are documented here to assist in migrating applications from Weblink Developer to Caché Server Pages (CSP).
For more information on each tag, please see the Weblink Developer Guide.
<WLD:A>
<WLD:A>...</WLD:A>
This rule implements the action= mechanism of WLD.
WLD:ACTIONSCRIPT
<WLD:ACTIONSCRIPT>...</WLD:ACTIONSCRIPT>
The <CSP:METHOD> tag is superseded by the <SCRIPT LANGUAGE=CACHE METHOD> tag.
Attribute Description Value
MethodName The name of the generated method. Must be a valid method name. methodName:STRING
ReturnType The return type of the method. dataType:STRING
Arguments A list of arguments for the method. spec:STRING
<WLD:DHTMLGRID>
<WLD:DHTMLGRID>...</WLD:DHTMLGRID>
This rule implements the <WLD:DHTMLGRID> tag.
Attribute Description Value
Name Name of the grid. A string.
Cols Maximum number of columns. An integer.
Rows Maximum number of rows. An integer.
<WLD:END>
<WLD:END>
Required. This tag should be put at the very end of any page that uses the WLD functionality, just before the </BODY> tag.
<WLD:ERROR>
<WLD:ERROR>
The <WLD:ERROR> tag displays the Error variable as an alert, as per Developer.
<WLD:FOCUS>
<WLD:FOCUS>
The <WLD:FOCUS> tag forces focus onto a field, as per WebLink Developer. Note the execution is carried out in the <WLD:END> tag.
Attribute Description Value
FIELD Field name on which to focus. A string.
FORM Form number containing field. A string.
<WLD:GRID>
<WLD:GRID>...</WLD:GRID>
This rule implements the standard <WLD:GRID> tag.
Attribute Description Value
Name Name of the grid. A string.
Cols Maximum number of columns. An integer.
Rows Maximum number of rows. An integer.
<WLD:GRIDDATA>
<WLD:GRID>...<WLD:GRIDDATA>...</WLD:GRIDATA>
The purpose of this empty rule is to instantiate the <WLD:GRIDDATA> tag into the DOM.
Attribute Description Value
Value Value to be displayed. A string.
Col Column of the cell where the value is to be displayed. An integer.
Row Row of the cell where the value is to be displayed. An integer.
<WLD:GRIDHEADING>
<WLD:GRID>...<WLD:GRIDHEADING>...</WLD:GRIDHEADING>
The purpose of this empty rule is to instantiate the <WLD:GRIDHEADING> tag into the DOM.
Attribute Description Value
Name The name for the column. A string.
Col The column where the heading is to be displayed. An integer.
<WLD:OPTIONS>
<LIST>...<WLD:OPTIONS>
The <WLD:OPTIONS> tag generates options from the LIST array.
<WLD:PostPageScript>
<WLD:PostPageScript>...</WLD:PostPageScript>
Script to run after this page.
<WLD:PrePageScript>
<WLD:PrePageScript>...</WLD:PrePageScript>
Script to run before this page.
<WLD:SHOWLV>
<WLD:SHOWLV>
The WLDSHOWLV displays all request and session object contents.
<WLD:START>
<WLD:START>
Required. This tag should be put at the start of any page that uses the WLD functionality.
<WLD:WRITETEXT>
<TEXTAREA>...<WLD:WRITETEXT>
The WLD:WRITETEXT generates text from the TEXTAREA array.
<XSQL:QUERY>
<XSQL:QUERY>...</XSQL:QUERY>
Oracle XSQL tag
Attribute Description Value
CONNECTION Connection identifier. A string.
MAX-ROWS Specifies the maximum number of rows to return (not including any rows skipped by skip-rows). A string.
ROW-ELEMENT Specifies the XML tag for each row, the default is ROW. A string.
ROWSET-ELEMENT Specifies the XML tag for the row set, the default is ROWSET. A string.
SKIP-ROWS Specifies the number of rows to skip before returning data. A string.
TAG-CASE Controls the case of the XML tags, 'upper' means uppercase and 'lower' specifies lowercase. The default is to use the case of the column names. A string.
XMLNS:XSQL Must be URN:ORACLE-XSQL for CSP support of Oracle A string.
<INPUT TYPE=WLDCHECKBOX>
<INPUT TYPE=WLDCHECKBOX>
This rule implements the mechanisms for emulating WLD's CHECKBOX button interface.
<INPUT SRC TYPE=WLDIMAGE>
<INPUT SRC TYPE=WLDIMAGE>
This rule implements the mechanisms for supporting WLD's NEXTPAGE= and ACTION= within a <input type=image> tag.
Attribute Description Value
Action Function or expression to be executed on submit. Function or expression.
Name Name of image. String.
NextPage Specifies the page to display next. URI.
OnClick A pointer button was clicked. JavaScript code.
<INPUT TYPE=WLDRADIO>
<INPUT TYPE=WLDRADIO>
This rule implements the mechanisms for emulating WLD's RADIO button interface.
Attribute Description Value
Name Name of radio button. String.
Onclick A pointer button was clicked. JavaScript code.
Value Value of radio button. String.
<INPUT TYPE=WLDSUBMIT>
<INPUT TYPE=WLDSUBMIT>
This rule implements the mechanisms for supporting WLD's NEXTPAGE= and ACTION= within a <input type=submit> tag.
Attribute Description Value
Action Function or expression to be executed on submit. Function or expression.
Name Name of form. String.
NextPage Specifies the page to display next. URI.
Onclick A pointer button was clicked. JavaScript code.