Caché I/O Device Guide
Sequential File I/O
[Back] [Next]
   
Server:docs1
Instance:LATEST
User:UnknownUser
 
-
Go to:
Search:    

This chapter describes using sequential files in Caché. Most of the information in this chapter applies to sequential files of all types on all operating systems. A few matters pertain specifically to Record Management System (RMS) files on OpenVMS systems; these are described in the section Using RMS Files.

All operating systems consider disk I/O files as sequential files. OpenVMS systems can treat a magnetic tape file as a sequential file using standard RMS file parameters. For further details on magnetic tape, refer to the Magnetic Tape I/O chapter of this manual. Windows systems consider printers as sequential file I/O devices (unless the printer is connected through a serial communications port). UNIX® and OpenVMS systems consider printers as terminal I/O devices. For further details on printers, refer to the Printers chapter of this manual.
Using Sequential Files
This section discusses how Caché processes sequential files. It provides an introduction to sequential file I/O and descriptions of the relevant commands.
OPEN Command
OPEN opens a Caché sequential file. Remember that you cannot use the OPEN command to open a Caché database file.
The OPEN command by itself does not prevent another process from opening the same sequential file. You can govern concurrent sequential file access by using the OPEN command “L” mode parameter and/or the Caché ObjectScript LOCK command. File locking support is provided by the file access rules of the underlying operating system.
Caché allocates each process' open file quota between database files and files opened with the Caché ObjectScript OPEN command. When an OPEN command causes too many files to be allocated to OPEN commands, a <TOOMANYFILES> error occurs. The Caché maximum number of open files for a process is 1,024. The actual maximum number of open files for each process is a platform-specific setting. For example, Windows defaults to a maximum of 998 open files per process. Consult the operating system documentation for your system.
OPEN Syntax
OPEN filename{{:({parameters{:reclength{:terminators}}})}{:timeout}}
where
Argument Description
filename Any valid file specification, enclosed in quotation marks. This file pathname must not exceed 255 characters (Windows and UNIX® platforms), or 244 characters (OpenVMS). Valid characters may be 8-bit ASCII, or ISO Latin-1 Unicode.
parameters Optional — A string of one-letter codes, enclosed in quotation marks, that define the file format and types of operations you can perform. (You may also specify parameters using keywords that begin with the slash (/) character.) See the table OPEN Mode Parameters,” for definitions of these codes. If the parameters do not include R or W, then R is the default. This system-wide default open mode can be configured by setting the OpenMode property of the Config.Miscellaneous class. To open a new file, you must specify the parameter N for new. Otherwise, the OPEN will hang or return unsuccessfully from a timeout. If the parameters do not include S, V, F, or U, then the default for a new Windows or UNIX® file is S (V for OpenVMS), and the default for an existing file is the mode specified when the file was created. If A is not specified, WRITE operations will overwrite the previous contents of the file. Parameters are applied in left-to-right order.
reclen Optional — For Windows and UNIX systems, specifies the maximum record length for (S) and (U) records, or the absolute record length for fixed-length (F) records. Ignored for variable-length (V) records. Default value is 32767. This argument is not available for OpenVMS RMS files.
terminators Optional — A string of user-defined record terminators that apply to stream mode only. They let you override the default terminators: carriage return, line feed, and form feed. User-defined terminators only apply to input, they do not affect how data is written to the file (terminators are written to a file as special characters). If there's more than one user-defined terminator, it is treated as a list of terminators, not a multi-character sequence to be used as a single terminator. This argument is not available for OpenVMS RMS files.
timeout Optional — Number of seconds during which Caché attempts to open the file. If it does not succeed within this interval, it sets $TEST to 0 and returns control to the process. If it succeeds, it sets $TEST to 1.
The timeout argument, though optional, is strongly recommended because the success or failure of OPEN is indicated by the value of the $TEST special variable, and $TEST is only set if timeout is specified. $TEST is set to 1 if the open attempt succeeds before the timeout expires; if the timeout expires, $TEST is set to 0.
OPEN Mode Parameters
You can specify the OPEN mode parameters in either of two ways:
When specifying a combination of letter code parameters and keyword parameters, specify the letter code string first, followed by the keyword parameter(s), separated with colons. The following example specifies three letter code parameters, followed by two keyword parameters, followed by the reclen and timeout arguments.
  OPEN "mytest":("WNS":/OBUFSIZE=65536:/GZIP=0:32767):10
OPEN Mode Parameters
Letter Code Keyword Description
N /NEW
New file. If the specified file does not exist, Caché creates the file. If the specified file already exists, Caché creates a new one with the same name. The status of the old file depends on which operating system you are using. On UNIX® and Windows, Caché deletes the old file. (Note that file locking should be used to prevent two concurrent processes using this parameter from overwriting the same file.) On OpenVMS, Caché supersedes the old file with a new file with a higher version number; the old file is not deleted.
If the “N” option is not specified and the file specified in OPEN does not exist, the Windows and UNIX® default is to not create a new file; the OpenVMS default is to create a new file. This behavior is configurable using the FileMode() method of the %SYSTEM.Process class. The system-wide default behavior can be established by setting the FileMode property of the Config.Miscellaneous class.
E
/CREATE
/CRE
UNIX® only — Create a file if it does not exist. Does not delete and recreate an existing file, as the N mode does. The default is to not create a new file. This default is overridden if the FileMode() method of the %SYSTEM.Process class, or the FileMode property of the Config.Miscellaneous class is enabled.
D
/DELETE[=n]
/DEL[=n]
Delete File: Specifies that the file should be automatically deleted when it is closed. /DELETE or /DELETE=n for nonzero values of n enable the parameter code. /DELETE=n for a zero value of n disables the parameter code. The default is to not delete a file.
R /READ Read: Caché permits READ access the file. Other processes may also access this file (however, see “L” parameter). If you attempt to open a nonexistent file in “R” mode, the process hangs. To prevent this situation, use a timeout. “R” is the default for all platforms (Windows, UNIX®, and OpenVMS). The system-wide default open mode can be configured by setting the OpenMode property of the Config.Miscellaneous class.
W
/WRITE
/WRI
Write: Caché permits WRITE access to the file. In Windows and UNIX®, “W” gives the process shared write access to the file, with exclusive write access to the record. Use “WL” to specify exclusive write access to the file. In OpenVMS, “W” gives exclusive write access to the file. If you attempt to open a nonexistent file in “W” mode, the process hangs until the file is created or the process is resolved by a timeout, a Process Terminate, or RESJOB. “R” is the default for all platforms. The system-wide default open mode can be configured by setting the OpenMode property of the Config.Miscellaneous class. In Windows, can be used with /OBUFSIZE.
L   Locked Exclusive: Use this option with the “W” (Write) option to specify exclusive write access to a file. “WL” or “WRL” specifies that the current process has exclusive write access to the file. A file opened with “RL” may still have shared read access. The effects of the “L” option on concurrent opens are different in Windows and UNIX®. Refer to the “OPEN Mode Locking” section, below, for further details. On UNIX® systems if one process specifies “WL” (or “WRL”) access to a file, other processes requesting read access to that file must specify “RL” so that UNIX® can coordinate file locking. This option is not used by OpenVMS, and is ignored. When OpenVMS opens a file in “W” mode, it provides exclusive write access by default. To override this OpenVMS behavior, use the RMS file “H” (shared) mode parameter.
A
/APPEND
/APP
Append: WRITE operations append data to the end of an existing file. The default is to overwrite existing data, rather than append.
S /STREAM Stream format with carriage return, line feed, or form feed as default terminators. Jobbed processes that inherit TCP devices are automatically set to “S” format. You can reset the format with the USE command. S, V, F, and U modes are mutually exclusive. Stream record format is the default on non-OpenVMS systems. Variable length (V) is the default on OpenVMS systems.
V /VARIABLE
Variable length: Each WRITE creates one record. For Windows and UNIX®, a variable record can be of any length; the reclen argument is ignored. For OpenVMS, the maximum variable length record is limited to 32,767 characters. OpenVMS automatically handles the actual record length during a WRITE.
Do not attempt to insert records at any point other than the end of a variable-length sequential file; a WRITE will render inaccessible all data in the file from the point after the WRITE on. S, V, F, and U modes are mutually exclusive. Stream record (S) format is the default on non-OpenVMS systems. Variable length is the default on OpenVMS systems.
A variable-length record written using a translation table, such as Unicode data using UTF8 translation, can result in a stored record with a different string length than the input data. Caché uses the original input string length when reading this record.
F
/FIXED
/FIX
Fixed length: Each record is the length specified in the reclen argument. All data written is packed into fixed-length records. For example:
OPEN "myfile":("NWF":4) USE "myfile" WRITE "ABC","DEF","GHI","JKL"
produces the following three records: ABCD EFGH IJKL. This works only for READ operations (not WRITE operations). S, V, F, and U modes are mutually exclusive. OpenVMS RMS files cannot use this option; use RMS “B” (block) mode instead.
U /UNDEFINED Undefined length: Specifies that file records have an undefined length and therefore READ operations must specify the number of characters to read. The maximum record length is specified in the reclen argument. No translation on output. Default value is the maximum string length. S, V, F, and U modes are mutually exclusive.
K\name\
Knum
/TRANSLATE[=n]: /IOTABLE[=name]
/TRA[=n]: /IOT[=name]
I/O Translation Mode: When you use the “K” parameter for a device, I/O translation will occur for that device if translation has been enabled system-wide. You identify the previously defined table on which the translation is based by specifying the table's name. When using keywords you specify /TRANSLATE to enable I/O translation (n=1 to enable; n=0 to disable), and /IOTABLE=name to specify the translation table to use. For a list of available translation tables, refer to Encoded Translation in the $ZCONVERT function documentation. The + and - options for turning protocols on and off are not available with the K protocol. (The older form Knum, where “num” represents the number of the slot the table is loaded in, is being phased out but is still supported. The system manager can display slot numbers in the %NLS utility in the selection window for each table type.) This parameter may be used with either the OPEN command or the USE command.
Y\name\
Ynum
/XYTABLE[=name]
/XYT[=name]
$X/$Y Action Mode: When you use the “Y” parameter for a device, the system uses the named $X/$Y Action Table. You identify the previously defined $X/$Y Action Table on which translation is based by specifying the table's name. $X/$Y action is always enabled. If “Y” is not specified and a system default $X/$Y is not defined, a built in $X/$Y action table is used. The + and - options for turning protocols on and off are not available with the Y protocol. (The older form Ynum, where “num” represents the number of the slot the table is loaded in, is being phased out but is still supported. The system manager can display slot numbers in the NLS utility in the selection window for each table type.) This parameter may be used with either the OPEN command or the USE command.
  /NOXY [=n] No $X and $Y processing: /NOXY or /NOXY=n (for nonzero values of n) disables $X and $Y processing. This can substantially improve performance of READ and WRITE operations. The values of the $X and $Y variables are indeterminate, and margin processing (which depends on $X) is disabled. /NOXY=0 enables $X and $Y processing; this is the default. This parameter may be used with either the OPEN command or the USE command.
  /OBUFSIZE=int Windows only — Output Buffering: Creates an output WRITE buffer. The int variable is an integer that specifies the size of the buffer in bytes. May only be used when the file is open for write only (“W”, not “R” or “RW”). May provide significant performance improvement when performing multiple small writes, especially over a WAN. However, data in buffer may be lost if a system crash occurs. Data in buffer is flushed to disk upon CLOSE, or WRITE *-1 or WRITE *-3.
  /GZIP [=n] GZIP Compression: Specifies GZIP-compatible stream data compression. /GZIP or /GZIP=n (for nonzero values of n) enables compression on WRITE and decompression on READ. /GZIP=0 disables compression and decompression. Before issuing /GZIP=0 to disable compression and decompression, check the $ZEOS special variable to make sure that a stream data read is not in progress. /GZIP compression has no effect on I/O translation, such as translation established using /IOTABLE. This is because compression is applied after all other translation (except encryption) and decompression is applied before all other translation (except encryption).
OPEN Argument Keywords
The following table describes the OPEN command argument keywords for sequential files:
OPEN Keyword Arguments for Sequential Files
Keyword Default Description
/PARAMS=str
/PAR=str
No default Corresponds to the parameters positional parameter. (It provides a way to specify a parameter letter code string in a position-independent way).
/RECORDSIZE=int
/REC=int
No default Corresponds to the reclen positional parameter, which establishes a record size for fixed-length records. (Currently only implemented for READ operations.)
/TERMINATOR=str
/TER=str
No default Corresponds to the terminators positional parameter, which establishes user-defined terminators. str is a string of user-defined record terminators that apply to stream mode only. They let you override the default terminators: carriage return, line feed, and form feed. User-defined terminators only apply to input, they do not affect how data is written to the file (terminators are written to a file as special characters). If there's more than one user-defined terminator, it is treated as a list of terminators, not a multi-character sequence to be used as a single terminator. This argument is not available for OpenVMS RMS files.
OPEN Mode Locking
When two processes attempt to open the same sequential file, the second OPEN succeeds or fails based on the mode used by the first OPEN. The following tables show the interactions between two opens using exclusive (“L”) and non-exclusive read and write modes. Note that the interpretation of these interactions is platform-dependent. Tables are provided for Windows operating systems and UNIX® operating systems; OpenVMS does not support the “L” mode parameter.
In the following tables, the horizontal axes indicates the open mode of the first OPEN and the vertical axis indicates the open mode of the second OPEN. A 1 indicates that the second OPEN succeeds; a 0 indicates that the second OPEN fails.
Windows OPEN Mode Interactions
  W RW RL WL RWL R
W 1 1 1 0 0 1
RW 1 1 1 0 0 1
RL 1 1 1 0 0 1
WL 0 0 0 0 0 0
RWL 0 0 0 0 0 0
R 1 1 1 0 0 1
For Windows systems, the interactions in this table apply equally to concurrent opens from the same Caché instance, concurrent opens from two different Caché instances, or concurrent opens by Caché and a non-Caché application (with restrictions on non-Caché applications, as described below).
UNIX® OPEN Mode Interactions
  W RW RL WL RWL R
W 1 1 1 1 1 1
RW 1 1 1 1 1 1
RL 1 1 1 0 0 1
WL 1 1 0 0 0 1
RWL 1 1 0 0 0 1
R 1 1 1 1 1 1
For UNIX® systems, the interactions in this table only to concurrent opens from the same Caché instance. They do not govern concurrent opens from two different Caché instances, or concurrent opens by Caché and a non-Caché application.
Interactions with Non-Caché Software
On Windows systems, opening a sequential file in Caché for “WL” write access generally prevents a non-Caché application from opening the sequential file for write access. Similarly, a non-Caché application opening a sequential file for write access generally prevents a Caché process from concurrent “WL” write access.
However, certain non-Caché applications, including the Notepad and WordPad applications, open a file, make a copy of the file in shared mode, and then immediately close it. Thus a Caché process could still open the file in “WL” mode. An error would occur when one of these non-Caché applications then either attempts to save changes from their copy to the original, or attempts to reopen the original file. A more serious situation can occur as follows: If one of these non-Caché applications opens a file, then Caché opens, modifies, and closes the file, then the non-Caché application saves changes to the file, the changes made by both processes are saved, and the integrity of the file data could be compromised.
On UNIX® systems, opening a sequential file in Caché for “WL” write access generally has no effect on the behavior of non-Caché applications. You must use locks to reliably restrict write access from non-Caché applications.
Examples
The following example opens the file “LUDWIG.B” in the current directory. Because it specifies no mode parameters, it opens the file with read access and in stream mode by default:
  OPEN "LUDWIG.B"
The following example opens a new file “LIST.FILE” in the current directory, with write access, in stream format. Notice that you do not need parentheses when you include only the first of the arguments they would normally enclose.
  OPEN "LIST.FILE":"WNS"
The following example opens a file “CARDS” in the current directory, with read and write access, and 80-character fixed-length records.
  OPEN "CARDS":("FRW":80)
The following example opens the stream-format file “STRNG” in the directory c:\usr\dir, with non-default terminators.
  OPEN "c:\usr\dir\STRNG":("S"::$CHAR(0)_$CHAR(255))
USE Command
The USE command makes an opened sequential file the current device. You can have many open sequential files, but you can use only one sequential file at a time.
Syntax
USE file:position
where
USE Command Parameters
Argument Description
file Any valid file specification, enclosed in quotation marks. The specified file must already have been opened.
position Optional — The position of the next READ or WRITE within the file. The position value is a numerical expression whose meaning depends on the record format of the file. For fixed-length records, position is the absolute record number, relative to zero, where each record contains the number of characters specified in the previous OPEN command. For stream or variable-length records, position is the absolute byte position relative to zero. The default is to read or write records sequentially from the beginning of the file. You cannot specify a position for OpenVMS RMS files.
USE-Only Command Keywords
In addition to the command keywords that it shares with OPEN, listed above, the USE command has its own set of keywords:
USE-Only Command Keywords for Sequential Files
Keyword Default Description
/POSITION=n Current file position. (The file pointer position is at the beginning of a file when it is first opened, unless the file was opened in append mode. In that case, the file pointer position is at the end of the file.) Corresponds to the positional parameter, which sets the position of the next READ or WRITE within a file.
READ and WRITE Commands
After a positioned READ or WRITE, subsequent READ or WRITE operations proceed sequentially until the next USE command with a position parameter.
READ Command
The READ command reads data from the current device, one record at a time. Reading past the end of file causes an <ENDOFFILE> error.
Syntax
READ X 
READ X:timeout 
READ X#n
where
Argument Description
X The variable that will hold the record read from the file.
n Number of characters to read.
timeout The number of seconds to wait for the read operation to complete before timing out. Either a numeric value or a variable that resolves to a numeric value.
The timeout argument, though optional, is strongly recommended because the success or failure of the READ is indicated by the value of the $TEST special variable if timeout is specified. $TEST is set to 1 if the read attempt succeeds before the timeout expires; if the timeout expires, $TEST is set to 0.
WRITE Command
The WRITE command writes data, one record at a time, to the sequential file that is the current device.
Syntax
WRITE x
where
Argument Description
x The data in variable x is written as one record in the sequential file.
Example
The following example reads the third, fourth, and fifth records of a fixed-length file:
   SET myfile="FIXED.LEN"
   OPEN myfile:("FR":100)
   USE myfile:2 
   READ var1(3),var1(4),var1(5)
CLOSE Command
The CLOSE command relinquishes ownership of a sequential file.
Syntax
CLOSE file
CLOSE file:"D"
CLOSE file:("R":newname)
Argument Description
file Closes the file with the name specified in the argument.
"D" Closes and deletes the file with the name specified in the argument.
("R":newname) Closes the file with the name specified in the argument and renames it newname.
CLOSE-Only Command Keywords
The following table describes the keywords for controlling sequential files with only the CLOSE command.
CLOSE-Only Command Keywords for Sequential Files
Keyword Default Description
/DELETE[=n]
/DEL[=n]
0, unless the file was marked for delete when it was opened. Corresponds to the D parameter code, which specifies that the file should be deleted. /DELETE or /DELETE=n for nonzero values of n enable the parameter code and /DELETE=n for a zero value of n disables the parameter code.
/RENAME=name
/REN=name
Do not rename the file. Corresponds to the R parameter code and the file name positional parameter. The R parameter code specifies that the file should be renamed and the file name positional parameter gives the new name of the file.
Using RMS Files
OpenVMS uses RMS (Record Management System) sequential files. RMS files can be stored on disk or magnetic tape. If the file is on a magnetic tape, the tape must first be mounted at the OpenVMS level without the FOREIGN parameter.
RMS sequential files are accessed using the same Caché ObjectScript commands as other sequential files: OPEN, USE, READ, WRITE, and CLOSE. The principal difference is that RMS supports several open mode parameters not supported for other sequential files.
Note:
Records in an RMS file must be no greater than 32,767 characters.
OPEN Command for RMS Files
The OPEN command obtains ownership of an RMS file for subsequent read and write operations.
The OPEN command by itself does not prevent another process from opening the same RMS sequential file, or creating a new version of the file. If your OPEN command specifies the “R” (read-only) parameter, it receives GET sharing; otherwise, it receives no sharing unless you specify the “H” parameter (share for any type of access). “R” (read-only) is the default. If you specify the “WN” parameters, sharing does not occur but each process creates a new version of the file. You can govern concurrent sequential file access by using the Caché ObjectScript LOCK command.
Syntax
OPEN "file"{:"parameters"{:timeout}}
where
Argument Description
file Any valid file specification, enclosed in quotation marks. This is the only required argument.
parameters A string of one-letter codes, enclosed in quotation marks, that define the file format and types of operations you can perform. See the “OPEN Mode Parameters” table below.
timeout Number of seconds to try to open the file. If Caché does not open the file in that amount of time, you receive a <NOTOPEN> error.
The following table lists the codes you can include in the parameter string of the OPEN command for an RMS file and lists the commands theses codes represent. You may specify any combination of these characters.
Note:
The OPEN parameters that define data format (S, V, F, B, or M) must be used consistently each time the file is opened. The data format default is V. If the data format parameter(s) that you specified when initially creating an RMS file do not match the data format parameter(s) specified (or defaulted to) in a subsequent OPEN operations, file operations follow neither the initial parameter setting, nor the setting of the most recent OPEN. Instead, RMS attempts to reconcile these two data formats, leading to often unexpected results. This behavior is established by OpenVMS, not Caché.
Additional OPEN Mode Parameters for RMS Files
Code Command Description
H /SHARED Open a file in Shared mode for any type of access. The default is not shared.
C
/CRFMT[=n]
/CRF[=n]
Carriage return, carriage control: Specifies carriage return/line feed format. /CRFMT or /CRFMT=n for nonzero values of n enable the parameter code. /CRFMT=n for a zero value of n disables the parameter code.
C can be used with either S or V data format. The RMS file default is V (variable length). C cannot be used with the B, F, or U mode.
M Emulate DSM The Caché default mode is that each WRITE operation creates a separate record. This parameter sets the DSM mode. When this parameter is specified, multiple WRITE operations write to the same record until you specify a record terminator. This parameter sets up a file type with a variable length records, the CR/CC (Carriage Return/Carriage Control) file attribute, and buffering of all output until a record terminator (that is, “!” or “#”) is encountered. The maximum record length for the file is 32KB. B, M, S, V, F, and U modes are mutually exclusive.
B Block Record length defaults to 32256. Block mode does not truncate the file on WRITE or CLOSE. B, M, S, V, F, and U modes are mutually exclusive.
Z /VBUF Connects a file to the view buffer. The default is no view buffer.