Working with Streams

docs.intersystems.com

Home / Application Development: Core Topics / Defining and Using Classes / Working with Streams

Defining and Using Classes

InterSystems: The power behind what matters

Streams provide a way to store extremely large amounts of data (longer than the string length limit). You can define stream properties in any object class. You can also define stand-alone stream objects for other purposes, such as for use as an method argument or return value. This chapter describes streams and stream properties. It covers the following topics:

Introduction to stream classes
How to define stream properties
How to use the stream interface
Stream classes for use with gzip files
How stream properties are projected to SQL (for persistent objects)

When viewing this book online, use the preface of this book to quickly find related topics.

Introduction to Stream Classes

InterSystems IRIS Data Platform™ provides the following stream classes:

%Stream.GlobalCharacter — use this to store character data in global nodes.
%Stream.GlobalBinary — use this to store binary data in global nodes.
%Stream.FileCharacter — use this to store character data in an external file.
%Stream.FileBinary — use to store binary data in an external file.
%Stream.TmpCharacter — use this when you need a stream to hold character data but do not need to save the data.
%Stream.TmpBinary — use this when you need a stream to hold binary data but do not need to save the data.

These classes all inherit from %Stream.Object, which defines the common stream interface.

The %Library package also includes stream classes, but those are deprecated. The class library includes additional stream classes, but those are not intended for general use.

Note that stream classes are object classes. Thus a stream is an object.

Important:

Many of the methods of these classes return status values. In all cases, consult the class reference for details. If a method returns a status value, your code should check that returned value and proceed appropriately. Similarly, for %Stream.FileCharacter and %Stream.FileBinary, if you set the Filename property, you should next check for an error by examining %objlasterror.

Declaring Stream Properties

InterSystems IRIS supports both binary streams and character streams. Binary streams contain the same sort of data as type %Binary, and are intended for very large binary objects such as pictures. Similarly, character streams contain the same sort of data as type %String, and are intended for storing large amounts of text. Character streams, like strings, may undergo a Unicode translation within client applications.

Stream data may be stored in an external file or an InterSystems IRIS global (or not at all), depending on how the stream property is defined:

The %Stream.FileCharacter and %Stream.FileBinary classes are used for streams stored as external files.
The %Stream.GlobalCharacter and %Stream.GlobalBinary classes are used for streams stored as globals.
The %Stream.TmpCharacter and %Stream.TmpBinary classes are used for streams that do not need to be saved.

The first four classes can use the optional LOCATION parameter to specify a default storage location.

In the following example, the JournalEntry class contains four stream properties (one for each of the first four stream classes), and specifies a default storage location for two of them:

Class testPkg.JournalEntry Extends %Persistent
{
Property DailyText As %Stream.FileCharacter;

Property DailyImage As %Stream.FileBinary(LOCATION = "C:/Images");

Property Text As %Stream.GlobalCharacter(LOCATION = "^MyText");

Property Picture As %Stream.GlobalBinary;
}

In this example, data for DailyImage is stored in a file (with an automatically generated name) in the C:/Images directory, while the data for the Text property is stored in a global named ^MyText.

Using the Stream Interface

All streams inherit a set of methods and properties used to manipulate the data they contain. The next section lists the commonly used methods and properties, and the following sections provide concrete examples using them:

Important:

Commonly Used Stream Methods and Properties

Some commonly used methods include the following:

Read() — Read a specified number of characters starting at the current position in the stream.
Write() — Append data to the stream, starting at the current position. Overwrites existing data if the position is not set to the end of the stream.
Rewind() — Move to the beginning of the stream.
MoveTo() — Move to a given position in the stream.
MoveToEnd() — Move to the end of the stream.
CopyFrom() — Copy the contents of a source stream into this stream.
NewFileName() — Specify a filename for a %Stream.FileCharacter or %Stream.FileBinary property.

Commonly used properties include the following:

AtEnd — Set to true when a Read encounters the end of the data source.
Id — The unique identifier for an instance of a stream within the extent specified by %Location.
Size — The current size of the stream (in bytes or characters, depending on the type of stream).

For detailed information on individual stream methods and properties, see the InterSystems Class Reference entries for the classes listed at the beginning of this chapter.

Reading and Writing Stream Data

At the core of the stream interface are the methods Read(), Write(), and Rewind() and the properties AtEnd and Size.

The following example reads data from the Person.Memo stream and writes it to the console, 100 characters at a time. The value of len is passed by reference, and is reset to 100 before each Read. The Read method attempts to read the number of characters specified by len, and then sets it to the actual number of characters read:

  Do person.Memo.Rewind()
  While (person.Memo.AtEnd = 0) {
    Set len = 100
    Write person.Memo.Read(.len)
  }

Similarly, you can write data into the stream:

  Do person.Memo.Write("This is some text. ")
  Do person.Memo.Write("This is some more text.")

Specifying a Translation Table

If you are reading or writing a stream of type %Stream.FileCharacter in any character set other than the native character set of the locale, you must set the TranslateTable property of the stream. For an overview of translation tables, see “Default I/O Tables” in the chapter “Localization Support” of the Orientation Guide for Server-Side Programming.

Copying Data between Streams

All streams contain a CopyFrom() method which allows one stream to fill itself from another stream. This can be used, for example, to copy data from a file into a stream property. In this case, one uses the %Library.File class, which is a wrapper around operating system commands and allows you to open a file as a stream. In this case, the code is:

  // open a text file using a %Library.File stream
  Set file = ##class(%File).%New("\data\textfile.txt")
  Do file.Open("RU") // same flags as the OPEN command

  // Open a Person object containing a Memo stream
  // and copy the file into Memo
  Set person = ##class(Person).%New()
  Do person.Memo.CopyFrom(file)

  Do person.%Save() // save the person object
  Set person = ""   // close the person object
  Set file = ""     // close the file

You can also copy data into a stream with the Set command:

  Set person2.Memo = person1.Memo

where the Memo property of the Person class holds an OREF for a stream and this command copies the contents of person1.Memo into person2.Memo.

Note:

Using Set with two streams in this manner does not copy the OREF of one stream to the other — it copies the stream contents exclusively. This differs from the behavior of the deprecated stream classes in %Library,.

Inserting Stream Data

Apart from the temporary stream classes (whose data cannot be saved), streams have both a temporary and a permanent storage location. All inserts go into the temporary storage area, which is only made permanent when you save the stream. If you start inserting into a stream, then decide that you want to abandon the insert, the data stored in the permanent location is not altered.

If you create a stream, start inserting, then do some reading, you can call MoveToEnd() and then continue appending to the temporary stream data. However, after you save the stream, the data is moved to the permanent storage location. If you then reload the stream and start inserting, it inserts into the temporary storage area, rather than appending to the permanently stored data.

If this is the behavior you want, you need to create a temporary stream, for example:

    Set test = ##class(Test).%OpenId(5)
    Set tmpstream = ##class(%Stream.GlobalCharacter).%New()
    Do tmpstream.CopyFrom(test.text)
    Do tmpstream.MoveToEnd()
    Do tmpstream.Write("append text")
    Set test.text = tmpstream
    Set tmpstream = ""
    // Now do whatever you want with the test object

In this example, we create a temporary stream of the required type, then copy from the stream stored in the Test object, which puts this data in the temporary storage area of our new stream. Then we append to this stream and put its OREF into the Test object and close it to keep the reference counts correct.

Finding Literal Values in a Stream

The stream interface includes the FindAt() method, which you can use to find the location of a given literal value. This method has the following arguments:

method FindAt(position As %Integer, target, ByRef tmpstr, caseinsensitive As %Boolean = 0) as %Integer

Where:

position is the position at which to start searching.
target is the literal value to search for.
tmpstr, which is passed by reference, returns information that can be used in the next call to FindAt(). Use this when you want to search the same stream repeatedly, starting from the last position where the target was found. In this scenario, specify position as –1 and pass tmpstr by reference in every call. Then each successive call to FindAt() will start where the last call left off.
caseinsensitive specifies whether to perform a case-insensitive search. By default, the method does not consider case.

The method returns the position at this match starting at the beginning of the stream. If it does not find a match, it returns -1.

Saving a Stream

When you use a stream class as an object property, the stream data is saved when you save the containing object.

When you use a stream class as a standalone object, use the %Save() method to save the stream data. (Note that for the temporary stream classes — %Stream.TmpCharacter and %Stream.TmpBinary — this method returns immediately and does not save any data.)

Using Streams in Object Applications

Stream properties are manipulated via a transient object that is created by the object that owns the stream property. Streams act as literal values (think of them as large strings). Two object instances cannot refer to the same stream.

In the following example, a long memo is created and then written to the console:

    // create object and stream
    Set p = ##class(Person).%New()
    Set p.Name = "Mo"
    Do p.Memo.Write("This is part one of a long memo")

    // ...

    Do p.Memo.Write("This is part 10000 of a long memo")
    Do p.%Save()
    Set p = ""

    // read object and stream
    Set p = ##class(Person).%Open(oid)
    Do p.Memo.Rewind() // not required first time

    // write contents of stream to console, 100 characters at a time
    While (p.Memo.AtEnd = 0) {
        Set len = 100
        Write p.Memo.Read(.len)
    }
    Set p = ""

Stream Classes for Use with gzip Files

The %Stream package also defines the specialized stream classes %Stream.FileBinaryGzip and %Stream.FileCharacterGzip, which you can use to read and write to gzip files. These use the same interface described earlier. Note the following points:

For these classes, the Size property returns the uncompressed size. When you access the Size property, InterSystems IRIS reads the data in order to calculate the size of the file and this can be an expensive operation
When you access the Size property, InterSystems IRIS rewinds the stream and leaves you at its start.

Projection of Stream Properties to SQL and ODBC

As described earlier in this book, a persistent class is projected as an SQL table. For such classes, character stream properties and binary stream properties are projected to SQL (and to ODBC clients) as BLOBs (binary large objects).

Stream properties are projected with the ODBC type LONG VARCHAR (or LONG VARBINARY). The ODBC driver/server uses a special protocol to read/write BLOBs. Typically you have to write BLOB applications by hand, because the standard reporting tools do not support them.

The following subsections describes how to use stream properties with SQL. It includes the following topics:

Stream fields have the following restrictions within SQL:

You cannot use a stream value in a WHERE clause, with a few specific exceptions. For further details, refer to the WHERE clause in the InterSystems SQL Reference.
You cannot UPDATE/INSERT multiple rows containing a stream; you must do it row by row.

For information on indexing a stream value, contact the InterSystems Worldwide Response Center.

Reading a Stream via Embedded SQL

You can use embedded SQL to read a stream as follows:

Use embedded SQL to select the ID of the stream:
```
  &sql(SELECT Memo INTO :memo FROM Person WHERE Person.ID = 12345)
```
This fetches the ID of the stream and places it into the memo host variable.
Then open the stream and process it as usual.

Writing a Stream via Embedded SQL

To write a stream via embedded SQL, you have several options. For the value to insert, you can use an object reference (OREF) of a stream, the string version of such an OREF, or a string literal.

The following examples show all these techniques. For these examples, assume that you have a table named Test.ClassWStream which has a column named Prop1, which expects a stream value.

The following example uses an object reference:

///use an OREF
ClassMethod Insert1()
{
    set oref=##class(%Stream.GlobalCharacter).%New()
    do oref.Write("Technique 1")

    //do the insert; this time use an actual OREF
    &sql(INSERT INTO Test.ClassWStreams (Prop1) VALUES (:oref))
}

The next example uses a string version of an object reference:

///use a string version of an OREF
ClassMethod Insert2()
{
    set oref=##class(%Stream.GlobalCharacter).%New()
    do oref.Write("Technique 2")

    //next line converts OREF to a string OREF
    set string=oref_""

    //do the insert
    &sql(INSERT INTO Test.ClassWStreams (Prop1) VALUES (:string))
}

The last example inserts a string literal into the stream Prop1:

///insert a string literal into the stream column
ClassMethod Insert3()
{
    set literal="Technique 3"

    //do the insert; use a string
    &sql(INSERT INTO Test.ClassWStreams (Prop1) VALUES (:literal))
}

Note:

The first character of the string literal cannot be a number. If it is a number, then SQL interprets this as an OREF and attempts to file it as such. Because the stream is not an OREF, this results in an SQL -415 error.