Serializing Large Dynamic Entities to Streams

Serializing Large Dynamic Entities to Streams

If a dynamic entity is large enough, the output of %ToJSON() may exceed the maximum possible length for a string (see “String Length Limit”). The examples in this section use a maximum length string named longStr. The following code fragment demonstrates how longStr is generated:

   set longStr=""
   for i=1:1:$SYSTEM.SYS.MaxLocalLength() { set longStr = longStr_"x" }
   write "Maximum string length = "_$LENGTH(longStr)

Maximum string length = 3641144

Whenever an expression uses the return value of %ToJSON(), the string is built on the program stack, which is subject to the string length limit. For example, a read/write statement such as write dyn.%ToJSON() or an assignment statement such as set x=dyn.%ToJSON() will attempt to put the string on the stack. The following example adds two copies of longStr to a dynamic array and attempts to assign the serialized string to a variable, causing ObjectScript to return a <MAXSTRING> error:

   set longArray = [(longStr),(longStr)]
   set tooBig = longArray.%ToJSON()

SET tooBig = longArray.%ToJSON()
^
<MAXSTRING>

The general solution to this problem is to pass the %ToJSON() output by reference in a DO command, without actually examining the return value. Output is written directly to the current device, and there is no limit on the length of the output. In the following examples, the device is a stream.

Writing to a file stream

This example writes dynamic object longObject to a file and then retrieves it. Variable longStr is the value defined at the beginning of this section:

   set longObject = {"a":(longStr),"b":(longStr)}
   set file=##class(%File).%New("c:\temp\longObjectFile.txt")
   do file.Open("WSN")
   do longObject.%ToJSON(file)
   do file.Close()

   do file.Open("RS")
   set newObject = {}.%FromJSONFile(file)
   write !,"Property newObject.a is "_$LENGTH(newObject.a)_" characters long."

Property newObject.a is 3641144 characters long.

This solution can also be used to read input from other streams.

Reading and writing global character streams

In this example, we serialize two large dynamic entities (using temporary streams because %ToJSON() can only serialize one entity per stream). Standard stream handling methods are used to store each temporary stream as a separate line in stream bigLines:

   set tmpArray = ##class(%Stream.GlobalCharacter).%New()
   set dyn = [(longStr),(longStr)]
   do dyn.%ToJSON(tmpArray)

   set tmpObject = ##class(%Stream.GlobalCharacter).%New()
   set dyn = {"a":(longStr),"b":(longStr),"c":(longStr)}
   do dyn.%ToJSON(tmpObject)

   set bigLines = ##class(%Stream.GlobalCharacter).%New()
   do bigLines.CopyFrom(tmpArray)
   do bigLines.WriteLine()
   do bigLines.CopyFrom(tmpObject)

Later, we can deserialize each dynamic entity from bigLines:

   do bigLines.Rewind()
   while ('bigLines.AtEnd) {
      write !,{}.%FromJSON(bigLines.ReadLineIntoStream())
   }

7@%Library.DynamicArray
7@%Library.DynamicObject