Invoking Callout Library Functions
A Callout library is a shared library (a DLL or SO file) that includes hooks to the $ZF Callout Interface, allowing various $ZF functions to load it at runtime and invoke its functions. The $ZF Callout Interface provides four different interfaces that can be used to load a Callout library at runtime and call functions from that library. These interfaces differ mainly in how the libraries are identified and loaded into memory:
-
Using $ZF() to Access the iriszf Callout Library describes how to use a special shared library named iriszf. When this library is available, its functions can be accessed by a call of the form $ZF("funcname",args), without previously loading the library or specifying the library name.
-
Using $ZF(-3) for Simple Library Function Calls describes how to load a library and invoke a function by specifying the library file path and function name. It is simple to use, but can only have one library at a time in virtual memory. Unlike the other interfaces, it does not require any initialization before a library function can be invoked.
-
Using $ZF(-5) to Access Libraries by System ID describes an interface that can be used to efficiently maintain and access more than one library at a time. Several libraries can be loaded and used simultaneously, each requiring significantly less processing overhead than $ZF(-3). Libraries in memory are identified by a system-defined ID generated when the library is loaded.
-
Using $ZF(-6) to Access Libraries by User Index describes the most efficient interface for handling a large set of Callout libraries. The interface provides access to libraries through a globally defined index table. The index is available to all processes in an instance of InterSystems IRIS, and several libraries can be in memory at the same time. Each indexed library is given a unique, user-defined index number, and the index table can be defined and modified at runtime. The filename associated with a given library ID can be changed when a library file is renamed or relocated, and this change will be transparent to applications that load the library by index number.
Using $ZF() to Access the iriszf Callout Library
When a Callout library named iriszf is available in an instance's <install_dir>/bin directory, its functions can be invoked by a $ZF call that specifies only the function name and arguments (for example, $ZF("functionName",arg1, arg2)). The iriszf functions can be called without previously loading the library, and are available to all processes in the instance.
A custom iriszf library is defined by creating a standard Callout library, moving it to your instance's <install_dir>/bin directory, and renaming it iriszf (specifically iriszf.dll or iriszf.so, depending on the platform).
Here are the steps to compile the simplecallout.c example (see “Creating an InterSystems Callout Library”) and set it up as an iriszf library. These examples assume an instance running under Linux, installed in a directory named /intersystems/iris, but the procedure is basically the same on all platforms:
-
Write and save simplecallout.c:
#define ZF_DLL #include "iris-cdzf.h" int AddTwoIntegers(int a, int b, int *outsum) { *outsum = a+b; /* set value to be returned by $ZF function call */ return IRIS_SUCCESS; /* set the exit status code */ } ZFBEGIN ZFENTRY("AddInt","iiP",AddTwoIntegers) ZFEND
-
Generate the Callout library file (simplecallout.so):
gcc -c -fPIC simplecallout.c -I /intersystems/iris/dev/iris-callin/include/ -o simplecallout.o gcc simplecallout.o -shared -o simplecallout.so
-
Test the library with $ZF(-3) from an InterSystems IRIS terminal session:
USER>write $ZF(-3,"/mytest/simplecallout.so","AddInt",1,4) 5
-
Now install the library for use with $ZF(). Copy simplecallout.so into <install_dir>/bin, renaming it iriszf.so:
cp simplecallout.so /intersystems/iris/bin/iriszf.so
-
Confirm that the code can be called with $ZF() from an InterSystems IRIS session:
USER>write $zf("AddInt",1,4) 5
The iriszf library is loaded once when first used, and never unloaded. It is completely independent of the other $ZF loading and unloading operations described earlier in this chapter.
Previous versions of the $ZF Callout Interface allowed code to be statically linked to the InterSystems kernel and called with $ZF(). Static linking is no longer supported, but the iriszf library provides the same functionality without the need to relink the kernel.
Using $ZF(-3) for Simple Library Function Calls
The $ZF(-3) function is used to load a Callout library and execute a specified function from that library. $ZF(-3) is most useful if you are only using one library, or aren’t making enough calls to worry about the overhead of loading libraries. It allows you to call any available library function by specifying the library name, the function name, and a comma-separated list of function arguments:
result = $ZF(-3, library_name[, function_name[, arguments]])
The specified library is loaded if it hasn’t already been loaded by a previous call to $ZF(-3). Only one library can be loaded at a time. When a subsequent $ZF(-3) call specifies a different library, the old library is unloaded and the new one replaces it. The library stays loaded as long as subsequent $ZF(-3) calls specify the same library. After a library has been loaded, the library name can be specified as a null string ("") in subsequent calls.
You can load or unload a library without calling a function. To load a new library, specify only the library name. To unload the current library without loading a new one, specify only a null string. In either case, $ZF(-3) returns a status code indicating whether the load or unload was successful.
The following ObjectScript code calls two different functions from each of two different libraries, and then unloads the current library:
// define Callout library paths
set libOne = "c:\intersystems\iris\bin\myfirstlibrary.dll"
set libTwo = "c:\intersystems\iris\bin\anotherlibrary.dll"
//load and call
SET result1=$ZF(-3,libOne,"FuncA",123) // loads libOne and calls FuncA
SET result2=$ZF(-3,"","FuncB","xyz") // calls FuncB from same library
//load, then call with null name
SET status=$ZF(-3,libTwo) // unloads libOne, loads libTwo
SET result1=$ZF(-3,"","FunctionOne","arg1")
SET result2=$ZF(-3,"","FunctionTwo","argA", "argB")
//unload
SET status=$ZF(-3,"") // unloads libTwo
-
For convenience, the library names are assigned to strings libOne and libTwo.
-
The first call to $ZF(-3) loads Callout library libOne and invokes function FuncA from that library.
-
The second call specifies a null string for the library name, indicating that currently loaded libOne should be used again, and invokes function FuncB from that library.
-
The third call to $ZF(-3) specifies only library name libTwo. This unloads libOne and loads libTwo, but does not invoke any library functions. The call returns a status code indicating whether libTwo was successfully loaded.
-
The fourth and fifth calls invoke library functions FunctionOne and FunctionTwo from currently loaded libTwo.
-
The final $ZF(-3) call does not invoke a library function, and specifies a null string for the library name. This unloads libTwo and does not load a new library. The call returns a status code indicating whether libTwo was successfully unloaded.
The following sections of this chapter describe $ZF functions that can load more than one library at a time. These functions will not conflict with $ZF(-3). You can always use $ZF(-3) as if it were loading and unloading its own private copy of a library.
Using $ZF(-5) to Access Libraries by System ID
The $ZF(-5) function uses system-defined library and function identifiers to invoke library functions. In applications that make many library function calls, this can significantly reduce processing overhead. Multiple libraries can be open at the same time. Each library only needs to be loaded once, and each library or function identifier only has to be generated once. Utility functions $ZF(-4,1), $ZF(-4,2) and $ZF(-4,3) are used to get the required identifiers and to load or unload libraries:
-
$ZF(-5) invokes a function referenced by system-defined library and function identifiers.
-
$ZF(-4,1) loads a library. It takes a library filename and returns a system-defined library ID value for the loaded library.
-
$ZF(-4,2) — unloads a Callout library specified by library ID.
-
$ZF(-4,3) — returns a function ID value for a given library ID and function name.
The $ZF(-4,1) and $ZF(-4,3) functions are used to load Callout libraries and get library and function identifiers. The syntax for $ZF(-4,1) is:
lib_id = $ZF(-4,1,lib_name) // get library ID
where lib_name is the full name and path of the shared library file, and lib_id is the returned library ID. The syntax for $ZF(-4,3) is:
func_id=$ZF(-4,3,lib_id, func_name) // get function ID
where lib_id is the library ID, func_name is the library function name, and func_id is the returned function ID value.
The following ObjectScript code loads Callout library mylibrary.dll and gets the library ID, then gets the function ID for "MyFunction" and invokes it with $ZF(-5):
set libID = $ZF(-4,1,"C:\calloutlibs\mylibrary.dll")
set funcID = $ZF(-4,3,libID, "MyFunction")
set x = $ZF(-5,libID, funcID, "arg1")
Once the identifiers have been defined, the library will remain loaded until unloaded by $ZF(-4,2), and the identifiers can be used without any further calls to $ZF(-4,1) or $ZF(-4,3). This eliminates a significant amount of processing overhead when functions from several libraries are invoked many times.
The following ObjectScript code loads two different libraries and invokes functions from both libraries in long loops. A function in inputlibrary.dll acquires data, and functions in outputlibrary.dll plot and store the data:
Method GraphSomeData(loopsize As %Integer=100000) As %Status
{
// load libraries and get system-defined ID values
set InputLibID = $ZF(-4,1,"c:\intersystems\iris\bin\inputlibrary.dll")
set OutputLibID = $ZF(-4,1,"c:\intersystems\iris\bin\outputlibrary.dll")
set fnGetData = $ZF(-4,3,InputLibID,"GetData")
set fnAnalyzeData = $ZF(-4,3,OutputLibID,"AnalyzeData")
set fnPlotPoint = $ZF(-4,3,OutputLibID,"PlotPoint")
set fnWriteData = $ZF(-4,3,OutputLibID,"WriteData")
// call functions from each library until we have 100000 good data items
set count = 0
do {
set datapoint = $ZF(-5,InputLibID,fnGetData)
set normalized = $ZF(-5,OutputLibID,fnAnalyzeData,datapoint)
if (normalized'="") { set flatdata($INCREMENT(count)) = normalized }
} while (count<loopsize)
set status = $ZF(-4,2,InputLibID) //unload "inputlibrary.dll"
// plot results of the previous loop and write to output
for point=1:1:count {
set list = $ZF(-5,OutputLibID,fnPlotPoint,flatdata(point))
set x = $PIECE(list,",",1)
set y = $PIECE(list,",",2)
set sc = $ZF(-5,OutputLibID,fnWriteData,flatdata(point),x,y,"outputfile.dat")
}
set status = $ZF(-4,2,OutputLibID) //unload "outputlibrary.dll"
quit 0
}
-
The calls to $ZF(-4,1) load Callout libraries inputlibrary.dll and outputlibrary.dll into virtual memory and return system-defined library IDs for them.
-
The calls to $ZF(-4,3) use the library IDs and function names to get IDs for the library functions. The returned function IDs are actually ZFEntry table sequence numbers (see “Creating a ZFEntry Table” in the previous chapter).
-
The first loop uses $ZF(-5) to call a function from each library:
-
The GetData() function from inputlibrary.dll reads raw data from some unspecified source.
-
The AnalyzeData() function from outputlibrary.dll either normalizes the raw data or rejects it and returns an empty string.
-
Each normalized datapoint is stored in flatdata(count) (where the first call to ObjectScript function $INCREMENT creates count and initializes it to 1).
By default, the loop fetches 100000 items. Since both libraries have been loaded and remain in memory, there is no processing overhead for switching between two different libraries.
-
-
After the first loop ends, library inputlibrary.dll is no longer needed, so $ZF(-4,2) is called to unload it. Library outputlibrary.dll will remain in memory.
-
The second loop processes each item from array flatdata and writes it to a file at some unspecified location:
-
Library function PlotPoint() reads the item and returns a comma-delimited string containing the coordinates at which it will be plotted (see “Introduction to Linkages” for a description of how multiple output parameters are returned by a library function).
-
The $PIECE function is used to extract coordinate values x and y from the string.
-
Library function WriteData() stores the item and coordinates in file outputfile.dat, which will be used by some other application to print a graph.
-
-
After the second loop finishes, $ZF(-4,2) is called again to unload library outputlibrary.dll.
The following section describes the $ZF(-6) interface, which loads libraries into the same virtual memory space as the $ZF(-5) interface.
Using $ZF(-6) to Access Libraries by User Index
The $ZF(-6) function provides an efficient interface that allows access to Callout libraries through a globally defined index, usable even by applications that do not know the location of the shared library files. The user-defined index table stores a key-value pair consisting of a library ID number and a corresponding library filename. The filename associated with a given library ID can be changed when a library file is renamed or relocated. This change will be transparent to applications that load the library by index number. Other $ZF functions are provided to create and maintain index tables, and to unload libraries loaded by $ZF(-6).
The following $ZF functions are discussed in this section:
-
$ZF(-6) — invokes a function from a Callout library referenced by user-specified index number. Automatically loads the library if it is not already loaded.
-
$ZF(-4,4) — unloads a Callout library specified by index number.
-
$ZF(-4,5) and $ZF(-4,6) — creates or deletes an entry in the system index table. The system index is globally available to all processes within an instance of InterSystems IRIS.
-
$ZF(-4,7) and $ZF(-4,8) — creates or deletes an entry in a process index table. Process tables are searched before the system table, so they can be used within a process to override system-wide definitions.
The $ZF(-6) interface is similar to the one used by $ZF(-5) (see “Using $ZF(-5) to Access Libraries by System ID”) with the following differences:
-
Before $ZF(-6) can be used, a library index table must be created. Library index values are user-defined, and can be changed or overridden at runtime.
-
Library names are stored in the index, which does not have to be defined by the application that loads the library. The name and location of the library file can be changed in the index without affecting dependent applications that load the library by index value.
-
There is no separate $ZF function to load a library. Instead, a library is loaded automatically by the first $ZF(-6) call that invokes one of its functions.
-
It is assumed that the developer will already know the library function IDs (which are determined by their order in the ZFEntry table), so there is no $ZF function that will return a function ID for a given name and library index value.
The following examples demonstrate how the $ZF(-6) interface is used. The first example defines a library ID in the system index table, and the second example (which may be called from a different application) uses the library ID to invoke a library function:
This example sets 100 as the library ID for mylibrary.dll in the system index table. If a definition already exists for that number, it is deleted and replaced.
set LibID = 100
set status=$ZF(-4,4,LibID) // unload any existing library with this ID value
set status = $ZF(-4,5,LibID,"C:\calloutlibs\mylibrary.dll") // set system ID
-
LibID is the index number chosen by the developer. It can be any integer greater than zero, except reserved system values 1024 through 2047.
-
If a library has already been loaded with index number 100, it should be unloaded before the entry is replaced.
-
The call to $ZF(-4,5) associates index number 100 with library file mylibrary.dll.
Once the library ID is defined in the system index table, it is globally available to all processes within the current instance of InterSystems IRIS.
This example uses the system index table created in the previous example. It uses $ZF(-6) to load the library and invoke a library function, then unloads the library. This code does not have to be called from the same application that defined the library ID in the system index:
set LibID = 100 // library ID in system index table
set FuncID = 2 // second function in library ZFEntry table
set x = $ZF(-6,LibID, FuncID, "arg1") // call function 2
set status = $ZF(-4,4,LibID) // unload the library
-
LibID is the library ID defined in the system index. This application does not have to know the library name or path in order to use library functions.
-
FuncID is the function identifier for the second function listed in the ZFEntry table of library LibID. It is assumed that the developer has access to the library code — the $ZF(-6) interface does not have a function to retrieve this number by specifying the library function name.
-
The call to $ZF(-6) specifies 100 as the library ID, 2 as the function ID and "arg1" as the argument passed to the function. This call will load Callout library mylibrary.dll if it isn’t already loaded, and will invoke the second function listed in the ZFEntry table.
-
The call to $ZF(-4,4) unloads the library. Each library loaded by $ZF(-6) will remain resident until the process ends or until unloaded by $ZF(-4,4).
Using the $ZF(-6) Interface to Encapsulate Library Functions
It would be simple to write an example for the $ZF(-6) interface that works just like the example for the $ZF(-5) interface (see “Using $ZF(-5) to Access Libraries by System ID” earlier in this chapter), but this would not demonstrate the advantages of using $ZF(-6). Instead, this section will present ObjectScript classes that allow an end user to perform exactly the same task without knowing anything about the contents or location of the Callout libraries.
The $ZF(-5) example invoked functions from Callout libraries inputlibrary.dll and outputlibrary.dll to process some experimental data and produce a two-dimensional array that could be used to draw a graph. The examples in this section perform the same tasks using the following ObjectScript code:
-
Class User.SystemIndex — encapsulates the file names and index numbers used to define entries in the system index table.
-
Class User.GraphData — provides methods that encapsulate functions from both libraries.
-
Method GetGraph() — is part of an end user program that calls the User.GraphData methods. The code in this method performs exactly the same task as the $ZF(-5) example, but never calls a $ZF function directly.
The User.SystemIndex class allows applications that use the Callout libraries to create and access system index entries without hard coding index numbers or file locations:
Class User.SystemIndex Extends %Persistent
{
/// Defines system index table entries for the User.GraphData libraries
ClassMethod InitGraphData() As %Status
{
// For each library, delete any existing system index entry and add a new one
set sc = $ZF(-4,4,..#InputLibraryID)
set sc = $ZF(-4,5,..#InputLibraryID,"c:\intersystems\iris\bin\inputlibrary.dll")
set sc = $ZF(-4,4,..#OutputLibraryID)
set sc = $ZF(-4,5,..#OutputLibraryID,"c:\intersystems\iris\bin\outputlibrary.dll")
quit 0
}
Parameter InputLibraryID = 100;
Parameter OutputLibraryID = 200;
}
-
The InitGraphData() method adds the libraries for User.GraphData to the system index table. It could be called automatically when the instance of InterSystems IRIS starts, making the libraries available to all processes within the instance.
-
The InputLibraryID and OutputLibraryID class parameters are made available so that dependent applications don’t have to hard code the index values (as demonstrated by the Init() method of User.GraphData in the following example).
The User.GraphData class allows end users to invoke library functions without knowing anything about the actual Callout libraries.
Class User.GraphData Extends %Persistent
{
/// Gets library IDs and updates the system index table for both libraries.
Method Init() As %Status
{
set InLibID = ##class(User.GraphDataIndex).%GetParameter("InputLibraryID")
set OutLibID = ##class(User.GraphDataIndex).%GetParameter("OutputLibraryID")
quit ##class(User.SystemIndex).InitGraphData()
}
Property InLibID As %Integer [Private];
Property OutLibID As %Integer [Private];
/// Calls function "FormatData" in library "inputlibrary.dll"
Method FormatData(rawdata As %Double) As %String
{
quit $ZF(-6,..InLibID,1,rawdata)
}
/// Calls function "RefineData" in library "outputlibrary.dll"
Method RefineData(midvalue As %String) As %String
{
quit $ZF(-6,..OutLibID,1,midvalue)
}
/// Calls function "PlotGraph" in library "outputlibrary.dll"
Method PlotGraph(datapoint As %String, xvalue As %Integer) As %String
{
quit $ZF(-6,..OutLibID,2,datapoint,xvalue)
}
/// Unloads both libraries
Method Unload() As %String
{
set sc = $ZF(-4,4,..InLibID) // unload "inputlibrary.dll"
set sc = $ZF(-4,4,..OutLibID) // unload "outputlibrary.dll"
quit 0
}
}
-
The Init() method calls a class method from User.SystemIndex that will set or update the system index entries for inputlibrary.dll and outputlibrary.dll. It also gets the current values for the library IDs. The developer of this class still needs to know something about the Callout library code, but future changes to the system index will be transparent.
-
Methods FormatData(), RefineData(), and PlotGraph() each encapsulate a call to one library function. Since they contain only the unconditional $ZF function calls, they will be optimized to run just as fast as the original $ZF calls.
-
The Unload() method unloads either or both libraries.
The following example demonstrates how an end user might use the methods in User.GraphData. The GetGraph() method uses the Callout libraries to perform exactly the same task as the GraphSomeData() method in the $ZF(-5) interface example (see “Using $ZF(-5) to Access Libraries by System ID” earlier in this chapter), but it does not directly call any $ZF functions:
Method GetGraph(loopsize As %Integer = 100000) As %Status
{
// Get an instance of class GraphData and initialize the system index
set graphlib = ##class(User.GraphData).%New()
set sc = graphlib.Init()
// call functions from both libraries repeatedly
// each library is loaded automatically on first call
for count=1:1:loopsize {
set midvalue = graphlib.FormatData(^rawdata(count))
set flatdata(count) = graphlib.RefineData(midvalue)
}
// plot results of the previous loop
for count=1:1:loopsize {
set x = graphlib.PlotGraph(flatdata(count),0)
set y = graphlib.PlotGraph(flatdata(count),x)
set ^graph(x,y) = flatdata(count)
}
//return after unloading all libraries loaded by $ZF(-6)
set status = graphlib.Unload()
quit 0
}
-
The User.GraphData class is instantiated as graphlib, and the Init() method is called to initialize the system index. This method does not necessarily have to be called here, since the system index only has to be initialized once for all processes in an instance of InterSystems IRIS.
-
The first loop indirectly uses $ZF(-6) to call a functions from each library, and $ZF(-6) automatically loads each library the first time it is needed. Library inputlibrary.dll is loaded by the first call to FormatData(), and outputlibrary.dll is loaded on the first call to RefineData().
-
The second loop invokes PlotGraph() from library outputlibrary.dll, which has already been loaded.
-
The call to Unload() indirectly calls $ZF(-4,4) on both libraries.
Using a Process Index for Testing
As previously mentioned, a process index table is searched before the system index table, so it can be used within a process to override system-wide definitions. The following example creates a process index that is used to test a new version of one of the libraries used in the previous section.
// Initialize the system index and generate output from standard library
set testlib = ##class(User.GraphData).%New()
set sc = testlib.Init()
set sc = graphgen.GetGraph() // get 100000 data items by default
merge testgraph1 = ^graph
kill ^graph
// create process index and test new library with same instance of testproc
set sc = $ZF(-4,4,100) // unload current copy of inputlib
set sc = $ZF(-4,8) // delete existing process index, if any
set sc = $ZF(-4,7,100, "c:\testfiles\newinputlibrary.dll") // override system index
set sc = graphgen.GetGraph()
merge testgraph2 = ^graph
// Now compare testdata1 and testdata2
-
In the first three lines, this test code initializes the system index and generates a graph, just like the previous example. The graph has been plotted using the standard version of inputlibrary.dll (identified by the system index entry with ID value 100), and has been saved to testgraph1.
-
The call to $ZF(-4,4) unloads inputlibrary.dll, which is identified by library ID 100 in the system index table.
-
$ZF(-4,8) is called without specifying a library ID, indicating that all entries in the current process index table are to be deleted.
-
The call to $ZF(-4,7) adds an entry to the process index table that sets 100 as the library ID for test library newinputlibrary.dll. This overrides the entry for that ID in the system index. Library ID 100 now points to newinputlibrary.dll rather than inputlibrary.dll.
-
GetGraph() is called again, using the same instance of User.GraphData. Nothing has changed except that the standard version of inputlibrary.dll has been unloaded, so GetGraph() will now load and use the new version of the library. The test then compares graphs testgraph1 and testgraph2 to verify that both versions are producing the same results.