Skip to main content

Invoking Callout Library Functions

The Callout Gateway provides three different interfaces that can be used to load an InterSystems Callout library at runtime and call functions from that library. This chapter covers the following topics:

Note:
Shared libraries and Callout libraries

In this book, the term shared library refers to a dynamically linked file (a DLL file on Windows or an SO file on UNIX® and related operating systems). A Callout library is a shared library that includes hooks to the Callout Gateway, allowing various $ZF functions to load it at runtime and invoke its functions. Instructions for writing Callout libraries are provided in “Creating an InterSystems Callout Library”.

Callout Library Interfaces

The $ZF(-3), $ZF(-5), and $ZF(-6) functions are all used to invoke Callout library functions, but each of these functions has its own strengths and weaknesses.

The $ZF(-3) Interface

The $ZF(-3) function loads a Callout library and invokes a library function by specifying the library file path and the function name. It is simple to use, but can only have one library at a time in virtual memory. Unlike the other interfaces, it does not require any initialization before a library function can be invoked. See “Using $ZF(-3) for Simple Library Function Calls” for details.

The $ZF(-5) Interface

The $ZF(-5) function invokes Callout library functions by specifying system-defined numeric identifiers for the Callout library and the library functions. It can keep several libraries in memory at the same time, and individual function calls are much more efficient than $ZF(-3). The $ZF(-5) interface also includes utility functions $ZF(-4,1), $ZF(-4,2), and $ZF(-4,3), which must be used to obtain library and function IDs and to load or unload libraries. See “Using $ZF(-5) to Access Libraries by System ID” for details.

The $ZF(-6) Interface

The $ZF(-6) interface is similar to $ZF(-5) except that Callout applications can load libraries without hard-coding the actual library filenames. Instead, a separate index table contains that information, and each indexed library is given a unique, user-defined index number. Once the index table is defined in an instance of Caché, it is available to all Callout applications in that instance. The $ZF(-6) function invokes a library function by specifying the index number and a function ID. If necessary, it automatically reads the index table and loads the specified library. The $ZF(-6) interface also includes utility functions $ZF(-4,4) through $ZF(-4,8), which must be used to unload libraries and create or maintain indexes. See “Using $ZF(-6) to Access Libraries by User Index” for details.

Using $ZF(-3) for Simple Library Function Calls

The $ZF(-3) function is used to load a Callout library and execute a specified function from that library. $ZF(-3) is most useful if you are only using one library, or aren’t making enough calls to worry about the overhead of loading libraries. It allows you to call any available library function by specifying the library name, the function name, and a comma-separated list of function arguments:

   result = $ZF(-3, library_name[, function_name[, arguments]])

The specified library is loaded if it hasn’t already been loaded by a previous call to $ZF(-3). Only one library can be loaded at a time. When a subsequent $ZF(-3) call specifies a different library, the old library is unloaded and the new one replaces it. The library stays loaded as long as subsequent $ZF(-3) calls specify the same library. After a library has been loaded, the library name can be specified as a null string ("") in subsequent calls.

You can load or unload a library without calling a function. To load a new library, specify only the library name. To unload the current library without loading a new one, specify only a null string. In either case, $ZF(-3) returns a status code indicating whether the load or unload was successful.

The following ObjectScript code calls two different functions from each of two different libraries, and then unloads the current library:

Using $ZF(-3) to load libraries and call functions
   // define Callout library paths
   set libOne = "c:\intersystems\cache\bin\myfirstlibrary.dll"
   set libTwo = "c:\intersystems\cache\bin\anotherlibrary.dll"

   //load and call
   SET result1=$ZF(-3,libOne,"FuncA",123)   // loads libOne and calls FuncA
   SET result2=$ZF(-3,"","FuncB","xyz")   // calls FuncB from same library

   //load, then call with null name
   SET status=$ZF(-3,libTwo)   // unloads libOne, loads libTwo
   SET result1=$ZF(-3,"","FunctionOne","arg1")
   SET result2=$ZF(-3,"","FunctionTwo","argA", "argB")

   //unload
   SET status=$ZF(-3,"")   // unloads libTwo
  • For convenience, the library names are assigned to strings libOne and libTwo.

  • The first call to $ZF(-3) loads Callout library libOne and invokes function FuncA from that library.

  • The second call specifies a null string for the library name, indicating that currently loaded libOne should be used again, and invokes function FuncB from that library.

  • The third call to $ZF(-3) specifies only library name libTwo. This unloads libOne and loads libTwo, but does not invoke any library functions. The call returns a status code indicating whether libTwo was successfully loaded.

  • The fourth and fifth calls invoke library functions FunctionOne and FunctionTwo from currently loaded libTwo.

  • The final $ZF(-3) call does not invoke a library function, and specifies a null string for the library name. This unloads libTwo and does not load a new library. The call returns a status code indicating whether libTwo was successfully unloaded.

The following sections of this chapter describe $ZF functions that can load more than one library at a time. These functions will not conflict with $ZF(-3). You can always use $ZF(-3) as if it were loading and unloading its own private copy of a library.

Using $ZF(-5) to Access Libraries by System ID

The $ZF(-5) function uses system-defined library and function identifiers to invoke library functions. Utility functions $ZF(-4,1), $ZF(-4,2) and $ZF(-4,3) are used to get the required identifiers and to load or unload libraries. Multiple libraries can be open at the same time. Unlike $ZF(-3) (see “Using $ZF(-3) for Simple Library Function Calls”), $ZF(-5) can not be used until the utility functions have been called to load libraries and get identifiers. However, each library only needs to be loaded once, and each library or function identifier only has to be generated once. In applications that make many library function calls, this can significantly reduce processing overhead.

The following $ZF functions are discussed in this section:

  • $ZF(-5) — invokes a Callout library function referenced by system-defined library and function identifiers.

  • $ZF(-4,1) — loads a Callout library specified by filename, and returns a system-defined ID value for it.

  • $ZF(-4,2) — unloads a Callout library specified by library ID.

  • $ZF(-4,3) — returns a function ID value for a given library ID and function name.

The $ZF(-4,1) and $ZF(-4,3) functions are used to load Callout libraries and get library and function identifiers. The syntax for $ZF(-4,1) is:

   lib_id = $ZF(-4,1,lib_name)   // get library ID

where lib_name is the full name and path of the shared library file, and lib_id is the returned library ID. The syntax for $ZF(-4,3) is:

   func_id=$ZF(-4,3,lib_id, func_name)   // get function ID

where lib_id is the library ID, func_name is the library function name, and func_id is the returned function ID value.

The following ObjectScript code loads Callout library mylibrary.dll and gets the library ID, then gets the function ID for "MyFunction" and invokes it with $ZF(-5):

Loading a library and invoking a function with $ZF(-5)
   set libID = $ZF(-4,1,"C:\calloutlibs\mylibrary.dll")
   set funcID = $ZF(-4,3,libID, "MyFunction")
   set x = $ZF(-5,libID, funcID, "arg1")

Once the identifiers have been defined, the library will remain loaded until unloaded by $ZF(-4,2), and the identifiers can be used without any further calls to $ZF(-4,1) or $ZF(-4,3). This eliminates a significant amount of processing overhead when functions from several libraries are invoked many times.

The following ObjectScript code loads two different libraries and invokes functions from both libraries in long loops. A function in inputlibrary.dll acquires data, and functions in outputlibrary.dll plot and store the data:

Using $ZF(-5) with multiple libraries and many function calls
Method GraphSomeData(loopsize As %Integer=100000) As %Status
 {
   // load libraries and get system-defined ID values
   set InputLibID = $ZF(-4,1,"c:\intersystems\cache\bin\inputlibrary.dll")
   set OutputLibID = $ZF(-4,1,"c:\intersystems\cache\bin\outputlibrary.dll")
   set fnGetData = $ZF(-4,3,InputLibID,"GetData")
   set fnAnalyzeData = $ZF(-4,3,OutputLibID,"AnalyzeData")
   set fnPlotPoint = $ZF(-4,3,OutputLibID,"PlotPoint")
   set fnWriteData = $ZF(-4,3,OutputLibID,"WriteData")

   // call functions from each library until we have 100000 good data items
   do {
      set datapoint = $ZF(-5,InputLibID,fnGetData)
      set normalized = $ZF(-5,OutputLibID,fnAnalyzeData,datapoint)
      if (normalized>"") { set flatdata($INCREMENT(count)) = normalized }
   } while (count<loopsize)
   set status = $ZF(-4,2,InputLibID)   //unload "inputlibrary.dll"

   // plot results of the previous loop and write to output
   for point=1:1:loopsize {
      set list = $ZF(-5,OutputLibID,fnPlotPoint,flatdata(point))
      set x = $PIECE(list,",",1)
      set y = $PIECE(list,",",2)
      set sc = $ZF(-5,OutputLibID,fnWriteData,flatdata(point),x,y,"outputfile.dat")
   }
   set status = $ZF(-4,2,OutputLibID)   //unload "outputlibrary.dll"
   quit 0
 }
  • The calls to $ZF(-4,1) load Callout libraries inputlibrary.dll and outputlibrary.dll into virtual memory and return system-defined library IDs for them.

  • The calls to $ZF(-4,3) use the library IDs and function names to get IDs for the library functions. The returned function IDs are actually ZFEntry table sequence numbers (see “Creating a ZFEntry Table” in the previous chapter).

  • The first loop uses $ZF(-5) to call a function from each library:

    • The GetData() function from inputlibrary.dll reads raw data from some unspecified source.

    • The AnalyzeData() function from outputlibrary.dll either normalizes the raw data or rejects it and returns an empty string.

    • Each normalized datapoint is stored in flatdata(count) (where the first call to Caché function $INCREMENT creates count and initializes it to 1).

    By default, the loop fetches 100000 items. Since both libraries have been loaded and remain in memory, there is no processing overhead for switching between two different libraries.

  • After the first loop ends, library inputlibrary.dll is no longer needed, so $ZF(-4,2) is called to unload it. Library outputlibrary.dll will remain in memory.

  • The second loop processes each item from array flatdata and writes it to a file at some unspecified location:

    • Library function PlotPoint() reads the item and returns a comma-delimited string containing the coordinates at which it will be plotted (see “Introduction to Linkages” for a description of how multiple output parameters are returned by a library function).

    • The $PIECE function is used to extract coordinate values x and y from the string.

    • Library function WriteData() stores the item and coordinates in file outputfile.dat, which will be used by some other application to print a graph.

  • After the second loop finishes, $ZF(-4,2) is called again to unload library outputlibrary.dll.

The following section describes the $ZF(-6) interface, which loads libraries into the same virtual memory space as the $ZF(-5) interface.

Using $ZF(-6) to Access Libraries by User Index

The $ZF(-6) function provides an efficient interface that allows access to Callout libraries through a globally defined index, usable even by applications that do not know the location of the shared library files. The user-defined index table stores a key/value pair consisting of a library ID number and a corresponding library filename. The filename associated with a given library ID can be changed when a library file is renamed or relocated. This change will be transparent to applications that load the library by index number. Other $ZF functions are provided to create and maintain index tables, and to unload libraries loaded by $ZF(-6).

The following $ZF functions are discussed in this section:

  • $ZF(-6) — invokes a function from a Callout library referenced by user-specified index number. Automatically loads the library if it is not already loaded.

  • $ZF(-4,4) — unloads a Callout library specified by index number.

  • $ZF(-4,5) and $ZF(-4,6) — creates or deletes an entry in the system index table. The system index is globally available to all processes within an instance of Caché.

  • $ZF(-4,7) and $ZF(-4,8) — creates or deletes an entry in a process index table. Process tables are searched before the system table, so they can be used within a process to override system-wide definitions.

The $ZF(-6) interface is similar to the one used by $ZF(-5) (see “Using $ZF(-5) to Access Libraries by System ID”) with the following differences:

  • Before $ZF(-6) can be used, a library index table must be created. Library index values are user-defined, and can be changed or overridden at runtime.

  • Library names are stored in the index, which does not have to be defined by the application that loads the library. The name and location of the library file can be changed in the index without affecting dependent applications that load the library by index value.

  • There is no separate $ZF function to load a library. Instead, a library is loaded automatically by the first $ZF(-6) call that invokes one of its functions.

  • It is assumed that the developer will already know the library function IDs (which are determined by their order in the ZFEntry table), so there is no $ZF function that will return a function ID for a given name and library index value.

The following examples demonstrate how the $ZF(-6) interface is used. The first example defines a library ID in the system index table, and the second example (which may be called from a different application) uses the library ID to invoke a library function:

Defining a system index entry with $ZF(-4,5) and $ZF(-4,6)

This example sets 100 as the library ID for mylibrary.dll in the system index table. If a definition already exists for that number, it is deleted and replaced.

   set LibID = 100
   set status = $ZF(-4,6,LibID)  // clear any old entries with this ID value
   set status = $ZF(-4,5,LibID,"C:\calloutlibs\mylibrary.dll")  // set system ID
  • LibID is the index number chosen by the developer. It can be any integer greater than zero, except reserved system values 1024 through 2047.

  • If the system index already contains an entry that uses index number 100, the call to $ZF(-4,6) will delete it. This is good practice, since index entries cannot be overwritten.

  • The call to $ZF(-4,5) associates index number 100 with library file mylibrary.dll.

Once the library ID is defined in the system index table, it is globally available to all processes within the current instance of Caché.

Invoking a function with $ZF(-6)

This example uses the system index table created in the previous example. It uses $ZF(-6) to load the library and invoke a library function, then unloads the library. This code does not have to be called from the same application that defined the library ID in the system index:

   set LibID = 100   // library ID in system index table
   set FuncID = 2    // second function in library ZFEntry table
   set x = $ZF(-6,LibID, FuncID, "arg1")   // call function 2
   set status = $ZF(-4,4,LibID)            // unload the library
  • LibID is the library ID defined in the system index. This application does not have to know the library name or path in order to use library functions.

  • FuncID is the function identifier for the second function listed in the ZFEntry table of library LibID. It is assumed that the developer has access to the library code — the $ZF(-6) interface does not have a function to retrieve this number by specifying the library function name.

  • The call to $ZF(-6) specifies 100 as the library ID, 2 as the function ID and "arg1" as the argument passed to the function. This call will load Callout library mylibrary.dll if it isn’t already loaded, and will invoke the second function listed in the ZFEntry table.

  • The call to $ZF(-4,4) unloads the library. Each library loaded by $ZF(-6) will remain resident until the process ends or until unloaded by $ZF(-4,4).

Using the $ZF(-6) Interface to Encapsulate Library Functions

It would be simple to write an example for the $ZF(-6) interface that works just like the example for the $ZF(-5) interface (see “Using $ZF(-5) to Access Libraries by System ID” earlier in this chapter), but this would not demonstrate the advantages of using $ZF(-6). Instead, this section will present ObjectScript classes that allow an end user to perform exactly the same task without knowing anything about the contents or location of the Callout libraries.

The $ZF(-5) example invoked functions from Callout libraries inputlibrary.dll and outputlibrary.dll to process some experimental data and produce a two-dimensional array that could be used to draw a graph. The examples in this section perform the same tasks using the following ObjectScript code:

  • Class User.SystemIndex — encapsulates the file names and index numbers used to define entries in the system index table.

  • Class User.GraphData — provides methods that encapsulate functions from both libraries.

  • Method GetGraph() — is part of an end user program that calls the User.GraphData methods. The code in this method performs exactly the same task as the $ZF(-5) example, but never calls a $ZF function directly.

The User.SystemIndex class allows applications that use the Callout libraries to create and access system index entries without hard coding index numbers or file locations:

ObjectScript Class User.SystemIndex
Class User.SystemIndex Extends %Persistent
{
/// Defines system index table entries for the User.GraphData libraries
ClassMethod InitGraphData() As %Status
{
 // For each library, delete any existing system index entry and add a new one
    set sc = $ZF(-4,6,..#InputLibraryID)
    set sc = $ZF(-4,5,..#InputLibraryID,"c:\intersystems\cache\bin\inputlibrary.dll")
    set sc = $ZF(-4,6,..#OutputLibraryID)
    set sc = $ZF(-4,5,..#OutputLibraryID,"c:\intersystems\cache\bin\outputlibrary.dll")
    quit 0
}

Parameter InputLibraryID = 100;
Parameter OutputLibraryID = 200;
}
  • The InitGraphData() method adds the libraries for User.GraphData to the system index table. It could be called automatically when the instance of Caché starts, making the libraries available to all processes within the instance.

  • The InputLibraryID and OutputLibraryID class parameters are made available so that dependent applications don’t have to hard code the index values (as demonstrated by the Init() method of User.GraphData in the following example).

The User.GraphData class allows end users to invoke library functions without knowing anything about the actual Callout libraries.

ObjectScript Class User.GraphData
Class User.GraphData Extends %Persistent
{
/// Gets library IDs and updates the system index table for both libraries.
Method Init() As %Status
{
    set InLibID = ##class(User.GraphDataIndex).%GetParameter("InputLibraryID")
    set OutLibID = ##class(User.GraphDataIndex).%GetParameter("OutputLibraryID")
    quit ##class(User.SystemIndex).InitGraphData()
}
Property InLibID As %Integer [Private]; 
Property OutLibID As %Integer [Private]; 

/// Calls function "FormatData" in library "inputlibrary.dll"
Method FormatData(rawdata As %Double) As %String
{
    quit $ZF(-6,..InLibID,1,rawdata)
}
/// Calls function "RefineData" in library "outputlibrary.dll"
Method RefineData(midvalue As %String) As %String
{
    quit $ZF(-6,..OutLibID,1,midvalue)
}
/// Calls function "PlotGraph" in library "outputlibrary.dll"
Method PlotGraph(datapoint As %String, xvalue As %Integer) As %String
{
    quit $ZF(-6,..OutLibID,2,datapoint,xvalue)
}
/// Unloads both libraries 
Method Unload() As %String
{
    set sc = $ZF(-4,4,..InLibID)   // unload "inputlibrary.dll"
    set sc = $ZF(-4,4,..OutLibID)   // unload "outputlibrary.dll"
    quit 0
}
}
  • The Init() method calls a class method from User.SystemIndex that will set or update the system index entries for inputlibrary.dll and outputlibrary.dll. It also gets the current values for the library IDs. The developer of this class still needs to know something about the Callout library code, but future changes to the system index will be transparent.

  • Methods FormatData(), RefineData(), and PlotGraph() each encapsulate a call to one library function. Since they contain only the unconditional $ZF function calls, the Caché compiler can optimize these methods to run just as fast as the original $ZF calls.

  • The Unload() method unloads either or both libraries.

The following example demonstrates how an end user might use the methods in User.GraphData. The GetGraph() method uses the Callout libraries to perform exactly the same task as the GraphSomeData() method in the $ZF(-5) interface example (see “Using $ZF(-5) to Access Libraries by System ID” earlier in this chapter), but it does not directly call any $ZF functions:

Method GetGraph()
Method GetGraph(loopsize As %Integer = 100000) As %Status
{
   // Get an instance of class GraphData and initialize the system index
   set graphlib = ##class(User.GraphData).%New()
   set sc = graphlib.Init()

   // call functions from both libraries repeatedly
   // each library is loaded automatically on first call
   for count=1:1:loopsize {
      set midvalue = graphlib.FormatData(^rawdata(count))
      set flatdata(count) = graphlib.RefineData(midvalue)
   }

   // plot results of the previous loop
   for count=1:1:loopsize {
      set x = graphlib.PlotGraph(flatdata(count),0)
      set y = graphlib.PlotGraph(flatdata(count),x)
      set ^graph(x,y) = flatdata(count)
   }

   //return after unloading all libraries loaded by $ZF(-6)
   set status = graphlib.Unload()
   quit 0
}
  • The User.GraphData class is instantiated as graphlib, and the Init() method is called to initialize the system index. This method does not necessarily have to be called here, since the system index only has to be initialized once for all processes in an instance of Caché.

  • The first loop indirectly uses $ZF(-6) to call a functions from each library, and $ZF(-6) automatically loads each library the first time it is needed. Library inputlibrary.dll is loaded by the first call to FormatData(), and outputlibrary.dll is loaded on the first call to RefineData().

  • The second loop invokes PlotGraph() from library outputlibrary.dll, which has already been loaded.

  • The call to Unload() indirectly calls $ZF(-4,4) on both libraries.

Using a Process Index for Testing

As previously mentioned, a process index table is searched before the system index table, so it can be used within a process to override system-wide definitions. The following example creates a process index that is used to test a new version of one of the libraries used in the previous section.

Using a process index to test a new version of "inputlibrary.dll"
   // Initialize the system index and generate output from standard library
   set testlib = ##class(User.GraphData).%New()
   set sc = testlib.Init()
   set sc = graphgen.GetGraph()   // get 100000 data items by default
   merge testgraph1 = ^graph
   kill ^graph

   // create process index and test new library with same instance of testproc
   set sc = $ZF(-4,4,100)   // unload current copy of inputlib
   set sc = $ZF(-4,8)   // delete existing process index, if any
   set sc = $ZF(-4,7,100, "c:\testfiles\newinputlibrary.dll")  // override system index
   set sc = graphgen.GetGraph()
   merge testgraph2 = ^graph

   // Now compare testdata1 and testdata2
  • In the first three lines, this test code initializes the system index and generates a graph, just like the previous example. The graph has been plotted using the standard version of inputlibrary.dll (identified by the system index entry with ID value 100), and has been saved to testgraph1.

  • The call to $ZF(-4,4) unloads inputlibrary.dll, which is identified by library ID 100 in the system index table.

  • $ZF(-4,8) is called without specifying a library ID, indicating that all entries in the current process index table are to be deleted.

  • The call to $ZF(-4,7) adds an entry to the process index table that sets 100 as the library ID for test library newinputlibrary.dll. This overrides the entry for that ID in the system index. Library ID 100 now points to newinputlibrary.dll rather than inputlibrary.dll.

  • GetGraph() is called again, using the same instance of User.GraphData. Nothing has changed except that the standard version of inputlibrary.dll has been unloaded, so GetGraph() will now load and use the new version of the library. The test then compares graphs testgraph1 and testgraph2 to verify that both versions are producing the same results.

FeedbackOpens in a new tab