Class Reference
Cache for UNIX 2018.1
InterSystems: The power behind what matters   
Documentation  Search
Private  Storage  

class %iKnow.Source.Loader extends %RegisteredObject

Deprecation note: Direct use of this class is still supported, but it is recommended to use either Domain Definitions (cf %iKnow.DomainDefinition) to manage a domain's contents or use the shorthand methods in %SYSTEM.iKnow to work with an existing domain not managed through a domain definition class.

This class coordinates the loading process.

For Batch Loading, use the ProcessBatch after one or more %iKnow.Source.Lister instances have been used to register one or more lists of sources to load into the domain using their AddListToBatch methods.

For Direct Loading, register a %iKnow.Source.Lister instance with this Loader instance and call ProcessList or ProcessVirtualList to add a small number of sources to the domain directly (in the same process).

For Direct Text Loading, you can simply queue %String data using the BufferSource and then add these directly (in the same process) to the domain using either ProcessBuffer or ProcessVirtualBuffer.

With both Direct Loading options, the IDs of the sources added in the last execution are available through GetSourceIds

Inventory

Parameters Properties Methods Queries Indices ForeignKeys Triggers
15 31


Summary

Properties
DomainId

Methods
%AddToSaveSet %ClassIsLatestVersion %ClassName %ConstructClone
%DispatchClassMethod %DispatchGetModified %DispatchGetProperty %DispatchMethod
%DispatchSetModified %DispatchSetMultidimProperty %DispatchSetProperty %Extends
%GetParameter %IsA %IsModified %New
%NormalizeObject %ObjectModified %OriginalNamespace %PackageName
%RemoveFromSaveSet %SerializeObject %SetModified %ValidateObject
BufferSource ClearBuffer ClearLogs DeleteAllVirtualSources
DeleteSource DeleteVirtualSource GetBuffer GetCurrentLists
GetErrors GetFailed GetSourceIds GetWarnings
LogProcess ProcessBatch ProcessBuffer ProcessList
ProcessVirtualBuffer ProcessVirtualList Reset SetLister


Properties

• property DomainId as %Integer;

Methods

• method BufferSource(ref As %String, data As %String) as %Status

Lines up the data supplied to this method to be indexed as a new source with the provided reference in a subsequent call to ProcessBuffer or ProcessVirtualBuffer.

Multiple calls to this method using the same ref before calling the Process* method, will result in the data being appended to what was previously buffered for that ref. All buffering and processing operations should happen within the same process.

• method ClearBuffer() as %Status

This method clears any data supplied so far through calls to BufferSource

• method ClearLogs(clearErrors As %Boolean = 1, clearFailed As %Boolean = 1, clearWarnings As %Boolean = 1) as %Status
Clears the logs, as directed by the boolean parameters.
• classmethod DeleteAllVirtualSources(domId As %Integer) as %Status

Deletes all Virtual Sources associated with this domain.

• classmethod DeleteSource(domId As %Integer, internalId As %Integer) as %Status

This method deletes a single source from the domain, including all its entity and CRC occurrences, sentences and paths. Unique entities and CRCs occurring in this source will have their frequency and spread updated.

If there's any metadata or matching results associated with this source, it will also be deleted.

• classmethod DeleteVirtualSource(domId As %Integer, vSrcId As %Integer) as %Status

This method deletes a single Virtual Source from the domain, including all its entity and CRC occurrences, sentences and paths.

• method GetBuffer(Output pBuffer) as %Status

This method returns all the data currently in the buffer, as entered through calls to BufferSource, since the last call to ProcessBuffer or ClearBuffer. Data is returned in a two-level array structure:

pBuffer(reference) = numberOfEntries
pBuffer(reference, n) = data

• method GetCurrentLists(ByRef result) as %Status

Simple utility method to retrieve an overview of all Lists as currently scheduled through AddListToBatch calls on %iKnow.Source.Lister objects for this Loaders domain.

• method GetErrors(ByRef result, returnAll As %Boolean = 0) as %Status

Returns all logged errors for this domain. By default only the errors logged since this Loader object's last load operation are returned, unless the returnAll parameter is set to 1. The returned result contains the following columns:

  1. Error message
  2. Location: indication of the class and method where the error occurred
  3. Timestamp: in $horolog format
• method GetFailed(ByRef result, returnAll As %Boolean = 0) as %Status

Returns all logged source failures for this domain. By default only the failures logged since this Loader object's last load operation are returned, unless the returnAll parameter is set to 1. The returned result contains the following columns:

  1. Composite reference: elements used to generate the external ID (in case it didn't get generated)
  2. External ID
  3. Location: indication of the class and method where the error occurred
  4. Error message
  5. Internal error ID: Contact WRC if this is a positive integer.
  6. Timestamp: in $horolog format
• method GetSourceIds() as %List

This method returns the Source IDs of the sources created in the last call to ProcessList, ProcessBuffer, ProcessVirtualList or ProcessVirtualBuffer.

• method GetWarnings(ByRef result, returnAll As %Boolean = 0) as %Status

Returns all logged warnings for this domain. By default only the warnings logged since this Loader object's last load operation are returned, unless the returnAll parameter is set to 1. The returned result contains the following columns:

  1. Composite reference: elements used to generate the external ID (in case it didn't get generated)
  2. External ID
  3. Location: indication of the class and method where the error occurred
  4. Error message
  5. Timestamp: in $horolog format
• classmethod LogProcess(domainId As %Integer, procId As %Integer, args As %List) as %Status
args(1) = number of sources. args(2) = the status global to log the process for LoadSources. args(3) = the time to queue work into the queue.
• method ProcessBatch() as %Status

This is the primary method for starting a Batch Load. When executed, this method will first go through the lists registered through AddListToBatch() calls on %iKnow.Source.Lister objects and call their ExpandList() method to get a complete list of all the sources to index. Then, it will instantiate the appropriate %iKnow.Source.Processor objects to read their contents and forward it to the iKnow engine (optionally through a %iKnow.Source.Converter instance). Finally, it will invoke the ^%iKnow.BuildGlobals routine to transform the output of the iKnow engine into the persistent domain data structures.

• method ProcessBuffer(cfg As %String = "") as %Status

Indexes all data buffered within this process through previous calls to BufferSource using the Direct Loading approach. This includes calling SetLister with a %iKnow.Source.Temp.Lister instance and initiating the Direct Load through ProcessList. The buffer is cleaned automatically at the end of successfully processing the data.

After this method has finished, the generated source IDs can be retrieved through GetSourceIds.

• method ProcessList(listargs...) as %Status

This method initiates a Direct Load, indexing the sources identified by the Lister registered with SetLister. The Lister Parameters passed as listargs to this method should correspond in type and number to what the currently registered Lister expects as its (implementation-specific) Lister Parameters.

All operations will execute within the current process and therefore has superior performance over Batch Loading when used for small numbers of sources.

After this method has finished, the generated source IDs can be retrieved through GetSourceIds.

• method ProcessVirtualBuffer(cfg As %String = "") as %Status

Same As ProcessBuffer, but buffered sources will be added as Virtual Sources.

• method ProcessVirtualList(listargs...) as %Status

Same As ProcessList, but listed sources will be added as Virtual Sources.

• method Reset() as %Status

This method resets the internal globals used during the load process.

• method SetLister(lister As %iKnow.Source.Lister) as %Status

This call registers the supplied %iKnow.Source.Lister object to be used for Direct Loading by subsequent calls to ProcessList.

The supplied Lister object should be fully configured, but cannot have any lists already scheduled for it (through calls to its AddListToBatch method).



Copyright © 1997-2018 InterSystems Corporation, Cambridge, MA