Skip to main content

%iKnow.Source.Loader

deprecated class %iKnow.Source.Loader extends %Library.RegisteredObject

Deprecation note: Direct use of this class is still supported, but it is recommended to use either Domain Definitions (cf %iKnow.DomainDefinition) to manage a domain's contents or use the shorthand methods in %SYSTEM.iKnow to work with an existing domain not managed through a domain definition class.

This class coordinates the loading process.

For Batch Loading, use the ProcessBatch() after one or more %iKnow.Source.Lister instances have been used to register one or more lists of sources to load into the domain using their AddListToBatch() methods.

For Direct Loading, register a %iKnow.Source.Lister instance with this Loader instance and call ProcessList() or ProcessVirtualList() to add a small number of sources to the domain directly (in the same process).

For Direct Text Loading, you can simply queue %String data using the BufferSource() and then add these directly (in the same process) to the domain using either ProcessBuffer() or ProcessVirtualBuffer().

With both Direct Loading options, the IDs of the sources added in the last execution are available through GetSourceIds()

Property Inventory

Method Inventory

Properties

property DomainId as %Integer;
Property methods: DomainIdDisplayToLogical(), DomainIdGet(), DomainIdIsValid(), DomainIdLogicalToDisplay(), DomainIdNormalize(), DomainIdSet()

Methods

method BufferSource(ref As %String, data As %String) as %Status

Lines up the data supplied to this method to be indexed as a new source with the provided reference in a subsequent call to ProcessBuffer() or ProcessVirtualBuffer().

Multiple calls to this method using the same ref before calling the Process* method, will result in the data being appended to what was previously buffered for that ref. All buffering and processing operations should happen within the same process.

method ClearBuffer() as %Status

This method clears any data supplied so far through calls to BufferSource()

method ClearLogs(clearErrors As %Boolean = 1, clearFailed As %Boolean = 1, clearWarnings As %Boolean = 1) as %Status
Clears the logs, as directed by the boolean parameters.
classmethod DeleteAllVirtualSources(domId As %Integer) as %Status

Deletes all Virtual Sources associated with this domain.

classmethod DeleteSource(domId As %Integer, internalId As %Integer) as %Status

This method deletes a single source from the domain, including all its entity and CRC occurrences, sentences and paths. Unique entities and CRCs occurring in this source will have their frequency and spread updated.

If there's any metadata or matching results associated with this source, it will also be deleted.

classmethod DeleteVirtualSource(domId As %Integer, vSrcId As %Integer) as %Status

This method deletes a single Virtual Source from the domain, including all its entity and CRC occurrences, sentences and paths.

method GetBuffer(Output pBuffer) as %Status

This method returns all the data currently in the buffer, as entered through calls to BufferSource(), since the last call to ProcessBuffer() or ClearBuffer(). Data is returned in a two-level array structure:

pBuffer(reference) = numberOfEntries
pBuffer(reference, n) = data

method GetCurrentLists(ByRef result) as %Status

Simple utility method to retrieve an overview of all Lists as currently scheduled through AddListToBatch calls on %iKnow.Source.Lister objects for this Loaders domain.

method GetErrors(ByRef result, returnAll As %Boolean = 0) as %Status

Returns all logged errors for this domain. By default only the errors logged since this Loader object's last load operation are returned, unless the returnAll parameter is set to 1. The returned result contains the following columns:

  1. Error message
  2. Location: indication of the class and method where the error occurred
  3. Timestamp: in $horolog format
method GetFailed(ByRef result, returnAll As %Boolean = 0) as %Status

Returns all logged source failures for this domain. By default only the failures logged since this Loader object's last load operation are returned, unless the returnAll parameter is set to 1. The returned result contains the following columns:

  1. Composite reference: elements used to generate the external ID (in case it didn't get generated)
  2. External ID
  3. Location: indication of the class and method where the error occurred
  4. Error message
  5. Internal error ID: Contact WRC if this is a positive integer.
  6. Timestamp: in $horolog format
method GetSourceIds() as %List

This method returns the Source IDs of the sources created in the last call to ProcessList(), ProcessBuffer(), ProcessVirtualList() or ProcessVirtualBuffer().

method GetWarnings(ByRef result, returnAll As %Boolean = 0) as %Status

Returns all logged warnings for this domain. By default only the warnings logged since this Loader object's last load operation are returned, unless the returnAll parameter is set to 1. The returned result contains the following columns:

  1. Composite reference: elements used to generate the external ID (in case it didn't get generated)
  2. External ID
  3. Location: indication of the class and method where the error occurred
  4. Error message
  5. Timestamp: in $horolog format
classmethod LogProcess(domainId As %Integer, procId As %Integer, args As %List) as %Status
args(1) = number of sources. args(2) = the status global to log the process for LoadSources. args(3) = the time to queue work into the queue.
method ProcessBatch() as %Status

This is the primary method for starting a Batch Load. When executed, this method will first go through the lists registered through AddListToBatch() calls on %iKnow.Source.Lister objects and call their ExpandList() method to get a complete list of all the sources to index. Then, it will instantiate the appropriate %iKnow.Source.Processor objects to read their contents and forward it to the iKnow engine (optionally through a %iKnow.Source.Converter instance). Finally, it will invoke the ^%iKnow.BuildGlobals routine to transform the output of the iKnow engine into the persistent domain data structures.

method ProcessBuffer(cfg As %String = "") as %Status

Indexes all data buffered within this process through previous calls to BufferSource() using the Direct Loading approach. This includes calling SetLister() with a %iKnow.Source.Temp.Lister instance and initiating the Direct Load through ProcessList(). The buffer is cleaned automatically at the end of successfully processing the data.

After this method has finished, the generated source IDs can be retrieved through GetSourceIds().

method ProcessList(listargs...) as %Status

This method initiates a Direct Load, indexing the sources identified by the Lister registered with SetLister(). The Lister Parameters passed as listargs to this method should correspond in type and number to what the currently registered Lister expects as its (implementation-specific) Lister Parameters.

All operations will execute within the current process and therefore has superior performance over Batch Loading when used for small numbers of sources.

After this method has finished, the generated source IDs can be retrieved through GetSourceIds().

method ProcessVirtualBuffer(cfg As %String = "") as %Status
method ProcessVirtualList(listargs...) as %Status

Same As ProcessList(), but listed sources will be added as Virtual Sources.

method Reset() as %Status

This method resets the internal globals used during the load process.

method SetLister(lister As %iKnow.Source.Lister) as %Status

This call registers the supplied %iKnow.Source.Lister object to be used for Direct Loading by subsequent calls to ProcessList().

The supplied Lister object should be fully configured, but cannot have any lists already scheduled for it (through calls to its AddListToBatch method).

Inherited Members

Inherited Methods

FeedbackOpens in a new tab