Class Reference
InterSystems: The power behind what matters   
Documentation  Search
 

abstract class %iKnow.Source.Lister extends %RegisteredObject

Deprecation note: Direct use of this class is still supported, but it is recommended to use either Domain Definitions (cf %iKnow.DomainDefinition) to manage a domain's contents or use the shorthand methods in %SYSTEM.iKnow to work with an existing domain not managed through a domain definition class.

This is the base class for any Lister implementations. Lister classes are used at the start of the load process to identify the sources that need to be loaded. Each Lister implemenation is specifically designed to be able to scan a certain type of source locations, for example a filesystem for %iKnow.Source.File.Lister and a SQL query for %iKnow.Source.SQL.Lister

Each Lister implementation class defines a set of Lister Parameters, specifying how to reach the sources in the particular type of location this Lister class is designed to work with. For a file lister, this would include parameters such as the directory where to look and the type of files to load from that directory. These Lister Parameters are documented by each Lister implementation in its class documentation.

These Lister Parameters are to be provided to the AddListToBatch method, when configuring a list for Batch Loading, or to the ProcessList method when executing Direct Loads.

Inventory

Parameters Properties Methods Queries Indices ForeignKeys Triggers
27 40


Summary

Properties
Configuration SrcList SrcMeta SrcTempMeta

Methods
%%OIDGet %AddToSaveSet %BindExport %BuildObjectGraph
%ClassIsLatestVersion %ClassName %Close %ConstructClone
%DispatchClassMethod %DispatchGetModified %DispatchGetProperty %DispatchMethod
%DispatchSetModified %DispatchSetMultidimProperty %DispatchSetProperty %Extends
%GetParameter %IncrementCount %IsA %IsModified
%New %NormalizeObject %ObjectModified %OriginalNamespace
%PackageName %RemoveFromSaveSet %SerializeObject %SetModified
%ValidateObject AddGroupStatic AddListToBatch BuildExtId
BuildExtIdFromName BuildFullRefFromComposite BuildSimpleRefFromComposite DefaultProcessor
ExpandList FormatParams GetAlias GetExtidPrefix
GetGroupId GetGroupName GetListerData Init
MaxListId ProcessList ProcessVirtualList RegisterFixedMetadata
RegisterMetadataKeysInternal ResetRef SetBatchMode SetConfig
SetConverter SetDataId SetProcessor SplitExtId
TestParams

Subclasses
%iKnow.DeepSee.Lister %iKnow.Source.Domain.Lister %iKnow.Source.File.Lister
%iKnow.Source.Global.Lister %iKnow.Source.RSS.Lister %iKnow.Source.SQL.Lister
%iKnow.Source.Temp.Lister

Properties

• property BatchMode as %Integer [ Private ];
• property Configuration as %String;
• property ConfigurationId as %Integer [ Private ];
• property ConvParams as %String [ Private ];
• property Converter as %String [ Private ];
• property ConverterId as %Integer [ Private ];
• property CurData as %Integer [ Private ];
• property CurMetaLength as %Integer [ Private ];
• property CurrentRef as %String [ Private ];
• property DataId as %Integer [ Private ];
• property DomainId as %Integer [ Private ];
• property FixedMetaKeys as %List [ Private ];
• property FixedMetaValues as %List [ Private ];
• property ListGlo as %String [ Private ];
• property ListId as %Integer [ Private ];
• property ListerClassId as %Integer [ Private ];
• property Loader as %iKnow.Source.Loader [ Private ];
Utility local Loader instance for the ProcessList wrapper call
• property MetaId as %Integer [ Private ];
• property Processor as %String [ Private ];
• property ProcessorId as %Integer [ Private ];
• property ProcessorParams as %String [ Private ];
• property SrcData as %String [ Private ];
• property SrcList as %String;
• property SrcMeta as %String;
• property SrcTemp as %String [ Private ];
• property SrcTempMeta as %String;
• property TempId as %Integer [ Private ];
surrogate domain ID for use in CacheTemp guaranteed to be unique across namespaces

Methods

• private method %OnNew(domId As %Integer) as %Status
This callback method is invoked by the %New method to provide notification that a new instance of an object is being created.

If this method returns an error then the object will not be created.

It is passed the arguments provided in the %New call. When customizing this method, override the arguments with whatever variables and types you expect to receive from %New(). For example, if you're going to call %New, passing 2 arguments, %OnNew's signature could be:

Method %OnNew(dob as %Date = "", name as %Name = "") as %Status If instead of returning a %Status code this returns an oref and this oref is a subclass of the current class then this oref will be the one returned to the caller of %New method.

• private final method AddGroup(groupName As %String) as %Integer

Utility method to store a (new) Group Name for this Lister, returning the corresponding ID. If the Group Name already exists for this domain and lister ID, it returns the existing groups ID.

• final classmethod AddGroupStatic(domainId As %Integer, listerId As %Integer, groupName As %String) as %Integer

Static version of AddGroup.

• final method AddListToBatch(listargs...) as %Status

This method is used to register a list the Lister implementation will later on scan through as part of the execution of the %iKnow.Source.Loader ProcessBatch process. This method takes this Lister implementations Lister Parameters as its arguments, as specified in the class documentation of this Lister implementation.

• final classmethod BuildExtId(domainId As %Integer, listerId As %Integer, groupId As %Integer, localRef As %String, Output sc As %Status = $$$OK) as %String

This method will assemble an External ID, starting from the lister ID, group ID and local reference.

The exact implementation of combining the group name and local reference into a full reference is delegated to BuildFullRef method of the specific Lister implementation referenced by the listerId.

• final classmethod BuildExtIdFromName(domainId As %Integer, groupName As %String, localRef As %String, Output sc As %Status = $$$OK) as %String

This method will, assuming it is called for the appropriate Lister class, build the External ID based on the group name and local reference.

WARNING: this method will throw an error if called from the abstract Lister class, as this combination of groupName and localRef is specific to the Lister implementation class.

• private classmethod BuildFullRef(domainId As %Integer, groupName As %String, localRef As %String) as %String

This method should be implemented by subclasses, taking into account the Full Reference structure for this particular Lister. It builds the Full Reference based on a Group Name and Local Reference.

• final classmethod BuildFullRefFromComposite(domainId As %Integer, ref As %List, Output sc As %Status = $$$OK) as %String
• classmethod BuildSimpleRefFromComposite(domainId As %Integer, ref As %List, Output sc As %Status = $$$OK) as %String
Builds a reference for use in $$$IKPSIMPLEEXTIDS domains. Defaults to the Local Ref
• classmethod DefaultProcessor() as %String

This method returns the default %iKnow.Source.Processor class to use (without any parameters) if none was registered explicitly through calls to SetProcessor. By default, this selects (if it exists) a class called Processor in the same package as this Lister implementation, but subclasses might wish to override this and hardcode a specific default Processor to use.

• method ExpandList(listparams As %List) as %Status

This method should be overridden by subclasses, implementing how they scan through the particular type of source location or structures they specialize in. The arguments for this method are the same Lister Parameters as those used when issuing the corresponding AddListToBatch call and are specified and documented by the Lister implementation.

When expanding the list, the Lister should build a "Composite reference": a %List containing the lister ID, the group ID and the local reference for each (candidate) source it found and pass that to either a StoreRef, StoreTemp or StoreTempStream call. Optionally, it can store metadata for each of these refs using the SetMetadataValues method.

NOTE: This method is used by the iKnow loading frameworkand should never be called by custom code.

• classmethod FormatParams(listerParams As %List) as %String

Utility method to format the parameter string passed in AddList for display by the %iKnow.Source.Loader.GetCurrentLists() method. Defaults to $lts() but should be overridden to nicen up complex datatypes in the parameter list.

• classmethod GetAlias() as %String

This method returns the simple alias for this Lister class, to be prefixed with a colon. If this method returns the empty string, the Lister Reference will start with the full Lister class name rather than this alias (prefixed with a colon)

• final classmethod GetExtidPrefix() as %String
This utility method provides the prefix that is prepended to the Full Reference
• final classmethod GetGroupId(domId As %Integer, listerId As %Integer, groupName As %String) as %Integer

Utility method to retrieve the group ID corresponding to a partiular Group Name, for a given domain and lister ID, or zero when the group name is not recognized.

• final classmethod GetGroupName(domId As %Integer, listerId As %Integer, groupId As %Integer, Output sc As %Status = $$$OK) as %String

Utility method to retrieve the Group Name corresponding to a particular group ID, for a given domain and lister ID. An error is passed through the sc output parameter if no group was found for this ID.

• final method GetListerData() as %List
• method Init(config As %String, processor As %String, processorParams As %List = "", converter As %String = "", converterParams As %List = "") as %Status

Shorthand method invoking SetConfig, SetProcessor and SetConverter in one call. A Lister can be re-initialized between calls to AddListToBatch or ProcessList, resulting in different settings to be used for the respective lists.

• final method MaxListId() as %Integer
• final method ProcessList(listerParams...) as %Status

Wrapper method to process a single List in one call. This method instantiates a %iKnow.Source.Loader object, registers itself with it and then calls the ProcessList methodon the Loader with the supplied parameters.

This method takes the Lister Parameters for this particular Lister implementation.

• final method ProcessVirtualList(listerParams...) as %Status

Same as ProcessList, but sources will be added as Virtual Sources.

• private method RefExists(groupName As %String, localRef As %String, checkStaging As %Boolean = 0) as %Boolean

Basic method to check if a given reference exists already in the domain, which can be used in the Lister process to take appropriate action, such as skipping this particular ref or raising an error. If checkStaging is set to 1, it will also loop through the sources listed so far (by ExpandList calls) to see if there is no name conflict with these either.

• final method RegisterFixedMetadata(pKeys As %List, pValues As %List) as %Status
Registers fixed metadata values pValues for the metadata fields in pKeys, for all sources to be listed as part of the next ProcessList or AddListToBatch calls, until the next call to RegistereFixedMetadata. The keys and values in pKeys and pValues should correspond positionally.
• private final method RegisterMetadataKeys(metaFieldNames As %List) as %Status

Registers the specified list of metadata field names to be used for all subsequent calls to SetMetadataValues until RegisterMetadataKeys is called again.

The entries in the supplied %List have to be existing metadata field names for the current domain, otherwise they will be ignored.

• final method RegisterMetadataKeysInternal(metaFieldNames As %List, forceRefresh As %Boolean = 0) as %Status
Internal method. Allows explicitly updating the current metadata list to ensure new values for fixed fields are picked up.
• private final method ReportError(ref As %List, message As %String, location As %String = "") as %Status
Utility method to report failures for a given source (identified through a Composite Ref)
• private final method ReportWarning(ref As %List, message As %String, location As %String = "") as %Status
Utility method to report warnings for a given source (identified through a Composite Ref)
• method ResetRef() as %Status
BDB304 - Reset the current ref, to be run after Loader.ProcessList()
• method SetBatchMode(BatchMode As %Integer)
• final method SetConfig(ByRef config As %String) as %Status

Registers the %iKnow.Source.Configuration name to use when indexing sources from from the lists registered through AddListToBatch or ProcessList calls onwards. Using this method to set a different Configuration between two AddListToBatch calls will not impact any lists already registered in earlier calls.

• final method SetConverter(converter As %String, convparams As %List) as %Status

Registers the %iKnow.Source.Converter class and parameters to use when indexing sources from from the lists registered through AddListToBatch or ProcessList calls onwards. Using this method to set a different Converter between two AddListToBatch calls will not impact any lists already registered in earlier calls.

• final method SetDataId(dataId As %Integer) as %Status
• private final method SetMetadataValues(ref As %List, metaValues As %List) as %Status

Stores the supplied metadata values for the source being listed identified by ref. The values in metavalues should correspond positionally to the metadata field names passed in the most recent call to RegisterMetadataKeys for this Lister instance.

• final method SetProcessor(processor As %String, processorParams As %List) as %Status

Registers the %iKnow.Source.Processor class and parameters to use when indexing sources from from the lists registered through AddListToBatch or ProcessList calls onwards. Using this method to set a different Processor between two AddListToBatch calls will not impact any lists already registered in earlier calls.

• final classmethod SplitExtId(domainId As %Integer, extId As %String, Output listerId As %Integer, Output groupId As %Integer, Output localRef As %String) as %Status

This method will split a complete External ID into the lister reference, group name and local reference and return the corresponding lister ID, group ID and local reference.

The exact implementation of splitting the full reference into a group name and local reference is delegated to SplitFullRef method of the specific Lister implementation as referenced by the lister reference, which is the first part of the full External ID.

• private classmethod SplitFullRef(domainId As %Integer, fullRef As %String, Output groupName As %String, Output localRef As %String) as %Status

This method should be implemented by subclasses, taking into account the Full Reference structure for this particular Lister implementation. It derives the Group Name and Local Reference from the supplied Full Reference.

• private final method StoreRef(ref As %List) as %Status
Stores the supplied reference for a source to be processed.
• private final method StoreTemp(ref As %List, data As %String) as %Status

Stores the supplied reference for a source to be loaded, along with the sources contents passed in through 'data' as a %String to be processed by %iKnow.Source.Temp.Processor (make sure it is set using SetProcessor at runtime or DefaultProcessor as the overall default!).

This method can be called with the same ref multiple times, which will append the %String data passed in.

• private final method StoreTempStream(ref As %List, streamOID As %String) as %Status

Stores the supplied reference for a source to be loaded, along with the sources contents passed in through 'data' in the form of a %Stream.GlobalCharacter to be processed by %iKnow.Source.Temp.Processor.

See also StoreTemp

• method TestParams(listerParams...) as %Status

This method is called at the start of ProcessList and AddListToBatch to validate the supplied Lister Parameters. Subclasses should override this method to execute the validation logic specific to their Lister Parameters.



Copyright © 1997-2018, InterSystems Corporation