Skip to main content

REST Interface

Important:

InterSystems has deprecatedOpens in a new tab InterSystems IRIS® Natural Language Processing (NLP). It may be removed from future versions of InterSystems products. The following documentation is provided as reference for existing users only. Existing users who would like assistance identifying an alternative solution should contact the WRCOpens in a new tab.

A REST API interface is provided for common NLP tasks in the %iKnow.REST.v1Opens in a new tab class of InterSystems IRIS data platform. This REST API covers basics (querying domains & configurations), classic queries (including semantics), simple domain operations (adding/deleting sources, skiplist management) and support for the matching and dictionary APIs.

Here are some of the main characteristics of the API:

  • Most API calls have a GET and POST variant. The GET variant takes default values for all non-essential parameters and should be easiest to invoke. The POST variant gives you access to all relevant arguments through a JSON string passed as the request object, which will complement what you put in the URL and override it in the event of an overlap. For example, some entity values that contain a slash cannot be used with a GET request, but can be supplied as a JSON property submitted in a POST request.

  • Available arguments for POST requests are described succinctly in the Class Reference in %iKnow.REST.v1Opens in a new tab and some shared arguments (filters & highlighting) in %iKnow.REST.BaseOpens in a new tab.

  • The entire URL is always in lowercase. POST arguments and returned JSON property names are in camelcase, with the first character in lowercase.

  • Returned JSON is typically comprehensive, returning much more information than is supplied by the corresponding flat (SQL-compatible) query. For example, if you ask for all sources containing an entity, you’ll get the list of sources, metadata, and optionally highlighted snippets back. You can exclude some of this functionality if desired for performance reasons, but by default most functionality is included automatically.

Swagger

The NLP REST API is fully documented using the OpenAPI SpecificationOpens in a new tab (also known as SwaggerOpens in a new tab). The description in YAML is available from the "/swagger" endpoint and can be loaded directly into swagger-uiOpens in a new tab for convenient GUI capabilities on top of this API.

To use it, either install swagger-ui or go to http://petstore.swagger.io and point to this endpoint.

  • Download swagger-ui

  • Unzip the swagger-ui download.

  • In the Swagger box specify either:

    • for InterSystems IRIS with Minimal security: http://localhost:57775/api/iknow/v1/samples/swagger (substituting your Webserver port number for 57775 and the desired namespace for samples).

    • for InterSystems IRIS with Normal security: http://localhost:57775/api/iknow/v1/samples/swagger?IRISUsername=myname&IRISPassword=mypassword (substituting your Webserver port number for 57775, the desired namespace for samples, your InterSystems IRIS user name for myname, and your InterSystems IRIS accounts password for mypassword).

  • Select the Explore button.

Returning NLP Data Using Swagger

The Swagger–UI display for each REST statement can be used to invoke the REST API directly, or to provide a Request URL value that can be used to invoke the REST API in a browser window (for GET requests).

To invoke a REST API directly, specify the required Parameters. Commonly, an NLP REST API requires the domain ID parameter, specified as an integer number. For example, to GET the domain details of the domain with the ID of 2 generates the following Request URL:

http://localhost:57775/api/iknow/v1/user/domain/2/details

Note that the Swagger interface infers the namespace to use from the URL you used to access the /swagger endpoint. For example, to invoke the REST API in namespace SAMPLES, load the swagger endpoint using the URL:

http://localhost:57775/api/iknow/v1/samples/swagger

The NLP data is returned to the browser in JSON format. For example, this Request URL returns the following JSON string for a domain named “crashes”:

{"id":"2","name":"crashes","parameters":{"DefaultConfig":"crashes.Configuration","DefinitionClass":"User.crashes","ManagedBy":"User.crashes"},"metadata":[{"id":1,"name":"DateIndexed","operators":["=","!=","<","<=",">",">="],"dataType":"date","storage":"normal","caseSensitive":0,"hidden":0}]}

REST Operations

The NLP REST operations are grouped into the following topics:

While the core REST API supports access to any defined domain (by specifying the domain id at the start of almost every URL), you can also generate a domain-specific REST API, allowing you to expose a single domain through your webapp. To do this, use the CreateDomainAPI()Opens in a new tab method found in the %iKnow.REST.BaseOpens in a new tab class, and then create a new web application referring to the newly created class as the “Dispatch Class”. This may be useful if you only want to enable access to a specific domain and have disabled the general /api/iknow/ web application.

For any of these REST operations, if the specified domain does not exist, the operation returns {"errors":[{"error":"ERROR #8021: Domain with id 7 does not exist","code":8021,"domain":"%ObjectErrors","id":"IKNoDomainWithId","params":["7"]}],"summary":"ERROR #8021: Domain with id 7 does not exist"}, unless stated otherwise.

Domains and Configurations

The NLP REST operations for Domains and Configurations (categorized as “miscellaneous”) are the following:

  • List Domains: GET and POST to retrieve information about all domains in the current namespace. See GetDomains()Opens in a new tab.

    If there are no defined domains, returns {"domains":[]}. Otherwise, returns a JSON array of domains in domain name order. For each domain, it lists the domain id, domain name, NLP version, definitionClass name, and sourceCount.

    Listing domains is further described in the “NLP Environment” chapter of this guide.

  • List Configurations: GET and POST to retrieve information about all configurations in the current namespace. See GetConfigurations()Opens in a new tab.

    An NLP-enabled namespace is assigned the configuration {"configurations":[{"id":1,"name":"DEFAULT","languages":["en"]}]}. You can explicitly define additional configurations. When you use the Domain Architect to create a domain, NLP creates a corresponding configuration.

    This option returns a JSON array of configurations in configuration name order. For each configuration it lists the configuration id, the configuration name, a JSON array of supported languages, and (optionally) a userDictionary JSON object. The language defaults to English: ["en"].

    Listing configurations is further described in the “NLP Environment” chapter of this guide.

  • Domain Details: GET and POST to retrieve detailed information about a specified domain. See GetDomainDetails()Opens in a new tab.

    Data returned includes the domain’s ID, name, parameters, and metadata.

    • parameters include DefaultConfig (domainname.Configuration), DefinitionClass, and ManagedBy (defaults to the DefinitionClass).

    • metadata includes metadata id, metadata name (defaults to DateIndexed), an array of supported operators, datatype (DateIndexed is datatype date), storage (defaults to normal), caseSensitive (a boolean value), and hidden (a boolean value).

Sources

The NLP REST operations for sources are the following:

  • Filtered Sources: GET and POST to retrieve all sources that pass a specified filter. By default, returns all sources. See GetSources()Opens in a new tab.

    If the domain exists but contains no sources that pass the filter, returns {"sources":[]}. If there are sources, returns the id, extId (external ID), metadata (the date and time when the source was indexed), and a snippet of the text for each source. This snippet consists of two sentences from the text. If the sentences are not consecutive, an ellipsis (three dots) is specified to indicate this gap. If there two or fewer sentences in the text, the snippet contains the full text.

    Filtering sources is further described in the “Filtering Sources” chapter of this guide.

  • Sources by Entity: GET and POST to retrieve sources containing a specified entity. See GetSourcesByEntity()Opens in a new tab

    If the domain exists but contains no sources that pass the filter, returns {"sources":[]}. If there are sources, returns the id, extId (external ID), metadata (the date and time when the source was indexed), and a snippet of the text for each source. This snippet consists of (at most) two sentences from the text that contain the specified entity. These sentences are not necessarily in consecutive order.

    Filtering sources by entity match is further described in the “Filtering Sources” chapter of this guide.

  • Sources by CRC: GET and POST to retrieve sources containing a specified CRC. The operations can be used to retrieve a CRC (concept:relation:concept), a CR (concept:relation), a RC (:relation:concept), or a CC (concept::concept). See GetSourcesByCRC()Opens in a new tab.

    If the domain exists but contains no sources that contain the CRC, returns {"sources":[]}. If there are sources that contain the CRC, returns the id, extId (external ID), metadata (the date and time when the source was indexed), and a snippet of the text for each source. This snippet consists of (at most) two sentences from the text that contain the specified CRC. These sentences are not necessarily in consecutive order.

    Note that a CR must be followed by a concept. Therefore the CR pilot:stated returns ”pilot stated the airplane”, pilot stated he” and “pilot stated \"I then". It does not return “pilot stated that” or “pilot stated to”.

  • Similar Sources: GET and POST to retrieve sources similar to a specified source. See GetSimilarSources()Opens in a new tab.

    If the domain exists but contains no similar sources, returns {"sources":[]}. If there are similar sources, returns a JSON array of entities in descending order by similarity score. If there are multiple similar entities that have the same similarity score, they are listed in ID order. For each similar source, returns the id, extId (external ID), score (similarity score), percentageMatched, percentageNew, numberOfEntitiesInRefSource, numberOfEntitiesInCommon, numberOfEntitiesInThisSource, metadata (the date and time when the source was indexed), and a snippet of the text for each source. This snippet consists of (at most) two sentences from the text that contain similar entities.

    Similar sources are further described in the “NLP Queries” chapter of this guide.

  • Source Details: GET and POST to retrieve details about a specified source. See GetSourceDetails()Opens in a new tab.

    For the specified source returns a virtual boolean, metadata (the date and time when the source was indexed), and the full text of the source.

  • Add a Source: POST to add a source to a domain. See AddSource()Opens in a new tab.

    Adding sources are further described in the “Loading Text Data Programmatically” chapter of this guide.

  • Delete a Source: DELETE to delete a source from a domain. See DropSource()Opens in a new tab.

    Deleting sources are further described in the “Loading Text Data Programmatically” chapter of this guide.

  • Source Metadata: POST to update metadata for a specific source. See SetMetadata()Opens in a new tab.

Entities

The NLP REST operations for entities are the following:

  • Top Entities: GET and POST to retrieve the top entities in the specified domain. See GetEntities()Opens in a new tab.

    If there are no defined sources containing entities, returns {"entities":[]}. If there are entities, returns a JSON array of entities in descending order by frequency. If there are multiple entities that have the same frequency, they are listed in ID order. Each entity is listed as a JSON object with key-value pairs for id, value, frequency, and spread.

    Top entities are further described in the “NLP Queries” chapter of this guide.

  • Similar Entities: GET and POST to retrieve entities similar to a specified string. See GetSimilarEntities()Opens in a new tab.

    If there are no similar entities, returns {"entities":[]}. If there are similar entities, returns a JSON array of entities in descending order by frequency. If there are multiple similar entities that have the same frequency, they are listed in ID order. Each entity is listed as a JSON object with key-value pairs for id, value, frequency, and spread.

    Similar entities are further described in the “NLP Queries” chapter of this guide.

  • Related Entities: GET and POST to retrieve entities related to a specified entity. See GetRelatedEntities()Opens in a new tab.

    If there are no related entities, returns {"entities":[]}. If there are related entities, returns a JSON array of entities in descending order by proximity. If there are multiple similar entities that have the same proximity, they are listed in ID order. Each entity is listed as a JSON object with key-value pairs for id, value, and proximity.

    Related entities are further described in the “NLP Queries” chapter of this guide.

  • Entity Details: GET and POST to retrieve details about a specified entity. See GetEntityDetails()Opens in a new tab.

    If the specified entity does not exist in the domain, or the specified domain does not exist, returns {"id":"","value":"aardvark"}. If the entity is found, returns a JSON object with key-value pairs for id, value, and the subarrays metricsAsConcept, metricsAsRelation, and metricsAsAny. Each of these subarrays lists frequency, spread, and tfidf.

    If the entity only occurs as a concept, the metricsAsRelation values are 0 and the metricsAsAny values are the same as the metricsAsConcept values. If the entity only occurs as a relation, the metricsAsConcept values are 0 and the metricsAsAny values are the same as the metricsAsRelation values. If the entity occurs as both a concept and a relation, the metricsAsAny values for frequency and spread are the totals of their concept and relation values. The tfidf values is calculated from the corresponding concept and relation values.

Sentences

The NLP REST operations for sentences are the following:

  • Sentences by Entity: GET and POST to retrieve sentences containing a specified entity. See GetSentencesByEntity()Opens in a new tab.

    If there are no sentences containing the specified entity, returns {"sentences":[]}. If there are sentences that contain the entity, returns a JSON array of sentences. For each sentence lists the sentence id, the source id, the text of the sentence, and the parts of the sentence. Parts are listed in sentence order, with each part an element of the parts array. For each part in the part array it lists partId, literal (with original letter case), role (concept, relation, pathRelevant, or nonRelevant), entityId, and entity (in lowercase letters). If the part is a nonRelevant, no entityId or entity are listed.

  • Sentence Details: GET and POST to retrieve details about a specified sentence. See GetSentenceDetails()Opens in a new tab.

    If the specified sentence exists, returns the sentence id, the parts of the sentence, the text, and the sourceId. Parts are listed in sentence order, with each part an element of the parts array. For each part in the part array it lists partId, literal (with original letter case), role (concept, relation, pathRelevant, or nonRelevant), entityId, and entity (in lowercase letters). If the part is a nonRelevant, no entityId or entity are listed.

Paths and CRCs

The NLP REST operations for Paths and CRCs (concept-relation-concept strings) are the following:

  • Top CRCs: GET and POST to retrieve the top CRCs in the specified domain. See GetCRCs()Opens in a new tab.

    If there are no sources containing CRCs, returns {"crcs":[]}. Returns a JSON array of CRCs in descending order by frequency. If there are multiple CRCs that have the same frequency, they are listed in ID order. For each CRC, it lists the id, head concept, relation, tail concept, frequency, and spread. For some CRCs, the head or the tail values are "" (the empty string). For example. a head key-value pair would have a value of "" in a CRC that begins with the relation “according to” or “during”, because the head of this CRC is unstated; a tail key-value pair would have a value of "" in a CRC that contains a relation such as “occurred” or “said”, because the tail of this CRC is unstated.

  • CRCs by Entity: GET and POST to retrieve CRCs containing a specified entity. See GetCRCsByEntity()Opens in a new tab.

    If there are no sources with CRCs containing a specified entity, returns {"crcs":[]}. Otherwise, returns a JSON array of CRCs in descending order by frequency. If there are multiple CRCs that have the same frequency, they are listed in ID order. For each CRC it lists the id, head concept, relation, tail concept, frequency, and spread. For some CRCs, the head or tail key:value pair is omitted.

  • Paths by Entity: GET and POST to retrieve Paths containing a specified entity. See GetPathsByEntity()Opens in a new tab.

    If there are no sources with paths containing a specified entity, returns {"paths":[]}. Otherwise, returns a JSON array of paths, each consisting of a path id, an entities array, a sourceId and a sentenceId. The entities array contains for each entity an entity id, a partId, an entity value, a role specifier (concept, relation, or pathRelevant), and (optionally) an attributes array. An attributes array consists of type, (negation), level (path, word), and, if level is word, a words key:value pair. For example, "attributes":[{"type":"negation","level":"path"}]}.

Dictionaries and Matching

The NLP REST operations for dictionaries are the following:

The NLP REST operations for matching operations using a dictionary are the following:

Skiplists

The NLP REST operations for skiplists are the following:

FeedbackOpens in a new tab