Skip to main content

DeepSee Cube Integration (Deprecated Form)

This appendix describes how to use older, deprecated form of integration between iKnow and DeepSee cubes. It discusses the following topics:

For information on using the newer form of integration, see “Using Unstructured Data in Cubes” in the Advanced DeepSee Modeling Guide.

Also see the chapter “iKnow KPIs and DeepSee Dashboards.”

Overview

This section provides an overview of the older, deprecated form of integration between iKnow and DeepSee cubes.

DeepSee is an analytics engine and reporting platform that is built into Caché in the same way that iKnow is. The primary purpose of DeepSee is to enable you to create interactive dashboards that you can embed on Zen pages or that users can access via the DeepSee User Portal (which is intended for all end users, not just end users of DeepSee).

Background

A DeepSee cube is a data model that enables you to create DeepSee queries by drag and drop actions. The queries are known as pivot tables. You create pivot tables so that you can display them on DeepSee dashboards. When the dashboard is displayed, the query is executed.

Each cube is based on a source table. You build a cube, and this process generates a fact table that has the same number of records as the source table (or possibly fewer if the model skips some records deliberately). The queries use the fact table rather than the source table. DeepSee provides multiple complementary mechanisms to keep the fact table synchronized with the source table.

generated description: cube and tables

The following figure shows an example pivot table that shows the number of patients and the average allergy count per patient, grouped by age and gender.

generated description: general pivot intro sample

Using this example as a reference, let us discuss the key terms in DeepSee cubes:

  • A level is used to group the records. In this case, we are grouping patients. Each record in the source table corresponds to one patient, as does each record in the fact table.

  • A measure is a value displayed in the body of the pivot table. It is based on values ultimately taken from the source data. For a given context, a measure aggregates the values for all applicable records and represents them with a single value (typically by summing them).

    For example, the measure Patient Count is the number of patients, and the measure Avg Allergy Count is the average number of allergies per patient.

A cube can also define detail listings, whose purpose is to display lowest-level data associated with the current context. To access a listing, you select one or more cells and click the Listing button (not shown in the previous picture). DeepSee then displays a table that lists selected fields for all the records used in the current context.

Relationship between a Cube and iKnow Sources

The iKnow/cube integration makes the following assumption: there is a single iKnow source associated with each record of the cube’s source class. That is, the cube serves as additional information about the iKnow sources. (Phrased differently, the iKnow sources contain additional information associated with each record of the cube’s source class.)

The goal is to be able to use DeepSee and iKnow filters interchangeably where possible.

You can use utility methods to do the following tasks:

Generating iKnow Source Metadata Fields for an Associated DeepSee Cube

If a DeepSee cube uses a source class that includes a reference to iKnow data source, you can use the levels of that cube as iKnow source metadata fields for those sources. This means that you can use DeepSee filters within iKnow queries (including when you access those queries via the DeepSee KPI mechanism as described in “Using Unstructured Data in Cubes” in the Advanced DeepSee Modeling Guide).

This section assumes that the cube is defined and has been built. The cube does not need to be modified or rebuilt.

Prerequisites

Before you can generate iKnow source metadata fields for a cube:

  1. Load the iKnow sources.

    When you do so, make sure that the external ID of each source can be readily associated with a specific record of the source class of the cube.

    For example:

    • If you are using a file lister, you might make sure that the file names match the patient identifiers.

    • If you are using an SQL lister, you might make sure that the IDs of the records match the patient identifiers.

  2. Make sure that the source class of the cube includes a property whose value is the external ID of the corresponding iKnow source.

    For example, this class could include a calculated property whose value follows the naming convention established in the preceding step.

    Property DocumentId As %String [ Calculated ];
    
    Method DocumentIdGet() As %String
    {
     quit ":FILE:c:\patient-data\patient"_..PatientNumber_".txt"
    }
    

Generating the Metadata Fields from the Cube

To generate the metadata fields, you use the GenerateMDFieldsFromDSDims() method of %iKnow.DeepSee.Utils. This method has the following signature:

ClassMethod GenerateMDFieldsFromDSDims(domainId As %Integer,
                                       cubeClassName As %String, 
                                       ByRef levels = 1, 
                                       extIdProperty = "", 
                                       killExistingMDFields As %Boolean = 0) As %Status

Where:

  • domainId is the integer ID of an iKnow domain.

  • cubeClassName is the name of the cube class.

  • levels controls which levels are processed. If you omit this argument, the system processes all levels, skipping the ones that are not supported by the iKnow engine.

    This argument, if specified, must be a multidimensional array of the following form:

    Node Value
    arrayname(level_name) where level_name is the complete MDX identifier of the level, in double quotes. For example: "[DocD].[H1].[Doctor]" Either 0 or 1. If you specify 1, the iKnow metadata field is generated as a bitstring, which is suitable when you expect only a small number of values.
  • extIdProperty is the property you added in the previous subsection. For example: DocumentId

    More generally, extIdProperty is a property (in the cube’s source class) whose value equals the external ID of an iKnow source in the given domain.

  • killExistingMDFields specifies whether to remove any existing iKnow metadata fields.

Using DeepSee Filters with an iKnow KPI

If you also create iKnow KPIs in this domain, then you can use DeepSee filters in the KPI, provided that the KPI and the cube are both associated with the same iKnow domain.

To use this mechanism:

  • Display the KPI in a widget on a dashboard.

  • In the DeepSee Analyzer, create a pivot table.

  • On the same dashboard, add a widget that displays the pivot table. Configure this widget to use one or more levels of the cube as filters, and specify the filter target as * (all widgets).

Generating DeepSee Cubes for an iKnow Domain

This release provides a preliminary version of a utility that generates DeepSee cubes from iKnow engine results. The generated cubes enable you to view entities, entity occurrences, and match results grouped in various ways. With a DeepSee cube, you can perform analysis and exploration via drag and drop actions in the DeepSee Analyzer.

You can use these DeepSee cubes in the same way that you use other DeepSee cubes; you can explore them in the Analyzer, create pivot tables, and add those pivot tables to DeepSee dashboards.

This section describes how to generate and (briefly) how to use DeepSee cubes, given an iKnow domain. It discusses the following topics:

If you can achieve a required result through an existing iKnow query, use that query rather than trying to create an equivalent query with the generated cubes. The iKnow queries are optimized to use internal and precalculated indices and are faster as a consequence.

Generating the Cubes

To generate DeepSee cubes for an iKnow domain, you use the GenerateSourceObjectAndCubes() method of %iKnow.DeepSee.Utils. This method has the following signature:

classmethod GenerateSourceObjectAndCubes(domainId, 
                                         packageName="User", 
                                         createCube=2, 
                                         overwriteExisting=0, 
                                         buildCubes=0) as %Status

Where:

  • domainId is the integer ID of an iKnow domain.

  • packageName is the package in which to place the generated classes.

  • createCube controls which classes are generated:

    • If createCube is 0, the system generates and compiles the classes on which the cubes are based, but does not generate other classes. You might use this option if you wanted to create DeepSee cubes manually in the DeepSee Architect.

      These classes define mappings to access the iKnow data.

    • If createCube is 1, Caché also generates and compiles a cube that represents the iKnow sources.

    • If createCube is 2, Caché also generates and compiles a cube that represents unique entities, and another cube that represents entity occurrences.

    • If createCube is 3, Caché also generates and compiles a cube that represents matching results.

  • overwriteExisting specifies whether to overwrite any existing classes.

  • buildCubes specifies whether to build the generated cubes. This has no effect if createCube is 0.

    The phrase build a cube refers to the process of populating the structures that the cube uses. This process makes it possible to use the cube.

A Brief Look at the Analyzer and the Cubes

This section briefly introduces the Analyzer and the generated cubes. To access the Analyzer and display one of these cubes:

  1. Click Home,DeepSee,Analyzer.

  2. Click the Change button generated description: button change subjectarea.

  3. Click the cube name.

  4. Click OK.

For example, the entity occurrence cube provides information about the occurrence of entities in the source documents. The name of this cube ends with EntityOccurrences. When you view this cube, the left area of the page displays its contents, as follows:

generated description: cube eo

This cube defines one measure, Count, which counts entity occurrences. This measure is used by default.

The Dimensions section shows three dimensions. GeneratedD1CubesSource and Entity are displayed differently from Entity Type because they are defined differently. You can use this dimensions as follows:

  • Entity Type — This dimension enables you to categorize entity occurrences by entity type. If you expand this folder and the Entity Type folder within it, the system displays the following:

    generated description: gencube entity type

  • GeneratedD1CubesSource — This dimension enables you to categorize entity occurrences by the source document to which they belong. If you expand this folder, the system displays the following:

    generated description: gencube sources

    If you expand the SourceID folder, the system displays something like this, depending on your data:

    generated description: gencube source id

  • Entity — This dimension enables you to categorize entity occurrences by entity value. If you expand this folder and the top-level folders in it, the system displays the following:

    generated description: gencube entity

    If you expand the Entity folder, the system displays something like this, depending on your data:

    generated description: gencube entity value

The right area of the page looks like this:

generated description: analyzer intro

The area at the bottom is a simple pivot table. It displays the total value for Count. In this example, there are 7118 entity occurrences in the domain.

The area above that enables you to create pivot tables. You can drag and drop items from the left to this area, and the system automatically modifies the pivot table. For example, if you drag Entity (the inner Entity folder) to the Rows box, the system displays the following:

generated description: pivot example

This pivot table shows both relations and concepts because we have not applied any filtering. To filter the pivot table to show only concepts:

  1. Expand the Entity Type dimension on the left:

    generated description: gencube entity type

  2. Drag concept and drop it into the Filters box on the right.

    This immediately updates the display.

Now you might see something like this:

generated description: pivot filtered

The EntityOccurrences cube defines a detail listing. To access it, select one or more cells and click the Listing button generated description: button listing. (In this example, we click the cell for ancient viruses.) The system then displays something like the following:

generated description: listing example

Rebuilding the Cubes

After you process more sources for this iKnow domain, it is not necessary to regenerate the classes but it is necessary to rebuild the DeepSee cubes. It is also necessary to rebuild the cubes in a specific order:

  1. Rebuild the sources cube, which is the cube whose name ends in Sources.

  2. Rebuild the other cubes, in any order.

To rebuild a cube, open it in the Architect and click Rebuild.

See Also

FeedbackOpens in a new tab