Compiling and Building Cubes

Specifying the Agent Count

When you build a cube asynchronously, DeepSee sets up a pool of agents to do the work, if it is possible to use parallel processing. This pool consists of a set of agents with high priority and the same number of agents with low priority. By default, the total number of high priority agents (or low priority agents) is four times the number of cores detected on the machine where DeepSee is running.

To specify a different agent count, call the class method %SetAgentCount() in the class %DeepSee.UtilsOpens in a new tab. The argument should be one of the following:

• The desired number of high priority agents — that is, half of the desired total agent count.

• Null — to restore the default behavior.

The method returns the current number of high priority agents — that is, half of the total agent count. Or it can return null, which means that the default behavior is in effect.

For example, the following command sets the total agent count to 10:

 d ##class(%DeepSee.Utils).%SetAgentCount(5)

To see the current number of agents, use the %GetAgentCount() method of the same class. This method returns the current number of high priority agents. Or it can return null, which means that the default behavior is in effect.

When the agents are running, they are visible in the Management Portal as separate processes. To see them, click System Operation and then click Processes. Look for instances of the %DeepSee.TaskMasterOpens in a new tab routine. They are not shut down automatically but are intended not to consume resources when idle.

On rare occasions, you might need to reset these agents. To do so, use the %Reset() method of %DeepSee.UtilsOpens in a new tab. This method also clears any pending tasks and clears the result cache for the current namespace, which would have an immediate impact on any users. This method is intended for use only during development.

Seeing Build Errors

When you build a cube in the Architect or in the Terminal, DeepSee indicates if there are any build errors. For example:

This user interface does not display all the build errors (the same is true if you build a cube in the Terminal).

To see all the recorded build errors, do either of the following:

• Look for the log file install-dir/mgr/DeepSeeUpdate_cube_name_NAMESPACE.log, where cube_name is the name of the cube, and NAMESPACE is the namespace in which this cube is defined.

  The time stamp in this file uses $NOW to write the local date and time, ignoring daylight saving time.

• Use the %PrintBuildErrors() method of %DeepSee.UtilsOpens in a new tab, as follows:

  do ##class(%DeepSee.Utils).%PrintBuildErrors(cubename)

  

  Where cubename is the logical name of the cube, in quotes.

  This method displays information about all build errors. For example (with added line breaks):

  SAMPLES>do ##class(%DeepSee.Utils).%PrintBuildErrors("patients")
      1   Source ID: 13
          ERROR #5001: Error inserting/updating fact: (Source ID:'13') 
  Field 'DeepSee_Model_PatientsCube.Fact.Dx3295243289' (value 'abc') failed validation
   
      2   Source ID: 22
          ERROR #5001: Error inserting/updating fact: (Source ID:'22') 
  Field 'DeepSee_Model_PatientsCube.Fact.Dx3295243289' (value 'abc') failed validation
   
      3   Source ID: 37
          ERROR #5001: Error inserting/updating fact: (Source ID:'37') 
  Field 'DeepSee_Model_PatientsCube.Fact.Dx3295243289' (value 'abc') failed validation
  ...
  81 build error(s) for 'patients'

  

Checking the Fact Count

When you build a cube, DeepSee reports the number of facts that it builds and indexes. For example, in the Architect:

Each fact is a record in the fact table. The fact table should have the same as the number of records in the base table, except in the following cases:

• You limit the fact count as discussed earlier in this chapter.

• The cube class also defines the %OnProcessFact() callback, which you can use to exclude records from the cube. See “Using Advanced Features of Cubes and Subject Areas” in the Advanced DeepSee Modeling Guide.

Also, when DeepSee builds the indices, the index count should equal the number of records in the fact table. For example, the Architect should show the same number for Building facts and for Building indices. If there is a discrepancy between these numbers, check the log files.

Possible Causes of Build Errors

If you see build errors or unexplained discrepancies in the fact count, do the following:

1. Examine any levels that use range expressions, and verify that these levels do not drop records. See “Validating Your Levels,” later in this book.

   An error of this kind affects the index count but not the fact count.

2. Try disabling selected dimensions or measures. Then recompile and rebuild to isolate the dimension or measure that is causing the problem.

<STORE> Errors

In some cases, the build log might include errors like the following:

ERROR #5002: Cache error: <STORE>%ConstructIndices+44^Cube.cube_name.Fact.1

This error can occur when a level has a very large number of members. By default, when DeepSee builds the indices, it uses local memory to store the indices in chunks and then write these to disk. If a level has a very large number of members, it is possible to run out of local memory, which causes the <STORE> errors.

To avoid such errors, try either of the following:

• Build the cube with a single process. To do so, use %BuildCube() in the Terminal, and use 0 for its second argument.

• In the <cube> element, specify bitmapChunkInMemory="false" (this is the default). When this cube is built using background processes, the system will use process-private globals instead of local variables (and will not be limited by local memory).

Missing Reference Errors

If your cubes have relationships to other cubes, the build log might include errors like the following:

ERROR #5001: Missing relationship reference in RelatedCubes/Patients: source ID 1 missing reference to RxHomeCity 4

This error can mean that you have built the cubes in the wrong order. See “Building Cubes That Have Relationships” in the Advanced DeepSee Modeling Guide. Note that if you use the Cube Manager, the Cube Manager determines an appropriate build order.

The missing relationship reference error can also occur when new source data becomes available during the cube build process — that is, after only some of the cubes have been built. For example, consider the sample cubes RelatedCubes/Cities and RelatedCubes/Patients (which are available in the SAMPLES namespace). Suppose that you build the cube RelatedCubes/Cities, and after that, the source table for RelatedCubes/Patients receives a record that uses a new city. When you build the cube RelatedCubes/Patients, there will be a missing relationship reference error.

If you are certain that you have built the cubes in the correct order, see the next section for information on recovering from the errors.

Recovering from Build Errors

DeepSee provides a way to rebuild only the records that previously generated build errors, rather than rebuilding the entire cube. To do this:

1. Correct the issues that cause these errors.

2. Use the %FixBuildErrors() methods of %DeepSee.UtilsOpens in a new tab, as follows:

   set sc=##class(%DeepSee.Utils).%FixBuildErrors(cubename)

   

   Where cubename is the logical name of the cube, in quotes. This method accepts a second argument, which specifies whether to display progress messages; for this argument, the default is true.

   For example:

   Fact '100' corrected
   Fact '500' corrected
   Fact '700' corrected
    
    
   3 fact(s) corrected for 'patients'
   0 error(s) remaining for 'patients'

   

   Or rebuild the entire cube.