Advanced DeepSee Modeling Guide
Defining KPIs with Filters and Listings
|
|
This chapter describes options for defining KPIs that include filters and listings. It discusses the following topics:
Typical dashboards include one or more filter controls, with which the user interacts with the widgets on the dashboard. Each filter control is typically displayed as a drop-down list. A filter can filter the data displayed by a widget, or it could affect the data source in some other different manner.
For a pivot table, any level in the cube or subject area can be used as filter. For a KPI, however, no filters are defined by default. To add filters to a KPI, you use the following system:
-
You define each filter; that is, you specify the filter names and the items in the filter lists.
You can skip this step if you intend to use filters provided by a pivot table.
-
You specify the KPI query programmatically so that it uses the filters. To do this, you implement a callback method in the KPI so that it overrides any hardcoded query. Within this method, you have access to the selected filter values.
In typical cases, you add a WHERE clause (or %FILTER, for an MDX query) that restricts the records used by the query. Then when the user refreshes the dashboard, DeepSee reruns the query and displays the results.
Because you construct the KPI query yourself, you can use filters in more general ways. That is, you are not required to use the filter selection to affect the WHERE (or %FILTER) clause of the query. For an extreme example, a filter could display the options 1, 2, and 3. Then the KPI could execute version 1, 2, or 3 of the query, which might be entirely different versions. For another example, a filter could display a list of measure names, and the KPI could include the selected measures in its query.
A dashboard can contain both KPIs and pivot tables, displayed in scorecards, pivot table widgets, or other widgets. Within a dashboard, a filter can be configured to affect multiple widgets.
It is possible to define filters that affect both KPIs and pivot tables.
Pivot tables send and receive filter values in a specific syntax, described in the
following subsection. To create interoperable filters, you must consider this requirement; the
second subsection describes two possible approaches.
The following table indicates the filter syntax used and expected by pivot tables. This syntax applies to pivot tables that are displayed in any kind of dashboard widget.
Other than the optional
%NOT string, the filter names and filter values are not case-sensitive.
There are two general ways to create filters that can affect both pivot tables and KPIs:
-
You can create a KPI that defines and uses filters in the format required by pivot tables.
-
You can define a KPI that converts filter names and values from the pivot table format to the format needed for the KPI query.
These approaches can work with either MDX-based or SQL-based KPIs, but more work is needed to convert values when you use an SQL-based KPI.
For an MDX example, see the dashboard
Demo Filter Interoperability in the
SAMPLES namespace. This dashboard demonstrates the former approach. This dashboard displays two pivot table widgets; the upper one displays a pivot table, and the lower one displays a KPI. For this dashboard, the
Filters worklist on the left includes the following filters:
-
The
Favorite Color filter is configured as part of the upper widget and is therefore defined by the pivot table that is displayed in that widget.
Favorite Color is a level in the cube on which this pivot table is based.
-
The
Doctor Group filter is configured as part of the lower widget and is therefore defined by the KPI that is displayed in that widget.
This filter is defined programmatically within this KPI. This filter is defined to use values in pivot table format, as given in the previous section.
For both of these filters, the target of this filter is
* (which refers to all widgets in this dashboard). As you can see by experimentation, both filters affect both of these widgets.
The KPI includes an additional filter (
Yaxis), which controls the y-axis of the MDX query used by the KPI. This filter has no effect on the pivot table.
To define filters in an MDX-based KPI, do the following:
-
One option is to hardcode the list of filters. To do so, add a set of
<filter> elements to the
<kpi> element. For example:
<kpi name="sample KPI"...>
<filter name="[aged].[h1].[age group]" displayName="Age Group"/>
<filter name="[gend].[h1].[gender]" displayName="Gender"/>
<filter name="[homed].[h1].[zip]" displayName="ZIP Code"/>
...
In this example, the logical filter names are
MDX level identifiers. It is convenient, but not required, to specify the logical names in this way.
-
-
You can enable users to select only one element from a filter list (instead of multiple elements, which is the default). To do so, add
multiSelect="false" to the
<filter> element. For example:
<filter name="[homed].[h1].[zip]" displayName="ZipCode" multiSelect="false" />
When multiselect is disabled, the
Exclude option is also disabled.
-
If the filter items are days, you can display a calendar control rather than the default drop-down list. To do so, add
searchType="day" to the
<filter> element. For example:
<filter name="[birthd].[h1].[day]" displayName="Day"
filterProperty="Day" searchType="day"/>
-
A filter can depend upon another filter. To indicate this, use the optional
dependsOn attribute. For example:
<filter name="[homed].[h1].[zip]" displayName="ZIP Code"/>
<filter name="[homed].[h1].[city]" displayName="City" dependsOn="[homed].[h1].[zip]"/>
-
It is useful if the logical names are the same as MDX member keys; otherwise, more work is needed to construct filter clauses, as described
later in this chapter.
ClassMethod %OnGetFilterMembers(pFilter As %String, Output pMembers As %List,pSearchKey As %String = "") As %Status
{
set status = $$$OK
try {
do ..%GetMembersForFilter("Patients.cube",pFilter,.pMembers,pSearchKey)
}
catch(ex) {
set status = ex.AsStatus()
}
quit status
}
Simply replace
Patients with the name of your cube. For example:
do ..%GetMembersForFilter("YourCube.cube",pFilter,.pMembers,pSearchKey)
-
classmethod %OnGetFilterMembers(pFilter As %String, Output pMembers As %List, pSearchKey As %String = "") as %Status
-
pFilter is the logical name of a filter.
-
pMembers specifies the members in a $LISTBUILD list. This list contains both the logical names and the display names. For details, see the class reference for
%DeepSee.KPI.
-
Tip:
You can use this method in any KPI, regardless of the type of query it uses.
classmethod %GetMembersForFilter(pCube As %String, pFilterSpec As %String, Output pMembers, pSearchKey As %String = "") as %Status
-
pCube is the logical name of a DeepSee cube, with
.cube appended to it.
-
-
pMembers, which is returned as an output parameter, is a list of members in the form required by
%OnGetFilterMembers(). In this list, the filter items are member keys.
-
This method is more useful for MDX-based KPIs than for SQL-based KPIs. For an SQL-based KPI, you would need to convert the filter values to a form suitable for use in your query.
To define filters in an SQL-based KPI:
-
-
-
-
This section describes other options for defining filter names and filter items. It discusses the following topics:
The easiest way to define the filter names is to hardcode them as described
earlier in this chapter. Another option is to define them at runtime. To do so, override the
%OnGetFilterList() method of your KPI class. This method has the following signature:
classmethod %OnGetFilterList(Output pFilters As %List,pDataSourceName As %String = "") As %Status
Where
pFilters is an array with the following nodes:
-
pFilters Specifies the number of filters.
-
pFilters(n) Specifies the details for the
nth filter. This is a
$LISTBUILD list that consists of the following items:
-
A string that equals the logical name of the filter.
-
A string that equals the display name of the filter.
-
A string that equals the filter property; the default is the logical name of the filter.
-
1 or 0 to indicate whether multiselect is enabled for this filter. Use 1 to enable multiselect or 0 to disable it.
ClassMethod %OnGetFilterList(Output pFilters As %List, pDataSourceName As %String = "") As %Status
{
set newfilter=$LB("New Filter","New Filter Display Name",,0)
set pFilters($I(pFilters))=newfilter
}
ClassMethod %OnGetFilterList(Output pFilters As %List, pDataSourceName As %String = "") As %Status
{
set tSC = ##class(%DeepSee.Dashboard.Utils).%GetFiltersForDataSource("patients.cube",.tFilters)
quit:$$$ISERR(tSC)
set i = ""
for {
set i = $order(tFilters(i), 1, data)
quit:i=""
set pFilters($i(pFilters)) = $lb($lg(data,2), $lg(data,1),,1)
}
quit $$$OK
}
Another way to build the list of filter items is to specify the
valueList attribute of the
<filter> element. Use a comma-separated list of logical names for the filter items. The order of this list determines the order in which filter controls list the filter items.
<filter name="ZipCode"
valueList="&[36711],&[34577],&[38928],&[32006],&[32007]"
displayList="36711,34577,38928,32006,32007"
/>
This attribute takes precedence over the
sql attribute.
If you specify this attribute, you can also specify the
displayList attribute, as shown above. If specified, this must be a comma-separated list of display names. If you do not specify this attribute, the logical names are also used as the display names.
Another way to build the list of filter items is to specify the
sql attribute of the
<filter> element. If specified, this must be an SQL query. The query can return either one or two columns. The first column in the returned dataset must provide the logical names for the filter items. The second column, if included, provides the corresponding display names. If you do not include a second column, the logical names are also used as the display names.
<filter name="ZipCode1" sql="SELECT DISTINCT PostalCode FROM DeepSee_Study.City"/>
If you specify the
sql attribute, do not specify the
displayList or
valueList attributes (see the previous subsection).
Another way to build the list of filter items is to implement
%OnGetFilterMembers() and your own logic to create the list. For example:
ClassMethod %OnGetFilterMembers(pFilter As %String, Output pMembers As %List, pSearchKey As %String = "") As %Status
{
Set status = $$$OK
Try {
If (pFilter = "AgeGroup") {
set pFilterSpec="[AgeD].[h1].[Age Group]"
} Elseif (pFilter="Gender") {
set pFilterSpec="[GenD].[h1].[Gender]"
} Elseif (pFilter="ZipCode") {
set pFilterSpec="[HomeD].[h1].[ZIP]"
}
do ##class(%DeepSee.KPI).%GetMembersForFilter("Patients",pFilterSpec,.pMembers,pSearchKey)
}
Catch(ex) {
Set status = ex.AsStatus()
}
Quit status
}
ClassMethod %OnGetFilterMembers(pFilter As %String, Output pMembers As %List, pSearchKey As %String = "") As %Status
{
Set status = $$$OK
Try {
If (pFilter = "AgeGroup") {
//get values from level table
Set sql = "SELECT DISTINCT DxAgeGroup FROM DeepSee_Model_PatientsCube.DxAgeGroup"
Set st = ##class(%SQL.Statement).%New()
Set status = st.%Prepare(sql)
If $$$ISERR(status) {Do $system.Status.DisplayError(status) Quit}
Set rs = st.%Execute()
While(rs.%Next(.status)) {
If $$$ISERR(status) {Do $system.Status.DisplayError(status) Quit}
Set display=rs.DxAgeGroup
Set actual="&["_display_"]"
Set pMembers($I(pMembers)) = $LB(display,actual)
}
If $$$ISERR(status) {Do $system.Status.DisplayError(status) Quit}
}
}
Catch(ex) {
Set status = ex.AsStatus()
}
Quit status
}
Note that this example does not use the
pSearchKey argument.
For an MDX-based KPI, to modify the query to use filters, implement the
%OnGetMDX() method. If you use the KPI wizard in Studio and you choose the source type as MDX, the generated KPI class includes the following stub definition for this method, which you can use as a starting point:
/// Return an MDX statement to execute.
Method %OnGetMDX(ByRef pMDX As %String) As %Status
{
Quit $$$OK
}
The variable
pMDX contains the MDX query specified in the
mdx attribute of
<kpi>. (If you did not specify that attribute,
pMDX is null.)
When you implement this method, do the following:
-
If there is no initial query, specify an initial value of the variable
pMDX. This should be an MDX SELECT query.
There is no initial query if you do not specify a query in the
mdx attribute, as described in the
previous chapter.
-
Examine each applicable filter and get its value.
-
Parse each filter value and convert it as needed to a form that can be used in an MDX %FILTER clause (if that is how you intend to use the filter). For a table that lists the suitable forms, see the
second subsection.
-
Modify the variable
pMDX to use the filter values, as appropriate for your needs. In the most common case, append a
%FILTER clause for each non-null filter, as follows:
Where
mdx_expression is the expression for that filter, as created in the previous step.
Or create an MDX
tuple expression that combines the expressions and then add a single %FILTER clause like this:
In more exotic cases, you could use the filter values to rewrite the query string in other ways.
-
-
It has one property for each filter in the KPI, with the same name (case-sensitive) as the filter.
-
The properties of
%filterValues contain values shown in the following table:
The following table lists appropriate MDX expressions to use in %FILTER clauses, depending on user selections.
Note that if multiselect is disabled, EXCLUDE is also disabled, and there are fewer possible forms of filter values. In this case, query construction is comparatively simple.
The following implementation of
%OnGetMDX() applies to a KPI in which multiselect is disabled:
Method %OnGetMDX(ByRef pMDX As %String) As %Status
{
if (..%filterValues."[aged].[h1].[age group]"'="")
{
set pMDX = pMDX _ " %FILTER [aged].[h1].[age group]." _..%filterValues."[aged].[h1].[age group]"
}
if (..%filterValues."[gend].[h1].[gender]"'="")
{
set pMDX = pMDX _ " %FILTER [gend].[h1].[gender]." _..%filterValues."[gend].[h1].[gender]"
}
if (..%filterValues."[homed].[h1].[zip]"'="")
{
set pMDX = pMDX _ " %FILTER [homed].[h1].[zip]." _..%filterValues."[homed].[h1].[zip]"
}
quit $$$OK
}
Because this method uses delimited property names, it can be confusing to read. Notes:
For an SQL-based KPI, to modify the query to use filters, implement the
%OnGetSQL() method. If you use the KPI wizard in Studio and you choose the source type as SQL, the generated KPI class includes the following stub definition for this method, which you can use as a starting point
/// Return an SQL statement to execute.
Method %OnGetSQL(ByRef pSQL As %String) As %Status
{
Quit $$$OK
}
The variable
pSQL contains the SQL query specified in the
sql attribute of
<kpi>. If you did not specify that attribute,
pSQL is null.
When you implement this method, do the following:
-
If there is no initial query, specify an initial value of the variable
pSQL. This should be an SQL SELECT query.
There is no initial query if you do not specify a query in the
sql attribute, as described in the
previous chapter.
-
Examine each applicable filter and get its value. The options are as follows:
-
Modify the variable
pSQL to use the filter values, as appropriate for your needs. In the most common case, modify the query to include an SQL WHERE clause.
In more exotic cases, you could use the filter values to rewrite the query string in other ways.
-
Note:
An SQL-based KPI cannot have more than 1000 rows; the system automatically limits the number of rows returned.
For an SQL-based KPI, you can use a method to access filter values in a format that is useful for including in your query:
method %GetSQLForFilter(sql_field_reference,filter_name) As %String
Examines the current filter selections and returns a string that you can use in the WHERE clause of an SQL query.
For example, consider the following method call:
..%GetSQLForFilter("City->Name","City")
The following table shows the values returned by this method call, in different scenarios.
The following example is from
DeepSee.Model.KPIs.DemoSQL. In this case, the filter adds GROUP BY and ORDER BY clauses to the SQL query.
Method %OnGetSQL(ByRef pSQL As %String) As %Status
{
//this is the start of the SQL query for this KPI
Set pSQL = "SELECT Count(*),AVG(Age) FROM DeepSee_Study.Patient "
Set where = ""
//look at %filterValues to see if a filter has been applied to this KPI instance
If $IsObject(..%filterValues) {
If (..%filterValues.ZipCode'="")
{
// Call utility method that returns filter data in convenient format
Set sqlstring=..%GetSQLForFilter("HomeCity->PostalCode","ZipCode")
Set where = "WHERE "_sqlstring
}
}
Set groupby="GROUP BY HomeCity "
Set orderby="ORDER BY HomeCity "
// assemble the SQL statement
Set pSQL=pSQL_where_groupby_orderby
Quit $$$OK
}
Method %OnGetSQL(ByRef pSQL As %String) As %Status
{
If $IsObject(..%filterValues) {
Set tWHERE = ""
If (..%filterValues.City'="") {
Set tWHERE = tWHERE _ $S(tWHERE="":"",1:" AND ") _ " Outlet->City = '" _ ..%filterValues.City _"'"
}
If (..%filterValues.Product'="") {
Set tWHERE = tWHERE _ $S(tWHERE="":"",1:" AND ") _ " Product = '" _ ..%filterValues.Product _"'"
}
If (tWHERE'="") {
// insert WHERE clase within query
Set tSQL1 = $P(pSQL,"GROUP BY",1)
Set tSQL2 = $P(pSQL,"GROUP BY",2)
Set pSQL = tSQL1 _ " WHERE " _ tWHERE
If (tSQL2 '= "") {
Set pSQL = pSQL _ " GROUP BY" _ tSQL2
}
}
}
Quit $$$OK
}
In this case, the KPI defines the initial query within the
sql attribute, as described in the
previous chapter. The
%OnGetSQL() method modifies that query.
This section discusses some other MDX KPI examples.
Class DeepSee.Model.KPIs.DemoMDXAutoFilters Extends DeepSee.Utils.MDXAutoFiltersKPI
{
Parameter CUBE = "PATIENTS";
Parameter DOMAIN = "PATIENTSAMPLE";
XData KPI [ XMLNamespace = "http://www.intersystems.com/deepsee/kpi" ]
{
<kpi name="DemoMDXAutoFilters" displayName="DemoMDXAutoFilters"
sourceType="mdx"
mdx="SELECT {[Measures].[%COUNT],[Measures].[Avg Age],[Measures].[Avg Allergy Count]} ON 0,
NON EMPTY [DiagD].[H1].[Diagnoses].Members ON 1 FROM [Patients]">
<property name="Patient Count" displayName="Patient Count" columnNo="1" />
<property name="Avg Age" displayName="Avg Age" columnNo="2" />
<property name="Avg Allergy Count" displayName="Avg Allergy Count" columnNo="3" />
</kpi>
}
}
Notice that this class does not directly define any filters, does not directly define the filter members, and defines only a hardcoded MDX query. When you display the test page for this KPI, however, you can use all the levels of the Patients cube as filters, and the KPI appends a suitable %WHERE clause to the query. For example:
-
-
-
The instance method
FilterBuilder() iterates through the cube-based filters, retrieves the current value of each, and then combines them into a string that is suitable for use as an MDX %FILTER clause.
-
%OnGetMDX() appends the %FILTER clause to the hardcoded query.
ClassMethod %OnGetFilterList(ByRef pFilters As %List, pDataSourceName As %String = "") As %Status
{
//call method in superclass so we can get filters of the associated cube
set tSC=##super(.pFilters,pDataSourceName)
quit:$$$ISERR(tSC) tSC
//update pFilters array to include the custom filter
set pFilters($i(pFilters)) = $lb("Yaxis","Yaxis",,0)
quit $$$OK
}
ClassMethod %OnGetFilterMembers(pFilter As %String, Output pMembers As %List, pSearchKey As %String = "",
pDataSourceName As %String = "") As %Status
{
set pMembers=""
if (pFilter="Yaxis") {
set pMembers($I(pMembers))=$LB("Home City","[homed].[h1].[city]")
set pMembers($I(pMembers))=$LB("Favorite Color","[colord].[h1].[favorite color]")
set pMembers($I(pMembers))=$LB("Profession","[profd].[h1].[profession]")
set pMembers($I(pMembers))=$LB("Diagnoses","[diagd].[h1].[diagnoses]")
} else {
//call method in superclass so we can get filter members for the associated cube
do ..%GetMembersForFilter(..#CUBE,pFilter,.pMembers)
}
quit $$$OK
}
Finally,
%OnGetMDX() constructs the MDX query. The
Yaxis filter determines which level is used for rows. Then the method appends the %FILTER clause to the query; the %FILTER clause uses any cube-based filters as in the previous example.
Method %OnGetMDX(ByRef pMDX As %String) As %Status
{
set yaxis=", NON EMPTY [profd].[h1].[profession].MEMBERS ON 1"
//check custom filter value
if (..%filterValues."Yaxis"'="") {
set yaxis=", NON EMPTY "_..%filterValues.Yaxis_".MEMBERS ON 1"
}
set pMDX="SELECT {MEASURES.[%COUNT],MEASURES.[avg age]} on 0"_yaxis_" FROM "_..#CUBE
/// append a %FILTER clause to handle any other filter values
Set pMDX = pMDX _ ..FilterBuilder()
Quit $$$OK
}
You can define a KPI so that it includes a listing option. In this case, if the KPI also includes filters, the listing definition must consider the filter selections.
To define the listing, you implement the
%OnGetListingSQL() method in your KPI class. This method has the following signature:
ClassMethod %OnGetListingSQL(ByRef pFilters As %String, ByRef pSelection As %String) As %String
This method returns the text of a listing query. The arguments are as follows:
-
pFilters is a multidimensional array that contains the current filter values. This array has the following nodes:
-
pSelection is a multidimensional array that contains the information about the current selection. This array has the following nodes:
The method should return an SQL SELECT query. In this query, you can also use Caché arrow syntax and SQL functions, as with other listings.
ClassMethod %OnGetListingSQL(ByRef pFilters As %String, ByRef pSelection As %String) As %String
{
Set tSQL = "SELECT TOP 1000 %ID,DateOfSale,Product FROM HoleFoods.SalesTransaction"
// apply sorting, if asked for
If (+$G(pSelection("sortColumn"))>0) {
Set tSQL = tSQL _ " ORDER BY " _ pSelection("sortColumn") _ " " _ $G(pSelection("sortDir"))
}
Quit tSQL
}