Drill down and Drill across in WEBI

While BusinessObjects provides a fairly simple way to drill down using hierarchies there are times where this is not good enough. For example if we take the Motors example, lets say when the user drills down from the showroom, they want to see a report that displays the demographic information and models for that particular showroom. This is usually hard to do just by using hierarchies and this is where hyperlink drill down is very useful.

But before we go to Hyperlink drill down (or as I call it drill across), let us quickly cover hierarchical drill down. The screenshot below shows 2 ways to define hierarchies in the universe.

The natural hierarchies which is automatically defined by the universe is based on the order or objects within a class. For example in my universe I have Year->Qtr->Month->Date as my time hierarchy. Now if I enable drill in my report I can drill down to specific dates as shown below.

I can also define a custom hierarchy as shown in the figure 1 where I have combined the showroom and the model into a single hierarchy such that when the user drill from the showroom they can see the models in the showroom.

Now coming to drill across. For example when I click on the revenue below I want to jump to another report that shows the sales by model in that country for the selected year as shown below.

Setting that up is fairly simple. I have to create 2 reports as shown below.

The 2nd report accepts the showroom country and year as prompts. Now I have to go to the summary report i.e. Showroom by year and setup the drill across. You can setup a hyperlink drill down by right click on the cell and selecting “New Hyperlink” as shown below.

When setting up the hyperlink you would need to associate the objects from the summary report to be sent as parameters to the prompts in the detailed report as shown above.

BOE Universe: Generating list of values based on data

Let us take a simple scenario. Let us say we have a countries dimension which has all the countries. If we want to use this dimension table to get a list of countries where we have customers, as well as use it to get a list of countries where we have offices we can create 2 aliases COUNTRY_OFFICE and COUNTRY_CUSTOMER and create objects from the aliases. No issues till now, except if we try to get a list of countries where we have offices. Since we have used the master country table it will list all the countries irrespective of whether we have office there or not.

Let us see how it works below.

In the example above I have a country table and I use it for both client and showroom country. Now if I query for showroom country i.e. countries where I have showrooms here is the query produced.

As you can see it lists all the countries in the countries table which is not what we wanted. In order to get the correct list of countries I have to join with the SHOWROOM table so that the countries list is restricted based on the SHOWROOM dimension table. This is done by specifying that whenever the Showroom Country object is used in a query, the Showroom table must also be inferred in the FROM clause of the SELECT statement. Providing that the Showroom_Country table is joined to the Showroom table the object is then guaranteed to only return countries in which showrooms exist.

Making this change ensures that we always get the correct set of countries when we query for showroom countries.

While this does not seem that critical it becomes very important especially when we want the user to select the showroom country in a prompt for example. We only want those countries to be in the prompt list where we have showrooms and making this change will ensure that we always get the correct list.

Outer Joins in Universe (BusinessObjects)

Lets take a simple scenario. I want to generate a report that shows the sales by showroom, and the report should display all the showrooms in the report. For the showroom with no sales it should display the region with a NULL value for sales.

The usual join (also called as inner join) will not work in this scenario. We need what is called as an outer join. If you want to know more about outer joins you can checkout the Wikipedia link here.

To enable outer join you need to first set the ANSI92 parameter to Yes. This will change the query from the simple join to an an inner join syntax with from clause as shown below.

You can also enable the FILTER_IN_FROM parameter. This pushes the where clause of the join inside the from which reduces the number of records in the join condition and will greatly improve performance.

Now we can setup our outer join. As shown below we can setup the right outer join betwen the fact table and the showroom dimension table.

The effect of this is that all showrooms irrespective of whether they had a sale or not will be returned by the query.

So in conclusion while it is easy to setup outer joins in the universe, one should be very careful when using outer joins as it can result in a cartesian product of all rows especially when using full outer join.

Is it my time yet ?

Over the last couple of weeks I have came across several posts in the BOB Board that revolve around time based analysis. Since the questions seem keep repeating it makes it an ideal candidate for a blog posting. Most of the analysis that I run into involve either analysis the most current data i.e. current day, current week, or current month. In fact most of the standard reports are probably built with that as the default selection parameter. And also in most of the cases this data is being compared with some other period like either last quarter or last year.

In my previous posts I have covered 2 very key topics

• Period to date analysis – Examples would be YTD or MTD type of analysis which I have covered here
• Prior Period Analysis – Covered here.

In this post I will cover how to make date selections easier for users, especially in scenarios where they want to analyse the most recent period. You might ask why all of this work when I can select dates using the filter criteria in WEBI. The answer is usability. As you can see below it is much simpler to select “Current Year” or “Last Week” from the prompt selection rather than having to go through a set of dates.

So how can we design something like this. It is quite simple actually. First off I define a derived table with the set of pre-defined date ranges that I want to make available for the users.

The code for the derived table is actually quite simple. For example in my case I have used the MAX function to determine the current date based on the dates in the dimension table.

Select 1 AS ITEM_INDEX, ‘All Days’ as DATE_RANGE, min(DATE) as DATE_RANGE_MIN, max(DATE) as DATE_RANGE_MAX from DATES_TABLE
union
Select 2 AS ITEM_INDEX, ‘Today’ as DATE_RANGE, max(DATE) as DATE_RANGE_MIN, max(DATE) as DATE_RANGE_MAX from DATES_TABLE
union
Select 3 AS ITEM_INDEX, ‘Last Week’ as DATE_RANGE, dateadd(dd,-7, max(DATE)) as DATE_RANGE_MIN, max(DATE) as DATE_RANGE_MAX from DATES_TABLE
union
Select 4 AS ITEM_INDEX,’Current Month’ as DATE_RANGE, cast(CAST(datepart(yyyy,max(DATE)) as varchar(10)) + ‘-‘ + CAST(datepart(mm,max(DATE)) as varchar(10)) + ‘-01’ as DATETIME) as DATE_RANGE_MIN, max(DATE) as DATE_RANGE_MAX from DATES_TABLE
union
Select 5 AS ITEM_INDEX,’Current Year’ as DATE_RANGE, cast(CAST(datepart(yyyy,max(DATE)) as varchar(10)) + ‘-01-01’ as DATETIME) as DATE_RANGE_MIN, max(DATES_TABLE.DATE) as DATE_RANGE_MAX from DATES_TABLE
union
Select 6 AS ITEM_INDEX,’Current Qtr’ as DATE_RANGE, ‘DATE_RANGE_MIN’ =
    case
        when datepart(qq,max(DATES_TABLE.DATE)) = 1 then cast(cast(datepart(yyyy,max(DATES_TABLE.DATE)) as varchar(10))+ ‘-01-01’ as datetime)
        when datepart(qq,max(DATES_TABLE.DATE)) = 2 then cast(cast(datepart(yyyy,max(DATES_TABLE.DATE)) as varchar(10))+ ‘-04-01’ as datetime)
        when datepart(qq,max(DATES_TABLE.DATE)) = 3 then cast(cast(datepart(yyyy,max(DATES_TABLE.DATE)) as varchar(10))+ ‘-07-01’ as datetime)
        When datepart(qq,max(DATES_TABLE.DATE)) = 4 then cast(cast(datepart(yyyy,max(DATES_TABLE.DATE)) as varchar(10))+ ‘-10-01’ as datetime)
        else cast(‘1900-01-01’ as datetime)
        end,
max(DATES_TABLE.DATE) as DATE_RANGE_MAX from DATES_TABLE

My code assumes that the dates dimension table is updated and contains only the valid dates. If that’s not the case then you would need to use either a system function like GetDate() to get the current date or use some form of control table for the current date information. This has been explained quite well by Dave in his blog.

This derived table has been joined to the fact table using a between clause as shown below.

I also need to define the contexts to resolve the loops created by the joins.

Now we are ready to add the "DATE_RANGE” column to the universe. In my specific example I have defined the object as a hidden object in the universe and defined a filter called DATE_RANGE with a @prompt as shown below. This is to make it easy to use. I do not want to clutter up the time hierarchy with unnecessary objects. However I want to give the flexibility to the users to easily pick a date range for their analysis.

DATE_RANGE.DATE_RANGE = case when @Prompt(‘Select Date Range for Analysis:’,’A’,’Date Range\Date Range’,mono,free) = ‘*’ then ‘All Days’ else @Prompt(‘Select Date Range for Analysis:’,’A’,’Date Range\Date Range’,mono,free) end

The prompt condition allows the user to either pick ‘*’ meaning all dates, or pick some other date range for analysis. Using the approach above ensures that during adhoc analysis the user has to drag the date range to the query filter and they will be prompted with a set of pre-defined filter conditions to restrict the data.

I have also gone ahead and defined another condition object called “Custom Date Range” that shows the calendar to the user and allows the user to pick any date range from a standard calendar. The custom date range prompts the user for a start and end date and filters the data based on the user selection.

DATES_TABLE.DATE >= @Prompt(‘Select Start date:’,’D’,’Period\Date’,Mono,free,not_persistent,{‘2001/01/01’}) AND DATES_TABLE.DATE <= @Prompt(‘Select End date:’,’D’,’Period\Date’,Mono,free,not_persistent)

So in summary using some of the techniques given here as well in the other posts around time slicing, you can implement quite sophisticated and flexible time based analysis. To access the other articles in the series click on the links below.

• Period to date analysis – Examples would be YTD or MTD type of analysis which I have covered here
• Prior Period Analysis – Covered here.

Managing performance by using Aggregate tables in the universe

In most large data warehouses one of the common strategies employed by DBA’s to speed up performance is to use aggregate tables. Generally aggregate tables contain information that has a coarser granularity than the detail data. For example in a retail datamart I might have information at the transaction level. However most of the analysis will be performed at the daily level by brand. Without aggregate tables the database will fetch the lowest level of data and will perform a group by at the day level for specific brands which can be a very expensive operation. Instead as part of the ETL process I can pre-aggregate the data at the daily level which would reduce the number of rows by a huge factor. The Sales_Receipts fact table would contain this detail data, but the records in that table might also be aggregated over various time periods to produce a set of aggregate tables (Sales_Daily, Sales_Monthly, and so on).

There are multiple ways of managing aggregates. One option is to create aggregate tables in the database as materialized views and let the query optimizer of the database handle the performance using seamless query rewrite. I will describe this in a separate post. In this post I will focus on using the aggregate awareness functionality of the universe.

My sample database tracks the sales of cars. My detailed fact table VW_SALE_MODEL is used to track the sales at the lowest level of detail i.e. client, showroom, model and color.

Using these I can create a family of aggregate fact tables. For example I have created a aggregate table called VW_SALE that aggregates the data at the client and showroom level. Similarly I can create additional aggregates at the year level.

Once I have create the aggregates I need to map them into the universe which is a 4 step process.

1. Add the aggregate tables and setup the joins with the dimension tables. I have not created standalone aggregate tables since I want to make sure that I can leverage the hierarchies defined in the dimension tables. This process is similar to adding any fact table in the universe.

2. Define the aggregated measures using the @aggregate_aware function. The @aggregate_aware function is used to setup aggregate awareness in the universe.

The syntax of the @Aggregate_Aware function is @Aggregate_Aware(sum(agg_table_1), sum(agg_table 2) …., sum(agg_table_n)) in the order of preference. For example agg_table1 should be the highest level aggregate, followed by agg_table 2 and so on. This is used by the universe to pick the best aggregate table to answer the query.

In my example I have stated that either try to get the sales total from the aggregate table or get it by calculating it using the detailed table.

3. Define the incompatibilities. For example in my structure the model and maker classes are not captured by the aggregate. Also the aggregate table only contains information about sales and not about the quantity sold. We need to define these incompatibilities so that when the user generates a query, the universe can quickly scan through the compatibility list to determine the best aggregate that can be used to answer the query.

When I define it as shown above all queries that include Model or Maker will go to the detailed table. All other queries will be satisfied by the aggregate table.

4. Resolve any loops. Since I have joined the dimension tables to both the fact I have created some loops in the universe which I need to resolve. I can do that by creating separate contexts for the aggregate fact and the detailed facts.

Now I am ready to using my aggregates. To illustrate let us go to WEBI and see the impact of our design. When I query for sales by showroom the entire query is answered by the aggregate table.

As soon as I add the maker to the query BO now retrieves the data from the detailed table instead.

The same approach can be used to capture additional aggregates such as Year level or Qtr level and BO will dynamically go from using the summary table to using the detailed table as the user is performing the drill down.

So in summary, aggregate tables are very powerful and necessary in most real implementations and BO provides a fairly simple way to model it within the universe.

WEBI – How to display “Others” in country field based on Rank

One of the frequent questions I come across is – I want to view the top 5, however I want everything else that is not in the top 5 to be placed in 1 bucket called others. I ran into the same question on the BOB board today and since it was simple enough here comes the solution.

Getting the rank within WEBI is quite simple using the RANK function. The syntax for the function is RANK(measure name; dimension name; top|bottom). For example of I wanted to find out the rank by sales I could define RANK([Sales Revenue], [City], top).

In order to make my life easier I have defined variable called Rank as shown below.

I can use that in a report filter to get only the top 3 regions by Sales Revenue.

Now to display all other regions I will use the WEBI function NOFILTER. This will return the total sales across all the regions irrespective of the filter. Using the formula below I can get the sales for all the regions that are not in top 3.

By placing this in the footer of the table I can get the report to show the sales for others.

How to perform YTD (or any Period to date) design in the Universe

Yet another post inspired by the BOBJ board. The idea is how to design a universe such that users could enter any date and get both the measure value for that period as well as YTD. Since we want to make it easy for adhoc users we need to do some design work in the universe to make it easy for the users performing adhoc analysis.

So I figured the easiest approach would be to define a separate set of measures for YTD similar to what I did for the YAGO computation in a previous post. So extending on the same example I followed a very similar approach and it turns out to be quite simple. All we need to do is make sure we are able to run multiple queries, once for getting the sales and another one that sums up the sales from the beginning of year to the selected date. So I know we have to define a separate context for the YTD sales, forcing the BI Server to automatically run 2 queries and join the results. That’s what I want to leverage.

1. To make my life easier in the universe I defined a separate reference table DATES_PERIOD that maps the date to its corresponding YTD start and end dates. This not only makes it simple, it also makes it possible for me use the same design for handling non standard calendars such as Fiscal calendar. Also if I want to do QTD or MTD instead of YTD I can use the same approach by just changing the start and end dates.

In this table I have gone ahead and filled up the start and end dates for YTD for every date in the DATES_TABLE my calendar table.

2.  In my universe I first go ahead and define an alias for the fact table called YTD_SALES. Now instead of joining it to my DATES_PERIOD (date dimension) table I have joined it to my DATES_PERIOD table using a complex join as shown below.

This ensures that I will always select all the sales from the time slice (start and end of YTD) rather than selecting a specific date. My universe is as shown below.

In my universe my time dimension objects such as Date or Qtr are driven by the DATES_TABLE. So in order to tie everything up I have joined the DATES_PERIOD to the DATES_TABLE on the date. This ensures that when the user selects the date, the corresponding period will be selected from the DATES_PERIOD and the BI Server will return the sales that fall in that period. This is the key part of the design.

Now I can setup up the rest of the joins with the rest of the dimension tables.

3. Now I have to define a new context for YTD as shown below. This is needed to make sure that when the user selects from Sales and YTD sales they are sent as separate queries.

After setting up the contexts I can define the YTD Sales revenue by pulling in the appropriate field from the YTD_SALES alias table.

Now checking to make sure that the logic is OK. I define 2 queries, YTD Sales till Dec 31-2004 and the sales for 2004. If the logic is correct both should come out same and it does.

Also I can pull them in the same query if I want. I know that there is a sale on 15-Mar-2004. Filtering on that date gives me both the sales value for that date as well as YTD sales.

The reason I love this design is because very versatile and it can be used for any period to date. The only thing to note is that it will work only if the user selects a date. If the user selects a Qtr or Month then the YTD value will be garbage. If you want to prevent this then you can force the user to select a date using a prompt.

Auditing III: How to enable auditing for Crystal Reports and WEBI viewing

A couple of weeks back I had written an article on enabling auditing in BOE. The same functionality is available in Crystal Reports server as well. However I had missed out what activities can be audited. So writing this post to complete my previous posting. Below are the most of the common scenarios that the administrators want to audit.

Crystal Reports Cache Server: Viewing of Crystal reports is audited by Crystal Cache Server.

Crystal Reports Jobs: Can be turned on by enabling auditing for Crystal reports job server.

Destination Job Server: Will audit all jobs that output to emails, ftp, and file system.

Event Server: Audits all events that are registered on the BOE or Crystal Reports server.

Publication Job Server: Will audit all publication jobs.

WEB Intelligence: Audits access to all WEBI reports.

For steps on how to turn on auditing and how to look at audit data you can go to my previous posts on the same topic.

Auditing I: How to enable audits logging in BOE XI 3.1

Auditing II: How to import the auditing reports in BOE

How to do Year Ago or Prior period type comparison in WEBI

Again a post inspired by the BOBJ board which talks about a generic requirement whereby making it easy for adhoc users to perform relative time period based analysis as easy as possible. The users should be able to select the date using a prompt or otherwise and ask for values for either the selected period or some other reference period such as Current Year Last Week or Prior Year by simply selecting a different measures named as such from the Universe.

This is actually quite simple once you get around to designing it. All we need to do is make sure we run multiple queries once for each time period and then join the queries at runtime in the BI Server. WEBI provides such functionality in the form of contexts. For example if we define the current period and YAGO (year ago) in 2 separate contexts, then when we pull Sales and YAGO sales in the same report BI Server will automatically run 2 queries and join the results. That’s what I want to leverage.

My example shows how to setup 2 measures, Sales and YAGO Sales. However this can be done for any number of measures as well as any number of reference periods as the process is quite simple. So here is what I have done.

1. For each relative time period we need to define an alias of the fact table in the universe. For example I need current period and YAGO therefore I have 2 fact tables – Sales and YAGO Sales.

2. It is also very important to have a proper dates/calendar table that you can use for your time based calculations. In my schema it is the DATES_TABLE with a structure as shown below.

3. Now join the 2 facts to the dates table. The main fact table for current period will be joined normally to the dates table, however the YAGO sales will be joined to the dates table with a lag.

 

Ofcourse you would also need to define all the other joins between the YAGO_SALES fact and the other dimension tables for completeness. Now that the 2 join conditions have been defined as you can define the measures from each of these tables. When user say runs the query for October 2010 then Sales revenue will be the value for Oct-2010, however YAGO will be Oct-2009 since YAGO sales will join with the date table with a 1 year lag due to our join condition.

Now in order to force the 2 queries we need to define the contexts – 1 for each fact table. This will resolve the loops as well as force the BI Server to issue 2 separate queries to the database.

Now in the query as will see the BI Server will issue 2 queries one for each context and combine them in the result set.

The beauty of this approach is that it is automatically level based. If the user selects a date in the prompt the results will be for same date last year, for month it will be same month LY, for Qtr will be same Qtr last year and for year will be last full year.

You can use the same mechanism to define last week or last month.

Duplicate values in Dashboard Prompts (OBIEE)

I have been working with OBIEE for several years now, and the one thing about this product that continues to amaze me is the BI Server’s query engine.

The situation:
I need to show a unique list of values for this prompt, and the BI Server will just not listen! Even when I explicitly use “Distinct”, the BI Server would simply ignore my command.

The solution:
There isn’t enough emphasis on how to build the Business Model, and this issue surfaces from that fact. Often times, when creating logical dimension tables, the logical primary key is not defined properly. The logical primary key field(s) is/are assumed, by the BI Server, to uniquely identify a dataset.

You will typically find duplicate values when you use a field that has been defined as the single primary key field in a logical dimension table although in reality, this field is not unique across rows in the underlying table.

The solution, therefore, is to define the primary key properly. If that is not an option, I will suggest that you add another logical column that maps to the same physical column but is not part of the logical primary key, and use this column instead in the prompt. Magically, this will use a “Distinct” automatically and bring a smile back to your face 🙂