My previous post described the Jazz reporting solutions, including the two data sources used by the Jazz Reporting Service (JRS): the Data Warehouse (DW) and the Lifecycle Query Engine (LQE). So which one should you use? It depends on your reporting needs. This post provides guidance on why you’d use one or the other – or maybe both.
A reminder of the overall reporting architecture:
The DW is the more mature data store, and has been part of the solution for many years (DCC is slightly more recent, debuting in 2014). It has a well-defined and documented schema. The JRS “ready-to-use” and “ready-to-copy” reports rely on the DW, as do many of the BIRT reports available in Rational Team Concert (RTC) and Rational Quality Manager (RQM). If you plan to use any of those reports, you need the DW. The DW includes some data not available in LQE, such as build data, and a rich set of metrics and history for trend reports, particularly for work items.
However, if you use configuration management for project areas in RQM or DOORS Next Generation (DNG), you must use the LQE (scoped by a configuration) data source for those project areas; the DW does not support versioned artifacts.
To use Rational Engineering Lifecycle Manager (RELM), you also need LQE as the data source.
You can use LQE to report on project areas that aren’t enabled for configurations too, and there are benefits to doing so. The data in LQE is refreshed in nearly real time, while scheduled DCC jobs typically update the DW less frequently. LQE constructs its metamodel dynamically based on your data, which can make some reports easier to build than with the predetermined DW schema. Some reports are easier to build with the “schema-less” model. With the best practice of defining external URIs for artifact types and attributes, you can equate attributes across project areas to facilitate cross-project reporting. LQE also includes some data not available in the DW, especially for RQM.
LQE does have some disadvantages. The DW offers much richer history and metrics data. LQE has limited sample reports, and customizing queries requires SPARQL knowledge, which might be less familiar than SQL.
If you need LQE for configuration-enabled project areas, you might choose to continue using the DW for some reports. In particular, data for RTC work items continues to be available in the DW (since work items aren’t versioned). Even if all of your DNG and RQM project areas are configuration-enabled, you can use the DW to run out-of-the-box and trend reports for work items.
For non-enabled DNG/RQM projects, you might choose to build some reports using the DW, and others using LQE to take advantage of the dynamic schema and frequent updates.
As you decide on data sources, you do need to consider system resources as well. The DW uses a database for storage and the Data Collection Component (DCC) application to extract and load the data; the LQE application acts as both the data indexer and data store. Both require adequate resources for your data and usage scale. (See the Jazz.net Deployment wiki for sizing strategy and performance reports for DCC and LQE.) With respect to sizing and system resources:
- If you’re not using the DW for any reporting, you can disable DCC jobs from running. If most of your RM and QM project areas are enabled for configuration management, you’ll need less space for the DW database to grow, since those project areas do not contribute to the DW.
- If you enable LQE, it collects data for all project areas in the registered application data sources; currently you can’t filter the data in the TRS feeds to reduce the size of the data store. However, from a query performance perspective, if you continue to use the DW for some number of reports, you reduce the reporting load on the LQE server, which could contribute to performance. That said, if you don’t use configuration management in RM or QM project areas, you might not want to invest in the extra resources for LQE.
There is another reporting option that uses neither of these data sources: Rational Publishing Engine (RPE) extracts data directly from the applications using the reportable REST API to generate document-style reports and spreadsheets. In some cases, RPE can access data not readily available in either the DW or LQE, and can also handle more complex data manipulation and formatting. RPE is available as a separate offering; it is not included with JRS.
In closing, carefully consider your reporting needs as you decide whether to use the DW or LQE – or maybe even both.