Reminder: Create & find trunk-related issues here. Legacy Trac tickets are still available. Problems? Workflow feedback? Need a module project? Open a support ticket.

BIRT Report Module

Report generation not handling date columns correctly (possible parsing error)

Details

  • Type: Bug Bug
  • Status: Needs Assessment Needs Assessment
  • Priority: Blocker Blocker
  • Resolution: Unresolved
  • Affects Version/s: None
  • Fix Version/s: 2.2.2
  • Component/s: None
  • Severity:
    3
  • Complexity:
    Undetermined
  • Keywords:
  • Description:
    Hide

    Email from Jeremy Keeton (jkeeton@pih.org) @ 7/16/2008 3:02 AM EDT

    ===== BEGIN email snippet

    I think I’ve figured out what’s going on here. I did the following:

    1. I generated the first PDF called “bobete_emr.pdf” directly from the emr
      2. I downloaded the report design and the dataset directly from the server and opened them up in the BIRT RCP designer
      3. I setup the dataset without editing the datatypes under “Select the file and columns for the dataset” and generated “bobete_wo_datatypes.pdf”
    2. Lastly I setup the dataset and edited the datatypes (only the ones that were dates) and generated “bobete_w_datatypes.pdf” –THIS generated the correct report with the correct numbers.

    So it looks like somehow BIRT in OpenMRS doesn’t know that the following report params are dates:

    Any of the report params with the word date, obs_datetime and the column Earliest ARV start

    So when it does a comparison against these dates (i.e. “DateTimeSpan.years(dataSetRow["Birthdate"],params[End Date"|"Report] )>14”) it sees “Birthdate” as a string and counts ‘0’ patients.

    One other note, there was a warning in the catalina.log file for every patient that says “There exists an error in toDate method” Unparsable date: “2007-08-16”. I’ve attached a sample, I couldn’t attach the whole log file because for some reason it was 760MB after running this, so I'm guessing this means there was a very long error for every patient.

    ===== END email snippet

    Show
    Email from Jeremy Keeton (jkeeton@pih.org) @ 7/16/2008 3:02 AM EDT ===== BEGIN email snippet I think I’ve figured out what’s going on here. I did the following:
    1. I generated the first PDF called “bobete_emr.pdf” directly from the emr 2. I downloaded the report design and the dataset directly from the server and opened them up in the BIRT RCP designer 3. I setup the dataset without editing the datatypes under “Select the file and columns for the dataset” and generated “bobete_wo_datatypes.pdf”
    2. Lastly I setup the dataset and edited the datatypes (only the ones that were dates) and generated “bobete_w_datatypes.pdf” –THIS generated the correct report with the correct numbers.
    So it looks like somehow BIRT in OpenMRS doesn’t know that the following report params are dates: Any of the report params with the word date, obs_datetime and the column Earliest ARV start So when it does a comparison against these dates (i.e. “DateTimeSpan.years(dataSetRow["Birthdate"],params[End Date"|"Report] )>14”) it sees “Birthdate” as a string and counts ‘0’ patients. One other note, there was a warning in the catalina.log file for every patient that says “There exists an error in toDate method” Unparsable date: “2007-08-16”. I’ve attached a sample, I couldn’t attach the whole log file because for some reason it was 760MB after running this, so I'm guessing this means there was a very long error for every patient. ===== END email snippet

Activity

Hide
Justin Miranda added a comment - 2008-07-17 17:56:03 EDT

During report generation, the module will write the new CSV dataset to the filesystem and then change the BIRT report design (XML) to references the new dataset. When overriding the flatfile dataset query to include the new filename, the module drops off an important part of the query which keeps track of the column data types.

The syntax for the flatfile dataset used to be a simple SQL-like query. The column data types must have been stored somewhere else in the report design.

SELECT columns FROM table

In some recent version of BIRT, the designer started tacking on the 'column_info' expression which includes the data type for every column.

SELECT columns FROM table : { column_info }

The BirtDataSetQuery class was only compatible with the former syntax, so it kept dropping off this critical section of the query during the report generation. This caused BIRT to essentially 'forget' the data type for each column, which means Integer, Date, and Boolean columns all became String columns.

Therefore, when performing Date operations on a String column (like the following):

DateTimeSpan.years(dataSetRow["Birthdate"],paramsReport[Date"|"End])>14

the BIRT report engine would complain

There exists an error in toDate method” Unparsable date: “2007-08-16”

The fix was to rewrite the query parsing utilities and make sure we maintain the extra data type column information.

Show
Justin Miranda added a comment - 2008-07-17 17:56:03 EDT During report generation, the module will write the new CSV dataset to the filesystem and then change the BIRT report design (XML) to references the new dataset. When overriding the flatfile dataset query to include the new filename, the module drops off an important part of the query which keeps track of the column data types. The syntax for the flatfile dataset used to be a simple SQL-like query. The column data types must have been stored somewhere else in the report design. SELECT columns FROM table In some recent version of BIRT, the designer started tacking on the 'column_info' expression which includes the data type for every column. SELECT columns FROM table : { column_info } The BirtDataSetQuery class was only compatible with the former syntax, so it kept dropping off this critical section of the query during the report generation. This caused BIRT to essentially 'forget' the data type for each column, which means Integer, Date, and Boolean columns all became String columns. Therefore, when performing Date operations on a String column (like the following): DateTimeSpan.years(dataSetRow["Birthdate"],paramsReport[Date"|"End])>14 the BIRT report engine would complain There exists an error in toDate method” Unparsable date: “2007-08-16” The fix was to rewrite the query parsing utilities and make sure we maintain the extra data type column information.

People

Dates

  • Created:
    2008-07-17 17:41:27 EDT
    Updated:
    2010-08-04 09:19:34 EDT