Re: [Dhis2-devs] periodstructure giving errors...

[Moemedi Ntunyane]

I cleared cache and even restarted my tomcat server

Moemedi Ntunyane
Co-founder KeyInConsulting


Date: Tue, 16 Sep 2014 06:09:43 +0200
Subject: Re: [Dhis2-devs] periodstructure giving errors...
From: jason.p.picker...@gmail.com
To: moemedi.ntuny...@hotmail.com
CC: larshe...@gmail.com; dhis2-devs@lists.launchpad.net

Did you clear your server cache and or restart DHIS2?
On Tue, Sep 16, 2014 at 5:50 AM, Moemedi Ntunyane 
 wrote:



I have eliminated all duplicates but this error still persist.
Process failed: PreparedStatementCallback; SQL [insert into _periodstructure 
values (?,?,?,?,?,?,?,?,?,?,?,?,?)]; ERROR: null value in column "iso" violates 
not-null constraint; nested exception is org.postgresql.util.PSQLException: 
ERROR: null value in column "iso" violates not-null constraint.

Moemedi Ntunyane
Co-founder KeyInConsulting


From: moemedi.ntuny...@hotmail.com
To: larshe...@gmail.com; jason.p.picker...@gmail.com
CC: dhis2-devs@lists.launchpad.net
Subject: RE: [Dhis2-devs] periodstructure giving errors...
Date: Mon, 15 Sep 2014 16:18:14 +0200




Hi Lars,

I have not done that. Im only capturing statistical data.

Regards,

Moemedi Ntunyane
Co-founder KeyInConsulting


Date: Mon, 15 Sep 2014 16:10:13 +0200
Subject: Re: [Dhis2-devs] periodstructure giving errors...
From: larshe...@gmail.com
To: jason.p.picker...@gmail.com
CC: moemedi.ntuny...@hotmail.com; dhis2-devs@lists.launchpad.net

Hi Moemedi,
may I ask, did you ever run the "aggregation queries" (producing aggregate 
values from event data) on your database?
regards,
Lars

On Mon, Sep 15, 2014 at 4:01 PM, Jason Pickering  
wrote:
I suspect you have more invalid periods then. You should follow the same 
procedure for other period types, i.e. checking to be sure that all months are 
one month, all weeks are one week, etc. 
You may want to try something like
 SELECT startdate, periodtypeid,COUNT(*) from period GROUP BY 
startdate,periodtypeid HAVING COUNT(*) > 1 ;

and see which periods and period types you have multiple records for. Ideally, 
this should not happen. 
Regards,Jason

On Mon, Sep 15, 2014 at 3:53 PM, Moemedi Ntunyane 
 wrote:



Hi Jason
Thnx, I had successfully elimated duplicate records the error still 
pesistit complaining of ISO column: is this column a date column??

Process
 failed: PreparedStatementCallback; SQL [insert into _periodstructure 
values (?,?,?,?,?,?,?,?,?,?,?,?,?)]; ERROR: null value in column "iso" 
violates not-null constraint; nested exception is 
org.postgresql.util.PSQLException: ERROR: null value in column "iso" 
violates not-null constraint.

Regards,

Moemedi Ntunyane
Co-founder KeyInConsulting


Date: Mon, 15 Sep 2014 12:22:31 +0200
Subject: Re: [Dhis2-devs] periodstructure giving errors...
From: jason.p.picker...@gmail.com
To: moemedi.ntuny...@hotmail.com
CC: dhis2-devs@lists.launchpad.net

So, the period 1589;6;"2010-02-01";"2010-02-28 , points to a yearly period 
type, which is not a year in duration. You are going to need to get rid of it. 
You will need to do something like
SELECT COUNT(*) FROM datavalue where periodid = 1589; 
If you have any records with this periodID, you are going to need to resolve 
these. This period looks to be monthly, so if the data is also monthly, you 
should be able to reassign it to a monthly period which starts in 2010-02-01 
and ends in 2010-02-28, if it is already there. You could try "SELECT * FROM 
period where startdate = '2010-02-01'::date;" and see if you get multiple 
periods for Feb 2010. If you have multiple periods for Feb 2010, then you need 
to do something like
UPDATE datavalue set periodid = ? 
where  is the periodid of the "real" Feb 2010 period. 
Then you should get rid of the bogus Feb 2010 period 
with 
DELETE FROM period where periodid = 1589;
After that, you should clear your cache from Data administration. 
Do not try any of this on a production database!!!
Regards,Jason

On Mon, Sep 15, 2014 at 12:14 PM, Moemedi Ntunyane 
 wrote:



Hi Jason

Thanx, I just run the query you gave and no records were available and removed 
not = year(!=year) and records where available. The date format is available as:

1573;6;"2010-01-01";"2010-01-31"
1589;6;"2010-02-01";"2010-02-28"

The error available shows that there is key violation for null values for iso 
column in the periodstructure table.



Moemedi Ntunyane
Co-founder KeyInConsulting


Date: Mon, 15 Sep 2014 11:46:07 +0200
Subject: Re: [Dhis2-devs] periodstructure giving errors...
From: jason.p.picker...@gmail.com
To: moemedi.ntuny...@hotmail.com
CC: dhis2-devs@lists.launchpad.net

Very likely you have an invalid period in the periods table. 
Can you run this and see what happens?
SELECT * FROM period where age(enddate,startdate) != '1 year'::interval and 
periodtypeid = (SELECT periodtypeid from periodtype where name = 'Yearly');

Also, you should get an error which will help you to decipher which period is 
not correct , something like 
Caused by: org.postgresql.util.PSQLException: ERROR: 

Re: [Dhis2-devs] periodstructure giving errors...

[Jason Pickering]

Yeah, well, I can't offer much more advice other in this case. We had
similar problems as I mentioned before with 2.16, but after carefully
cleaning up all of the faulty periods, things worked OK.

One thing which I would suggest would be to turn on full logging of all
postgresql queries, and see exactly where things are failing.

You should also see something like

Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value
violates unique constraint "in_periodstructure_iso"
  Detail: Key (iso)=(1995) already exists.

in the Tomcat log, which again, gives you a clue about which period may be
causing the problem. I know you posted a portion of the log before, but I
did not see anything like this in your log. Turning on the postgresql logs
may help you to track down which period is causing this error.

Maybe one of the developers can add more information? The procedure worked
for us, but required a bit of detective work.

Regards,
Jason




On Tue, Sep 16, 2014 at 9:12 AM, Moemedi Ntunyane <
moemedi.ntuny...@hotmail.com> wrote:

> I cleared cache and even restarted my tomcat server
>
>
> *Moemedi NtunyaneCo-founder KeyInConsulting*
>
>
> --
> Date: Tue, 16 Sep 2014 06:09:43 +0200
> Subject: Re: [Dhis2-devs] periodstructure giving errors...
> From: jason.p.picker...@gmail.com
> To: moemedi.ntuny...@hotmail.com
> CC: larshe...@gmail.com; dhis2-devs@lists.launchpad.net
>
>
> Did you clear your server cache and or restart DHIS2?
>
> On Tue, Sep 16, 2014 at 5:50 AM, Moemedi Ntunyane <
> moemedi.ntuny...@hotmail.com> wrote:
>
> I have eliminated all duplicates but this error still persist.
>
> Process failed: PreparedStatementCallback; SQL [insert into
> _periodstructure values (?,?,?,?,?,?,?,?,?,?,?,?,?)]; ERROR: null value in
> column "iso" violates not-null constraint; nested exception is
> org.postgresql.util.PSQLException: ERROR: null value in column "iso"
> violates not-null constraint.
>
>
> *Moemedi NtunyaneCo-founder KeyInConsulting*
>
>
> --
> From: moemedi.ntuny...@hotmail.com
> To: larshe...@gmail.com; jason.p.picker...@gmail.com
> CC: dhis2-devs@lists.launchpad.net
> Subject: RE: [Dhis2-devs] periodstructure giving errors...
> Date: Mon, 15 Sep 2014 16:18:14 +0200
>
>
> Hi Lars,
>
> I have not done that. Im only capturing statistical data.
>
> Regards,
>
>
> *Moemedi NtunyaneCo-founder KeyInConsulting*
>
>
> --
> Date: Mon, 15 Sep 2014 16:10:13 +0200
> Subject: Re: [Dhis2-devs] periodstructure giving errors...
> From: larshe...@gmail.com
> To: jason.p.picker...@gmail.com
> CC: moemedi.ntuny...@hotmail.com; dhis2-devs@lists.launchpad.net
>
> Hi Moemedi,
>
> may I ask, did you ever run the "aggregation queries" (producing aggregate
> values from event data) on your database?
>
> regards,
>
> Lars
>
>
> On Mon, Sep 15, 2014 at 4:01 PM, Jason Pickering <
> jason.p.picker...@gmail.com> wrote:
>
> I suspect you have more invalid periods then. You should follow the same
> procedure for other period types, i.e. checking to be sure that all months
> are one month, all weeks are one week, etc.
>
> You may want to try something like
>
>  SELECT startdate, periodtypeid,COUNT(*) from period GROUP BY
> startdate,periodtypeid HAVING COUNT(*) > 1 ;
>
> and see which periods and period types you have multiple records for.
> Ideally, this should not happen.
>
> Regards,
> Jason
>
>
> On Mon, Sep 15, 2014 at 3:53 PM, Moemedi Ntunyane <
> moemedi.ntuny...@hotmail.com> wrote:
>
> Hi Jason
> Thnx, I had successfully elimated duplicate records the error still
> pesistit complaining of ISO column: is this column a date column??
>
> Process failed: PreparedStatementCallback; SQL [insert into
> _periodstructure values (?,?,?,?,?,?,?,?,?,?,?,?,?)]; ERROR: null value in
> column "iso" violates not-null constraint; nested exception is
> org.postgresql.util.PSQLException: ERROR: null value in column "iso"
> violates not-null constraint.
>
> Regards,
>
>
> *Moemedi NtunyaneCo-founder KeyInConsulting*
>
>
> --
> Date: Mon, 15 Sep 2014 12:22:31 +0200
>
> Subject: Re: [Dhis2-devs] periodstructure giving errors...
> From: jason.p.picker...@gmail.com
> To: moemedi.ntuny...@hotmail.com
> CC: dhis2-devs@lists.launchpad.net
>
> So, the period 1589;6;"2010-02-01";"2010-02-28 , points to a yearly
> period type, which is not a year in duration. You are going to need to get
> rid of it. You will need to do something like
>
> SELECT COUNT(*) FROM datavalue where periodid = 1589;
>
> If you have any records with this periodID, you are going to need to
> resolve these. This period looks to be monthly, so if the data is also
> monthly, you should be able to reassign it to a monthly period which starts
> in 2010-02-01 and ends in 2010-02-28, if it is already there. You could try
> "SELECT * FROM period where startdate = '2010-02-01'::date;" and see if you
> get multiple periods for Feb 2010. If you have multiple periods 

Re: [Dhis2-devs] periodstructure giving errors...

[Moemedi Ntunyane]

Thanks, I was upgrading from 2.0.5 to 2.14. I can only see this error its not 
complaining of duplication but null value error:
org.postgresql.util.PSQLException: ERROR: null value in column "iso" violates 
not-null constraint.



Moemedi Ntunyane
Co-founder KeyInConsulting


Date: Tue, 16 Sep 2014 09:18:20 +0200
Subject: Re: [Dhis2-devs] periodstructure giving errors...
From: jason.p.picker...@gmail.com
To: moemedi.ntuny...@hotmail.com
CC: larshe...@gmail.com; dhis2-devs@lists.launchpad.net

Yeah, well, I can't offer much more advice other in this case. We had similar 
problems as I mentioned before with 2.16, but after carefully cleaning up all 
of the faulty periods, things worked OK. 
One thing which I would suggest would be to turn on full logging of all 
postgresql queries, and see exactly where things are failing. 
You should also see something like 
Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value 
violates unique constraint "in_periodstructure_iso"  Detail: Key (iso)=(1995) 
already exists.
in the Tomcat log, which again, gives you a clue about which period may be 
causing the problem. I know you posted a portion of the log before, but I did 
not see anything like this in your log. Turning on the postgresql logs may help 
you to track down which period is causing this error. 
Maybe one of the developers can add more information? The procedure worked for 
us, but required a bit of detective work.
Regards,Jason



On Tue, Sep 16, 2014 at 9:12 AM, Moemedi Ntunyane 
 wrote:



I cleared cache and even restarted my tomcat server

Moemedi Ntunyane
Co-founder KeyInConsulting


Date: Tue, 16 Sep 2014 06:09:43 +0200
Subject: Re: [Dhis2-devs] periodstructure giving errors...
From: jason.p.picker...@gmail.com
To: moemedi.ntuny...@hotmail.com
CC: larshe...@gmail.com; dhis2-devs@lists.launchpad.net

Did you clear your server cache and or restart DHIS2?
On Tue, Sep 16, 2014 at 5:50 AM, Moemedi Ntunyane 
 wrote:



I have eliminated all duplicates but this error still persist.
Process failed: PreparedStatementCallback; SQL [insert into _periodstructure 
values (?,?,?,?,?,?,?,?,?,?,?,?,?)]; ERROR: null value in column "iso" violates 
not-null constraint; nested exception is 

Moemedi Ntunyane
Co-founder KeyInConsulting


From: moemedi.ntuny...@hotmail.com
To: larshe...@gmail.com; jason.p.picker...@gmail.com
CC: dhis2-devs@lists.launchpad.net
Subject: RE: [Dhis2-devs] periodstructure giving errors...
Date: Mon, 15 Sep 2014 16:18:14 +0200




Hi Lars,

I have not done that. Im only capturing statistical data.

Regards,

Moemedi Ntunyane
Co-founder KeyInConsulting


Date: Mon, 15 Sep 2014 16:10:13 +0200
Subject: Re: [Dhis2-devs] periodstructure giving errors...
From: larshe...@gmail.com
To: jason.p.picker...@gmail.com
CC: moemedi.ntuny...@hotmail.com; dhis2-devs@lists.launchpad.net

Hi Moemedi,
may I ask, did you ever run the "aggregation queries" (producing aggregate 
values from event data) on your database?
regards,
Lars

On Mon, Sep 15, 2014 at 4:01 PM, Jason Pickering  
wrote:
I suspect you have more invalid periods then. You should follow the same 
procedure for other period types, i.e. checking to be sure that all months are 
one month, all weeks are one week, etc. 
You may want to try something like
 SELECT startdate, periodtypeid,COUNT(*) from period GROUP BY 
startdate,periodtypeid HAVING COUNT(*) > 1 ;

and see which periods and period types you have multiple records for. Ideally, 
this should not happen. 
Regards,Jason

On Mon, Sep 15, 2014 at 3:53 PM, Moemedi Ntunyane 
 wrote:



Hi Jason
Thnx, I had successfully elimated duplicate records the error still 
pesistit complaining of ISO column: is this column a date column??

Process
 failed: PreparedStatementCallback; SQL [insert into _periodstructure 
values (?,?,?,?,?,?,?,?,?,?,?,?,?)]; ERROR: null value in column "iso" 
violates not-null constraint; nested exception is 
org.postgresql.util.PSQLException: ERROR: null value in column "iso" 
violates not-null constraint.

Regards,

Moemedi Ntunyane
Co-founder KeyInConsulting


Date: Mon, 15 Sep 2014 12:22:31 +0200
Subject: Re: [Dhis2-devs] periodstructure giving errors...
From: jason.p.picker...@gmail.com
To: moemedi.ntuny...@hotmail.com
CC: dhis2-devs@lists.launchpad.net

So, the period 1589;6;"2010-02-01";"2010-02-28 , points to a yearly period 
type, which is not a year in duration. You are going to need to get rid of it. 
You will need to do something like
SELECT COUNT(*) FROM datavalue where periodid = 1589; 
If you have any records with this periodID, you are going to need to resolve 
these. This period looks to be monthly, so if the data is also monthly, you 
should be able to reassign it to a monthly period which starts in 2010-02-01 
and ends in 2010-02-28, if it is already there. You could try "SELECT * FROM 
period where startdate = '2010-02-01'::date;" and see if you get multiple 
periods for Feb 2010. If you have multiple periods for Feb 2010, then you need 
to do 

Re: [Dhis2-devs] periodstructure giving errors...

[Jason Pickering]

Yes, you will see that error, but immediately after that exception in the
logs, should be another one. If you do not see it, as I said, you may need
to resort to postgresql logging to figure out exactly what is causing the
problem.

On Tue, Sep 16, 2014 at 9:27 AM, Moemedi Ntunyane <
moemedi.ntuny...@hotmail.com> wrote:

> Thanks, I was upgrading from 2.0.5 to 2.14. I can only see this error its
> not complaining of duplication but null value error:
>
> org.postgresql.util.PSQLException: ERROR: null value in column "iso"
> violates not-null constraint.
>
>
>
>
> *Moemedi NtunyaneCo-founder KeyInConsulting*
>
>
> --
> Date: Tue, 16 Sep 2014 09:18:20 +0200
>
> Subject: Re: [Dhis2-devs] periodstructure giving errors...
> From: jason.p.picker...@gmail.com
> To: moemedi.ntuny...@hotmail.com
> CC: larshe...@gmail.com; dhis2-devs@lists.launchpad.net
>
> Yeah, well, I can't offer much more advice other in this case. We had
> similar problems as I mentioned before with 2.16, but after carefully
> cleaning up all of the faulty periods, things worked OK.
>
> One thing which I would suggest would be to turn on full logging of all
> postgresql queries, and see exactly where things are failing.
>
> You should also see something like
>
> Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value
> violates unique constraint "in_periodstructure_iso"
>   Detail: Key (iso)=(1995) already exists.
>
> in the Tomcat log, which again, gives you a clue about which period may be
> causing the problem. I know you posted a portion of the log before, but I
> did not see anything like this in your log. Turning on the postgresql logs
> may help you to track down which period is causing this error.
>
> Maybe one of the developers can add more information? The procedure worked
> for us, but required a bit of detective work.
>
> Regards,
> Jason
>
>
>
>
> On Tue, Sep 16, 2014 at 9:12 AM, Moemedi Ntunyane <
> moemedi.ntuny...@hotmail.com> wrote:
>
> I cleared cache and even restarted my tomcat server
>
>
> *Moemedi NtunyaneCo-founder KeyInConsulting*
>
>
> --
> Date: Tue, 16 Sep 2014 06:09:43 +0200
> Subject: Re: [Dhis2-devs] periodstructure giving errors...
> From: jason.p.picker...@gmail.com
> To: moemedi.ntuny...@hotmail.com
> CC: larshe...@gmail.com; dhis2-devs@lists.launchpad.net
>
>
> Did you clear your server cache and or restart DHIS2?
>
> On Tue, Sep 16, 2014 at 5:50 AM, Moemedi Ntunyane <
> moemedi.ntuny...@hotmail.com> wrote:
>
> I have eliminated all duplicates but this error still persist.
>
> Process failed: PreparedStatementCallback; SQL [insert into
> _periodstructure values (?,?,?,?,?,?,?,?,?,?,?,?,?)]; ERROR: null value in
> column "iso" violates not-null constraint; nested exception is
>
>
> *Moemedi NtunyaneCo-founder KeyInConsulting*
>
>
> --
> From: moemedi.ntuny...@hotmail.com
> To: larshe...@gmail.com; jason.p.picker...@gmail.com
> CC: dhis2-devs@lists.launchpad.net
> Subject: RE: [Dhis2-devs] periodstructure giving errors...
> Date: Mon, 15 Sep 2014 16:18:14 +0200
>
>
> Hi Lars,
>
> I have not done that. Im only capturing statistical data.
>
> Regards,
>
>
> *Moemedi NtunyaneCo-founder KeyInConsulting*
>
>
> --
> Date: Mon, 15 Sep 2014 16:10:13 +0200
> Subject: Re: [Dhis2-devs] periodstructure giving errors...
> From: larshe...@gmail.com
> To: jason.p.picker...@gmail.com
> CC: moemedi.ntuny...@hotmail.com; dhis2-devs@lists.launchpad.net
>
> Hi Moemedi,
>
> may I ask, did you ever run the "aggregation queries" (producing aggregate
> values from event data) on your database?
>
> regards,
>
> Lars
>
>
> On Mon, Sep 15, 2014 at 4:01 PM, Jason Pickering <
> jason.p.picker...@gmail.com> wrote:
>
> I suspect you have more invalid periods then. You should follow the same
> procedure for other period types, i.e. checking to be sure that all months
> are one month, all weeks are one week, etc.
>
> You may want to try something like
>
>  SELECT startdate, periodtypeid,COUNT(*) from period GROUP BY
> startdate,periodtypeid HAVING COUNT(*) > 1 ;
>
> and see which periods and period types you have multiple records for.
> Ideally, this should not happen.
>
> Regards,
> Jason
>
>
> On Mon, Sep 15, 2014 at 3:53 PM, Moemedi Ntunyane <
> moemedi.ntuny...@hotmail.com> wrote:
>
> Hi Jason
> Thnx, I had successfully elimated duplicate records the error still
> pesistit complaining of ISO column: is this column a date column??
>
> Process failed: PreparedStatementCallback; SQL [insert into
> _periodstructure values (?,?,?,?,?,?,?,?,?,?,?,?,?)]; ERROR: null value in
> column "iso" violates not-null constraint; nested exception is
> org.postgresql.util.PSQLException: ERROR: null value in column "iso"
> violates not-null constraint.
>
> Regards,
>
>
> *Moemedi NtunyaneCo-founder KeyInConsulting*
>
>
> --
> Date: Mon, 15 Sep 2014 12:22:31 +0200
>
> Subject: Re: [Dhis2-devs] periodstruc

Re: [Dhis2-devs] periodstructure giving errors...

[Knut Staring]

Depending on how much of configuration you have in your database, one
option could be to start afresh - doing metadata and data exports, and
import to an empty database. You would lose certain things (like users),
but also possibly get rid of some issues, rather than operating directly in
the database.



On Tue, Sep 16, 2014 at 9:27 AM, Moemedi Ntunyane <
moemedi.ntuny...@hotmail.com> wrote:

> Thanks, I was upgrading from 2.0.5 to 2.14. I can only see this error its
> not complaining of duplication but null value error:
>
> org.postgresql.util.PSQLException: ERROR: null value in column "iso"
> violates not-null constraint.
>
>
>
>
> *Moemedi NtunyaneCo-founder KeyInConsulting*
>
>
> --
> Date: Tue, 16 Sep 2014 09:18:20 +0200
>
> Subject: Re: [Dhis2-devs] periodstructure giving errors...
> From: jason.p.picker...@gmail.com
> To: moemedi.ntuny...@hotmail.com
> CC: larshe...@gmail.com; dhis2-devs@lists.launchpad.net
>
> Yeah, well, I can't offer much more advice other in this case. We had
> similar problems as I mentioned before with 2.16, but after carefully
> cleaning up all of the faulty periods, things worked OK.
>
> One thing which I would suggest would be to turn on full logging of all
> postgresql queries, and see exactly where things are failing.
>
> You should also see something like
>
> Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value
> violates unique constraint "in_periodstructure_iso"
>   Detail: Key (iso)=(1995) already exists.
>
> in the Tomcat log, which again, gives you a clue about which period may be
> causing the problem. I know you posted a portion of the log before, but I
> did not see anything like this in your log. Turning on the postgresql logs
> may help you to track down which period is causing this error.
>
> Maybe one of the developers can add more information? The procedure worked
> for us, but required a bit of detective work.
>
> Regards,
> Jason
>
>
>
>
> On Tue, Sep 16, 2014 at 9:12 AM, Moemedi Ntunyane <
> moemedi.ntuny...@hotmail.com> wrote:
>
> I cleared cache and even restarted my tomcat server
>
>
> *Moemedi NtunyaneCo-founder KeyInConsulting*
>
>
> --
> Date: Tue, 16 Sep 2014 06:09:43 +0200
> Subject: Re: [Dhis2-devs] periodstructure giving errors...
> From: jason.p.picker...@gmail.com
> To: moemedi.ntuny...@hotmail.com
> CC: larshe...@gmail.com; dhis2-devs@lists.launchpad.net
>
>
> Did you clear your server cache and or restart DHIS2?
>
> On Tue, Sep 16, 2014 at 5:50 AM, Moemedi Ntunyane <
> moemedi.ntuny...@hotmail.com> wrote:
>
> I have eliminated all duplicates but this error still persist.
>
> Process failed: PreparedStatementCallback; SQL [insert into
> _periodstructure values (?,?,?,?,?,?,?,?,?,?,?,?,?)]; ERROR: null value in
> column "iso" violates not-null constraint; nested exception is
>
>
> *Moemedi NtunyaneCo-founder KeyInConsulting*
>
>
> --
> From: moemedi.ntuny...@hotmail.com
> To: larshe...@gmail.com; jason.p.picker...@gmail.com
> CC: dhis2-devs@lists.launchpad.net
> Subject: RE: [Dhis2-devs] periodstructure giving errors...
> Date: Mon, 15 Sep 2014 16:18:14 +0200
>
>
> Hi Lars,
>
> I have not done that. Im only capturing statistical data.
>
> Regards,
>
>
> *Moemedi NtunyaneCo-founder KeyInConsulting*
>
>
> --
> Date: Mon, 15 Sep 2014 16:10:13 +0200
> Subject: Re: [Dhis2-devs] periodstructure giving errors...
> From: larshe...@gmail.com
> To: jason.p.picker...@gmail.com
> CC: moemedi.ntuny...@hotmail.com; dhis2-devs@lists.launchpad.net
>
> Hi Moemedi,
>
> may I ask, did you ever run the "aggregation queries" (producing aggregate
> values from event data) on your database?
>
> regards,
>
> Lars
>
>
> On Mon, Sep 15, 2014 at 4:01 PM, Jason Pickering <
> jason.p.picker...@gmail.com> wrote:
>
> I suspect you have more invalid periods then. You should follow the same
> procedure for other period types, i.e. checking to be sure that all months
> are one month, all weeks are one week, etc.
>
> You may want to try something like
>
>  SELECT startdate, periodtypeid,COUNT(*) from period GROUP BY
> startdate,periodtypeid HAVING COUNT(*) > 1 ;
>
> and see which periods and period types you have multiple records for.
> Ideally, this should not happen.
>
> Regards,
> Jason
>
>
> On Mon, Sep 15, 2014 at 3:53 PM, Moemedi Ntunyane <
> moemedi.ntuny...@hotmail.com> wrote:
>
> Hi Jason
> Thnx, I had successfully elimated duplicate records the error still
> pesistit complaining of ISO column: is this column a date column??
>
> Process failed: PreparedStatementCallback; SQL [insert into
> _periodstructure values (?,?,?,?,?,?,?,?,?,?,?,?,?)]; ERROR: null value in
> column "iso" violates not-null constraint; nested exception is
> org.postgresql.util.PSQLException: ERROR: null value in column "iso"
> violates not-null constraint.
>
> Regards,
>
>
> *Moemedi NtunyaneCo-founder KeyInConsulting*
>
>
> --
> Date:

[Dhis2-devs] DHIS2 - Custom categories subaggregated results

[Robin Martens]

Hi devs,

Just an idea for your feedback.

We sometimes have DE with multiple categories where, for indicator 
calculations, we only need sub-aggregated results. Let me give an example:

DE: Workers
Categories: Gender (2 options), Sector (10 options)

I have my indicator: number of women working in the country. Currently, the 
indicator equation definition box only shows an overall total (over both 
dimensions) and fully disaggregated (over both dimensions), such as this:

Workers (total)
Workers (male, sector1)
Workers (male, sector2)
...

Would it be possible to add subaggregated results to the list to facilitate 
indicator definition? The options lacking in this case would be:

Workers (total, sector1)
Workers (total, sector2)
...
Workers (male, total)
Workers (female, total)

It would be much easier to use the total number of female workers in indicator 
calculations.

Regards,

Robin

___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

Re: [Dhis2-devs] DHIS2 - Indicator calculation over dimensions

[Jason Pickering]

Hi Robin,

I think that is the real issue, namely that you are applying DHIS2 in a
domain which is slightly different than it's typical domain, namely health.
I have been involved in some other projects with DHIS2 on the fringes of
what it can do out of the box in food security, water and sanitation, even
using it for recording golf handicap scores. What I have seen in each of
these domains is that there are some challenges with the way that the data
is aggregated. Lots of things work out of the box, like data collection,
user management and security, etc. But sometimes, the analysis needs to be
done externally through other means. Of course, it would be great if DHIS2
could do all of this for all domains, but since its primary focus is on
collection and management of health data, that is where things work most
often (although there are some challenges there as well, particular on data
which needs to be averaged or handled different in time or across orgunits,
such as ART current count).  Contributions from the community are of course
welcome! :)

Regards,
Jason


On Fri, Sep 12, 2014 at 8:55 AM, Robin Martens  wrote:

>  Hi Jason,
>
>
>
> Thanks for taking the time to read through my email.
>
>
>
> I'll have a look at the different possibilities you proposed, and we'll be
> looking forward to any future upgrade of the calculation method (for now or
> later). I guess it's just that some sectors need more complex indicators
> than others (our project is in forest management).
>
>
>
> Have a nice day,
>
>
>
> Robin
>
>
>
> *From:* Jason Pickering [mailto:jason.p.picker...@gmail.com]
> *Sent:* 11 September 2014 19:00
>
> *To:* Robin Martens
> *Cc:* Lars Helge Øverland; dhis2-us...@lists.launchpad.net; dhis2-devs
> *Subject:* Re: [Dhis2-devs] DHIS2 - Indicator calculation over dimensions
>
>
>
> Hi Robin,
>
>
>
> Your mail is dense and will need some digestion. :)
>
>
>
>  You give a very good level of detail however of you problem in this mail
> and will be very useful as this type of functionality is attempted to be
> implemented.
>
>
>
> To respond immediately to how you might be able to solve the issue, you
> should possibly consider using the WebAPI to extract your data, process it
> as you need, and then inject it back into DHIS2. The WebAPI is described in
> detail here .
> I have also written a chapter on the use of the R programming language with
> DHIS2, which is particularly well suited to do the type of custom
> calculations you are describing here. It is available here
> . Of course,
> other language/methods may also be more suited to your situation, such as
> Python. Lastly, you can have a look at the DHIS2 Ad-hoc tool
> 
>  which
> would allow interaction with the service layer of DHIS2. Another approach
> could be SQL which interacts directly with the database. I am sure there
> are many other means as well. So short answer is, right now there is no
> in-built way to achieve what you need I think, and it will take some coding
> on your side.
>
>
>
> We have run into similar issues in the water and sanitation sector, where
> we need to work with the "latest reported data", which DHIS2 does not
> handle really. We pull out the data via the WebAPI, do the aggregation
> externally, and then inject everything back into the system to get the
> figures we need. It would be nice if the system did it automatically, but
> given the nature of the project, there are many feature requests and
> limited resources. Contributions of course are welcome.
>
>
>
> The current aggregation engine handles the "easy" cases of sums and
> averages pretty well, but for more complex stuff, external routes may be
> the only solution for now.
>
>
>
> We should certainly try and distill some of your ideas into a concrete
> blueprint.
>
>
>
> Best regards,
>
> Jason
>
>
>
>
>
> On Thu, Sep 11, 2014 at 6:15 PM, Robin Martens  wrote:
>
> Hi Jason,
>
>
>
> I appreciate your help as this is very important for our project, thanks.
>
>
>
> Some of our indicators are indeed quite complex and might need some custom
> coding if not too complicated. However, can you give some basic steps on
> how to achieve this (and on how hard this is in terms of programming as
> we're not experts here)?
>
>
>
> ---
>
>
>
> The rest of this mail is about the specific issue I'm having here, it's
> basically related to three things:
>
>
>
> 1.   The absence of "cross-product" calculations in DHIS2 (I think
> it's what you call compulsory pairs of data).
>
> 2.   The fact that when no data exists on a disaggregated level, the
> value is taken to be zero instead of the aggregated (for custom dimensions
> only I think).
>
> 3.   The average function only exists over the time dimension (as
> discussed by Lars previously this week).
>
>
>
> A s

[Dhis2-devs] Question around Analytics and Data Mart export.

[Mansi Singhal]

Hello All,

Just wanted to know if there is any api that is exposed to "Start Export"
for "Analytics And Data Mart" ?


-- 
Regards,
Mansi Singhal
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

Re: [Dhis2-devs] Question around Analytics and Data Mart export.

[Lars Helge Øverland]

Hi Mansi,

yes it is, see here:

https://www.dhis2.org/doc/snapshot/en/user/html/ch31s24.html

Lars


On Tue, Sep 16, 2014 at 12:09 PM, Mansi Singhal 
wrote:

>
> Hello All,
>
> Just wanted to know if there is any api that is exposed to "Start Export"
> for "Analytics And Data Mart" ?
>
>
> --
> Regards,
> Mansi Singhal
>
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

Re: [Dhis2-devs] Question around Analytics and Data Mart export.

[Mansi Singhal]

Thanks Lars .

On Tue, Sep 16, 2014 at 3:45 PM, Lars Helge Øverland 
wrote:

> Hi Mansi,
>
> yes it is, see here:
>
> https://www.dhis2.org/doc/snapshot/en/user/html/ch31s24.html
>
> Lars
>
>
> On Tue, Sep 16, 2014 at 12:09 PM, Mansi Singhal  > wrote:
>
>>
>> Hello All,
>>
>> Just wanted to know if there is any api that is exposed to "Start Export"
>> for "Analytics And Data Mart" ?
>>
>>
>> --
>> Regards,
>> Mansi Singhal
>>
>
>


-- 
Regards,
Mansi Singhal

+91 9900246052
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

Re: [Dhis2-devs] Question around Analytics and Data Mart export.

[Mansi Singhal]

Hey Lars,

How will i come to know if the initiated server side process is completed ?

On Tue, Sep 16, 2014 at 3:46 PM, Mansi Singhal 
wrote:

> Thanks Lars .
>
> On Tue, Sep 16, 2014 at 3:45 PM, Lars Helge Øverland 
> wrote:
>
>> Hi Mansi,
>>
>> yes it is, see here:
>>
>> https://www.dhis2.org/doc/snapshot/en/user/html/ch31s24.html
>>
>> Lars
>>
>>
>> On Tue, Sep 16, 2014 at 12:09 PM, Mansi Singhal <
>> msing...@thoughtworks.com> wrote:
>>
>>>
>>> Hello All,
>>>
>>> Just wanted to know if there is any api that is exposed to "Start
>>> Export" for "Analytics And Data Mart" ?
>>>
>>>
>>> --
>>> Regards,
>>> Mansi Singhal
>>>
>>
>>
>
>
> --
> Regards,
> Mansi Singhal
>
> +91 9900246052
>



-- 
Regards,
Mansi Singhal

+91 9900246052
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

Re: [Dhis2-devs] Question around Analytics and Data Mart export.

[Jason Pickering]

Hi Mansi,
You can see this thread

https://lists.launchpad.net/dhis2-devs/msg17759.html

for some approaches of how to see what is going on with the analytics
process.

This info may be a bit dated, but essentially, the approach is the same.

Regards,
Jason


On Tue, Sep 16, 2014 at 12:40 PM, Mansi Singhal 
wrote:

> Hey Lars,
>
> How will i come to know if the initiated server side process is completed ?
>
> On Tue, Sep 16, 2014 at 3:46 PM, Mansi Singhal 
> wrote:
>
>> Thanks Lars .
>>
>> On Tue, Sep 16, 2014 at 3:45 PM, Lars Helge Øverland > > wrote:
>>
>>> Hi Mansi,
>>>
>>> yes it is, see here:
>>>
>>> https://www.dhis2.org/doc/snapshot/en/user/html/ch31s24.html
>>>
>>> Lars
>>>
>>>
>>> On Tue, Sep 16, 2014 at 12:09 PM, Mansi Singhal <
>>> msing...@thoughtworks.com> wrote:
>>>

 Hello All,

 Just wanted to know if there is any api that is exposed to "Start
 Export" for "Analytics And Data Mart" ?


 --
 Regards,
 Mansi Singhal

>>>
>>>
>>
>>
>> --
>> Regards,
>> Mansi Singhal
>>
>> +91 9900246052
>>
>
>
>
> --
> Regards,
> Mansi Singhal
>
> +91 9900246052
>
> ___
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to : dhis2-devs@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>
>


-- 
Jason P. Pickering
email: jason.p.picker...@gmail.com
tel:+46764147049
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

Re: [Dhis2-devs] Question around Analytics and Data Mart export.

[Mansi Singhal]

Thanks Jason.

On Tue, Sep 16, 2014 at 4:13 PM, Jason Pickering <
jason.p.picker...@gmail.com> wrote:

> Hi Mansi,
> You can see this thread
>
> https://lists.launchpad.net/dhis2-devs/msg17759.html
>
> for some approaches of how to see what is going on with the analytics
> process.
>
> This info may be a bit dated, but essentially, the approach is the same.
>
> Regards,
> Jason
>
>
> On Tue, Sep 16, 2014 at 12:40 PM, Mansi Singhal  > wrote:
>
>> Hey Lars,
>>
>> How will i come to know if the initiated server side process is completed
>> ?
>>
>> On Tue, Sep 16, 2014 at 3:46 PM, Mansi Singhal > > wrote:
>>
>>> Thanks Lars .
>>>
>>> On Tue, Sep 16, 2014 at 3:45 PM, Lars Helge Øverland <
>>> larshe...@gmail.com> wrote:
>>>
 Hi Mansi,

 yes it is, see here:

 https://www.dhis2.org/doc/snapshot/en/user/html/ch31s24.html

 Lars


 On Tue, Sep 16, 2014 at 12:09 PM, Mansi Singhal <
 msing...@thoughtworks.com> wrote:

>
> Hello All,
>
> Just wanted to know if there is any api that is exposed to "Start
> Export" for "Analytics And Data Mart" ?
>
>
> --
> Regards,
> Mansi Singhal
>


>>>
>>>
>>> --
>>> Regards,
>>> Mansi Singhal
>>>
>>> +91 9900246052
>>>
>>
>>
>>
>> --
>> Regards,
>> Mansi Singhal
>>
>> +91 9900246052
>>
>> ___
>> Mailing list: https://launchpad.net/~dhis2-devs
>> Post to : dhis2-devs@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~dhis2-devs
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
>
> --
> Jason P. Pickering
> email: jason.p.picker...@gmail.com
> tel:+46764147049
>



-- 
Regards,
Mansi Singhal

+91 9900246052
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

Re: [Dhis2-devs] [Dhis2-users] Dhis2 Demo Login Credentials

[Inés Bebea]

Hi!

I am trying to access the demo site but login credentials admin / district
are not working. Can you please reset?

Thanks!
Inés



2014-08-04 10:23 GMT+02:00 Knut Staring :

> It seems that a lot of people are inclined to change the admin password
>
> I have now set it to "District1" again.
>
> Knut
>
>
> On Mon, Aug 4, 2014 at 10:17 AM, Stanley Kalyati 
> wrote:
>
>> Knut,
>>
>> I have tried the login credentials but they are not working.
>>
>> Please advice.
>>
>> Regards
>>
>>
>>
>> *Stanley Kalyati | Monitoring and Evaluation Technical Lead *
>>
>> *PSI/Malawi*
>> Head Office & Main Warehouse,Off M1 Road,Kanengo,Area 28 Plot 198
>>
>> PO Box 30132, Capital City, Lilongwe 3 Malawi
>>
>>
>> Tel: +265 (0) 1 711 484  |  Fax: +265 (0) 1 711 486| Cell: 0992 313
>> 978/0993 855 635
>>
>> skaly...@psimalawi.org  |
>> www.psi.org/malawi
>>
>> Skype: stanley.kalyati1  | Gmail: skaly...@gmail.com
>>
>>
>>
>>
>> On 1 August 2014 15:02, Knut Staring  wrote:
>>
>>>  Hello,
>>>
>>> You can now log in as admin with password "District1"
>>>
>>> (from midnight it should be back to just "district")
>>>
>>> Knut
>>>
>>>
>>> On Fri, Aug 1, 2014 at 2:47 PM, benjamin dubdabasoduba mwalimu <
>>> dubdabasod...@gmail.com> wrote:
>>>
 Hi guys!
 The admin/ district login credentials arent working for the demo site
 can anyone please have a look on that.

 Regards,
 Benjamin Mwalimu

 ___
 Mailing list: https://launchpad.net/~dhis2-devs
 Post to : dhis2-devs@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~dhis2-devs
 More help   : https://help.launchpad.net/ListHelp


>>>
>>>
>>> --
>>> Knut Staring
>>> Dept. of Informatics, University of Oslo
>>> +4791880522
>>> http://dhis2.org
>>>
>>> ___
>>> Mailing list: https://launchpad.net/~dhis2-users
>>> Post to : dhis2-us...@lists.launchpad.net
>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>
>
>
> --
> Knut Staring
> Dept. of Informatics, University of Oslo
> +4791880522
> http://dhis2.org
>
> ___
> Mailing list: https://launchpad.net/~dhis2-users
> Post to : dhis2-us...@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~dhis2-users
> More help   : https://help.launchpad.net/ListHelp
>
>


-- 
Inés Bebea
@bbaobabb
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

Re: [Dhis2-devs] [Dhis2-users] Dhis2 Demo Login Credentials

[Ola Hodne Titlestad]

I have changed the admin password to "District123" for now.
It will be reset to "district" at midnight (Oslo time).

Ola
---


--
Ola Hodne Titlestad (Mr)
HISP
Department of Informatics
University of Oslo

Mobile: +47 48069736
Home address: Eftasåsen 68, 0687 Oslo, Norway. Googlemaps link


On 16 September 2014 13:06, Inés Bebea  wrote:

> Hi!
>
> I am trying to access the demo site but login credentials admin / district
> are not working. Can you please reset?
>
> Thanks!
> Inés
>
>
>
> 2014-08-04 10:23 GMT+02:00 Knut Staring :
>
>> It seems that a lot of people are inclined to change the admin
>> password
>>
>> I have now set it to "District1" again.
>>
>> Knut
>>
>>
>> On Mon, Aug 4, 2014 at 10:17 AM, Stanley Kalyati 
>> wrote:
>>
>>> Knut,
>>>
>>> I have tried the login credentials but they are not working.
>>>
>>> Please advice.
>>>
>>> Regards
>>>
>>>
>>>
>>> *Stanley Kalyati | Monitoring and Evaluation Technical Lead *
>>>
>>> *PSI/Malawi*
>>> Head Office & Main Warehouse,Off M1 Road,Kanengo,Area 28 Plot 198
>>>
>>> PO Box 30132, Capital City, Lilongwe 3 Malawi
>>>
>>>
>>> Tel: +265 (0) 1 711 484  |  Fax: +265 (0) 1 711 486| Cell: 0992 313
>>> 978/0993 855 635
>>>
>>> skaly...@psimalawi.org  |
>>> www.psi.org/malawi
>>>
>>> Skype: stanley.kalyati1  | Gmail: skaly...@gmail.com
>>>
>>>
>>>
>>>
>>> On 1 August 2014 15:02, Knut Staring  wrote:
>>>
  Hello,

 You can now log in as admin with password "District1"

 (from midnight it should be back to just "district")

 Knut


 On Fri, Aug 1, 2014 at 2:47 PM, benjamin dubdabasoduba mwalimu <
 dubdabasod...@gmail.com> wrote:

> Hi guys!
> The admin/ district login credentials arent working for the demo site
> can anyone please have a look on that.
>
> Regards,
> Benjamin Mwalimu
>
> ___
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to : dhis2-devs@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>
>


 --
 Knut Staring
 Dept. of Informatics, University of Oslo
 +4791880522
 http://dhis2.org

 ___
 Mailing list: https://launchpad.net/~dhis2-users
 Post to : dhis2-us...@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~dhis2-users
 More help   : https://help.launchpad.net/ListHelp


>>>
>>
>>
>> --
>> Knut Staring
>> Dept. of Informatics, University of Oslo
>> +4791880522
>> http://dhis2.org
>>
>> ___
>> Mailing list: https://launchpad.net/~dhis2-users
>> Post to : dhis2-us...@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~dhis2-users
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
>
> --
> Inés Bebea
> @bbaobabb
>
>
> ___
> Mailing list: https://launchpad.net/~dhis2-users
> Post to : dhis2-us...@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~dhis2-users
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

[Dhis2-devs] DHIS2 - Import data from CSV - bug

[Robin Martens]

Hi devs,

I'm having what seems like a bug with the import module:

When importing data from CSV, I get the following error message and the batch 
doesn't run. However, this only happens when importing a data element over 
different categoryoptioncombos and only when it is new data, not updates. In 
other words, I'm getting my import done piece by piece, i.e. per data 
element/categoryoptioncombo combination.

Also note that the import dry run doesn't detect any issues.

Any idea how to solve this?

Kind regards,

Robin



* INFO  2014-09-16 13:47:11,377 [Level: INFO, category: DATAVALUE_IMPORT, time: 
Tue Sep 16 13:47:11 CAT 2014, message: Process started] (InMemoryNotifier.java 
[taskScheduler-6])
* INFO  2014-09-16 13:47:11,469 [Level: INFO, category: DATAVALUE_IMPORT, time: 
Tue Sep 16 13:47:11 CAT 2014, message: Importing data values] 
(InMemoryNotifier.java [taskScheduler-6])
* INFO  2014-09-16 13:47:11,470 importing data values 
(DefaultDataValueSetService.java [taskScheduler-6])
* ERROR 2014-09-16 13:47:12,396 java.lang.RuntimeException: Failed to flush 
BatchHandler
at 
org.amplecode.quick.batchhandler.AbstractBatchHandler.flush(AbstractBatchHandler.java:311)
at 
org.hisp.dhis.dxf2.datavalueset.DefaultDataValueSetService.saveDataValueSet(DefaultDataValueSetService.java:613)
at 
org.hisp.dhis.dxf2.datavalueset.DefaultDataValueSetService.saveDataValueSetCsv(DefaultDataValueSetService.java:388)
at 
org.hisp.dhis.importexport.action.util.ImportDataValueTask.run(ImportDataValueTask.java:78)
at 
org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:53)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value 
violates unique constraint "datavalue_pkey"
  Detail: Key (dataelementid, periodid, sourceid, categoryoptioncomboid, 
attributeoptioncomboid)=(2443, 2079, 59, 16, 16) already exists.
at 
org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2161)
at 
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1890)
at 
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:560)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:403)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:331)
at 
org.amplecode.quick.batchhandler.AbstractBatchHandler.flush(AbstractBatchHandler.java:295)
... 11 more
(DefaultDataValueSetService.java [taskScheduler-6])
* INFO  2014-09-16 13:47:12,396 [Level: ERROR, category: DATAVALUE_IMPORT, 
time: Tue Sep 16 13:47:12 CAT 2014, message: Process failed: Failed to flush 
BatchHandler] (InMemoryNotifier.java [taskScheduler-6])
* ERROR 2014-09-16 13:47:15,261 Left side ($summary.conflicts.size()) of '>' 
operation has null value at /dhis-web-importexport/importSummary.vm[line 35, 
column 33] (Log4JLogChute.java [http-bio-8080-exec-2])
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug

[Olav Poppe]

Hi,
my experience is that when this happens (dry run works, but actual import fails 
with an error similar to this), it is usually caused by duplicate values in the 
source file i.e. multiple values with the same orgunit, data element, category 
and period. 

Olav



> 16. sep. 2014 kl. 14.22 skrev Robin Martens :
> 
> Hi devs,
>  
> I'm having what seems like a bug with the import module:
>  
> When importing data from CSV, I get the following error message and the batch 
> doesn't run. However, this only happens when importing a data element over 
> different categoryoptioncombos and only when it is new data, not updates. In 
> other words, I'm getting my import done piece by piece, i.e. per data 
> element/categoryoptioncombo combination.
>  
> Also note that the import dry run doesn't detect any issues.
>  
> Any idea how to solve this?
>  
> Kind regards,
>  
> Robin
>  
>  
>  
> * INFO  2014-09-16 13:47:11,377 [Level: INFO, category: DATAVALUE_IMPORT, 
> time: Tue Sep 16 13:47:11 CAT 2014, message: Process started] 
> (InMemoryNotifier.java [taskScheduler-6])
> * INFO  2014-09-16 13:47:11,469 [Level: INFO, category: DATAVALUE_IMPORT, 
> time: Tue Sep 16 13:47:11 CAT 2014, message: Importing data values] 
> (InMemoryNotifier.java [taskScheduler-6])
> * INFO  2014-09-16 13:47:11,470 importing data values 
> (DefaultDataValueSetService.java [taskScheduler-6])
> * ERROR 2014-09-16 13:47:12,396 java.lang.RuntimeException: Failed to flush 
> BatchHandler
> at 
> org.amplecode.quick.batchhandler.AbstractBatchHandler.flush(AbstractBatchHandler.java:311)
> at 
> org.hisp.dhis.dxf2.datavalueset.DefaultDataValueSetService.saveDataValueSet(DefaultDataValueSetService.java:613)
> at 
> org.hisp.dhis.dxf2.datavalueset.DefaultDataValueSetService.saveDataValueSetCsv(DefaultDataValueSetService.java:388)
> at 
> org.hisp.dhis.importexport.action.util.ImportDataValueTask.run(ImportDataValueTask.java:78)
> at 
> org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:53)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value 
> violates unique constraint "datavalue_pkey"
>   Detail: Key (dataelementid, periodid, sourceid, categoryoptioncomboid, 
> attributeoptioncomboid)=(2443, 2079, 59, 16, 16) already exists.
> at 
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2161)
> at 
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1890)
> at 
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
> at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:560)
> at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:403)
> at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:331)
> at 
> org.amplecode.quick.batchhandler.AbstractBatchHandler.flush(AbstractBatchHandler.java:295)
> ... 11 more
> (DefaultDataValueSetService.java [taskScheduler-6])
> * INFO  2014-09-16 13:47:12,396 [Level: ERROR, category: DATAVALUE_IMPORT, 
> time: Tue Sep 16 13:47:12 CAT 2014, message: Process failed: Failed to flush 
> BatchHandler] (InMemoryNotifier.java [taskScheduler-6])
> * ERROR 2014-09-16 13:47:15,261 Left side ($summary.conflicts.size()) of '>' 
> operation has null value at /dhis-web-importexport/importSummary.vm[line 35, 
> column 33] (Log4JLogChute.java [http-bio-8080-exec-2])
> ___
> Mailing list: https://launchpad.net/~dhis2-devs 
> 
> Post to : dhis2-devs@lists.launchpad.net 
> 
> Unsubscribe : https://launchpad.net/~dhis2-devs 
> 
> More help   : https://help.launchpad.net/ListHelp 
> 
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~

Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug

[Olav Poppe]

Hi, 
I see that your log file actually confirms this:
Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value 
violates unique constraint "datavalue_pkey"
  Detail: Key (dataelementid, periodid, sourceid, categoryoptioncomboid, 
attributeoptioncomboid)=(2443, 2079, 59, 16, 16) already exists.

Olav



> 16. sep. 2014 kl. 15.23 skrev Olav Poppe :
> 
> Hi,
> my experience is that when this happens (dry run works, but actual import 
> fails with an error similar to this), it is usually caused by duplicate 
> values in the source file i.e. multiple values with the same orgunit, data 
> element, category and period. 
> 
> Olav
> 
> 
> 
>> 16. sep. 2014 kl. 14.22 skrev Robin Martens > >:
>> 
>> Hi devs,
>>  
>> I'm having what seems like a bug with the import module:
>>  
>> When importing data from CSV, I get the following error message and the 
>> batch doesn't run. However, this only happens when importing a data element 
>> over different categoryoptioncombos and only when it is new data, not 
>> updates. In other words, I'm getting my import done piece by piece, i.e. per 
>> data element/categoryoptioncombo combination.
>>  
>> Also note that the import dry run doesn't detect any issues.
>>  
>> Any idea how to solve this?
>>  
>> Kind regards,
>>  
>> Robin
>>  
>>  
>>  
>> * INFO  2014-09-16 13:47:11,377 [Level: INFO, category: DATAVALUE_IMPORT, 
>> time: Tue Sep 16 13:47:11 CAT 2014, message: Process started] 
>> (InMemoryNotifier.java [taskScheduler-6])
>> * INFO  2014-09-16 13:47:11,469 [Level: INFO, category: DATAVALUE_IMPORT, 
>> time: Tue Sep 16 13:47:11 CAT 2014, message: Importing data values] 
>> (InMemoryNotifier.java [taskScheduler-6])
>> * INFO  2014-09-16 13:47:11,470 importing data values 
>> (DefaultDataValueSetService.java [taskScheduler-6])
>> * ERROR 2014-09-16 13:47:12,396 java.lang.RuntimeException: Failed to flush 
>> BatchHandler
>> at 
>> org.amplecode.quick.batchhandler.AbstractBatchHandler.flush(AbstractBatchHandler.java:311)
>> at 
>> org.hisp.dhis.dxf2.datavalueset.DefaultDataValueSetService.saveDataValueSet(DefaultDataValueSetService.java:613)
>> at 
>> org.hisp.dhis.dxf2.datavalueset.DefaultDataValueSetService.saveDataValueSetCsv(DefaultDataValueSetService.java:388)
>> at 
>> org.hisp.dhis.importexport.action.util.ImportDataValueTask.run(ImportDataValueTask.java:78)
>> at 
>> org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:53)
>> at 
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> at 
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>> at 
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>> at 
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at 
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:744)
>> Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value 
>> violates unique constraint "datavalue_pkey"
>>   Detail: Key (dataelementid, periodid, sourceid, categoryoptioncomboid, 
>> attributeoptioncomboid)=(2443, 2079, 59, 16, 16) already exists.
>> at 
>> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2161)
>> at 
>> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1890)
>> at 
>> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
>> at 
>> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:560)
>> at 
>> org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:403)
>> at 
>> org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:331)
>> at 
>> org.amplecode.quick.batchhandler.AbstractBatchHandler.flush(AbstractBatchHandler.java:295)
>> ... 11 more
>> (DefaultDataValueSetService.java [taskScheduler-6])
>> * INFO  2014-09-16 13:47:12,396 [Level: ERROR, category: DATAVALUE_IMPORT, 
>> time: Tue Sep 16 13:47:12 CAT 2014, message: Process failed: Failed to flush 
>> BatchHandler] (InMemoryNotifier.java [taskScheduler-6])
>> * ERROR 2014-09-16 13:47:15,261 Left side ($summary.conflicts.size()) of '>' 
>> operation has null value at /dhis-web-importexport/importSummary.vm[line 35, 
>> column 33] (Log4JLogChute.java [http-bio-8080-exec-2])
>> ___
>> Mailing list: https://launchpad.net/~dhi

[Dhis2-devs] [Bug 1370057] [NEW] Last month is current in Event Reports and Event Visualizer

[Lars Helge Øverland]

Public bug reported:

In Event Reports and Event Visualizer, when "last month" is selected,
the current month is selected for the report instead of last. For
example, in September, then September is selected for the report instead
of August.

** Affects: dhis2
 Importance: Medium
 Assignee: Jan Henrik Øverland (janhenrik-overland)
 Status: Confirmed

** Changed in: dhis2
 Assignee: (unassigned) => Jan Henrik Øverland (janhenrik-overland)

** Changed in: dhis2
   Importance: Undecided => Medium

** Changed in: dhis2
   Status: New => Confirmed

** Changed in: dhis2
Milestone: None => 2.16

-- 
You received this bug notification because you are a member of DHIS 2
developers, which is subscribed to DHIS.
https://bugs.launchpad.net/bugs/1370057

Title:
  Last month is current in Event Reports and Event Visualizer

Status in DHIS 2:
  Confirmed

Bug description:
  In Event Reports and Event Visualizer, when "last month" is selected,
  the current month is selected for the report instead of last. For
  example, in September, then September is selected for the report
  instead of August.

To manage notifications about this bug go to:
https://bugs.launchpad.net/dhis2/+bug/1370057/+subscriptions

___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

Re: [Dhis2-devs] periodstructure giving errors...

[Moemedi Ntunyane]

Thanks a lot for your assistance, I just used a fresh database and imported 
meta data and the data itself eventhough other info was missing but managed to 
fix everything. Now its working ok on my local instance.

Moemedi Ntunyane
Co-founder KeyInConsulting


From: knu...@gmail.com
Date: Tue, 16 Sep 2014 09:37:36 +0200
Subject: Re: [Dhis2-devs] periodstructure giving errors...
To: moemedi.ntuny...@hotmail.com
CC: jason.p.picker...@gmail.com; dhis2-devs@lists.launchpad.net

Depending on how much of configuration you have in your database, one option 
could be to start afresh - doing metadata and data exports, and import to an 
empty database. You would lose certain things (like users), but also possibly 
get rid of some issues, rather than operating directly in the database.


On Tue, Sep 16, 2014 at 9:27 AM, Moemedi Ntunyane 
 wrote:



Thanks, I was upgrading from 2.0.5 to 2.14. I can only see this error its not 
complaining of duplication but null value error:
org.postgresql.util.PSQLException: ERROR: null value in column "iso" violates 
not-null constraint.



Moemedi Ntunyane
Co-founder KeyInConsulting


Date: Tue, 16 Sep 2014 09:18:20 +0200
Subject: Re: [Dhis2-devs] periodstructure giving errors...
From: jason.p.picker...@gmail.com
To: moemedi.ntuny...@hotmail.com
CC: larshe...@gmail.com; dhis2-devs@lists.launchpad.net

Yeah, well, I can't offer much more advice other in this case. We had similar 
problems as I mentioned before with 2.16, but after carefully cleaning up all 
of the faulty periods, things worked OK. 
One thing which I would suggest would be to turn on full logging of all 
postgresql queries, and see exactly where things are failing. 
You should also see something like 
Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value 
violates unique constraint "in_periodstructure_iso"  Detail: Key (iso)=(1995) 
already exists.
in the Tomcat log, which again, gives you a clue about which period may be 
causing the problem. I know you posted a portion of the log before, but I did 
not see anything like this in your log. Turning on the postgresql logs may help 
you to track down which period is causing this error. 
Maybe one of the developers can add more information? The procedure worked for 
us, but required a bit of detective work.
Regards,Jason



On Tue, Sep 16, 2014 at 9:12 AM, Moemedi Ntunyane 
 wrote:



I cleared cache and even restarted my tomcat server

Moemedi Ntunyane
Co-founder KeyInConsulting


Date: Tue, 16 Sep 2014 06:09:43 +0200
Subject: Re: [Dhis2-devs] periodstructure giving errors...
From: jason.p.picker...@gmail.com
To: moemedi.ntuny...@hotmail.com
CC: larshe...@gmail.com; dhis2-devs@lists.launchpad.net

Did you clear your server cache and or restart DHIS2?
On Tue, Sep 16, 2014 at 5:50 AM, Moemedi Ntunyane 
 wrote:



I have eliminated all duplicates but this error still persist.
Process failed: PreparedStatementCallback; SQL [insert into _periodstructure 
values (?,?,?,?,?,?,?,?,?,?,?,?,?)]; ERROR: null value in column "iso" violates 
not-null constraint; nested exception is 

Moemedi Ntunyane
Co-founder KeyInConsulting


From: moemedi.ntuny...@hotmail.com
To: larshe...@gmail.com; jason.p.picker...@gmail.com
CC: dhis2-devs@lists.launchpad.net
Subject: RE: [Dhis2-devs] periodstructure giving errors...
Date: Mon, 15 Sep 2014 16:18:14 +0200




Hi Lars,

I have not done that. Im only capturing statistical data.

Regards,

Moemedi Ntunyane
Co-founder KeyInConsulting


Date: Mon, 15 Sep 2014 16:10:13 +0200
Subject: Re: [Dhis2-devs] periodstructure giving errors...
From: larshe...@gmail.com
To: jason.p.picker...@gmail.com
CC: moemedi.ntuny...@hotmail.com; dhis2-devs@lists.launchpad.net

Hi Moemedi,
may I ask, did you ever run the "aggregation queries" (producing aggregate 
values from event data) on your database?
regards,
Lars

On Mon, Sep 15, 2014 at 4:01 PM, Jason Pickering  
wrote:
I suspect you have more invalid periods then. You should follow the same 
procedure for other period types, i.e. checking to be sure that all months are 
one month, all weeks are one week, etc. 
You may want to try something like
 SELECT startdate, periodtypeid,COUNT(*) from period GROUP BY 
startdate,periodtypeid HAVING COUNT(*) > 1 ;

and see which periods and period types you have multiple records for. Ideally, 
this should not happen. 
Regards,Jason

On Mon, Sep 15, 2014 at 3:53 PM, Moemedi Ntunyane 
 wrote:



Hi Jason
Thnx, I had successfully elimated duplicate records the error still 
pesistit complaining of ISO column: is this column a date column??

Process
 failed: PreparedStatementCallback; SQL [insert into _periodstructure 
values (?,?,?,?,?,?,?,?,?,?,?,?,?)]; ERROR: null value in column "iso" 
violates not-null constraint; nested exception is 
org.postgresql.util.PSQLException: ERROR: null value in column "iso" 
violates not-null constraint.

Regards,

Moemedi Ntunyane
Co-founder KeyInConsulting


Date: Mon, 15 Sep 2014 12:22:31 +0200
Subject: Re: [Dhis2-devs] peri

Re: [Dhis2-devs] Tracker - Patient dashboard problem

[sumudu weerasinghe]

Sorry, my system detail as follow
*Version: 2.16**Build revision:16414**Database type:MySQL*
*Java version:1.7.0_04**OS name:Windows 7*


On Tue, Sep 16, 2014 at 7:23 PM, sumudu weerasinghe 
wrote:

> Dear all ,
> I have updated my instance from 2.15 to 2.16  over the old mysql database.
> Then i list the clinic patient . But it was unable to go for patient
> dashboard through the list and give error as exception occur (shown in
> screen shot). Log also show some error. Log and screenshots are attached
> here.
>
> Regards,
> Sumudu
>
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug

[Robin Martens]

Hi Olav,

Thanks for the answer but I don't think that's the issue.

First of all, when checking my CSV with Excel's "Duplicate data" functionality 
there's no duplicates found.
Secondly, when importing the same CSV file but piece by piece (per data 
element-catoptcombo set) I don't have this issue.

Regards,

Robin

From: Olav Poppe [mailto:olav.po...@me.com]
Sent: 16 September 2014 15:26
To: Robin Martens
Cc: dhis2-devs@lists.launchpad.net
Subject: Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug

Hi,
I see that your log file actually confirms this:
Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value 
violates unique constraint "datavalue_pkey"
  Detail: Key (dataelementid, periodid, sourceid, categoryoptioncomboid, 
attributeoptioncomboid)=(2443, 2079, 59, 16, 16) already exists.

Olav



16. sep. 2014 kl. 15.23 skrev Olav Poppe 
mailto:olav.po...@me.com>>:

Hi,
my experience is that when this happens (dry run works, but actual import fails 
with an error similar to this), it is usually caused by duplicate values in the 
source file i.e. multiple values with the same orgunit, data element, category 
and period.

Olav



16. sep. 2014 kl. 14.22 skrev Robin Martens 
mailto:mart...@sher.be>>:

Hi devs,

I'm having what seems like a bug with the import module:

When importing data from CSV, I get the following error message and the batch 
doesn't run. However, this only happens when importing a data element over 
different categoryoptioncombos and only when it is new data, not updates. In 
other words, I'm getting my import done piece by piece, i.e. per data 
element/categoryoptioncombo combination.

Also note that the import dry run doesn't detect any issues.

Any idea how to solve this?

Kind regards,

Robin



* INFO  2014-09-16 13:47:11,377 [Level: INFO, category: DATAVALUE_IMPORT, time: 
Tue Sep 16 13:47:11 CAT 2014, message: Process started] (InMemoryNotifier.java 
[taskScheduler-6])
* INFO  2014-09-16 13:47:11,469 [Level: INFO, category: DATAVALUE_IMPORT, time: 
Tue Sep 16 13:47:11 CAT 2014, message: Importing data values] 
(InMemoryNotifier.java [taskScheduler-6])
* INFO  2014-09-16 13:47:11,470 importing data values 
(DefaultDataValueSetService.java [taskScheduler-6])
* ERROR 2014-09-16 13:47:12,396 java.lang.RuntimeException: Failed to flush 
BatchHandler
at 
org.amplecode.quick.batchhandler.AbstractBatchHandler.flush(AbstractBatchHandler.java:311)
at 
org.hisp.dhis.dxf2.datavalueset.DefaultDataValueSetService.saveDataValueSet(DefaultDataValueSetService.java:613)
at 
org.hisp.dhis.dxf2.datavalueset.DefaultDataValueSetService.saveDataValueSetCsv(DefaultDataValueSetService.java:388)
at 
org.hisp.dhis.importexport.action.util.ImportDataValueTask.run(ImportDataValueTask.java:78)
at 
org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:53)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value 
violates unique constraint "datavalue_pkey"
  Detail: Key (dataelementid, periodid, sourceid, categoryoptioncomboid, 
attributeoptioncomboid)=(2443, 2079, 59, 16, 16) already exists.
at 
org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2161)
at 
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1890)
at 
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:560)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:403)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:331)
at 
org.amplecode.quick.batchhandler.AbstractBatchHandler.flush(AbstractBatchHandler.java:295)
... 11 more
(DefaultDataValueSetService.java [taskScheduler-6])
* INFO  2014-09-16 13:47:12,396 [Level: ERROR, category: DATAVALUE_IMPORT, 
time: Tue Sep 16 13:47:12 CAT 2014, message: Process failed: Failed to flush 
BatchHandler] (InMemoryNotifier.java [taskScheduler-6])
* ERROR 2014-09-16 13:47:15,261 Left side ($sum

Re: [Dhis2-devs] [Dhis2-users] Tracker - Patient dashboard problem

[Lars Helge Øverland]

Hi Sumudu,

we are migrating the old individual case module to the new "tracker
capture" app in 2.16. Could you please try out the new app and see if it
meets your needs?

regards,

Lars


On Tue, Sep 16, 2014 at 3:56 PM, sumudu weerasinghe 
wrote:

> Sorry, my system detail as follow
> *Version: 2.16**Build revision:16414**Database type:MySQL*
> *Java version:1.7.0_04**OS name:Windows 7*
>
>
> On Tue, Sep 16, 2014 at 7:23 PM, sumudu weerasinghe 
> wrote:
>
>> Dear all ,
>> I have updated my instance from 2.15 to 2.16  over the old mysql
>> database. Then i list the clinic patient . But it was unable to go for
>> patient dashboard through the list and give error as exception occur (shown
>> in screen shot). Log also show some error. Log and screenshots are attached
>> here.
>>
>> Regards,
>> Sumudu
>>
>
>
> ___
> Mailing list: https://launchpad.net/~dhis2-users
> Post to : dhis2-us...@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~dhis2-users
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug

[Olav Poppe]

Hi, 
if you are sure there are no duplicates it must be something else. 

However, the fact that it works when you import piece by piece also points to 
duplicates being the problem: if you split the file so that each of the 
duplicate values are in different batches your import will work.

Olav


> 16. sep. 2014 kl. 15.59 skrev Robin Martens :
> 
> Hi Olav,
>  
> Thanks for the answer but I don't think that's the issue.
>  
> First of all, when checking my CSV with Excel's "Duplicate data" 
> functionality there's no duplicates found.
> Secondly, when importing the same CSV file but piece by piece (per data 
> element-catoptcombo set) I don't have this issue.
>  
> Regards,
>  
> Robin
>  
> From: Olav Poppe [mailto:olav.po...@me.com] 
> Sent: 16 September 2014 15:26
> To: Robin Martens
> Cc: dhis2-devs@lists.launchpad.net
> Subject: Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug
>  
> Hi, 
> I see that your log file actually confirms this:
> Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value 
> violates unique constraint "datavalue_pkey"
>   Detail: Key (dataelementid, periodid, sourceid, categoryoptioncomboid, 
> attributeoptioncomboid)=(2443, 2079, 59, 16, 16) already exists.
>  
> Olav
>  
>  
>  
> 16. sep. 2014 kl. 15.23 skrev Olav Poppe  >:
>  
> Hi,
> my experience is that when this happens (dry run works, but actual import 
> fails with an error similar to this), it is usually caused by duplicate 
> values in the source file i.e. multiple values with the same orgunit, data 
> element, category and period. 
>  
> Olav
>  
>  
>  
> 16. sep. 2014 kl. 14.22 skrev Robin Martens  >:
>  
> Hi devs,
>  
> I'm having what seems like a bug with the import module:
>  
> When importing data from CSV, I get the following error message and the batch 
> doesn't run. However, this only happens when importing a data element over 
> different categoryoptioncombos and only when it is new data, not updates. In 
> other words, I'm getting my import done piece by piece, i.e. per data 
> element/categoryoptioncombo combination.
>  
> Also note that the import dry run doesn't detect any issues.
>  
> Any idea how to solve this?
>  
> Kind regards,
>  
> Robin
>  
>  
>  
> * INFO  2014-09-16 13:47:11,377 [Level: INFO, category: DATAVALUE_IMPORT, 
> time: Tue Sep 16 13:47:11 CAT 2014, message: Process started] 
> (InMemoryNotifier.java [taskScheduler-6])
> * INFO  2014-09-16 13:47:11,469 [Level: INFO, category: DATAVALUE_IMPORT, 
> time: Tue Sep 16 13:47:11 CAT 2014, message: Importing data values] 
> (InMemoryNotifier.java [taskScheduler-6])
> * INFO  2014-09-16 13:47:11,470 importing data values 
> (DefaultDataValueSetService.java [taskScheduler-6])
> * ERROR 2014-09-16 13:47:12,396 java.lang.RuntimeException: Failed to flush 
> BatchHandler
> at 
> org.amplecode.quick.batchhandler.AbstractBatchHandler.flush(AbstractBatchHandler.java:311)
> at 
> org.hisp.dhis.dxf2.datavalueset.DefaultDataValueSetService.saveDataValueSet(DefaultDataValueSetService.java:613)
> at 
> org.hisp.dhis.dxf2.datavalueset.DefaultDataValueSetService.saveDataValueSetCsv(DefaultDataValueSetService.java:388)
> at 
> org.hisp.dhis.importexport.action.util.ImportDataValueTask.run(ImportDataValueTask.java:78)
> at 
> org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:53)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value 
> violates unique constraint "datavalue_pkey"
>   Detail: Key (dataelementid, periodid, sourceid, categoryoptioncomboid, 
> attributeoptioncomboid)=(2443, 2079, 59, 16, 16) already exists.
> at 
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2161)
> at 
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1890)
> at 
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
> at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:560)
> at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.executeWi

Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug

[Lars Helge Øverland]

Hi Robin,

usually the problem is internal duplicates in the import file like Olav
says.

Importing piece by piece is not really proof in this case - if the
duplicates are spread across two pieces then the second piece will simply
update the record imported in the first instead of crashing.

We do imports in SQL bulks to improve performance and hence its a bit
tricky to check for duplicates.

regards,

Lars







On Tue, Sep 16, 2014 at 3:59 PM, Robin Martens  wrote:

>  Hi Olav,
>
>
>
> Thanks for the answer but I don't think that's the issue.
>
>
>
> First of all, when checking my CSV with Excel's "Duplicate data"
> functionality there's no duplicates found.
>
> Secondly, when importing the same CSV file but piece by piece (per data
> element-catoptcombo set) I don't have this issue.
>
>
>
> Regards,
>
>
>
> Robin
>
>
>
> *From:* Olav Poppe [mailto:olav.po...@me.com]
> *Sent:* 16 September 2014 15:26
> *To:* Robin Martens
> *Cc:* dhis2-devs@lists.launchpad.net
> *Subject:* Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug
>
>
>
> Hi,
>
> I see that your log file actually confirms this:
>
> Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value
> violates unique constraint "datavalue_pkey"
>
>   Detail: Key (dataelementid, periodid, sourceid, categoryoptioncomboid,
> attributeoptioncomboid)=(2443, 2079, 59, 16, 16) already exists.
>
>
>
> Olav
>
>
>
>
>
>
>
>  16. sep. 2014 kl. 15.23 skrev Olav Poppe :
>
>
>
> Hi,
>
> my experience is that when this happens (dry run works, but actual import
> fails with an error similar to this), it is *usually* caused by duplicate
> values in the source file i.e. multiple values with the same orgunit, data
> element, category and period.
>
>
>
> Olav
>
>
>
>
>
>
>
>  16. sep. 2014 kl. 14.22 skrev Robin Martens :
>
>
>
> Hi devs,
>
>
>
> I'm having what seems like a bug with the import module:
>
>
>
> When importing data from CSV, I get the following error message and the
> batch doesn't run. However, this only happens when importing a data element
> over different categoryoptioncombos and only when it is new data, not
> updates. In other words, I'm getting my import done piece by piece, i.e.
> per data element/categoryoptioncombo combination.
>
>
>
> Also note that the import dry run doesn't detect any issues.
>
>
>
> Any idea how to solve this?
>
>
>
> Kind regards,
>
>
>
> Robin
>
>
>
>
>
>
>
> * INFO  2014-09-16 13:47:11,377 [Level: INFO, category: DATAVALUE_IMPORT,
> time: Tue Sep 16 13:47:11 CAT 2014, message: Process started]
> (InMemoryNotifier.java [taskScheduler-6])
>
> * INFO  2014-09-16 13:47:11,469 [Level: INFO, category: DATAVALUE_IMPORT,
> time: Tue Sep 16 13:47:11 CAT 2014, message: Importing data values]
> (InMemoryNotifier.java [taskScheduler-6])
>
> * INFO  2014-09-16 13:47:11,470 importing data values
> (DefaultDataValueSetService.java [taskScheduler-6])
>
> * ERROR 2014-09-16 13:47:12,396 java.lang.RuntimeException: Failed to
> flush BatchHandler
>
> at
> org.amplecode.quick.batchhandler.AbstractBatchHandler.flush(AbstractBatchHandler.java:311)
>
> at
> org.hisp.dhis.dxf2.datavalueset.DefaultDataValueSetService.saveDataValueSet(DefaultDataValueSetService.java:613)
>
> at
> org.hisp.dhis.dxf2.datavalueset.DefaultDataValueSetService.saveDataValueSetCsv(DefaultDataValueSetService.java:388)
>
> at
> org.hisp.dhis.importexport.action.util.ImportDataValueTask.run(ImportDataValueTask.java:78)
>
> at
> org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:53)
>
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:744)
>
> Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value
> violates unique constraint "datavalue_pkey"
>
>   Detail: Key (dataelementid, periodid, sourceid, categoryoptioncomboid,
> attributeoptioncomboid)=(2443, 2079, 59, 16, 16) already exists.
>
> at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2161)
>
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1890)
>
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
>
> at
> org.postgresql.jdbc2.Ab

Re: [Dhis2-devs] Tracker - Patient dashboard problem

[Tran Chau]

Hi Sumudu,

Could you please run these statements below in your database directly to
fix this bug ? Then restart tomcat again ?

*ALTER TABLE programstageinstance MODIFY status INTEGER;*
*UPDATE programstageinstance SET status='ACTIVE' WHERE status='0';*
*UPDATE programstageinstance SET status='COMPLETED' WHERE status='1';*
*UPDATE programstageinstance SET status='SKIPPED' WHERE status='5';*


Best regards,

Châu Thu Trân
HISP Viet Nam
Email: tran.hispviet...@gmail.com

On Tue, Sep 16, 2014 at 8:56 PM, sumudu weerasinghe 
wrote:

> Sorry, my system detail as follow
> *Version: 2.16**Build revision:16414**Database type:MySQL*
> *Java version:1.7.0_04**OS name:Windows 7*
>
>
> On Tue, Sep 16, 2014 at 7:23 PM, sumudu weerasinghe 
> wrote:
>
>> Dear all ,
>> I have updated my instance from 2.15 to 2.16  over the old mysql
>> database. Then i list the clinic patient . But it was unable to go for
>> patient dashboard through the list and give error as exception occur (shown
>> in screen shot). Log also show some error. Log and screenshots are attached
>> here.
>>
>> Regards,
>> Sumudu
>>
>
>
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug

[Jason Pickering]

Hi Robin,

I have seem this happen in Excel before. AFAIK, there is no way to check
for discordant duplicates, which have the same orgunit/period/data
element/category combo but different values. Excel will not detect these as
being duplicates, if you include the value field as part of the criteria.
It will only work if the values are different. I suspect in your case, you
have values which are not the same, and thus, the import will fail as a
single batch but possibly work as multiple batches, depending on whether or
not the duplicates happen to be part of different batches.

Regards,
Jason


On Tue, Sep 16, 2014 at 4:08 PM, Lars Helge Øverland 
wrote:

> Hi Robin,
>
> usually the problem is internal duplicates in the import file like Olav
> says.
>
> Importing piece by piece is not really proof in this case - if the
> duplicates are spread across two pieces then the second piece will simply
> update the record imported in the first instead of crashing.
>
> We do imports in SQL bulks to improve performance and hence its a bit
> tricky to check for duplicates.
>
> regards,
>
> Lars
>
>
>
>
>
>
>
> On Tue, Sep 16, 2014 at 3:59 PM, Robin Martens  wrote:
>
>>  Hi Olav,
>>
>>
>>
>> Thanks for the answer but I don't think that's the issue.
>>
>>
>>
>> First of all, when checking my CSV with Excel's "Duplicate data"
>> functionality there's no duplicates found.
>>
>> Secondly, when importing the same CSV file but piece by piece (per data
>> element-catoptcombo set) I don't have this issue.
>>
>>
>>
>> Regards,
>>
>>
>>
>> Robin
>>
>>
>>
>> *From:* Olav Poppe [mailto:olav.po...@me.com]
>> *Sent:* 16 September 2014 15:26
>> *To:* Robin Martens
>> *Cc:* dhis2-devs@lists.launchpad.net
>> *Subject:* Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug
>>
>>
>>
>> Hi,
>>
>> I see that your log file actually confirms this:
>>
>> Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value
>> violates unique constraint "datavalue_pkey"
>>
>>   Detail: Key (dataelementid, periodid, sourceid, categoryoptioncomboid,
>> attributeoptioncomboid)=(2443, 2079, 59, 16, 16) already exists.
>>
>>
>>
>> Olav
>>
>>
>>
>>
>>
>>
>>
>>  16. sep. 2014 kl. 15.23 skrev Olav Poppe :
>>
>>
>>
>> Hi,
>>
>> my experience is that when this happens (dry run works, but actual import
>> fails with an error similar to this), it is *usually* caused by
>> duplicate values in the source file i.e. multiple values with the same
>> orgunit, data element, category and period.
>>
>>
>>
>> Olav
>>
>>
>>
>>
>>
>>
>>
>>  16. sep. 2014 kl. 14.22 skrev Robin Martens :
>>
>>
>>
>> Hi devs,
>>
>>
>>
>> I'm having what seems like a bug with the import module:
>>
>>
>>
>> When importing data from CSV, I get the following error message and the
>> batch doesn't run. However, this only happens when importing a data element
>> over different categoryoptioncombos and only when it is new data, not
>> updates. In other words, I'm getting my import done piece by piece, i.e.
>> per data element/categoryoptioncombo combination.
>>
>>
>>
>> Also note that the import dry run doesn't detect any issues.
>>
>>
>>
>> Any idea how to solve this?
>>
>>
>>
>> Kind regards,
>>
>>
>>
>> Robin
>>
>>
>>
>>
>>
>>
>>
>> * INFO  2014-09-16 13:47:11,377 [Level: INFO, category: DATAVALUE_IMPORT,
>> time: Tue Sep 16 13:47:11 CAT 2014, message: Process started]
>> (InMemoryNotifier.java [taskScheduler-6])
>>
>> * INFO  2014-09-16 13:47:11,469 [Level: INFO, category: DATAVALUE_IMPORT,
>> time: Tue Sep 16 13:47:11 CAT 2014, message: Importing data values]
>> (InMemoryNotifier.java [taskScheduler-6])
>>
>> * INFO  2014-09-16 13:47:11,470 importing data values
>> (DefaultDataValueSetService.java [taskScheduler-6])
>>
>> * ERROR 2014-09-16 13:47:12,396 java.lang.RuntimeException: Failed to
>> flush BatchHandler
>>
>> at
>> org.amplecode.quick.batchhandler.AbstractBatchHandler.flush(AbstractBatchHandler.java:311)
>>
>> at
>> org.hisp.dhis.dxf2.datavalueset.DefaultDataValueSetService.saveDataValueSet(DefaultDataValueSetService.java:613)
>>
>> at
>> org.hisp.dhis.dxf2.datavalueset.DefaultDataValueSetService.saveDataValueSetCsv(DefaultDataValueSetService.java:388)
>>
>> at
>> org.hisp.dhis.importexport.action.util.ImportDataValueTask.run(ImportDataValueTask.java:78)
>>
>> at
>> org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:53)
>>
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>
>> at
>> java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>>
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>>
>> at
>> java.util.concurrent.ThreadPoolEx

Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug

[Bob Jolliffe]

I can also confirm that duplicates are the most likely cause of this
(having hit it quite a few times myself).  We should consider catching this
exception and reporting such to the user ...

On 16 September 2014 15:31, Jason Pickering 
wrote:

> Hi Robin,
>
> I have seem this happen in Excel before. AFAIK, there is no way to check
> for discordant duplicates, which have the same orgunit/period/data
> element/category combo but different values. Excel will not detect these as
> being duplicates, if you include the value field as part of the criteria.
> It will only work if the values are different. I suspect in your case, you
> have values which are not the same, and thus, the import will fail as a
> single batch but possibly work as multiple batches, depending on whether or
> not the duplicates happen to be part of different batches.
>
> Regards,
> Jason
>
>
> On Tue, Sep 16, 2014 at 4:08 PM, Lars Helge Øverland 
> wrote:
>
>> Hi Robin,
>>
>> usually the problem is internal duplicates in the import file like Olav
>> says.
>>
>> Importing piece by piece is not really proof in this case - if the
>> duplicates are spread across two pieces then the second piece will simply
>> update the record imported in the first instead of crashing.
>>
>> We do imports in SQL bulks to improve performance and hence its a bit
>> tricky to check for duplicates.
>>
>> regards,
>>
>> Lars
>>
>>
>>
>>
>>
>>
>>
>> On Tue, Sep 16, 2014 at 3:59 PM, Robin Martens  wrote:
>>
>>>  Hi Olav,
>>>
>>>
>>>
>>> Thanks for the answer but I don't think that's the issue.
>>>
>>>
>>>
>>> First of all, when checking my CSV with Excel's "Duplicate data"
>>> functionality there's no duplicates found.
>>>
>>> Secondly, when importing the same CSV file but piece by piece (per data
>>> element-catoptcombo set) I don't have this issue.
>>>
>>>
>>>
>>> Regards,
>>>
>>>
>>>
>>> Robin
>>>
>>>
>>>
>>> *From:* Olav Poppe [mailto:olav.po...@me.com]
>>> *Sent:* 16 September 2014 15:26
>>> *To:* Robin Martens
>>> *Cc:* dhis2-devs@lists.launchpad.net
>>> *Subject:* Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug
>>>
>>>
>>>
>>> Hi,
>>>
>>> I see that your log file actually confirms this:
>>>
>>> Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value
>>> violates unique constraint "datavalue_pkey"
>>>
>>>   Detail: Key (dataelementid, periodid, sourceid, categoryoptioncomboid,
>>> attributeoptioncomboid)=(2443, 2079, 59, 16, 16) already exists.
>>>
>>>
>>>
>>> Olav
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>  16. sep. 2014 kl. 15.23 skrev Olav Poppe :
>>>
>>>
>>>
>>> Hi,
>>>
>>> my experience is that when this happens (dry run works, but actual
>>> import fails with an error similar to this), it is *usually* caused by
>>> duplicate values in the source file i.e. multiple values with the same
>>> orgunit, data element, category and period.
>>>
>>>
>>>
>>> Olav
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>  16. sep. 2014 kl. 14.22 skrev Robin Martens :
>>>
>>>
>>>
>>> Hi devs,
>>>
>>>
>>>
>>> I'm having what seems like a bug with the import module:
>>>
>>>
>>>
>>> When importing data from CSV, I get the following error message and the
>>> batch doesn't run. However, this only happens when importing a data element
>>> over different categoryoptioncombos and only when it is new data, not
>>> updates. In other words, I'm getting my import done piece by piece, i.e.
>>> per data element/categoryoptioncombo combination.
>>>
>>>
>>>
>>> Also note that the import dry run doesn't detect any issues.
>>>
>>>
>>>
>>> Any idea how to solve this?
>>>
>>>
>>>
>>> Kind regards,
>>>
>>>
>>>
>>> Robin
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> * INFO  2014-09-16 13:47:11,377 [Level: INFO, category:
>>> DATAVALUE_IMPORT, time: Tue Sep 16 13:47:11 CAT 2014, message: Process
>>> started] (InMemoryNotifier.java [taskScheduler-6])
>>>
>>> * INFO  2014-09-16 13:47:11,469 [Level: INFO, category:
>>> DATAVALUE_IMPORT, time: Tue Sep 16 13:47:11 CAT 2014, message: Importing
>>> data values] (InMemoryNotifier.java [taskScheduler-6])
>>>
>>> * INFO  2014-09-16 13:47:11,470 importing data values
>>> (DefaultDataValueSetService.java [taskScheduler-6])
>>>
>>> * ERROR 2014-09-16 13:47:12,396 java.lang.RuntimeException: Failed to
>>> flush BatchHandler
>>>
>>> at
>>> org.amplecode.quick.batchhandler.AbstractBatchHandler.flush(AbstractBatchHandler.java:311)
>>>
>>> at
>>> org.hisp.dhis.dxf2.datavalueset.DefaultDataValueSetService.saveDataValueSet(DefaultDataValueSetService.java:613)
>>>
>>> at
>>> org.hisp.dhis.dxf2.datavalueset.DefaultDataValueSetService.saveDataValueSetCsv(DefaultDataValueSetService.java:388)
>>>
>>> at
>>> org.hisp.dhis.importexport.action.util.ImportDataValueTask.run(ImportDataValueTask.java:78)
>>>
>>> at
>>> org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:53)
>>>
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executor

Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug

[Robin Martens]

Hi all,

I understand your comments, however I'm pretty sure there are no duplicates. I 
put the CSV file in annex, please have a look for yourself. Note that I used 
the "code" setting for both data elements and orgunits.

About Excel: when you select only the columns in which you want to find 
duplicates (in this case column A to D), it only looks at these values without 
looking at the values (in column F). No duplicates found. Can it be DHIS2 
considers them as duplicates only based on the fields of dataelement, period, 
and orgunit (without the catoptcomboid field)?

SQL importing would be an alternative but is much less user-friendly.

Regards,

Robin

From: Jason Pickering [mailto:jason.p.picker...@gmail.com]
Sent: 16 September 2014 16:31
To: Lars Helge Øverland
Cc: Robin Martens; dhis2-devs@lists.launchpad.net
Subject: Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug

Hi Robin,

I have seem this happen in Excel before. AFAIK, there is no way to check for 
discordant duplicates, which have the same orgunit/period/data element/category 
combo but different values. Excel will not detect these as being duplicates, if 
you include the value field as part of the criteria. It will only work if the 
values are different. I suspect in your case, you have values which are not the 
same, and thus, the import will fail as a single batch but possibly work as 
multiple batches, depending on whether or not the duplicates happen to be part 
of different batches.

Regards,
Jason


On Tue, Sep 16, 2014 at 4:08 PM, Lars Helge Øverland 
mailto:larshe...@gmail.com>> wrote:
Hi Robin,

usually the problem is internal duplicates in the import file like Olav says.

Importing piece by piece is not really proof in this case - if the duplicates 
are spread across two pieces then the second piece will simply update the 
record imported in the first instead of crashing.

We do imports in SQL bulks to improve performance and hence its a bit tricky to 
check for duplicates.

regards,

Lars







On Tue, Sep 16, 2014 at 3:59 PM, Robin Martens 
mailto:mart...@sher.be>> wrote:
Hi Olav,

Thanks for the answer but I don't think that's the issue.

First of all, when checking my CSV with Excel's "Duplicate data" functionality 
there's no duplicates found.
Secondly, when importing the same CSV file but piece by piece (per data 
element-catoptcombo set) I don't have this issue.

Regards,

Robin

From: Olav Poppe [mailto:olav.po...@me.com]
Sent: 16 September 2014 15:26
To: Robin Martens
Cc: dhis2-devs@lists.launchpad.net
Subject: Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug

Hi,
I see that your log file actually confirms this:
Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value 
violates unique constraint "datavalue_pkey"
  Detail: Key (dataelementid, periodid, sourceid, categoryoptioncomboid, 
attributeoptioncomboid)=(2443, 2079, 59, 16, 16) already exists.

Olav



16. sep. 2014 kl. 15.23 skrev Olav Poppe 
mailto:olav.po...@me.com>>:

Hi,
my experience is that when this happens (dry run works, but actual import fails 
with an error similar to this), it is usually caused by duplicate values in the 
source file i.e. multiple values with the same orgunit, data element, category 
and period.

Olav



16. sep. 2014 kl. 14.22 skrev Robin Martens 
mailto:mart...@sher.be>>:

Hi devs,

I'm having what seems like a bug with the import module:

When importing data from CSV, I get the following error message and the batch 
doesn't run. However, this only happens when importing a data element over 
different categoryoptioncombos and only when it is new data, not updates. In 
other words, I'm getting my import done piece by piece, i.e. per data 
element/categoryoptioncombo combination.

Also note that the import dry run doesn't detect any issues.

Any idea how to solve this?

Kind regards,

Robin



* INFO  2014-09-16 13:47:11,377 [Level: INFO, category: DATAVALUE_IMPORT, time: 
Tue Sep 16 13:47:11 CAT 2014, message: Process started] (InMemoryNotifier.java 
[taskScheduler-6])
* INFO  2014-09-16 13:47:11,469 [Level: INFO, category: DATAVALUE_IMPORT, time: 
Tue Sep 16 13:47:11 CAT 2014, message: Importing data values] 
(InMemoryNotifier.java [taskScheduler-6])
* INFO  2014-09-16 13:47:11,470 importing data values 
(DefaultDataValueSetService.java [taskScheduler-6])
* ERROR 2014-09-16 13:47:12,396 java.lang.RuntimeException: Failed to flush 
BatchHandler
at 
org.amplecode.quick.batchhandler.AbstractBatchHandler.flush(AbstractBatchHandler.java:311)
at 
org.hisp.dhis.dxf2.datavalueset.DefaultDataValueSetService.saveDataValueSet(DefaultDataValueSetService.java:613)
at 
org.hisp.dhis.dxf2.datavalueset.DefaultDataValueSetService.saveDataValueSetCsv(DefaultDataValueSetService.java:388)
at 
org.hisp.dhis.importexport.action.util.ImportDataValueTask.run(ImportDataValueTask.java:78)
at 
or

Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug

[Jason Pickering]

Hi Robin,

I agree, it seems, you have no duplicates here, but the spec for the CSV
file is different I think. Are you not using UIDs?

It should look something like this.

"dataelement","period","orgunit","categoryoptioncombo","attributeoptioncombo","value","storedby","timestamp","comment","followup"
"DUSpd8Jq3M7","201202","gP6hn503KUX","Prlt0C1RF0s",,"7","bombali","2010-04-17",,"false"
"DUSpd8Jq3M7","201202","gP6hn503KUX","V6L425pT3A0",,"10","bombali","2010-04-17",,"false"
"DUSpd8Jq3M7","201202","OjTS752GbZE","V6L425pT3A0",,"9","bombali","2010-04-06",,"false"

Direct injection with SQL is of course also possible by transforming this
file (which appears to use internal IDs) into insert statements.

Regards,
Jason


On Tue, Sep 16, 2014 at 4:47 PM, Robin Martens  wrote:

>  Hi all,
>
>
>
> I understand your comments, however I'm pretty sure there are no
> duplicates. I put the CSV file in annex, please have a look for yourself.
> Note that I used the "code" setting for both data elements and orgunits.
>
>
>
> About Excel: when you select only the columns in which you want to find
> duplicates (in this case column A to D), it only looks at these values
> without looking at the values (in column F). No duplicates found. Can it be
> DHIS2 considers them as duplicates only based on the fields of dataelement,
> period, and orgunit (without the catoptcomboid field)?
>
>
>
> SQL importing would be an alternative but is much less user-friendly.
>
>
>
> Regards,
>
>
>
> Robin
>
>
>
> *From:* Jason Pickering [mailto:jason.p.picker...@gmail.com]
> *Sent:* 16 September 2014 16:31
> *To:* Lars Helge Øverland
> *Cc:* Robin Martens; dhis2-devs@lists.launchpad.net
>
> *Subject:* Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug
>
>
>
> Hi Robin,
>
>
>
> I have seem this happen in Excel before. AFAIK, there is no way to check
> for discordant duplicates, which have the same orgunit/period/data
> element/category combo but different values. Excel will not detect these as
> being duplicates, if you include the value field as part of the criteria.
> It will only work if the values are different. I suspect in your case, you
> have values which are not the same, and thus, the import will fail as a
> single batch but possibly work as multiple batches, depending on whether or
> not the duplicates happen to be part of different batches.
>
>
>
> Regards,
>
> Jason
>
>
>
>
>
> On Tue, Sep 16, 2014 at 4:08 PM, Lars Helge Øverland 
> wrote:
>
> Hi Robin,
>
>
>
> usually the problem is internal duplicates in the import file like Olav
> says.
>
>
>
> Importing piece by piece is not really proof in this case - if the
> duplicates are spread across two pieces then the second piece will simply
> update the record imported in the first instead of crashing.
>
>
>
> We do imports in SQL bulks to improve performance and hence its a bit
> tricky to check for duplicates.
>
>
>
> regards,
>
>
>
> Lars
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Tue, Sep 16, 2014 at 3:59 PM, Robin Martens  wrote:
>
> Hi Olav,
>
>
>
> Thanks for the answer but I don't think that's the issue.
>
>
>
> First of all, when checking my CSV with Excel's "Duplicate data"
> functionality there's no duplicates found.
>
> Secondly, when importing the same CSV file but piece by piece (per data
> element-catoptcombo set) I don't have this issue.
>
>
>
> Regards,
>
>
>
> Robin
>
>
>
> *From:* Olav Poppe [mailto:olav.po...@me.com]
> *Sent:* 16 September 2014 15:26
> *To:* Robin Martens
> *Cc:* dhis2-devs@lists.launchpad.net
> *Subject:* Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug
>
>
>
> Hi,
>
> I see that your log file actually confirms this:
>
> Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value
> violates unique constraint "datavalue_pkey"
>
>   Detail: Key (dataelementid, periodid, sourceid, categoryoptioncomboid,
> attributeoptioncomboid)=(2443, 2079, 59, 16, 16) already exists.
>
>
>
> Olav
>
>
>
>
>
>
>
>  16. sep. 2014 kl. 15.23 skrev Olav Poppe :
>
>
>
> Hi,
>
> my experience is that when this happens (dry run works, but actual import
> fails with an error similar to this), it is *usually* caused by duplicate
> values in the source file i.e. multiple values with the same orgunit, data
> element, category and period.
>
>
>
> Olav
>
>
>
>
>
>
>
>  16. sep. 2014 kl. 14.22 skrev Robin Martens :
>
>
>
> Hi devs,
>
>
>
> I'm having what seems like a bug with the import module:
>
>
>
> When importing data from CSV, I get the following error message and the
> batch doesn't run. However, this only happens when importing a data element
> over different categoryoptioncombos and only when it is new data, not
> updates. In other words, I'm getting my import done piece by piece, i.e.
> per data element/categoryoptioncombo combination.
>
>
>
> Also note that the import dry run doesn't detect any issues.
>
>
>
> Any idea how to solve this?
>
>
>
> Kind regards,
>
>
>
> Robin
>
>
>
>
>
>
>
> * INFO  2014-09-16 13:47:11,377 [Level: INFO, category: DATAVALUE_IMPORT,

Re: [Dhis2-devs] Tracker - Patient dashboard problem

[sumudu weerasinghe]

Hi Tran,
I tried it but there is error in some query. My database "status" is a
integer.
I got following error?















*mysql> ALTER TABLE programstageinstance MODIFY status INTEGER;Query OK, 0
rows affected (0.10 sec)Records: 0  Duplicates: 0  Warnings: 0mysql> UPDATE
programstageinstance SET status='ACTIVE' WHERE status='0';ERROR 1366
(HY000): Incorrect integer value: 'ACTIVE' for column 'status' at
row 2mysql> UPDATE programstageinstance SET status='COMPLETED' WHERE
status='1';ERROR 1366 (HY000): Incorrect integer value: 'COMPLETED' for
column 'status' atrow 1mysql> UPDATE programstageinstance SET
status='SKIPPED' WHERE status='5';Query OK, 0 rows affected (0.00 sec)Rows
matched: 0  Changed: 0  Warnings: 0mysql>*

Sumudu


On Tue, Sep 16, 2014 at 7:47 PM, Tran Chau 
wrote:

> Hi Sumudu,
>
> Could you please run these statements below in your database directly to
> fix this bug ? Then restart tomcat again ?
>
> *ALTER TABLE programstageinstance MODIFY status INTEGER;*
> *UPDATE programstageinstance SET status='ACTIVE' WHERE status='0';*
> *UPDATE programstageinstance SET status='COMPLETED' WHERE status='1';*
> *UPDATE programstageinstance SET status='SKIPPED' WHERE status='5';*
>
>
> Best regards,
> 
> Châu Thu Trân
> HISP Viet Nam
> Email: tran.hispviet...@gmail.com
>
> On Tue, Sep 16, 2014 at 8:56 PM, sumudu weerasinghe 
> wrote:
>
>> Sorry, my system detail as follow
>> *Version: 2.16**Build revision:16414**Database type:MySQL*
>> *Java version:1.7.0_04**OS name:Windows 7*
>>
>>
>> On Tue, Sep 16, 2014 at 7:23 PM, sumudu weerasinghe 
>> wrote:
>>
>>> Dear all ,
>>> I have updated my instance from 2.15 to 2.16  over the old mysql
>>> database. Then i list the clinic patient . But it was unable to go for
>>> patient dashboard through the list and give error as exception occur (shown
>>> in screen shot). Log also show some error. Log and screenshots are attached
>>> here.
>>>
>>> Regards,
>>> Sumudu
>>>
>>
>>
>
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug

[Robin Martens]

Hi Jason,

Thanks for confirming on the duplicates. I've got the mistake now.

I'm using the CSV importer of DHIS2 itself, where you can set the "Data element 
ID scheme" and "Org unit ID scheme" under "more options". I used "code" for 
both. However, I used the PK for catoptcombos instead of the ID which is 
probably why DHIS2 didn't recognize them as different entries (invalid 
catoptcombo or something). Now it works fine.

Thanks guys!

Robin

From: Jason Pickering [mailto:jason.p.picker...@gmail.com]
Sent: 16 September 2014 17:28
To: Robin Martens
Cc: Lars Helge Øverland; dhis2-devs@lists.launchpad.net
Subject: Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug

Hi Robin,

I agree, it seems, you have no duplicates here, but the spec for the CSV file 
is different I think. Are you not using UIDs?

It should look something like this.


"dataelement","period","orgunit","categoryoptioncombo","attributeoptioncombo","value","storedby","timestamp","comment","followup"

"DUSpd8Jq3M7","201202","gP6hn503KUX","Prlt0C1RF0s",,"7","bombali","2010-04-17",,"false"

"DUSpd8Jq3M7","201202","gP6hn503KUX","V6L425pT3A0",,"10","bombali","2010-04-17",,"false"

"DUSpd8Jq3M7","201202","OjTS752GbZE","V6L425pT3A0",,"9","bombali","2010-04-06",,"false"
Direct injection with SQL is of course also possible by transforming this file 
(which appears to use internal IDs) into insert statements.

Regards,
Jason


On Tue, Sep 16, 2014 at 4:47 PM, Robin Martens 
mailto:mart...@sher.be>> wrote:
Hi all,

I understand your comments, however I'm pretty sure there are no duplicates. I 
put the CSV file in annex, please have a look for yourself. Note that I used 
the "code" setting for both data elements and orgunits.

About Excel: when you select only the columns in which you want to find 
duplicates (in this case column A to D), it only looks at these values without 
looking at the values (in column F). No duplicates found. Can it be DHIS2 
considers them as duplicates only based on the fields of dataelement, period, 
and orgunit (without the catoptcomboid field)?

SQL importing would be an alternative but is much less user-friendly.

Regards,

Robin

From: Jason Pickering 
[mailto:jason.p.picker...@gmail.com]
Sent: 16 September 2014 16:31
To: Lars Helge Øverland
Cc: Robin Martens; 
dhis2-devs@lists.launchpad.net

Subject: Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug

Hi Robin,

I have seem this happen in Excel before. AFAIK, there is no way to check for 
discordant duplicates, which have the same orgunit/period/data element/category 
combo but different values. Excel will not detect these as being duplicates, if 
you include the value field as part of the criteria. It will only work if the 
values are different. I suspect in your case, you have values which are not the 
same, and thus, the import will fail as a single batch but possibly work as 
multiple batches, depending on whether or not the duplicates happen to be part 
of different batches.

Regards,
Jason


On Tue, Sep 16, 2014 at 4:08 PM, Lars Helge Øverland 
mailto:larshe...@gmail.com>> wrote:
Hi Robin,

usually the problem is internal duplicates in the import file like Olav says.

Importing piece by piece is not really proof in this case - if the duplicates 
are spread across two pieces then the second piece will simply update the 
record imported in the first instead of crashing.

We do imports in SQL bulks to improve performance and hence its a bit tricky to 
check for duplicates.

regards,

Lars







On Tue, Sep 16, 2014 at 3:59 PM, Robin Martens 
mailto:mart...@sher.be>> wrote:
Hi Olav,

Thanks for the answer but I don't think that's the issue.

First of all, when checking my CSV with Excel's "Duplicate data" functionality 
there's no duplicates found.
Secondly, when importing the same CSV file but piece by piece (per data 
element-catoptcombo set) I don't have this issue.

Regards,

Robin

From: Olav Poppe [mailto:olav.po...@me.com]
Sent: 16 September 2014 15:26
To: Robin Martens
Cc: dhis2-devs@lists.launchpad.net
Subject: Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug

Hi,
I see that your log file actually confirms this:
Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value 
violates unique constraint "datavalue_pkey"
  Detail: Key (dataelementid, periodid, sourceid, categoryoptioncomboid, 
attributeoptioncomboid)=(2443, 2079, 59, 16, 16) already exists.

Olav



16. sep. 2014 kl. 15.23 skrev Olav Poppe 
mailto:olav.po...@me.com>>:

Hi,
my experience is that when this happens (dry run works, but actual import fails 
with an error similar to this), it is usually caused by duplicate values in the 
source file i.e. multiple values with the same orgunit, data element, category 
and period.

Olav



16. sep. 2014 kl. 14.22 skrev Robin Martens 
mailto:mart...@sher.be>>:

Hi devs,

I'm having what seems like a bu

Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug

[Bob Jolliffe]

Glad that is sorted.  You are right the catoptcombo doesn't currently
support a code.  Its something we should look at.

I'm not sure why the importer threw the duplicate key exception rather than
something more helpful like unknown or invalid catoptcombo.  Maybe Lars
would have a better idea.  I guess it is something to do with the way that
column is parsed that reduces it to some common value?

On 16 September 2014 16:40, Robin Martens  wrote:

>  Hi Jason,
>
>
>
> Thanks for confirming on the duplicates. I've got the mistake now.
>
>
>
> I'm using the CSV importer of DHIS2 itself, where you can set the "Data
> element ID scheme" and "Org unit ID scheme" under "more options". I used
> "code" for both. However, I used the PK for catoptcombos instead of the ID
> which is probably why DHIS2 didn't recognize them as different entries
> (invalid catoptcombo or something). Now it works fine.
>
>
>
> Thanks guys!
>
>
>
> Robin
>
>
>
> *From:* Jason Pickering [mailto:jason.p.picker...@gmail.com]
> *Sent:* 16 September 2014 17:28
> *To:* Robin Martens
> *Cc:* Lars Helge Øverland; dhis2-devs@lists.launchpad.net
>
> *Subject:* Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug
>
>
>
> Hi Robin,
>
>
>
> I agree, it seems, you have no duplicates here, but the spec for the CSV
> file is different I think. Are you not using UIDs?
>
>
>
> It should look something like this.
>
>
>
> "dataelement","period","orgunit","categoryoptioncombo","attributeoptioncombo","value","storedby","timestamp","comment","followup"
>
> "DUSpd8Jq3M7","201202","gP6hn503KUX","Prlt0C1RF0s",,"7","bombali","2010-04-17",,"false"
>
> "DUSpd8Jq3M7","201202","gP6hn503KUX","V6L425pT3A0",,"10","bombali","2010-04-17",,"false"
>
> "DUSpd8Jq3M7","201202","OjTS752GbZE","V6L425pT3A0",,"9","bombali","2010-04-06",,"false"
>
>   Direct injection with SQL is of course also possible by transforming
> this file (which appears to use internal IDs) into insert statements.
>
>
>
> Regards,
>
> Jason
>
>
>
>
>
> On Tue, Sep 16, 2014 at 4:47 PM, Robin Martens  wrote:
>
> Hi all,
>
>
>
> I understand your comments, however I'm pretty sure there are no
> duplicates. I put the CSV file in annex, please have a look for yourself.
> Note that I used the "code" setting for both data elements and orgunits.
>
>
>
> About Excel: when you select only the columns in which you want to find
> duplicates (in this case column A to D), it only looks at these values
> without looking at the values (in column F). No duplicates found. Can it be
> DHIS2 considers them as duplicates only based on the fields of dataelement,
> period, and orgunit (without the catoptcomboid field)?
>
>
>
> SQL importing would be an alternative but is much less user-friendly.
>
>
>
> Regards,
>
>
>
> Robin
>
>
>
> *From:* Jason Pickering [mailto:jason.p.picker...@gmail.com]
> *Sent:* 16 September 2014 16:31
> *To:* Lars Helge Øverland
> *Cc:* Robin Martens; dhis2-devs@lists.launchpad.net
>
>
> *Subject:* Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug
>
>
>
> Hi Robin,
>
>
>
> I have seem this happen in Excel before. AFAIK, there is no way to check
> for discordant duplicates, which have the same orgunit/period/data
> element/category combo but different values. Excel will not detect these as
> being duplicates, if you include the value field as part of the criteria.
> It will only work if the values are different. I suspect in your case, you
> have values which are not the same, and thus, the import will fail as a
> single batch but possibly work as multiple batches, depending on whether or
> not the duplicates happen to be part of different batches.
>
>
>
> Regards,
>
> Jason
>
>
>
>
>
> On Tue, Sep 16, 2014 at 4:08 PM, Lars Helge Øverland 
> wrote:
>
> Hi Robin,
>
>
>
> usually the problem is internal duplicates in the import file like Olav
> says.
>
>
>
> Importing piece by piece is not really proof in this case - if the
> duplicates are spread across two pieces then the second piece will simply
> update the record imported in the first instead of crashing.
>
>
>
> We do imports in SQL bulks to improve performance and hence its a bit
> tricky to check for duplicates.
>
>
>
> regards,
>
>
>
> Lars
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Tue, Sep 16, 2014 at 3:59 PM, Robin Martens  wrote:
>
> Hi Olav,
>
>
>
> Thanks for the answer but I don't think that's the issue.
>
>
>
> First of all, when checking my CSV with Excel's "Duplicate data"
> functionality there's no duplicates found.
>
> Secondly, when importing the same CSV file but piece by piece (per data
> element-catoptcombo set) I don't have this issue.
>
>
>
> Regards,
>
>
>
> Robin
>
>
>
> *From:* Olav Poppe [mailto:olav.po...@me.com]
> *Sent:* 16 September 2014 15:26
> *To:* Robin Martens
> *Cc:* dhis2-devs@lists.launchpad.net
> *Subject:* Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug
>
>
>
> Hi,
>
> I see that your log file actually confirms this:
>
> Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value
> violates unique const

Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug

[Lars Helge Øverland]

On Tue, Sep 16, 2014 at 6:29 PM, Bob Jolliffe  wrote:

> Glad that is sorted.  You are right the catoptcombo doesn't currently
> support a code.  Its something we should look at.
>

Actually trunk supports setting codes on category option combos, as well as
identifier schemes for data export as of Sunday ;)


>
> I'm not sure why the importer threw the duplicate key exception rather
> than something more helpful like unknown or invalid catoptcombo.  Maybe
> Lars would have a better idea.  I guess it is something to do with the way
> that column is parsed that reduces it to some common value?
>

Yes agree that we should give more helpful feedback to the user.

Lars

>
>
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug

[Robin Martens]

Damn, I was thinking about upgrading today... Well, good to know for next time! 
:)

From: Lars Helge Øverland [mailto:larshe...@gmail.com]
Sent: 16 September 2014 18:35
To: Bob Jolliffe
Cc: Robin Martens; dhis2-devs@lists.launchpad.net
Subject: Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug



On Tue, Sep 16, 2014 at 6:29 PM, Bob Jolliffe 
mailto:bobjolli...@gmail.com>> wrote:
Glad that is sorted.  You are right the catoptcombo doesn't currently support a 
code.  Its something we should look at.

Actually trunk supports setting codes on category option combos, as well as 
identifier schemes for data export as of Sunday ;)


I'm not sure why the importer threw the duplicate key exception rather than 
something more helpful like unknown or invalid catoptcombo.  Maybe Lars would 
have a better idea.  I guess it is something to do with the way that column is 
parsed that reduces it to some common value?

Yes agree that we should give more helpful feedback to the user.

Lars


___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug

[Robin Martens]

Ps: we're close to our project deadlines now so I just wanted to say I 
appreciated your fast replies of the last days.

Have a good night!

Robin

From: Lars Helge Øverland [mailto:larshe...@gmail.com]
Sent: 16 September 2014 18:35
To: Bob Jolliffe
Cc: Robin Martens; dhis2-devs@lists.launchpad.net
Subject: Re: [Dhis2-devs] DHIS2 - Import data from CSV - bug



On Tue, Sep 16, 2014 at 6:29 PM, Bob Jolliffe 
mailto:bobjolli...@gmail.com>> wrote:
Glad that is sorted.  You are right the catoptcombo doesn't currently support a 
code.  Its something we should look at.

Actually trunk supports setting codes on category option combos, as well as 
identifier schemes for data export as of Sunday ;)


I'm not sure why the importer threw the duplicate key exception rather than 
something more helpful like unknown or invalid catoptcombo.  Maybe Lars would 
have a better idea.  I guess it is something to do with the way that column is 
parsed that reduces it to some common value?

Yes agree that we should give more helpful feedback to the user.

Lars


___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

[Dhis2-devs] [Branch ~dhis2-devs-core/dhis2/trunk] Rev 16730: Model change. Added list of Indicators to Section. Impl maintenance UI.

[noreply]

revno: 16730
committer: Lars Helge Overland 
branch nick: dhis2
timestamp: Tue 2014-09-16 18:40:46 +0200
message:
  Model change. Added list of Indicators to Section. Impl maintenance UI.
added:
  
dhis-2/dhis-web/dhis-web-commons/src/main/java/org/hisp/dhis/commons/action/GetSectionIndicatorsAction.java
modified:
  dhis-2/dhis-api/src/main/java/org/hisp/dhis/dataset/Section.java
  
dhis-2/dhis-services/dhis-service-core/src/main/resources/org/hisp/dhis/dataset/hibernate/Section.hbm.xml
  
dhis-2/dhis-web/dhis-web-commons-resources/src/main/webapp/dhis-web-commons/ajax/jsonSection.vm
  
dhis-2/dhis-web/dhis-web-commons/src/main/java/org/hisp/dhis/commons/action/GetSectionDataElementsAction.java
  dhis-2/dhis-web/dhis-web-commons/src/main/resources/META-INF/dhis/beans.xml
  dhis-2/dhis-web/dhis-web-commons/src/main/resources/dhis-web-commons.xml
  
dhis-2/dhis-web/dhis-web-maintenance/dhis-web-maintenance-dataset/src/main/java/org/hisp/dhis/dataset/action/section/AddSectionAction.java
  
dhis-2/dhis-web/dhis-web-maintenance/dhis-web-maintenance-dataset/src/main/java/org/hisp/dhis/dataset/action/section/EditSectionAction.java
  
dhis-2/dhis-web/dhis-web-maintenance/dhis-web-maintenance-dataset/src/main/java/org/hisp/dhis/dataset/action/section/GetSectionAction.java
  
dhis-2/dhis-web/dhis-web-maintenance/dhis-web-maintenance-dataset/src/main/java/org/hisp/dhis/dataset/action/section/UpdateSectionAction.java
  
dhis-2/dhis-web/dhis-web-maintenance/dhis-web-maintenance-dataset/src/main/resources/META-INF/dhis/beans.xml
  
dhis-2/dhis-web/dhis-web-maintenance/dhis-web-maintenance-dataset/src/main/webapp/dhis-web-maintenance-dataset/addSection.vm
  
dhis-2/dhis-web/dhis-web-maintenance/dhis-web-maintenance-dataset/src/main/webapp/dhis-web-maintenance-dataset/editSection.vm
  
dhis-2/dhis-web/dhis-web-maintenance/dhis-web-maintenance-dataset/src/main/webapp/dhis-web-maintenance-dataset/javascript/addSection.js
  
dhis-2/dhis-web/dhis-web-maintenance/dhis-web-maintenance-dataset/src/main/webapp/dhis-web-maintenance-dataset/javascript/editSection.js


--
lp:dhis2
https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk

Your team DHIS 2 developers is subscribed to branch lp:dhis2.
To unsubscribe from this branch go to 
https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk/+edit-subscription
=== modified file 'dhis-2/dhis-api/src/main/java/org/hisp/dhis/dataset/Section.java'
--- dhis-2/dhis-api/src/main/java/org/hisp/dhis/dataset/Section.java	2014-08-15 07:40:20 +
+++ dhis-2/dhis-api/src/main/java/org/hisp/dhis/dataset/Section.java	2014-09-16 16:40:46 +
@@ -42,6 +42,7 @@
 import org.hisp.dhis.dataelement.DataElement;
 import org.hisp.dhis.dataelement.DataElementCategoryCombo;
 import org.hisp.dhis.dataelement.DataElementOperand;
+import org.hisp.dhis.indicator.Indicator;
 
 import com.fasterxml.jackson.annotation.JsonProperty;
 import com.fasterxml.jackson.annotation.JsonView;
@@ -66,6 +67,9 @@
 @Scanned
 private List dataElements = new ArrayList<>();
 
+@Scanned
+private List indicators = new ArrayList<>();
+
 private Set greyedFields = new HashSet<>();
 
 private int sortOrder;
@@ -215,6 +219,21 @@
 this.dataElements = dataElements;
 }
 
+@JsonProperty
+@JsonSerialize(contentAs = BaseIdentifiableObject.class)
+@JsonView({ DetailedView.class, ExportView.class })
+@JacksonXmlElementWrapper(localName = "indicators", namespace = DxfNamespaces.DXF_2_0)
+@JacksonXmlProperty(localName = "indicator", namespace = DxfNamespaces.DXF_2_0)
+public List getIndicators()
+{
+return indicators;
+}
+
+public void setIndicators( List indicators )
+{
+this.indicators = indicators;
+}
+
 public int getSortOrder()
 {
 return sortOrder;

=== modified file 'dhis-2/dhis-services/dhis-service-core/src/main/resources/org/hisp/dhis/dataset/hibernate/Section.hbm.xml'
--- dhis-2/dhis-services/dhis-service-core/src/main/resources/org/hisp/dhis/dataset/hibernate/Section.hbm.xml	2014-07-21 11:37:05 +
+++ dhis-2/dhis-services/dhis-service-core/src/main/resources/org/hisp/dhis/dataset/hibernate/Section.hbm.xml	2014-09-16 16:40:46 +
@@ -31,6 +31,14 @@
   foreign-key="fk_section_dataelementid" />
 
 
+
+  
+  
+  
+  
+
+
 
   
   

=== modified file 'dhis-2/dhis-web/dhis-web-commons-resources/src/main/webapp/dhis-web-commons/ajax/jsonSection.vm'
--- dhis-2/dhis-web/dhis-web-commons-resources/src/main/webapp/dhis-web-commons/ajax/jsonSection.vm	2014-06-13 07:55:17 +
+++ dhis-2/dhis-web/dhis-web-commons-resources/src/main/webapp/dhis-web-commons/ajax/jsonSection.vm	2014-09-16 16:40:46 +
@@ -5,6 +5,7 @@
 "name": "$!encoder.jsonEncode( ${section.displayName} )",
 "dataSet": "$!encoder.jsonEncode( ${section.dataSet.name} )",
 "categoryCombo": "$!encoder.jsonEncode( ${categoryCombo.name} )",
-	"dataEle

[Dhis2-devs] [Bug 1370241] [NEW] Memory leaks in while deploying dhis2

[John Eric]

Public bug reported:

The web application [/dhis] appears to have started a thread named
[SimpleAsyncTaskExecutor-64] but has failed to stop it. This is very
likely to create a memory leak.

** Affects: dhis2
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of DHIS 2
developers, which is subscribed to DHIS.
https://bugs.launchpad.net/bugs/1370241

Title:
  Memory leaks in while deploying dhis2

Status in DHIS 2:
  New

Bug description:
  The web application [/dhis] appears to have started a thread named
  [SimpleAsyncTaskExecutor-64] but has failed to stop it. This is very
  likely to create a memory leak.

To manage notifications about this bug go to:
https://bugs.launchpad.net/dhis2/+bug/1370241/+subscriptions

___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

[Dhis2-devs] [Branch ~dhis2-devs-core/dhis2/trunk] Rev 16731: Validation. Explicit printing of stacktrace for validation. Spring is poor at printing stacktrace...

[noreply]

revno: 16731
committer: Lars Helge Overland 
branch nick: dhis2
timestamp: Wed 2014-09-17 00:33:15 +0200
message:
  Validation. Explicit printing of stacktrace for validation. Spring is poor at 
printing stacktraces for async processes so needs some help.
modified:
  
dhis-2/dhis-services/dhis-service-core/src/main/java/org/hisp/dhis/validation/ValidatorThread.java


--
lp:dhis2
https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk

Your team DHIS 2 developers is subscribed to branch lp:dhis2.
To unsubscribe from this branch go to 
https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk/+edit-subscription
=== modified file 'dhis-2/dhis-services/dhis-service-core/src/main/java/org/hisp/dhis/validation/ValidatorThread.java'
--- dhis-2/dhis-services/dhis-service-core/src/main/java/org/hisp/dhis/validation/ValidatorThread.java	2014-08-15 07:40:20 +
+++ dhis-2/dhis-services/dhis-service-core/src/main/java/org/hisp/dhis/validation/ValidatorThread.java	2014-09-16 22:33:15 +
@@ -56,6 +56,7 @@
 import org.hisp.dhis.period.CalendarPeriodType;
 import org.hisp.dhis.period.Period;
 import org.hisp.dhis.period.PeriodType;
+import org.hisp.dhis.system.util.DebugUtils;
 import org.hisp.dhis.system.util.MathUtils;
 
 /**
@@ -87,6 +88,20 @@
 @Override
 public void run()
 {
+try
+{
+runInternal();
+}
+catch ( RuntimeException ex )
+{
+log.error( DebugUtils.getStackTrace( ex ) );
+
+throw ex;
+}
+}
+
+private void runInternal()
+{
 if ( context.getValidationResults().size() < ( ValidationRunType.INTERACTIVE == context.getRunType() ?
 ValidationRuleService.MAX_INTERACTIVE_ALERTS : ValidationRuleService.MAX_SCHEDULED_ALERTS) )
 {

___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

Re: [Dhis2-devs] Tracker - Patient dashboard problem

[Tran Chau]

Hi Sumudu,

I am sorry I sent wrong statements by mistake. The *status *column should
be change to* varchar(25)*.
Could you please run the statements below and re-start tomcat ?

*ALTER TABLE programstageinstance MODIFY status varchar(25);*
*UPDATE programstageinstance SET status='ACTIVE' WHERE status='0';*
*UPDATE programstageinstance SET status='COMPLETED' WHERE status='1';*
*UPDATE programstageinstance SET status='SKIPPED' WHERE status='5';*

Best regards,

Châu Thu Trân
HISP Viet Nam
Email: tran.hispviet...@gmail.com

On Tue, Sep 16, 2014 at 10:37 PM, sumudu weerasinghe 
wrote:

> Hi Tran,
> I tried it but there is error in some query. My database "status" is a
> integer.
> I got following error?
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *mysql> ALTER TABLE programstageinstance MODIFY status INTEGER;Query OK, 0
> rows affected (0.10 sec)Records: 0  Duplicates: 0  Warnings: 0mysql> UPDATE
> programstageinstance SET status='ACTIVE' WHERE status='0';ERROR 1366
> (HY000): Incorrect integer value: 'ACTIVE' for column 'status' at
> row 2mysql> UPDATE programstageinstance SET status='COMPLETED' WHERE
> status='1';ERROR 1366 (HY000): Incorrect integer value: 'COMPLETED' for
> column 'status' atrow 1mysql> UPDATE programstageinstance SET
> status='SKIPPED' WHERE status='5';Query OK, 0 rows affected (0.00 sec)Rows
> matched: 0  Changed: 0  Warnings: 0mysql>*
>
> Sumudu
>
>
> On Tue, Sep 16, 2014 at 7:47 PM, Tran Chau 
> wrote:
>
>> Hi Sumudu,
>>
>> Could you please run these statements below in your database directly to
>> fix this bug ? Then restart tomcat again ?
>>
>> *ALTER TABLE programstageinstance MODIFY status INTEGER;*
>> *UPDATE programstageinstance SET status='ACTIVE' WHERE status='0';*
>> *UPDATE programstageinstance SET status='COMPLETED' WHERE status='1';*
>> *UPDATE programstageinstance SET status='SKIPPED' WHERE status='5';*
>>
>>
>> Best regards,
>> 
>> Châu Thu Trân
>> HISP Viet Nam
>> Email: tran.hispviet...@gmail.com
>>
>> On Tue, Sep 16, 2014 at 8:56 PM, sumudu weerasinghe 
>> wrote:
>>
>>> Sorry, my system detail as follow
>>> *Version: 2.16**Build revision:16414**Database type:MySQL*
>>> *Java version:1.7.0_04**OS name:Windows 7*
>>>
>>>
>>> On Tue, Sep 16, 2014 at 7:23 PM, sumudu weerasinghe >> > wrote:
>>>
 Dear all ,
 I have updated my instance from 2.15 to 2.16  over the old mysql
 database. Then i list the clinic patient . But it was unable to go for
 patient dashboard through the list and give error as exception occur (shown
 in screen shot). Log also show some error. Log and screenshots are attached
 here.

 Regards,
 Sumudu

>>>
>>>
>>
>
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp

[Dhis2-devs] [Branch ~dhis2-devs-core/dhis2/trunk] Rev 16732: NPE check for UserCredentials

[noreply]

revno: 16732
committer: Morten Olav Hansen 
branch nick: dhis2
timestamp: Wed 2014-09-17 13:15:25 +0700
message:
  NPE check for UserCredentials
modified:
  
dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/webapi/controller/AccountController.java


--
lp:dhis2
https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk

Your team DHIS 2 developers is subscribed to branch lp:dhis2.
To unsubscribe from this branch go to 
https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk/+edit-subscription
=== modified file 'dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/webapi/controller/AccountController.java'
--- dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/webapi/controller/AccountController.java	2014-08-15 07:40:20 +
+++ dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/webapi/controller/AccountController.java	2014-09-17 06:15:25 +
@@ -28,18 +28,7 @@
  * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
-import java.io.IOException;
-import java.util.Collection;
-import java.util.Date;
-import java.util.HashMap;
-import java.util.HashSet;
-import java.util.Map;
-import java.util.Set;
-
-import javax.servlet.http.HttpServletRequest;
-import javax.servlet.http.HttpServletResponse;
-import javax.servlet.http.HttpSession;
-
+import com.fasterxml.jackson.databind.ObjectMapper;
 import org.apache.commons.lang.StringUtils;
 import org.apache.commons.logging.Log;
 import org.apache.commons.logging.LogFactory;
@@ -71,7 +60,16 @@
 import org.springframework.web.bind.annotation.RequestParam;
 import org.springframework.web.client.RestTemplate;
 
-import com.fasterxml.jackson.databind.ObjectMapper;
+import javax.servlet.http.HttpServletRequest;
+import javax.servlet.http.HttpServletResponse;
+import javax.servlet.http.HttpSession;
+import java.io.IOException;
+import java.util.Collection;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Set;
 
 /**
  * @author Lars Helge Overland
@@ -109,7 +107,7 @@
 
 @Autowired
 private SystemSettingManager systemSettingManager;
-
+
 private ObjectMapper objectMapper = new ObjectMapper();
 
 @RequestMapping( value = "/recovery", method = RequestMethod.POST )
@@ -133,7 +131,7 @@
 ContextUtils.conflictResponse( response, "User does not exist: " + username );
 return;
 }
-
+
 boolean recover = securityService.sendRestoreMessage( credentials, rootPath, RestoreOptions.RECOVER_PASSWORD_OPTION );
 
 if ( !recover )
@@ -181,7 +179,7 @@
 ContextUtils.conflictResponse( response, "User does not exist: " + username );
 return;
 }
-
+
 boolean restore = securityService.restore( credentials, token, code, password, RestoreType.RECOVER_PASSWORD );
 
 if ( !restore )
@@ -214,7 +212,7 @@
 {
 UserCredentials credentials = null;
 
-boolean invitedByEmail = ( inviteUsername != null && !inviteUsername.isEmpty() );
+boolean invitedByEmail = (inviteUsername != null && !inviteUsername.isEmpty());
 
 boolean canChooseUsername = true;
 
@@ -314,7 +312,7 @@
 
 if ( email == null || !ValidationUtils.emailIsValid( email ) )
 {
-ContextUtils.badRequestResponse( response,  "Email is not specified or invalid" );
+ContextUtils.badRequestResponse( response, "Email is not specified or invalid" );
 return;
 }
 
@@ -456,6 +454,15 @@
 Map result = new HashMap<>();
 result.put( "status", "OK" );
 
+if ( credentials == null )
+{
+result.put( "status", "NON_EXPIRED" );
+result.put( "message", "Username is not valid, redirecting to login." );
+
+ContextUtils.badRequestResponse( response, objectMapper.writeValueAsString( result ) );
+return;
+}
+
 if ( userService.credentialsNonExpired( credentials ) )
 {
 result.put( "status", "NON_EXPIRED" );

___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help   : https://help.launchpad.net/ListHelp