i just noticed that FlexFeature uses AttributeType.DATE, which is
java.sql.Date.class to determine if conversion is needed or not. that might
lead to conversion of java.util.Date objects which is definitely unwanted.
btw. found this commit
https://sourceforge.net/p/jump-pilot/code/6129/
where i disabled parsing of spatialite dates (at request ;). maybe we should at
least reenable
ret = rs.getTimestamp(columnIndex);
and only keep expensive FlexDateParsing disabled?
to reproduce/check i need a test dataset though. maybe an sqlite file? ..ede
ps. here are the two threads wrt. slow date attrib loading
https://www.mail-archive.com/jump-pilot-devel@lists.sourceforge.net/msg18295.html
https://www.mail-archive.com/jump-pilot-devel@lists.sourceforge.net/msg18317.html
On 19.08.2020 15:17, ede wrote:
> Mike,
>
> saving by itself is not the root problem here. this is proven by the second
> run (when all dates are converted).
>
> it is a combination of opening/editing/saving. if a data reader (eg. JML or
> GeoJSON now) uses FlexibleFeature's (FF) lazy conversion, time is saved by
> simply not converting expensive data types in the loader but having FF do
> that later. in the worst case this is during saving to another format, as you
> noticed.
>
> if saving data loaded from a database is slow and it is indeed caused by the
> FlexDateParser, then the loader must be at fault, because it does by design
> or flaw not convert to java.sql.Date during loading. what database do you
> load from?
>
> ..ede
>
> On 19.08.2020 14:06, michael michaud wrote:
>> A beanshell test just comparing writing shp from BasicFeature and from
>> FlexibleFeature.
>> (to be executed from the Beanshell Editor)
>>
>>
>> Attachments:
>>
>> -
>> [test_flexible_feature.bsh](https://sourceforge.net/p/jump-pilot/bugs/_discuss/thread/62e6f051ce/6cc8/attachment/test_flexible_feature.bsh)
>> (1.7 kB; application/octet-stream)
>>
>>
>> ---
>>
>> ** [bugs:#497] Shapefile export slowed down because of FlexibleDateParser**
>>
>> **Status:** open
>> **Created:** Mon Aug 17, 2020 11:47 AM UTC by michael michaud
>> **Last Updated:** Wed Aug 19, 2020 10:29 AM UTC
>> **Owner:** michael michaud
>>
>>
>> Exporting a big dataset to shapefile (1 300 000 objets - around 225+115 Mb)
>> with version 1.15 lasted more than 20 minutes.
>> Most of the time is used by FlexibleDateParser.parse().
>>
>>
>> ---
>>
>> Sent from sourceforge.net because you indicated interest in
>> <https://sourceforge.net/p/jump-pilot/bugs/497/>
>>
>>
>>
>> To unsubscribe from further messages, please visit
>> <https://sourceforge.net/auth/subscriptions/>
>>
>
>
>
> ---
>
> ** [bugs:#497] Shapefile export slowed down because of FlexibleDateParser**
>
> **Status:** open
> **Created:** Mon Aug 17, 2020 11:47 AM UTC by michael michaud
> **Last Updated:** Wed Aug 19, 2020 12:06 PM UTC
> **Owner:** michael michaud
>
>
> Exporting a big dataset to shapefile (1 300 000 objets - around 225+115 Mb)
> with version 1.15 lasted more than 20 minutes.
> Most of the time is used by FlexibleDateParser.parse().
>
>
> ---
>
> Sent from sourceforge.net because you indicated interest in
> <https://sourceforge.net/p/jump-pilot/bugs/497/>
>
>
>
> To unsubscribe from further messages, please visit
> <https://sourceforge.net/auth/subscriptions/>
>
---
** [bugs:#497] Shapefile export slowed down because of FlexibleDateParser**
**Status:** open
**Created:** Mon Aug 17, 2020 11:47 AM UTC by michael michaud
**Last Updated:** Wed Aug 19, 2020 12:06 PM UTC
**Owner:** michael michaud
Exporting a big dataset to shapefile (1 300 000 objets - around 225+115 Mb)
with version 1.15 lasted more than 20 minutes.
Most of the time is used by FlexibleDateParser.parse().
---
Sent from sourceforge.net because jump-pilot-devel@lists.sourceforge.net is
subscribed to https://sourceforge.net/p/jump-pilot/bugs/
To unsubscribe from further messages, a project admin can change settings at
https://sourceforge.net/p/jump-pilot/admin/bugs/options. Or, if this is a
mailing list, you can unsubscribe from the mailing list.
_______________________________________________
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
