Discussion:
Limiting rendering for large resultsets
(too old to reply)
PREVOSTO, Laurent
2014-08-06 09:39:09 UTC
Permalink
Hello,
We have geoserver based WMS services that display a quite large network (lots of points and polylines stored in an Oracle Spatial database).

When in the cities, the number of items can be very large
In the countryside, of course, the density of polylines is quite low.

If I try to render a whole city, since there are too many items to render, the request will timeout.
But if I set a scale condition to avoid drawing thousands of lines when in a large city, users complain that they can't find the cables when in the countryside...

So my question was : is there a way to tell geoserver to render a blank image if there are more than n items in the Oracle resultset ?
That way, I could keep a large scale in my TLD without going timeout in areas where there is a high density of items.

Regards,

Laurent
Jonathan Moules
2014-08-06 10:06:18 UTC
Permalink
Hi Laurent,
One possible solution - two datasets.

- Use a urban-areas polygon to clip the data into two datasets - one with
urban areas, the other rural.

- Style them both with different scale thresholds.

- Put both of them into the same layerGroup.

To your users it'll appear as a single layer while being optimised for
viewing at both scales.

---
Alternately rather than splitting the dataset, add an attribute/field to
the dataset that indicates if it is urban/rural. Load as a single layer
with an SLD which has two rules, and use a filter to distinguish between
them.

Hopefully that helps,
Jonathan
Post by PREVOSTO, Laurent
Hello,
We have geoserver based WMS services that display a quite large network
(lots of points and polylines stored in an Oracle Spatial database).
When in the cities, the number of items can be very large
In the countryside, of course, the density of polylines is quite low.
If I try to render a whole city, since there are too many items to
render, the request will timeout.
But if I set a scale condition to avoid drawing thousands of lines when in
a large city, users complain that they can't find the cables when in the
countryside...
So my question was : is there a way to tell geoserver to render a blank
image if there are more than n items in the Oracle resultset ?
That way, I could keep a large scale in my TLD without going timeout in
areas where there is a high density of items.
Regards,
Laurent
------------------------------------------------------------------------------
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls.
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk
_______________________________________________
Geoserver-users mailing list
https://lists.sourceforge.net/lists/listinfo/geoserver-users
--
This transmission is intended for the named addressee(s) only and may
contain confidential, sensitive or personal information and should be
handled accordingly. Unless you are the named addressee (or authorised to
receive it for the addressee) you may not copy or use it, or disclose it to
anyone else. If you have received this transmission in error please notify
the sender immediately. All email traffic sent to or from us, including
without limitation all GCSX traffic, may be subject to recording and/or
monitoring in accordance with relevant legislation.
Andrea Aime
2014-08-06 10:24:06 UTC
Permalink
Post by PREVOSTO, Laurent
Hello,
We have geoserver based WMS services that display a quite large network
(lots of points and polylines stored in an Oracle Spatial database).
When in the cities, the number of items can be very large
In the countryside, of course, the density of polylines is quite low.
If I try to render a whole city, since there are too many items to
render, the request will timeout.
You can increase the max rendering time in the WMS panel
Post by PREVOSTO, Laurent
But if I set a scale condition to avoid drawing thousands of lines when
in a large city, users complain that they can’t find the cables when in the
countryside

So my question was : is there a way to tell geoserver to render a blank
image if there are more than n items in the Oracle resultset ?
That way, I could keep a large scale in my TLD without going timeout in
areas where there is a high density of items.
No, there is no such functionality. It is of course possible to add it, but
we'd need to make
an extension to SLD, the limit would have to be specified at the
FeatureTypeStyle or Rule levels

Cheers
Andrea
--
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.
==

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------
PREVOSTO, Laurent
2014-08-06 14:09:38 UTC
Permalink
I agree that would be the perfect solution
 unfortunately it is not the way works today ☹

No dirty workaround ? damned


Regards,

Laurent

De : ***@gmail.com [mailto:***@gmail.com] De la part de Andrea Aime
Envoyé : mercredi 6 août 2014 12:24
À : PREVOSTO, Laurent
Cc : geoserver-***@lists.sourceforge.net
Objet : Re: [Geoserver-users] Limiting rendering for large resultsets

On Wed, Aug 6, 2014 at 11:39 AM, PREVOSTO, Laurent <***@sfr.com<mailto:***@sfr.com>> wrote:
Hello,
We have geoserver based WMS services that display a quite large network (lots of points and polylines stored in an Oracle Spatial database).

When in the cities, the number of items can be very large
In the countryside, of course, the density of polylines is quite low.

If I try to render a whole city, since there are too many items to render, the request will timeout.

You can increase the max rendering time in the WMS panel

But if I set a scale condition to avoid drawing thousands of lines when in a large city, users complain that they can’t find the cables when in the countryside


So my question was : is there a way to tell geoserver to render a blank image if there are more than n items in the Oracle resultset ?
That way, I could keep a large scale in my TLD without going timeout in areas where there is a high density of items.

No, there is no such functionality. It is of course possible to add it, but we'd need to make
an extension to SLD, the limit would have to be specified at the FeatureTypeStyle or Rule levels

Cheers
Andrea
--
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.
==

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------
cmaul
2014-08-07 01:12:32 UTC
Permalink
Laurent,

I've got the same problem with our cadastre, i.e. rural and urban
properties. My way around was to filter using the 'Local Government'
attribute. The result is a bit crude and the SLD is large, but it works
well. If you have got a similar attribute in your dataset that would be
simplest solution.

Disadvantaqe is: Users suddenly see nothing where they know that there are
parcel.

Increasing the time-out is not a good idea because after 60 secs (which is
the default) people are long gone and the only effect is you put a load on
your DB server. Nobody is that patient anymore.

I have actually a time-out of 30 secs.

Cheers

Christian



-----
____________________________

Dr Christian Maul
Project Manager

Information Services Branch
Department Environment and Primary Industries
Level13, Marland House, 570 Bourke Street
Melbourne 3000

PO Box 500, East Melbourne Vic 3002


Telephone: +61-3-8636 2325
Telefax: +61-3-8636 2813
--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Limiting-rendering-for-large-resultsets-tp5155236p5155351.html
Sent from the GeoServer - User mailing list archive at Nabble.com.
Mike Grogan
2014-08-08 16:03:39 UTC
Permalink
A crude approach I have used in the past is to add an integer attribute to
my points or lines and then to assign a random attribute value from a range
that is based on the density of features in the area. I then specify in
the SLD that only features greater than some number should be displayed for
particular zoom levels.

For instance,

For your city features, you might assign a random integer attribute to each
of them from a range between, say, 0 and 1000.

For your countryside features, you might assign a random integer attribute
that have a range between 700 and 1000.

Then, say for a zoom level 5 or 6, you might specify in the SLD to only
show features whose attribute > 700.

Doing this, you end up keeping all of the countryside features, but
randomly filtering out city features.

At greater zoom levels ... just say 9, 10, or so (depending on your
situation), you then show features whose attribute > 500, 400, etc.,
thereby allowing more city features to be shown as you zoom in.

I could probably have done some statistics to tell me the exact ranges to
use, but trial & error worked fine for me.

- Mike Grogan
Post by PREVOSTO, Laurent
Hello,
We have geoserver based WMS services that display a quite large network
(lots of points and polylines stored in an Oracle Spatial database).
When in the cities, the number of items can be very large
In the countryside, of course, the density of polylines is quite low.
If I try to render a whole city, since there are too many items to
render, the request will timeout.
But if I set a scale condition to avoid drawing thousands of lines when in
a large city, users complain that they can’t find the cables when in the
countryside

So my question was : is there a way to tell geoserver to render a blank
image if there are more than n items in the Oracle resultset ?
That way, I could keep a large scale in my TLD without going timeout in
areas where there is a high density of items.
Regards,
Laurent
------------------------------------------------------------------------------
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls.
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk
_______________________________________________
Geoserver-users mailing list
https://lists.sourceforge.net/lists/listinfo/geoserver-users
PREVOSTO, Laurent
2014-08-12 18:01:47 UTC
Permalink
Hi,
I understand your approach but i definately prefer to draw nothing or an “error” tile than just draw a random part of the network (a wrong information)
Because, users tend to believe what they see ☺

In my case, I am tempted to lower the timeout rendering of geoserver and consider that a tile that needs more than, let’s say, 10 sec to draw is not worth rendering.
But then Apache mod_jk load-balancer put that very Tomcat in FAIL state and I could not figure out a configuration that actually works.

Regards,

Laurent

De : Mike Grogan [mailto:***@gmail.com]
Envoyé : vendredi 8 août 2014 18:04
À : PREVOSTO, Laurent
Cc : geoserver-***@lists.sourceforge.net
Objet : Re: [Geoserver-users] Limiting rendering for large resultsets

A crude approach I have used in the past is to add an integer attribute to my points or lines and then to assign a random attribute value from a range that is based on the density of features in the area. I then specify in the SLD that only features greater than some number should be displayed for particular zoom levels.

For instance,

For your city features, you might assign a random integer attribute to each of them from a range between, say, 0 and 1000.

For your countryside features, you might assign a random integer attribute that have a range between 700 and 1000.

Then, say for a zoom level 5 or 6, you might specify in the SLD to only show features whose attribute > 700.

Doing this, you end up keeping all of the countryside features, but randomly filtering out city features.

At greater zoom levels ... just say 9, 10, or so (depending on your situation), you then show features whose attribute > 500, 400, etc., thereby allowing more city features to be shown as you zoom in.

I could probably have done some statistics to tell me the exact ranges to use, but trial & error worked fine for me.

- Mike Grogan




On Wed, Aug 6, 2014 at 5:39 AM, PREVOSTO, Laurent <***@sfr.com<mailto:***@sfr.com>> wrote:
Hello,
We have geoserver based WMS services that display a quite large network (lots of points and polylines stored in an Oracle Spatial database).

When in the cities, the number of items can be very large
In the countryside, of course, the density of polylines is quite low.

If I try to render a whole city, since there are too many items to render, the request will timeout.
But if I set a scale condition to avoid drawing thousands of lines when in a large city, users complain that they can’t find the cables when in the countryside


So my question was : is there a way to tell geoserver to render a blank image if there are more than n items in the Oracle resultset ?
That way, I could keep a large scale in my TLD without going timeout in areas where there is a high density of items.

Regards,

Laurent

------------------------------------------------------------------------------
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls.
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk
_______________________________________________
Geoserver-users mailing list
Geoserver-***@lists.sourceforge.net<mailto:Geoserver-***@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/geoserver-users
Daniel Bevilacqua
2014-08-12 18:09:43 UTC
Permalink
Chris Bennight
2014-08-13 00:36:55 UTC
Permalink
As another take on this same problem we've been working on a Z-order
occlusion based distributed renderer. (This is for an accumulo <->
geoserver datastore implementation).

Currently working is a point only implementation (well, it works for
polygons/linestrings, but calculates occlusion for the centroid only - so
only works "right" for points))

It's implemented as a WPS render transform process
https://github.com/ngageoint/geowave/blob/f40a547a5227540d1b68bd75ab2057ecc7ed00cd/geowave-gt/src/main/java/mil/nga/giat/geowave/gt/DecimationProcess.java

Here's the query setup code
https://github.com/ngageoint/geowave/blob/f40a547a5227540d1b68bd75ab2057ecc7ed00cd/geowave-gt/src/main/java/mil/nga/giat/geowave/gt/SpatialDecimationQuery.java

Index thinning
https://github.com/ngageoint/geowave/blob/f40a547a5227540d1b68bd75ab2057ecc7ed00cd/geowave-gt/src/main/java/mil/nga/giat/geowave/gt/FixedCardinalitySkippingIterator.java

This implementation doesn't distribute the rendering process; rather it
defines a pixel space -> index range transform function, and when a feature
(point in this case) is found it's returned, and the seek function in the
database skips to the next pixel

Here's a graphic example:
Loading Image...

--

The next phase, and what I think a general solution to the problem you
describe is to distribute the rendering process (in accumulo the database
"nodes" sit on top of multiple instances for easy horizontal scalability).

This portion has to take into account the SLD selected as well, as the
style can have a big impact on how features are rendered.

Basically each featurestylerule results in a distributed call for that
feature type - and the tile for that rule is rendered local to the majority
of the data (best effort) in the cluster. When the tile is completely
covered (or passes some heuristic, i.e. 80%, etc.) the seek stops and the
"full" tile returns. (There is some skipping here to - when a feature is
rendered an internal table of index ranges <-> pixel transforms is modified
to remove the covered pixels/ranges - so the next feature always overlaps a
non-colored region.)

All the tiles are now composted together in geoserver based on the ordering
from the SLD - and hopefully we have now skipped reading and writing a
bunch of data we don't have a "pixel budget" to actually display.

This portion is still in work - I think we might have an upcoming geoserver
pull request, as we need to make a few private methods protected so we can
distribute the render process across the cluster. Our internal
implementation just cuts and pastes a bunch of GPL code, and I didn't want
to include that since the rest of the project is apache 2.0 and, well, I
didn't want to explain the intermingled licenses.

Our torture case right now is basically one of the openstreetmap SLD's at ~
zoom level 17 rules, rendered globally (with the full osm planet dump).


--

That said, these methods have to be implemented at the geotools datastore
level - and specifically you have to have control enough of the indexing so
that you can short circuit reads. But it's something that could
conceptually (implementation would be slightly different) be to other
systems as well (certainly on postgis, since it's open source)
Post by PREVOSTO, Laurent
Hi,
I understand your approach but i definately prefer to draw nothing or an
“error” tile than just draw a random part of the network (a wrong
information)
Because, users tend to believe what they see J
In my case, I am tempted to lower the timeout rendering of geoserver and
consider that a tile that needs more than, let’s say, 10 sec to draw is not
worth rendering.
But then Apache mod_jk load-balancer put that very Tomcat in FAIL state
and I could not figure out a configuration that actually works.
Regards,
Laurent
*Envoyé :* vendredi 8 août 2014 18:04
*À :* PREVOSTO, Laurent
*Objet :* Re: [Geoserver-users] Limiting rendering for large resultsets
A crude approach I have used in the past is to add an integer attribute to
my points or lines and then to assign a random attribute value from a range
that is based on the density of features in the area. I then specify in
the SLD that only features greater than some number should be displayed for
particular zoom levels.
For instance,
For your city features, you might assign a random integer attribute to
each of them from a range between, say, 0 and 1000.
For your countryside features, you might assign a random integer attribute
that have a range between 700 and 1000.
Then, say for a zoom level 5 or 6, you might specify in the SLD to only
show features whose attribute > 700.
Doing this, you end up keeping all of the countryside features, but
randomly filtering out city features.
At greater zoom levels ... just say 9, 10, or so (depending on your
situation), you then show features whose attribute > 500, 400, etc.,
thereby allowing more city features to be shown as you zoom in.
I could probably have done some statistics to tell me the exact ranges to
use, but trial & error worked fine for me.
- Mike Grogan
On Wed, Aug 6, 2014 at 5:39 AM, PREVOSTO, Laurent <
Hello,
We have geoserver based WMS services that display a quite large network
(lots of points and polylines stored in an Oracle Spatial database).
When in the cities, the number of items can be very large
In the countryside, of course, the density of polylines is quite low.
If I try to render a whole city, since there are too many items to
render, the request will timeout.
But if I set a scale condition to avoid drawing thousands of lines when in
a large city, users complain that they can’t find the cables when in the
countryside

So my question was : is there a way to tell geoserver to render a blank
image if there are more than n items in the Oracle resultset ?
That way, I could keep a large scale in my TLD without going timeout in
areas where there is a high density of items.
Regards,
Laurent
------------------------------------------------------------------------------
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls.
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk
_______________________________________________
Geoserver-users mailing list
https://lists.sourceforge.net/lists/listinfo/geoserver-users
------------------------------------------------------------------------------
_______________________________________________
Geoserver-users mailing list
https://lists.sourceforge.net/lists/listinfo/geoserver-users
Jody Garnett
2014-08-13 06:22:11 UTC
Permalink
Jonathan Moules
2014-08-13 15:16:28 UTC
Permalink
Post by unknown
Note that if you are using PostGIS (or Oracle?) GeoTools will ask the
database to simplify
Andrea will know better (of course!), but I think last time this came up
the notion was that Oracle's no-where near as good as PostGIS at this.

Cheers,
Jonathan
Post by unknown
Interesting, I am reminded of the gvSig renderer which can "draw" the
shapefile index - getting off the ride when the index node gets down below
the size of a pixel. Think that is one of the only optimisations they have
left that we did not implement.
Note that if you are using PostGIS (or Oracle?) GeoTools will ask the
database to simplify (stepping size based on the distance a pixel covers at
current zoom level). This is great as it allows less data to be sent over
from the database. For most other data stores it will simplify in Java
before transform and rendering.
Chris it would be *great* if you could write the occasional blog for the
geotools website - this is really cool stuff you are working on.
Jody Garnett
Post by Chris Bennight
As another take on this same problem we've been working on a Z-order
occlusion based distributed renderer. (This is for an accumulo <->
geoserver datastore implementation).
Currently working is a point only implementation (well, it works for
polygons/linestrings, but calculates occlusion for the centroid only - so
only works "right" for points))
It's implemented as a WPS render transform process
https://github.com/ngageoint/geowave/blob/f40a547a5227540d1b68bd75ab2057ecc7ed00cd/geowave-gt/src/main/java/mil/nga/giat/geowave/gt/DecimationProcess.java
Here's the query setup code
https://github.com/ngageoint/geowave/blob/f40a547a5227540d1b68bd75ab2057ecc7ed00cd/geowave-gt/src/main/java/mil/nga/giat/geowave/gt/SpatialDecimationQuery.java
Index thinning
https://github.com/ngageoint/geowave/blob/f40a547a5227540d1b68bd75ab2057ecc7ed00cd/geowave-gt/src/main/java/mil/nga/giat/geowave/gt/FixedCardinalitySkippingIterator.java
This implementation doesn't distribute the rendering process; rather it
defines a pixel space -> index range transform function, and when a feature
(point in this case) is found it's returned, and the seek function in the
database skips to the next pixel
https://dl.dropboxusercontent.com/u/6649380/decimation.png
--
The next phase, and what I think a general solution to the problem you
describe is to distribute the rendering process (in accumulo the database
"nodes" sit on top of multiple instances for easy horizontal scalability).
This portion has to take into account the SLD selected as well, as the
style can have a big impact on how features are rendered.
Basically each featurestylerule results in a distributed call for that
feature type - and the tile for that rule is rendered local to the majority
of the data (best effort) in the cluster. When the tile is completely
covered (or passes some heuristic, i.e. 80%, etc.) the seek stops and the
"full" tile returns. (There is some skipping here to - when a feature is
rendered an internal table of index ranges <-> pixel transforms is modified
to remove the covered pixels/ranges - so the next feature always overlaps a
non-colored region.)
All the tiles are now composted together in geoserver based on the
ordering from the SLD - and hopefully we have now skipped reading and
writing a bunch of data we don't have a "pixel budget" to actually display.
This portion is still in work - I think we might have an upcoming
geoserver pull request, as we need to make a few private methods protected
so we can distribute the render process across the cluster. Our internal
implementation just cuts and pastes a bunch of GPL code, and I didn't want
to include that since the rest of the project is apache 2.0 and, well, I
didn't want to explain the intermingled licenses.
Our torture case right now is basically one of the openstreetmap SLD's at
~ zoom level 17 rules, rendered globally (with the full osm planet dump).
--
That said, these methods have to be implemented at the geotools datastore
level - and specifically you have to have control enough of the indexing so
that you can short circuit reads. But it's something that could
conceptually (implementation would be slightly different) be to other
systems as well (certainly on postgis, since it's open source)
On Tue, Aug 12, 2014 at 2:01 PM, PREVOSTO, Laurent <
Post by PREVOSTO, Laurent
Hi,
I understand your approach but i definately prefer to draw nothing or an
"error" tile than just draw a random part of the network (a wrong
information)
Because, users tend to believe what they see J
In my case, I am tempted to lower the timeout rendering of geoserver and
consider that a tile that needs more than, let's say, 10 sec to draw is not
worth rendering.
But then Apache mod_jk load-balancer put that very Tomcat in FAIL state
and I could not figure out a configuration that actually works.
Regards,
Laurent
*Envoyé :* vendredi 8 août 2014 18:04
*À :* PREVOSTO, Laurent
*Objet :* Re: [Geoserver-users] Limiting rendering for large resultsets
A crude approach I have used in the past is to add an integer attribute
to my points or lines and then to assign a random attribute value from a
range that is based on the density of features in the area. I then specify
in the SLD that only features greater than some number should be displayed
for particular zoom levels.
For instance,
For your city features, you might assign a random integer attribute to
each of them from a range between, say, 0 and 1000.
For your countryside features, you might assign a random integer
attribute that have a range between 700 and 1000.
Then, say for a zoom level 5 or 6, you might specify in the SLD to only
show features whose attribute > 700.
Doing this, you end up keeping all of the countryside features, but
randomly filtering out city features.
At greater zoom levels ... just say 9, 10, or so (depending on your
situation), you then show features whose attribute > 500, 400, etc.,
thereby allowing more city features to be shown as you zoom in.
I could probably have done some statistics to tell me the exact ranges
to use, but trial & error worked fine for me.
- Mike Grogan
On Wed, Aug 6, 2014 at 5:39 AM, PREVOSTO, Laurent <
Hello,
We have geoserver based WMS services that display a quite large network
(lots of points and polylines stored in an Oracle Spatial database).
When in the cities, the number of items can be very large
In the countryside, of course, the density of polylines is quite low.
If I try to render a whole city, since there are too many items to
render, the request will timeout.
But if I set a scale condition to avoid drawing thousands of lines when
in a large city, users complain that they can't find the cables when in the
countryside...
So my question was : is there a way to tell geoserver to render a blank
image if there are more than n items in the Oracle resultset ?
That way, I could keep a large scale in my TLD without going timeout in
areas where there is a high density of items.
Regards,
Laurent
------------------------------------------------------------------------------
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls.
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk
_______________________________________________
Geoserver-users mailing list
https://lists.sourceforge.net/lists/listinfo/geoserver-users
------------------------------------------------------------------------------
_______________________________________________
Geoserver-users mailing list
https://lists.sourceforge.net/lists/listinfo/geoserver-users
------------------------------------------------------------------------------
_______________________________________________
Geoserver-users mailing list
https://lists.sourceforge.net/lists/listinfo/geoserver-users
------------------------------------------------------------------------------
_______________________________________________
Geoserver-users mailing list
https://lists.sourceforge.net/lists/listinfo/geoserver-users
--
This transmission is intended for the named addressee(s) only and may
contain confidential, sensitive or personal information and should be
handled accordingly. Unless you are the named addressee (or authorised to
receive it for the addressee) you may not copy or use it, or disclose it to
anyone else. If you have received this transmission in error please notify
the sender immediately. All email traffic sent to or from us, including
without limitation all GCSX traffic, may be subject to recording and/or
monitoring in accordance with relevant legislation.
Andrea Aime
2014-08-15 14:24:27 UTC
Permalink
Post by Chris Bennight
As another take on this same problem we've been working on a Z-order
occlusion based distributed renderer. (This is for an accumulo <->
geoserver datastore implementation).
Currently working is a point only implementation (well, it works for
polygons/linestrings, but calculates occlusion for the centroid only - so
only works "right" for points))
It's implemented as a WPS render transform process
https://github.com/ngageoint/geowave/blob/f40a547a5227540d1b68bd75ab2057ecc7ed00cd/geowave-gt/src/main/java/mil/nga/giat/geowave/gt/DecimationProcess.java
Here's the query setup code
https://github.com/ngageoint/geowave/blob/f40a547a5227540d1b68bd75ab2057ecc7ed00cd/geowave-gt/src/main/java/mil/nga/giat/geowave/gt/SpatialDecimationQuery.java
Index thinning
https://github.com/ngageoint/geowave/blob/f40a547a5227540d1b68bd75ab2057ecc7ed00cd/geowave-gt/src/main/java/mil/nga/giat/geowave/gt/FixedCardinalitySkippingIterator.java
This implementation doesn't distribute the rendering process; rather it
defines a pixel space -> index range transform function, and when a feature
(point in this case) is found it's returned, and the seek function in the
database skips to the next pixel
https://dl.dropboxusercontent.com/u/6649380/decimation.png
What you are doing here in GeoTools is a rendering/store hint called
ScreenMap, and it's normally applied in memory.
However, theoretically we could do the same as, say, MinVisitor, and have
the store use a native and faster
implementation of it, like JDBCDataStore does (turning it into a select
min).

To make it possible we'd need to have ScreenMap provide its internal
information I supposed, the rendering rectangle,
and the grid to world transformation.

The advantage of doing so would be that your users would not have to add a
rendering transformation in the SLD,
it would just work automatically (downsize, there would be no way to turn
it off).

Cheers
Andrea
--
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.
==

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------
Chris Bennight
2014-08-16 17:19:07 UTC
Permalink
Post by Andrea Aime
What you are doing here in GeoTools is a rendering/store hint called
ScreenMap, and it's normally applied in memory.
However, theoretically we could do the same as, say, MinVisitor, and have
the store use a native and faster
implementation of it, like JDBCDataStore does (turning it into a select
min).
To make it possible we'd need to have ScreenMap provide its internal
information I supposed, the rendering rectangle,
and the grid to world transformation.
The advantage of doing so would be that your users would not have to add a
rendering transformation in the SLD,
it would just work automatically (downsize, there would be no way to turn
it off).
Cheers
Andrea
So essentially using the ScreenMap information as a way to pass down the
grid <-> world transform function information to a place where pushdown
support on a datastore can be implemented.

That definitely fits in a little more smoothly than requiring the WPS in
the SLD (as you mention). The inability to turn it off (which you also
highlight) would be my concern. To make it more "fire and forget" I
think the distributed rendering option I talked about would be required.
(To deal with symbolizers in grid space that aren't equivalent to the
native geometry in that same grid space). There's the efficiency
optimization if the symbolizer is larger than the geometry - but the bigger
concern is the "incorrectness" if the symbolizer is smaller than the
geometry. For points that's not really a huge issue (only case I think it
would break would be with offsets). But for lines and polygons it's a
definite issues. For the distributed rendering portion this is definitely
something I want to dig in to.
I saw your latest commit, and this seems to be like a good pointer into
digging into this (ScreenMap) functionality:
https://github.com/geoserver/geoserver/commit/b64b03f7e5f02ddddb090d8ff45c0a75cd823e20

Thanks! (for the idea, and for the unrelated but fortuitous fix)
unknown
1970-01-01 00:00:00 UTC
Permalink
--f46d043c08eaa3ece905007294c2
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi,

You can apply simplification functions from Oracle or PostGIS. These
function will return a simplified objects.
Post by PREVOSTO, Laurent
Hi,
I understand your approach but i definately prefer to draw nothing or an
“error” tile than just draw a random part of the network (a wrong
information)
Because, users tend to believe what they see J
In my case, I am tempted to lower the timeout rendering of geoserver and
consider that a tile that needs more than, let’s say, 10 sec to draw is not
worth rendering.
But then Apache mod_jk load-balancer put that very Tomcat in FAIL state
and I could not figure out a configuration that actually works.
Regards,
Laurent
*Envoyé :* vendredi 8 août 2014 18:04
*À :* PREVOSTO, Laurent
*Objet :* Re: [Geoserver-users] Limiting rendering for large resultsets
A crude approach I have used in the past is to add an integer attribute to
my points or lines and then to assign a random attribute value from a range
that is based on the density of features in the area. I then specify in
the SLD that only features greater than some number should be displayed for
particular zoom levels.
For instance,
For your city features, you might assign a random integer attribute to
each of them from a range between, say, 0 and 1000.
For your countryside features, you might assign a random integer attribute
that have a range between 700 and 1000.
Then, say for a zoom level 5 or 6, you might specify in the SLD to only
show features whose attribute > 700.
Doing this, you end up keeping all of the countryside features, but
randomly filtering out city features.
At greater zoom levels ... just say 9, 10, or so (depending on your
situation), you then show features whose attribute > 500, 400, etc.,
thereby allowing more city features to be shown as you zoom in.
I could probably have done some statistics to tell me the exact ranges to
use, but trial & error worked fine for me.
- Mike Grogan
On Wed, Aug 6, 2014 at 5:39 AM, PREVOSTO, Laurent <
Hello,
We have geoserver based WMS services that display a quite large network
(lots of points and polylines stored in an Oracle Spatial database).
When in the cities, the number of items can be very large
In the countryside, of course, the density of polylines is quite low.
If I try to render a whole city, since there are too many items to
render, the request will timeout.
But if I set a scale condition to avoid drawing thousands of lines when in
a large city, users complain that they can’t find the cables when in the
countryside…
So my question was : is there a way to tell geoserver to render a blank
image if there are more than n items in the Oracle resultset ?
That way, I could keep a large scale in my TLD without going timeout in
areas where there is a high density of items.
Regards,
Laurent
------------------------------------------------------------------------------
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls.
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk
_______________________________________________
Geoserver-users mailing list
https://lists.sourceforge.net/lists/listinfo/geoserver-users
------------------------------------------------------------------------------
_______________________________________________
Geoserver-users mailing list
https://lists.sourceforge.net/lists/listinfo/geoserver-users
--
Daniel Bevilacqua

cel +55 19 81383157

--f46d043c08eaa3ece905007294c2
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable <div dir="ltr">Hi,<div><br></div><div>You can apply simplification functions from Oracle or PostGIS. These function will return a simplified objects.</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Aug 12, 2014 at 3:01 PM, PREVOSTO, Laurent <span dir="ltr">&lt;<a href="mailto:***@sfr.com" target="_blank">***@sfr.com</a>&gt;</span> wrote:<br>

<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">





<div lang="FR" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">Hi,<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">I understand your approach but i definately prefer to draw nothing or an “error” tile than just draw a random part of the network (a wrong information)<u></u><u></u></span></p>


<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">Because, users tend to believe what they see
</span><span lang="EN-US" style="font-size:11.0pt;font-family:Wingdings;color:#1f497d">J</span><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d"><u></u><u></u></span></p>


<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">In my case, I am tempted to lower the timeout rendering of geoserver and consider that a tile that needs more than, let’s say, 10 sec to draw is
not worth rendering.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">But then Apache mod_jk load-balancer put that very Tomcat in FAIL state and I could not figure out a configuration that actually works.<u></u><u></u></span></p>


<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">Regards,<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">Laurent<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;">De :</span></b><span style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;"> Mike Grogan [mailto:<a href="mailto:***@gmail.com" target="_blank">***@gmail.com</a>]
<br>
<b>Envoyé :</b> vendredi 8 août 2014 18:04</span></p><div class=""><br>
<b>À :</b> PREVOSTO, Laurent<br>
<b>Cc :</b> <a href="mailto:geoserver-***@lists.sourceforge.net" target="_blank">geoserver-***@lists.sourceforge.net</a><br>
<b>Objet :</b> Re: [Geoserver-users] Limiting rendering for large resultsets<u></u><u></u></div><p></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">A crude approach I have used in the past is to add an integer attribute to my points or lines and then to assign a random attribute value from a range that is based on the density of features in the area.  I then specify in the SLD that
only features greater than some number should be displayed for particular zoom levels.<u></u><u></u></p><div><div class="h5"> <div> <p class="MsoNormal"><u></u>?<u></u></p> </div> <div> <p class="MsoNormal">For instance,?<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u>?<u></u></p> </div> <div> <p class="MsoNormal">For your city features, you might assign a random integer attribute to each of them from a range between, say, 0 and 1000.?<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u>?<u></u></p> </div> <div> <p class="MsoNormal">For your countryside features, you might assign a random integer attribute that have a range between 700 and 1000.?<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u>?<u></u></p> </div> <div> <p class="MsoNormal">Then, say for a zoom level 5 or 6, you might specify in the SLD to only show features whose attribute &gt; 700. ?<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u>?<u></u></p> </div> <div> <p class="MsoNormal">Doing this, you end up keeping all of the countryside features, but randomly filtering out city features.?<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u>?<u></u></p> </div> <div> <p class="MsoNormal">At greater zoom levels ... just say 9, 10, or so (depending on your situation), you then show features whose attribute &gt; 500, 400, etc., thereby allowing more city features to be shown as you zoom in.�<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u>�<u></u></p> </div> <div> <p class="MsoNormal">I could probably have done some statistics to tell me the exact ranges to use, but trial &amp; error worked fine for me.�<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u>�<u></u></p> </div> <div> <p class="MsoNormal">- Mike Grogan<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u>�<u></u></p> </div> <div> <p class="MsoNormal"><u></u>�<u></u></p> </div> <div> <p class="MsoNormal"><u></u>�<u></u></p> <div> <p class="MsoNormal" style="margin-bottom:12.0pt"><u></u>�<u></u></p> <div> <p class="MsoNormal">On Wed, Aug 6, 2014 at 5:39 AM, PREVOSTO, Laurent &lt;<a href="mailto:***@sfr.com" target="_blank">***@sfr.com</a>&gt; wrote:<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">Hello,</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">We have geoserver based WMS services that display a quite large network (lots of points and polylines stored in an Oracle Spatial database).</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">When in the cities, the number of items can be very large</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">In the countryside, of course, the density of polylines is quite low.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">If I try to render  a whole city, since there are too many items to render, the request will timeout.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">But if I set a scale condition to avoid drawing thousands of lines when in a large city, users complain that they can’t find the cables when in the countryside…</span><u></u><u></u></p>


<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">So my question was : is there a way to tell geoserver to render a blank image if there are more than n items in the Oracle resultset ?</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">That way, I could keep a large scale in my TLD without going timeout in areas where there is a high density of items.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">Regards,</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#888888"> </span><span style="color:#888888"><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#888888">Laurent</span><span style="color:#888888"><u></u><u></u></span></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
------------------------------------------------------------------------------<br>
Infragistics Professional<br>
Build stunning WinForms apps today!<br>
Reboot your WinForms applications with our WinForms controls.<br>
Build a bridge from your legacy apps to the future.<br>
<a href="http://pubads.g.doubleclick.net/gampad/clk?id=153845071&amp;iu=/4140/ostg.clktrk" target="_blank">http://pubads.g.doubleclick.net/gampad/clk?id=153845071&amp;iu=/4140/ostg.clktrk</a><br>
_______________________________________________<br>
Geoserver-users mailing list<br>
<a href="mailto:Geoserver-***@lists.sourceforge.net" target="_blank">Geoserver-***@lists.sourceforge.net</a><br>
<a href="https://lists.sourceforge.net/lists/listinfo/geoserver-users" target="_blank">https://lists.sourceforge.net/lists/listinfo/geoserver-users</a><u></u><u></u></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div></div></div>
</div>
</div>

<br>------------------------------------------------------------------------------<br>
<br>_______________________________________________<br>
Geoserver-users mailing list<br>
<a href="mailto:Geoserver-***@lists.sourceforge.net">Geoserver-***@lists.sourceforge.net</a><br>
<a href="https://lists.sourceforge.net/lists/listinfo/geoserver-users" target="_blank">https://lists.sourceforge.net/lists/listinfo/geoserver-users</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr"><div><font size="1" color="#999999">Daniel Bevilacqua</font></div><div><font size="1" color="#999999"><br></font></div><div><font size="1" color="#999999">cel +55 19 81383157</font></div>

<font size="1" color="#999999"><br></font><div><font size="1" color="#999999"><br></font></div></div>
</div>

--f46d043c08eaa3ece905007294c2--
unknown
1970-01-01 00:00:00 UTC
Permalink
--001a11c348f0f9a88705007cce7c
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Interesting, I am reminded of the gvSig renderer which can "draw" the
shapefile index - getting off the ride when the index node gets down below
the size of a pixel. Think that is one of the only optimisations they have
left that we did not implement.

Note that if you are using PostGIS (or Oracle?) GeoTools will ask the
database to simplify (stepping size based on the distance a pixel covers at
current zoom level). This is great as it allows less data to be sent over
from the database. For most other data stores it will simplify in Java
before transform and rendering.

Chris it would be *great* if you could write the occasional blog for the
geotools website - this is really cool stuff you are working on.

Jody Garnett
Post by Chris Bennight
As another take on this same problem we've been working on a Z-order
occlusion based distributed renderer. (This is for an accumulo <->
geoserver datastore implementation).
Currently working is a point only implementation (well, it works for
polygons/linestrings, but calculates occlusion for the centroid only - so
only works "right" for points))
It's implemented as a WPS render transform process
https://github.com/ngageoint/geowave/blob/f40a547a5227540d1b68bd75ab2057ecc7ed00cd/geowave-gt/src/main/java/mil/nga/giat/geowave/gt/DecimationProcess.java
Here's the query setup code
https://github.com/ngageoint/geowave/blob/f40a547a5227540d1b68bd75ab2057ecc7ed00cd/geowave-gt/src/main/java/mil/nga/giat/geowave/gt/SpatialDecimationQuery.java
Index thinning
https://github.com/ngageoint/geowave/blob/f40a547a5227540d1b68bd75ab2057ecc7ed00cd/geowave-gt/src/main/java/mil/nga/giat/geowave/gt/FixedCardinalitySkippingIterator.java
This implementation doesn't distribute the rendering process; rather it
defines a pixel space -> index range transform function, and when a feature
(point in this case) is found it's returned, and the seek function in the
database skips to the next pixel
https://dl.dropboxusercontent.com/u/6649380/decimation.png
--
The next phase, and what I think a general solution to the problem you
describe is to distribute the rendering process (in accumulo the database
"nodes" sit on top of multiple instances for easy horizontal scalability).
This portion has to take into account the SLD selected as well, as the
style can have a big impact on how features are rendered.
Basically each featurestylerule results in a distributed call for that
feature type - and the tile for that rule is rendered local to the majority
of the data (best effort) in the cluster. When the tile is completely
covered (or passes some heuristic, i.e. 80%, etc.) the seek stops and the
"full" tile returns. (There is some skipping here to - when a feature is
rendered an internal table of index ranges <-> pixel transforms is modified
to remove the covered pixels/ranges - so the next feature always overlaps a
non-colored region.)
All the tiles are now composted together in geoserver based on the
ordering from the SLD - and hopefully we have now skipped reading and
writing a bunch of data we don't have a "pixel budget" to actually display.
This portion is still in work - I think we might have an upcoming
geoserver pull request, as we need to make a few private methods protected
so we can distribute the render process across the cluster. Our internal
implementation just cuts and pastes a bunch of GPL code, and I didn't want
to include that since the rest of the project is apache 2.0 and, well, I
didn't want to explain the intermingled licenses.
Our torture case right now is basically one of the openstreetmap SLD's at
~ zoom level 17 rules, rendered globally (with the full osm planet dump).
--
That said, these methods have to be implemented at the geotools datastore
level - and specifically you have to have control enough of the indexing so
that you can short circuit reads. But it's something that could
conceptually (implementation would be slightly different) be to other
systems as well (certainly on postgis, since it's open source)
On Tue, Aug 12, 2014 at 2:01 PM, PREVOSTO, Laurent <
Post by PREVOSTO, Laurent
Hi,
I understand your approach but i definately prefer to draw nothing or an
“error” tile than just draw a random part of the network (a wrong
information)
Because, users tend to believe what they see J
In my case, I am tempted to lower the timeout rendering of geoserver and
consider that a tile that needs more than, let’s say, 10 sec to draw is not
worth rendering.
But then Apache mod_jk load-balancer put that very Tomcat in FAIL state
and I could not figure out a configuration that actually works.
Regards,
Laurent
*Envoyé :* vendredi 8 août 2014 18:04
*À :* PREVOSTO, Laurent
*Objet :* Re: [Geoserver-users] Limiting rendering for large resultsets
A crude approach I have used in the past is to add an integer attribute
to my points or lines and then to assign a random attribute value from a
range that is based on the density of features in the area. I then specify
in the SLD that only features greater than some number should be displayed
for particular zoom levels.
For instance,
For your city features, you might assign a random integer attribute to
each of them from a range between, say, 0 and 1000.
For your countryside features, you might assign a random integer
attribute that have a range between 700 and 1000.
Then, say for a zoom level 5 or 6, you might specify in the SLD to only
show features whose attribute > 700.
Doing this, you end up keeping all of the countryside features, but
randomly filtering out city features.
At greater zoom levels ... just say 9, 10, or so (depending on your
situation), you then show features whose attribute > 500, 400, etc.,
thereby allowing more city features to be shown as you zoom in.
I could probably have done some statistics to tell me the exact ranges to
use, but trial & error worked fine for me.
- Mike Grogan
On Wed, Aug 6, 2014 at 5:39 AM, PREVOSTO, Laurent <
Hello,
We have geoserver based WMS services that display a quite large network
(lots of points and polylines stored in an Oracle Spatial database).
When in the cities, the number of items can be very large
In the countryside, of course, the density of polylines is quite low.
If I try to render a whole city, since there are too many items to
render, the request will timeout.
But if I set a scale condition to avoid drawing thousands of lines when
in a large city, users complain that they can’t find the cables when in the
countryside…
So my question was : is there a way to tell geoserver to render a blank
image if there are more than n items in the Oracle resultset ?
That way, I could keep a large scale in my TLD without going timeout in
areas where there is a high density of items.
Regards,
Laurent
------------------------------------------------------------------------------
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls.
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk
_______________________________________________
Geoserver-users mailing list
https://lists.sourceforge.net/lists/listinfo/geoserver-users
------------------------------------------------------------------------------
_______________________________________________
Geoserver-users mailing list
https://lists.sourceforge.net/lists/listinfo/geoserver-users
------------------------------------------------------------------------------
_______________________________________________
Geoserver-users mailing list
https://lists.sourceforge.net/lists/listinfo/geoserver-users
--001a11c348f0f9a88705007cce7c
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable <div dir="ltr">Interesting, I am reminded of the gvSig renderer which can &quot;draw&quot; the shapefile index - getting off the ride when the index node gets down below the size of a pixel. Think that is one of the only optimisations they have left that we did not implement.<div> <br></div><div>Note that if you are using PostGIS (or Oracle?) GeoTools will ask the database to simplify (stepping size based on the distance a pixel covers at current zoom level). This is great as it allows less data to be sent over from the database. For most other data stores it will simplify in Java before transform and rendering.</div> <div><br></div><div>Chris it would be *great* if you could write the occasional blog for the geotools website - this is really cool stuff you are working on.</div></div><div class="gmail_extra"><br clear="all"><div><div dir="ltr"> <div>Jody Garnett</div></div></div> <br><br><div class="gmail_quote">On Tue, Aug 12, 2014 at 5:36 PM, Chris Bennight <span dir="ltr">&lt;<a href="mailto:***@slowcar.net" target="_blank">***@slowcar.net</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div dir="ltr">As another take on this same problem we&#39;ve been working on a Z-order occlusion based distributed renderer. �(This is for an accumulo &lt;-&gt; geoserver datastore implementation).<div><br></div><div>Currently working is a point only implementation (well, it works for polygons/linestrings, but calculates occlusion for the centroid only - so only works &quot;right&quot; for points))</div> <div><br></div><div>It&#39;s implemented as a WPS render transform process</div><div><a href="https://github.com/ngageoint/geowave/blob/f40a547a5227540d1b68bd75ab2057ecc7ed00cd/geowave-gt/src/main/java/mil/nga/giat/geowave/gt/DecimationProcess.java" target="_blank">https://github.com/ngageoint/geowave/blob/f40a547a5227540d1b68bd75ab2057ecc7ed00cd/geowave-gt/src/main/java/mil/nga/giat/geowave/gt/DecimationProcess.java</a><br> </div><div><br></div><div>Here&#39;s the query setup code</div><div><a href="https://github.com/ngageoint/geowave/blob/f40a547a5227540d1b68bd75ab2057ecc7ed00cd/geowave-gt/src/main/java/mil/nga/giat/geowave/gt/SpatialDecimationQuery.java" target="_blank">https://github.com/ngageoint/geowave/blob/f40a547a5227540d1b68bd75ab2057ecc7ed00cd/geowave-gt/src/main/java/mil/nga/giat/geowave/gt/SpatialDecimationQuery.java</a><br> </div><div><br></div><div>Index thinning</div><div><a href="https://github.com/ngageoint/geowave/blob/f40a547a5227540d1b68bd75ab2057ecc7ed00cd/geowave-gt/src/main/java/mil/nga/giat/geowave/gt/FixedCardinalitySkippingIterator.java" target="_blank">https://github.com/ngageoint/geowave/blob/f40a547a5227540d1b68bd75ab2057ecc7ed00cd/geowave-gt/src/main/java/mil/nga/giat/geowave/gt/FixedCardinalitySkippingIterator.java</a><br> </div><div><br></div><div>This implementation doesn&#39;t distribute the rendering process; rather it defines a pixel space -&gt; index range transform function, and when a feature (point in this case) is found it&#39;s returned, and the seek function in the database skips to the next pixel</div>

<div><br></div><div>Here&#39;s a graphic example:</div><div><a href="https://dl.dropboxusercontent.com/u/6649380/decimation.png" target="_blank">https://dl.dropboxusercontent.com/u/6649380/decimation.png</a><br></div><div>
<br></div><div>
--</div><div><br></div><div>The next phase, and what I think a general solution to the problem you describe is to distribute the rendering process ?(in accumulo the database &quot;nodes&quot; sit on top of multiple instances for easy horizontal scalability).</div> <div><br></div><div>This portion has to take into account the SLD selected as well, as the style can have a big impact on how features are rendered. �</div><div><br></div><div>Basically each featurestylerule results in a distributed call for that feature type - and the tile for that rule is rendered local to the majority of the data (best effort) in the cluster. �When the tile is completely covered (or passes some heuristic, i.e. 80%, etc.) �the seek stops and the &quot;full&quot; tile returns. � (There is some skipping here to - when a feature is rendered an internal table of index ranges &lt;-&gt; pixel transforms is modified to remove the covered pixels/ranges - so the next feature always overlaps a non-colored region.)</div> <div><br></div><div>All the tiles are now composted together in geoserver based on the ordering from the SLD - and hopefully we have now skipped reading and writing a bunch of data we don&#39;t have a &quot;pixel budget&quot; to actually display.</div> <div><br></div><div>This portion is still in work - I think we might have an upcoming geoserver pull request, as we need to make a few private methods protected so we can distribute the render process across the cluster. � Our internal implementation just cuts and pastes a bunch of GPL code, and I didn&#39;t want to include that since the rest of the project is apache 2.0 and, well, I didn&#39;t want to explain the intermingled licenses. ��</div> <div><br></div><div>Our torture case right now is basically one of the openstreetmap SLD&#39;s at ~ zoom level 17 rules, rendered globally (with the full osm planet dump). ��</div><div><br></div><div><br></div><div>--</div> <div><br></div><div>That said, these methods have to be implemented at the geotools datastore level - and specifically you have to have control enough of the indexing so that you can short circuit reads. � But it&#39;s something that could conceptually (implementation would be slightly different) be to other systems as well �(certainly on postgis, since it&#39;s open source)</div> </div><div class="gmail_extra"><br><br><div class="gmail_quote"><div class="">On Tue, Aug 12, 2014 at 2:01 PM, PREVOSTO, Laurent <span dir="ltr">&lt;<a href="mailto:***@sfr.com" target="_blank">***@sfr.com</a>&gt;</span> wrote:<br>

</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5">





<div lang="FR" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">Hi,<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">I understand your approach but i definately prefer to draw nothing or an “error” tile than just draw a random part of the network (a wrong information)<u></u><u></u></span></p>


<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">Because, users tend to believe what they see
</span><span lang="EN-US" style="font-size:11.0pt;font-family:Wingdings;color:#1f497d">J</span><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d"><u></u><u></u></span></p>


<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">In my case, I am tempted to lower the timeout rendering of geoserver and consider that a tile that needs more than, let’s say, 10 sec to draw is
not worth rendering.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">But then Apache mod_jk load-balancer put that very Tomcat in FAIL state and I could not figure out a configuration that actually works.<u></u><u></u></span></p>


<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">Regards,<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,&quot;sans-serif&quot;;color:#1f497d">Laurent<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;">De :</span></b><span style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;"> Mike Grogan [mailto:<a href="mailto:***@gmail.com" target="_blank">***@gmail.com</a>]
<br>
<b>Envoyé :</b> vendredi 8 août 2014 18:04</span></p><div><br>
<b>À :</b> PREVOSTO, Laurent<br>
<b>Cc :</b> <a href="mailto:geoserver-***@lists.sourceforge.net" target="_blank">geoserver-***@lists.sourceforge.net</a><br>
<b>Objet :</b> Re: [Geoserver-users] Limiting rendering for large resultsets<u></u><u></u></div><p></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">A crude approach I have used in the past is to add an integer attribute to my points or lines and then to assign a random attribute value from a range that is based on the density of features in the area.  I then specify in the SLD that
only features greater than some number should be displayed for particular zoom levels.<u></u><u></u></p><div><div> <div> <p class="MsoNormal"><u></u>?<u></u></p> </div> <div> <p class="MsoNormal">For instance,?<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u>?<u></u></p> </div> <div> <p class="MsoNormal">For your city features, you might assign a random integer attribute to each of them from a range between, say, 0 and 1000.?<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u>?<u></u></p> </div> <div> <p class="MsoNormal">For your countryside features, you might assign a random integer attribute that have a range between 700 and 1000.?<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u>?<u></u></p> </div> <div> <p class="MsoNormal">Then, say for a zoom level 5 or 6, you might specify in the SLD to only show features whose attribute &gt; 700. ?<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u>?<u></u></p> </div> <div> <p class="MsoNormal">Doing this, you end up keeping all of the countryside features, but randomly filtering out city features.?<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u>?<u></u></p> </div> <div> <p class="MsoNormal">At greater zoom levels ... just say 9, 10, or so (depending on your situation), you then show features whose attribute &gt; 500, 400, etc., thereby allowing more city features to be shown as you zoom in.�<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u>�<u></u></p> </div> <div> <p class="MsoNormal">I could probably have done some statistics to tell me the exact ranges to use, but trial &amp; error worked fine for me.�<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u>�<u></u></p> </div> <div> <p class="MsoNormal">- Mike Grogan<u></u><u></u></p> </div> <div> <p class="MsoNormal"><u></u>�<u></u></p> </div> <div> <p class="MsoNormal"><u></u>�<u></u></p> </div> <div> <p class="MsoNormal"><u></u>�<u></u></p> <div> <p class="MsoNormal" style="margin-bottom:12.0pt"><u></u>�<u></u></p> <div> <p class="MsoNormal">On Wed, Aug 6, 2014 at 5:39 AM, PREVOSTO, Laurent &lt;<a href="mailto:***@sfr.com" target="_blank">***@sfr.com</a>&gt; wrote:<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">Hello,</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">We have geoserver based WMS services that display a quite large network (lots of points and polylines stored in an Oracle Spatial database).</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">When in the cities, the number of items can be very large</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">In the countryside, of course, the density of polylines is quite low.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">If I try to render  a whole city, since there are too many items to render, the request will timeout.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">But if I set a scale condition to avoid drawing thousands of lines when in a large city, users complain that they can’t find the cables when in the countryside…</span><u></u><u></u></p>


<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">So my question was : is there a way to tell geoserver to render a blank image if there are more than n items in the Oracle resultset ?</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">That way, I could keep a large scale in my TLD without going timeout in areas where there is a high density of items.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">Regards,</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#888888"> </span><span style="color:#888888"><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#888888">Laurent</span><span style="color:#888888"><u></u><u></u></span></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
------------------------------------------------------------------------------<br>
Infragistics Professional<br>
Build stunning WinForms apps today!<br>
Reboot your WinForms applications with our WinForms controls.<br>
Build a bridge from your legacy apps to the future.<br>
<a href="http://pubads.g.doubleclick.net/gampad/clk?id=153845071&amp;iu=/4140/ostg.clktrk" target="_blank">http://pubads.g.doubleclick.net/gampad/clk?id=153845071&amp;iu=/4140/ostg.clktrk</a><br>
_______________________________________________<br>
Geoserver-users mailing list<br>
<a href="mailto:Geoserver-***@lists.sourceforge.net" target="_blank">Geoserver-***@lists.sourceforge.net</a><br>
<a href="https://lists.sourceforge.net/lists/listinfo/geoserver-users" target="_blank">https://lists.sourceforge.net/lists/listinfo/geoserver-users</a><u></u><u></u></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div></div></div>
</div>
</div>

<br></div></div>------------------------------------------------------------------------------<div class=""><br>
<br>_______________________________________________<br>
Geoserver-users mailing list<br>
<a href="mailto:Geoserver-***@lists.sourceforge.net" target="_blank">Geoserver-***@lists.sourceforge.net</a><br>
<a href="https://lists.sourceforge.net/lists/listinfo/geoserver-users" target="_blank">https://lists.sourceforge.net/lists/listinfo/geoserver-users</a><br>
<br></div></blockquote></div><br></div>
<br>------------------------------------------------------------------------------<br>
<br>_______________________________________________<br>
Geoserver-users mailing list<br>
<a href="mailto:Geoserver-***@lists.sourceforge.net">Geoserver-***@lists.sourceforge.net</a><br>
<a href="https://lists.sourceforge.net/lists/listinfo/geoserver-users" target="_blank">https://lists.sourceforge.net/lists/listinfo/geoserver-users</a><br>
<br></blockquote></div><br></div>

--001a11c348f0f9a88705007cce7c--
Continue reading on narkive:
Loading...