[Geoserver-users] Reference download in geoserver WPS extension

[chadi jaber]

Hello,


I am currently using the WPSextension for geoserver (2.10) and the python script hooks in order to add my own processes. After adding this simple process:


from geoserver.wps import process
from geoscript.layer import GeoTIFF


@process(
inputs={'geom': (GeoTIFF, 'The geometry to buffer')},
outputs={'result': (str, 'The buffered geometry')},
title='Buffer',
description='Buffers a geometry'
)
def run(geom):
s = str(geom.getsize())
return s;


and the following wps execute request:

<?xml version="1.0" encoding="UTF-8"?><wps:Execute version="1.0.0" service="WPS" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.opengis.net/wps/1.0.0" xmlns:wfs="http://www.opengis.net/wfs" xmlns:wps="http://www.opengis.net/wps/1.0.0" xmlns:ows="http://www.opengis.net/ows/1.1" xmlns:gml="http://www.opengis.net/gml" xmlns:ogc="http://www.opengis.net/ogc" xmlns:wcs="http://www.opengis.net/wcs/1.1.1" xmlns:xlink="http://www.w3.org/1999/xlink" xsi:schemaLocation="http://www.opengis.net/wps/1.0.0 http://schemas.opengis.net/wps/1.0.0/wpsAll.xsd">
<ows:Identifier>Chadi:buffer</ows:Identifier>
<wps:DataInputs>
<wps:Input>
<ows:Identifier>geom</ows:Identifier>
<wps:Reference mimeType="image/tiff" xlink:href="http://download.osgeo.org/geotiff/samples/misc/tjpeg.tif" method="GET"/>
</wps:Input>
</wps:DataInputs>
<wps:ResponseForm>
<wps:RawDataOutput>
<ows:Identifier>result</ows:Identifier>
</wps:RawDataOutput>
</wps:ResponseForm>
</wps:Execute>


I noticed that geoserver download the image before giving a handle on the "geom" object in the python script.

is it possible to disable this feature and to have only the URL reference passed to the script hook ?

If no, is it possible to download locally the file before executing the script ?


Thanks in advance,

Chadi

[P O'Toole]

Chadi -

>Hello,

>I am currently using the WPSextension for geoserver (2.10) and the python script hooks in order to add my own processes.

>I noticed that geoserver downloads the image before giving a handle on the "geom" object in the python script.

>Is it possible to disable this feature and to have only the URL reference passed to the script hook ?

>If no, is it possible to download locally the file before executing the script ?

One way around this would be to declare the 'geom' input-parameter as a string, and then simply instantiate objects as-needed within Python. That way, the Java/WPS container will not try to do any downloading for you if you don't want it to, since it would just see a string, rather than a remote resource. Jython should allow you to import the GeoTIFF class from Java as you would a regular Python module so you can build it yourself as needed (e.g. `from org.postgresql import PGStatement` and `import java.sql.SQLException` work fine from Jython, despite those being Java packages rather than Python modules.)

If that approach doesn't fit your requirements, I would recommend finding the handler-class that takes these requests and prepares to give them to Python, since it sounds like it happens before long before Python starts to do any work on the request. In either case, the could should reveal whether there are branches that skip downloads under certain conditions. If there aren't, you may need to create a subclass that has the behavior you want.

- Patrick

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

Geoserver-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

[Andrea Aime]

Hi,
in the Java world a process that wants to do its own encoding/decoding can
be done using
the RawData class as input or output, as follows:

@DescribeResult(name = "result", description = "Output raster", meta = {
"mimeTypes=application/json,text/xml",
"chosenMimeType=outputMimeType" })
public RawData execute(
@DescribeParameter(name = "data", description = "Input
features", meta = { "mimeTypes=application/json,text/xml" }) final RawData
input,
@DescribeParameter(name = "outputMimeType", min = 0) final
String outputMimeType,
@DescribeParameter(name = "returnNull", min = 0, defaultValue =
"false") final boolean returnNull) {

However, I don't know if this can be translated into the scripting modules
support for WPS, or not (these
modules are unsupported and have not been modified in quite some time, not
sure what is their
working state... contributors helping to revive them warmly welcomed :-) ).

Cheers
Andrea


On Thu, Jul 6, 2017 at 10:24 PM, P O'Toole <***@uwyo.edu> wrote:

> Chadi -
>
> >Hello,
>
> >I am currently using the WPSextension for geoserver (2.10) and the python
> script hooks in order to add my own processes.
>
> >I noticed that geoserver downloads the image before giving a handle on
> the "geom" object in the python script.
>
> >Is it possible to disable this feature and to have only the URL reference
> passed to the script hook ?
>
> >If no, is it possible to download locally the file before executing the
> script ?
>
> One way around this would be to declare the 'geom' input-parameter as a
> string, and then simply instantiate objects as-needed within Python. That
> way, the Java/WPS container will not try to do any downloading
> for you if you don't want it to, since it would just see a string, rather
> than a remote resource. Jython should allow you to import the GeoTIFF class
> from Java as you would a regular Python module so you can build it yourself
> as needed (e.g. `from org.postgresql import PGStatement` and `import
> java.sql.SQLException` work fine from Jython, despite those being Java
> packages rather than Python modules.)
>
> If that approach doesn't fit your requirements, I would recommend finding
> the handler-class that takes these requests and prepares to give them to
> Python, since it sounds like it happens before long before Python starts
> to do any work on the request. In either case, the could should reveal
> whether there are branches that skip downloads under certain conditions. If
> there aren't, you may need to create a subclass that has the behavior you
> want.
>
> - Patrick
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Geoserver-users mailing list
>
> Please make sure you read the following two resources before posting to
> this list:
> - Earning your support instead of buying it, but Ian Turton:
> http://www.ianturton.com/talks/foss4g.html#/
> - The GeoServer user list posting guidelines: http://geoserver.org/comm/
> userlist-guidelines.html
>
> Geoserver-***@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/geoserver-users
>



--

Regards,

Andrea Aime

==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V
for more information.
==

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: [phone]
fax: [phone]
mob: [phone]

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo Ú consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

[chadi jaber]

Hello Guys Thanks for your answers,


@Patrice<mailto:inmail-hit-***@linkedin.com>

I considered using a string to replace the geom parameter but we will loose the wps interface typing the geotiff is seen as a LitteralData string type (when we do the describe process request for example).

@Andrea

I think I ll try your solution knowing that there is something like this already implemented in the examples (https://github.com/geoserver/geoserver/blob/master/src/community/script/py/src/test/resources/org/geoserver/script/py/raw.py)

This solution create a more strongly typed XML (relatively to the first solution) however we ll loose the possibility to pass the data as a reference (through the xlink attribute in the input element)


Thanks again,

Chadi



________________________________
De : ***@gmail.com <***@gmail.com> de la part de Andrea Aime <***@geo-solutions.it>
Envoyé : vendredi 7 juillet 2017 09:42
À : P O'Toole
Cc : geoserver-***@lists.sourceforge.net; ***@hotmail.com
Objet : Re: [Geoserver-users] Reference download in geoserver WPS extension

Hi,
in the Java world a process that wants to do its own encoding/decoding can be done using
the RawData class as input or output, as follows:

@DescribeResult(name = "result", description = "Output raster", meta = {
"mimeTypes=application/json,text/xml", "chosenMimeType=outputMimeType" })
public RawData execute(
@DescribeParameter(name = "data", description = "Input features", meta = { "mimeTypes=application/json,text/xml" }) final RawData input,
@DescribeParameter(name = "outputMimeType", min = 0) final String outputMimeType,
@DescribeParameter(name = "returnNull", min = 0, defaultValue = "false") final boolean returnNull) {

However, I don't know if this can be translated into the scripting modules support for WPS, or not (these
modules are unsupported and have not been modified in quite some time, not sure what is their
working state... contributors helping to revive them warmly welcomed :-) ).

Cheers
Andrea


On Thu, Jul 6, 2017 at 10:24 PM, P O'Toole <***@uwyo.edu<mailto:***@uwyo.edu>> wrote:
Chadi -

>Hello,

>I am currently using the WPSextension for geoserver (2.10) and the python script hooks in order to add my own processes.

>I noticed that geoserver downloads the image before giving a handle on the "geom" object in the python script.

>Is it possible to disable this feature and to have only the URL reference passed to the script hook ?

>If no, is it possible to download locally the file before executing the script ?

One way around this would be to declare the 'geom' input-parameter as a string, and then simply instantiate objects as-needed within Python. That way, the Java/WPS container will not try to do any downloading for you if you don't want it to, since it would just see a string, rather than a remote resource. Jython should allow you to import the GeoTIFF class from Java as you would a regular Python module so you can build it yourself as needed (e.g. `from org.postgresql import PGStatement` and `import java.sql.SQLException` work fine from Jython, despite those being Java packages rather than Python modules.)

If that approach doesn't fit your requirements, I would recommend finding the handler-class that takes these requests and prepares to give them to Python, since it sounds like it happens before long before Python starts to do any work on the request. In either case, the could should reveal whether there are branches that skip downloads under certain conditions. If there aren't, you may need to create a subclass that has the behavior you want.

- Patrick

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

Geoserver-***@lists.sourceforge.net<mailto:Geoserver-***@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/geoserver-users



--

Regards,

Andrea Aime

==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
==

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: [phone]
fax: [phone]
mob: [phone]

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

[Andrea Aime]

Hi,
in origin the processes would only receive Java objects, however some
needed access to the
raw contents, so RawData was created.
Do you really need a link to the original data, in other words, a raw
representation of what the
user entered (which could be anything, ranging from a link, to a binary
blob, a POST request, or
even a call to a sub-process).
That seems a bit extreme to me, what's the use case?

As far as I know, there is no way in WPS to declare that you will only
accept a link, besides accepting a string
that's the link itself (and then use the documentation to explain the link
structure, what it should return, and so on).

Cheers
Andrea


On Wed, Jul 12, 2017 at 3:01 PM, chadi jaber <***@hotmail.com>
wrote:

> Hello Guys Thanks for your answers,
>
>
> @Patrice <inmail-hit-***@linkedin.com>
>
> I considered using a string to replace the geom parameter but we will
> loose the wps interface typing the geotiff is seen as a LitteralData string
> type (when we do the describe process request for example).
>
> @Andrea
>
> I think I ll try your solution knowing that there is something like this
> already implemented in the examples (https://github.com/geoserver/
> geoserver/blob/master/src/community/script/py/src/test/
> resources/org/geoserver/script/py/raw.py)
>
> This solution create a more strongly typed XML (relatively to the first
> solution) however we ll loose the possibility to pass the data as a
> reference (through the xlink attribute in the input element)
>
>
> Thanks again,
>
> Chadi
>
>
>
>
> ------------------------------
> *De :* ***@gmail.com <***@gmail.com> de la part de Andrea
> Aime <***@geo-solutions.it>
> *Envoyé :* vendredi 7 juillet 2017 09:42
> *À :* P O'Toole
> *Cc :* geoserver-***@lists.sourceforge.net; ***@hotmail.com
> *Objet :* Re: [Geoserver-users] Reference download in geoserver WPS
> extension
>
> Hi,
> in the Java world a process that wants to do its own encoding/decoding can
> be done using
> the RawData class as input or output, as follows:
>
> @DescribeResult(name = "result", description = "Output raster", meta =
> {
> "mimeTypes=application/json,text/xml", "chosenMimeType=outputMimeType"
> })
> public RawData execute(
> @DescribeParameter(name = "data", description = "Input
> features", meta = { "mimeTypes=application/json,text/xml" }) final
> RawData input,
> @DescribeParameter(name = "outputMimeType", min = 0) final
> String outputMimeType,
> @DescribeParameter(name = "returnNull", min = 0, defaultValue
> = "false") final boolean returnNull) {
>
> However, I don't know if this can be translated into the scripting modules
> support for WPS, or not (these
> modules are unsupported and have not been modified in quite some time, not
> sure what is their
> working state... contributors helping to revive them warmly welcomed :-) ).
>
> Cheers
> Andrea
>
>
> On Thu, Jul 6, 2017 at 10:24 PM, P O'Toole <***@uwyo.edu> wrote:
>
>> Chadi -
>>
>> >Hello,
>>
>> >I am currently using the WPSextension for geoserver (2.10) and the
>> python script hooks in order to add my own processes.
>>
>> >I noticed that geoserver downloads the image before giving a handle on
>> the "geom" object in the python script.
>>
>> >Is it possible to disable this feature and to have only the URL
>> reference passed to the script hook ?
>>
>> >If no, is it possible to download locally the file before executing the
>> script ?
>>
>> One way around this would be to declare the 'geom' input-parameter as a
>> string, and then simply instantiate objects as-needed within Python. That
>> way, the Java/WPS container will not try to do any downloading
>> for you if you don't want it to, since it would just see a string, rather
>> than a remote resource. Jython should allow you to import the GeoTIFF class
>> from Java as you would a regular Python module so you can build it yourself
>> as needed (e.g. `from org.postgresql import PGStatement` and `import
>> java.sql.SQLException` work fine from Jython, despite those being Java
>> packages rather than Python modules.)
>>
>> If that approach doesn't fit your requirements, I would recommend finding
>> the handler-class that takes these requests and prepares to give them to
>> Python, since it sounds like it happens before long before Python starts
>> to do any work on the request. In either case, the could should reveal
>> whether there are branches that skip downloads under certain conditions. If
>> there aren't, you may need to create a subclass that has the behavior you
>> want.
>>
>> - Patrick
>>
>> ------------------------------------------------------------
>> ------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> _______________________________________________
>> Geoserver-users mailing list
>>
>> Please make sure you read the following two resources before posting to
>> this list:
>> - Earning your support instead of buying it, but Ian Turton:
>> http://www.ianturton.com/talks/foss4g.html#/
>> - The GeoServer user list posting guidelines:
>> http://geoserver.org/comm/userlist-guidelines.html
>>
>> Geoserver-***@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/geoserver-users
>>
>
>
>
> --
>
> Regards,
>
> Andrea Aime
>
> ==
> GeoServer Professional Services from the experts! Visit
> http://goo.gl/it488V for more information.
> ==
>
> Ing. Andrea Aime
> @geowolf
> Technical Lead
>
> GeoSolutions S.A.S.
> Via di Montramito 3/A
> 55054 Massarosa (LU)
> phone: [phone] <+39%200584%20962313>
> fax: [phone] <+39%200584%20166%200272>
> mob: [phone] <+39%20339%20884%204549>
>
> http://www.geo-solutions.it
> http://twitter.com/geosolutions_it
>
> AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
>
> Le informazioni contenute in questo messaggio di posta elettronica e/o
> nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
> loro utilizzo Ú consentito esclusivamente al destinatario del messaggio,
> per le finalità indicate nel messaggio stesso. Qualora riceviate questo
> messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
> darcene notizia via e-mail e di procedere alla distruzione del messaggio
> stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
> divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
> utilizzarlo per finalità diverse, costituisce comportamento contrario ai
> principi dettati dal D.Lgs. 196/2003.
>
> The information in this message and/or attachments, is intended solely for
> the attention and use of the named addressee(s) and may be confidential or
> proprietary in nature or covered by the provisions of privacy act
> (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
> Code).Any use not in accord with its purpose, any disclosure, reproduction,
> copying, distribution, or either dissemination, either whole or partial, is
> strictly forbidden except previous formal approval of the named
> addressee(s). If you are not the intended recipient, please contact
> immediately the sender by telephone, fax or e-mail and delete the
> information in this message that has been received in error. The sender
> does not give any warranty or accept liability as the content, accuracy or
> completeness of sent messages and accepts no responsibility for changes
> made after they were sent or for other risks which arise as a result of
> e-mail transmission, viruses, etc.
>
>


--

Regards,

Andrea Aime

==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V
for more information.
==

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: [phone]
fax: [phone]
mob: [phone]

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo Ú consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.