[Geoserver-users] Is app-schema-cache is no longer automatically created

Discussion:

Michael Sexton

2017-05-15 05:59:29 UTC

Hi

I was trying to build a new GeoServer instance today using 2.10.3, and I found that the app-schema-cache needs to be present before you could import your mapping files. Is this a feature or a bug? I could have sworn that in previous versions (2.9 and previously) the folder was made for you.

Thanks

Michael

Ben Caradoc-Davies

2017-05-15 22:01:27 UTC

Permalink

Michael,

I just tested with the app-schema tutorial on master and
app-schema-cache was automatically created as soon as I accessed an
app-schema feature type. I had to fix the schemaUri because
www.geosciml.org was renamed to schemas.geosciml.org and there are still
broken imports of cgiutilities, but I can see the automatic download
working.

- What application schema are you using?

- Do you see any exceptions in the logs?

Kind regards,
Ben.

Post by Michael Sexton
Hi
I was trying to build a new GeoServer instance today using 2.10.3, and I found that the app-schema-cache needs to be present before you could import your mapping files. Is this a feature or a bug? I could have sworn that in previous versions (2.9 and previously) the folder was made for you.
Thanks
Michael
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Geoserver-users mailing list
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Ben Caradoc-Davies <***@transient.nz>
Director
Transient Software Limited <http://transient.nz/>
New Zealand

Ben Caradoc-Davies

2017-05-15 22:11:34 UTC

Permalink

Michael,

this is the old GeoSciML 2.0 schemaUri, which no longer works:
http://www.geosciml.org/geosciml/2.0/xsd/geosciml.xsd

The new GeoSciML 2.0 schemaUri has a new host name and removes the
"xsd" path element:
http://schemas.geosciml.org/geosciml/2.0/geosciml.xsd

Other versions of GeoSciML might have been affected by these changes.

Kind regards,
Ben.

Post by Ben Caradoc-Davies
Michael,
I just tested with the app-schema tutorial on master and
app-schema-cache was automatically created as soon as I accessed an
app-schema feature type. I had to fix the schemaUri because
www.geosciml.org was renamed to schemas.geosciml.org and there are still
broken imports of cgiutilities, but I can see the automatic download
working.
- What application schema are you using?
- Do you see any exceptions in the logs?
Kind regards,
Ben.

Post by Michael Sexton
Hi
I was trying to build a new GeoServer instance today using 2.10.3, and
I found that the app-schema-cache needs to be present before you could
import your mapping files. Is this a feature or a bug? I could have
sworn that in previous versions (2.9 and previously) the folder was
made for you.
Thanks
Michael
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Geoserver-users mailing list
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Ben Caradoc-Davies <***@transient.nz>
Director
Transient Software Limited <http://transient.nz/>
New Zealand

Michael Sexton

2017-05-16 00:35:46 UTC

Permalink

Hi Ben

Iâm using GeoSciML Basic 4 (http://schemas.opengis.net/gsml/4.1/geoSciMLBasic.xsd <http://schemas.opengis.net/gsml/4.1/geoSciMLBasic.xsd>)

As far as I can tell, this resolves. But there may be other problems in the schema documents I am not aware of.

On the case of exception logs, I get:

Caused by: java.lang.RuntimeException: Failed to resolve http://schemas.opengis.net/gsml/4.1/geoSciMLBasic.xsd
at org.geotools.xml.resolver.SchemaResolver.resolve(SchemaResolver.java:207)

I inspected this using the debugger in Eclipse, and the problem is that the following clause in SchemaResolver evaluates to false, because the cache variable is null.

// Use download cache.
if (resolvedLocation == null && cache != null) {
resolvedLocation = cache.resolveLocation(location);
}

After creating the app-schema-cache directory, the cache variable is of type org.geotools.xml.resolver.SchemaCache and points to the correct directory.

Post by Ben Caradoc-Davies
Michael,
I just tested with the app-schema tutorial on master and app-schema-cache was automatically created as soon as I accessed an app-schema feature type. I had to fix the schemaUri because www.geosciml.org was renamed to schemas.geosciml.org and there are still broken imports of cgiutilities, but I can see the automatic download working.
- What application schema are you using?
- Do you see any exceptions in the logs?
Kind regards,
Ben.

--
Director
Transient Software Limited <http://transient.nz/>
New Zealand

Ben Caradoc-Davies

2017-05-16 01:13:46 UTC

Permalink

Michael,

because there is no interface to allow GeoTools to discover the data
directory of an enclosing GeoServer, the schema cache searches for a
suitable cache location starting from the mapping file and searching all
ancestor directories for something that looks like a GeoServer data
directory:
https://github.com/geotools/geotools/blob/master/modules/library/xml/src/main/java/org/geotools/xml/resolver/SchemaCache.java#L420

A directory is recognised as suitable if it contains a subdirectory
"app-schema-cache" or it contains all the files "global.xml",
"wcs.xml","wfs.xml", and "wms.xml" and both subdirectories "styles" and
"workspaces".

By manually creating "app-schema-cache", you satisfy the first criterion.

Does your data directory contain:
(1) all the files "global.xml", "wcs.xml","wfs.xml", and "wms.xml", and
(2) both directories "styles" and "workspaces"?

It is possible to start GeoServer with an empty data directory. Did you
do this? After starting GeoServer once, are any of those files missing?
We might need to adjust the test.

Kind regards,
Ben.

Post by Michael Sexton
Hi Ben
I’m using GeoSciML Basic 4 (http://schemas.opengis.net/gsml/4.1/geoSciMLBasic.xsd <http://schemas.opengis.net/gsml/4.1/geoSciMLBasic.xsd>)
As far as I can tell, this resolves. But there may be other problems in the schema documents I am not aware of.
Caused by: java.lang.RuntimeException: Failed to resolve http://schemas.opengis.net/gsml/4.1/geoSciMLBasic.xsd
at org.geotools.xml.resolver.SchemaResolver.resolve(SchemaResolver.java:207)
I inspected this using the debugger in Eclipse, and the problem is that the following clause in SchemaResolver evaluates to false, because the cache variable is null.
// Use download cache.
if (resolvedLocation == null && cache != null) {
resolvedLocation = cache.resolveLocation(location);
}
After creating the app-schema-cache directory, the cache variable is of type org.geotools.xml.resolver.SchemaCache and points to the correct directory.

Post by Ben Caradoc-Davies
Michael,
I just tested with the app-schema tutorial on master and app-schema-cache was automatically created as soon as I accessed an app-schema feature type. I had to fix the schemaUri because www.geosciml.org was renamed to schemas.geosciml.org and there are still broken imports of cgiutilities, but I can see the automatic download working.
- What application schema are you using?
- Do you see any exceptions in the logs?
Kind regards,
Ben.

--
Director
Transient Software Limited <http://transient.nz/>
New Zealand

--
Ben Caradoc-Davies <***@transient.nz>
Director
Transient Software Limited <http://transient.nz/>
New Zealand

Michael Sexton

2017-05-16 01:44:46 UTC

Permalink

Ben

That looks like the issue. I have been starting Geoserver with empty data directories as opposed to our previous method of inheriting default data directories from one implementation to the next, and cleaning out the relevant folders (workspaces, styles, app-schema-cache) per implementation. I must have falsely assumed in previous cases where app-schema-cache failed that we were having another problem with our corporate proxy (and just copy-pasted earlier app-schema-caches as a temporary workaround, which turns out solved them problem but in a roundabout way).

On initial Geoserver start, the following files and subdirectories are created in the data directory

gwc
gwc-gs.xml
logging.xml
security
styles

So all the requisite XML files and the workspaces folder is missing on startup. At point of App Schema Data Store creation the following files and subdirectories exist:

global.xml
gwc-gs.xml
security
workspaces
gwc
logging.xml
styles

So there are still files missing that the schema cache will be looking for.

Given the cache test is the last clause in the schema resolver logic before an error is thrown, would it be possible to alert the user that the failure is in an inability to find the data directory, as opposed to what it says now in not being able to resolve the schema URI?

Thanks

Michael

Post by Ben Caradoc-Davies
Michael,
https://github.com/geotools/geotools/blob/master/modules/library/xml/src/main/java/org/geotools/xml/resolver/SchemaCache.java#L420 <https://github.com/geotools/geotools/blob/master/modules/library/xml/src/main/java/org/geotools/xml/resolver/SchemaCache.java#L420>
A directory is recognised as suitable if it contains a subdirectory "app-schema-cache" or it contains all the files "global.xml", "wcs.xml","wfs.xml", and "wms.xml" and both subdirectories "styles" and "workspaces".
By manually creating "app-schema-cache", you satisfy the first criterion.
(1) all the files "global.xml", "wcs.xml","wfs.xml", and "wms.xml", and
(2) both directories "styles" and "workspaces"?
It is possible to start GeoServer with an empty data directory. Did you do this? After starting GeoServer once, are any of those files missing? We might need to adjust the test.
Kind regards,
Ben.

Post by Michael Sexton
Hi Ben
Iâm using GeoSciML Basic 4 (http://schemas.opengis.net/gsml/4.1/geoSciMLBasic.xsd <http://schemas.opengis.net/gsml/4.1/geoSciMLBasic.xsd><http://schemas.opengis.net/gsml/4.1/geoSciMLBasic.xsd <http://schemas.opengis.net/gsml/4.1/geoSciMLBasic.xsd>>)
As far as I can tell, this resolves. But there may be other problems in the schema documents I am not aware of.
Caused by: java.lang.RuntimeException: Failed to resolve http://schemas.opengis.net/gsml/4.1/geoSciMLBasic.xsd
at org.geotools.xml.resolver.SchemaResolver.resolve(SchemaResolver.java:207)
I inspected this using the debugger in Eclipse, and the problem is that the following clause in SchemaResolver evaluates to false, because the cache variable is null.
// Use download cache.
if (resolvedLocation == null && cache != null) {
resolvedLocation = cache.resolveLocation(location);
}
After creating the app-schema-cache directory, the cache variable is of type org.geotools.xml.resolver.SchemaCache and points to the correct directory.

Post by Ben Caradoc-Davies
Michael,
I just tested with the app-schema tutorial on master and app-schema-cache was automatically created as soon as I accessed an app-schema feature type. I had to fix the schemaUri because www.geosciml.org was renamed to schemas.geosciml.org and there are still broken imports of cgiutilities, but I can see the automatic download working.
- What application schema are you using?
- Do you see any exceptions in the logs?
Kind regards,
Ben.

--
Director
Transient Software Limited <http://transient.nz/>
New Zealand

--
Director
Transient Software Limited <http://transient.nz/ <http://transient.nz/>>
New Zealand

Ben Caradoc-Davies

2017-05-16 02:30:03 UTC

Permalink

Thanks for the confirmation, Michael. If I start with a data directory
that is empty except for the app-schema tutorial workspaces directory, I
see even fewer files at the moment the cache is constructed, just a
logging.xml and the workspaces directory. The logic needs to change to
support an empty data directory.

I like your idea of the warning, but I think it should be logged when
the data store is initially configured. Logging when resolution fails
will cause failed OASIS catalog tests to give misleading warnings, not
to mention that this class is used elsewhere and may not be inside
GeoServer at all (the GeoTools wfs-ng client can use it). I think it is
more robust and general to warn when the app-schema data store is loaded.

I have created a Jira issue:

[GEOT-5729] Schema cache autoconfiguration fails for empty GeoServer
data directory
https://osgeo-org.atlassian.net/browse/GEOT-5729

Kind regards,
Ben.

Post by Michael Sexton
Ben
That looks like the issue. I have been starting Geoserver with empty data directories as opposed to our previous method of inheriting default data directories from one implementation to the next, and cleaning out the relevant folders (workspaces, styles, app-schema-cache) per implementation. I must have falsely assumed in previous cases where app-schema-cache failed that we were having another problem with our corporate proxy (and just copy-pasted earlier app-schema-caches as a temporary workaround, which turns out solved them problem but in a roundabout way).
On initial Geoserver start, the following files and subdirectories are created in the data directory
gwc
gwc-gs.xml
logging.xml
security
styles
global.xml
gwc-gs.xml
security
workspaces
gwc
logging.xml
styles
So there are still files missing that the schema cache will be looking for.
Given the cache test is the last clause in the schema resolver logic before an error is thrown, would it be possible to alert the user that the failure is in an inability to find the data directory, as opposed to what it says now in not being able to resolve the schema URI?
Thanks
Michael

Post by Michael Sexton
Hi Ben
I’m using GeoSciML Basic 4 (http://schemas.opengis.net/gsml/4.1/geoSciMLBasic.xsd <http://schemas.opengis.net/gsml/4.1/geoSciMLBasic.xsd><http://schemas.opengis.net/gsml/4.1/geoSciMLBasic.xsd <http://schemas.opengis.net/gsml/4.1/geoSciMLBasic.xsd>>)
As far as I can tell, this resolves. But there may be other problems in the schema documents I am not aware of.
Caused by: java.lang.RuntimeException: Failed to resolve http://schemas.opengis.net/gsml/4.1/geoSciMLBasic.xsd
at org.geotools.xml.resolver.SchemaResolver.resolve(SchemaResolver.java:207)
I inspected this using the debugger in Eclipse, and the problem is that the following clause in SchemaResolver evaluates to false, because the cache variable is null.
// Use download cache.
if (resolvedLocation == null && cache != null) {
resolvedLocation = cache.resolveLocation(location);
}
After creating the app-schema-cache directory, the cache variable is of type org.geotools.xml.resolver.SchemaCache and points to the correct directory.

Post by Ben Caradoc-Davies
Michael,
I just tested with the app-schema tutorial on master and app-schema-cache was automatically created as soon as I accessed an app-schema feature type. I had to fix the schemaUri because www.geosciml.org was renamed to schemas.geosciml.org and there are still broken imports of cgiutilities, but I can see the automatic download working.
- What application schema are you using?
- Do you see any exceptions in the logs?
Kind regards,
Ben.

--
Director
Transient Software Limited <http://transient.nz/>
New Zealand

--
Director
Transient Software Limited <http://transient.nz/ <http://transient.nz/>>
New Zealand

--
Ben Caradoc-Davies <***@transient.nz>
Director
Transient Software Limited <http://transient.nz/>
New Zealand