-
Notifications
You must be signed in to change notification settings - Fork 22
ESGFInterfaceGroups
Contents
- Overview
- 1. High Priority Interfaces 1. 1.1 Registry 2. 1.2 Search API 3. 1.3 Security 4. 1.4 THREDDS Profile
- 2. Lower Priority Interfaces 1. 2.1 Information Architecture (Data Model) 2. 2.2 Metadata 3. 2.3 Quality Control and DOIs 4. 2.4 Notification
8 interface documentation groups have been identified. Each group has a lead who is responsible for coordinating discussions and writing the documentation. We have to balance this work with ongoing development and operations of ESGF so the groups are ranked into high and lower priority. Lower priority groups do not imply no work should be done on them but that they are not on the critical path for current deployments and immediate plans.
High priority groups should report on progress at the June 21st ESGF teleconference. Lower priority groups can report briefly on June 21st or can postpone to a future teleconference.
Anyone interested in a group should put themselves down as an interested party and/or contact the group lead.
** Interface Document Groups **
** Lead **
** Priority **
[ Registry Group ](https://github.com/ESGF/esgf.github.io/wiki/ESGFInterfaceGr oups|RegistryGroup)
Gavin
High
Luca
High
[ Security Group ](https://github.com/ESGF/esgf.github.io/wiki/ESGFInterfaceGr oups|SecurityGroup)
Rachana
High
[ Threeds / Publishing Group ](https://github.com/ESGF/esgf.github.io/wiki/ESG FInterfaceGroups|ThreddsGroup)
Bob
High
[ Information Architecture (Data Model) Group ](https://github.com/ESGF/esgf.g ithub.io/wiki/ESGFInterfaceGroups|InformationArchitecture)
Stephen
Lower
[ Metadata Group ](https://github.com/ESGF/esgf.github.io/wiki/ESGFInterfaceGr oups|MetadataGroup)
Mark
Lower
Martina
Lower
[ Notification Group ](https://github.com/ESGF/esgf.github.io/wiki/ESGFInterfa ceGroups|NotificationGroup)
Gavin
Lower
(note: group pages are initially empty, see below for suggested tasks)
_ See the roadmap page for specific activities taking place and their POCs (point of contacts) _
These are interfaces that require urgent work as they impact current deployments of ESGF or imediate future plans. Multiple implementations exist or are being developed and therefore there is a good chance of rapid progress on a common interface.
Lead
Interested Parties
Nate, Phil, Luca , Feiyi, Rick, Neill
Page
[ Registry Group ](https://github.com/ESGF/esgf.github.io/wiki/ESGFInterfaceGr oups|RegistryGroup)
The registry is a concept that spans both P2P and Gateway architectures and is therefore vital to establishing a migration path to the P2P system. It is also part of the latest Gateway releases so impacts current test deployments and future operational deployments of the CMIP5 archive.
Items that need addressing include:
- bootstraping
- interoperability between the central registry and P2P registry designs
- maintaining security
- schema
Lead
Interested Parties
Mark Morgan, Eric, Stephan Kindermann, Feiyi, John Harney
Page
Until recently faceted search has been embedded in the Gateway UI. There is now 2 independend implementations of faceted search: Gateway and P2P. Also there are a couple of search interfaces built on top of other THREDDS harvesters at IPSL and DKRZ. All of these implementations should expose a common interface to faceted search.
This interface could be based on SOLr's capabilities but shouldn't rely on SOLr-specific formats and protocols in the long term. Another source could be opensearch ( http://www.opensearch.org ) which has an extensible mechanism for search parameters.
We also need to standardise how searchable information is federated and how search results for the same data from different sources will be represented (i.e. replicas). There are 2 basic models for federated search, harvesting or distributed query.
Lead
Rachana
Interested Parties
Phil, Luca, Nate, Gavin , Neill
Page
[ Security Group ](https://github.com/ESGF/esgf.github.io/wiki/ESGFInterfaceGr oups|SecurityGroup)
A security interface control document already exists but needs extending to cover some features required for the P2P development that are currently only implemented in the Gateway:
- Group interface: registration, discovery, etc.
- LAS integration
Lead
Interested Parties
Roland, Stephen Pascoe, Stephan Kindermann, Henrik Wiberg, Bob Drach, Hans Ramthum, Luca, John Harney, Feiyi Wang
Page
[ Thredds Group ](https://github.com/ESGF/esgf.github.io/wiki/ESGFInterfaceGro ups|ThreddsGroup)
THREDDS XML is a flexible XML schema with many features that are not used in ESGF. It is not clear whether these features are compatible with ESGF software. For instance use of the element to inherit file-dataset properties from their container.
There are multiple tools being developed to process ESGF THREDDS catalogues and each of these rely on the implied profile of THREDDS XML created by esgpublisher. These implementations include:
- Gateway
- P2P Index node
- IPSL internal datanode interface
- IS-ENES Portal developed at DKRZ
New applications of the ESGF stack are being developed that will rely on variations to the CMIP5 THREDDS XML encoding. These developments need to be kept consistent with any formalisation of the ESGF THREDDS profile:
- CORDEX data
- Observation data
There is also a need to document the publication interface as there are now 2 independent implementations of the server:
- Gateway
- P2P Index node
ESGF THREDDS XML interacts closely with the ESGF search interface and therefore the relationship between THREDDS elements and ESGF facets needs defining.
These are important for future integration of ESGF components but agreement is not critical to current deployments.
Lead
Stephen Pascoe
Interested Parties
Luca, Sebastian, Gavin
Page
[ Information Architecture Group ](https://github.com/ESGF/esgf.github.io/wiki /ESGFInterfaceGroups|InformationArchitecture)
What is an ESGF dataset? How do they relate to data files? What's the difference between a dataset it's location (given it may be in many places at once). What does dataset versioning mean? How do we organise facets and associate them with datasets? How do we associate complex metadata (e.g. Metafor CIM) with datasets. How do we associate services with datasets (e.g. OPeNDAP, gridftp, LAS)
Lots of thorny issues here.
Lead
Mark Morgan
Interested Parties
Nate, Stephen Pascoe, Charlotte Pascoe?, Luca, Cecelia, Sylvia, Sebastian
Page: [ Metadata Group ](https://github.com/ESGF/esgf.github.io/wiki/ESGFInter faceGroups|MetadataGroup)
Display of Metafor CIM metadata and it's association with ESGF datasets is an important requirement for CMIP5 and presently there is no alternative to the Gateway implementation. However, the Metafor services portal will expose APIs for accessing CIM metadata. This could be a future integration point for both the Gateway and P2P systems.
Lead
Martina
Interested Parties
Kevin Marsh?, Sebastien?, Luca, Sebastian
Page
Good documentation exists on Quality Control for CMIP5. It would be useful to generalise this to other data activities such as observations and CORDEX and to place it on the ESGF wiki.
Some issues are still ill defined. E.g. how a DOIs relate to datasets. What is meant by the "Simulation Level". How DOI landing pages will be implemented. How DOIs interacts with version changes.
Lead
Interested Parties
Gavin ,
Page
[ NotificationGroup ](https://github.com/ESGF/esgf.github.io/wiki/ESGFInterfac eGroups|NotificationGroup)
The notification and metrics system should be documented at some point.
Please also see the RoadMap page for more status on the implementations of the interfaces we have come up with.