Polar Data Discovery Enhancement Research (POLDER)
Making data discovery easier by developing aggregated search tools
Polder (Dutch verb): to work collaboratively to achieve a common goal
Federated metadata search for the polar regions will dramatically simplify data discovery for polar scientists. Instead of searching dozens of metadata catalogues individually, a user should be able to search them all from a single search page.
The Polar Federated Search Working Group (POLDER) is a collaboration between the Arctic Data Committee (ADC), Standing Committee on Antarctic Data Management (SCADM), and Southern Ocean Observing System (SOOS), to develop the tools and resources to support metadata aggregation and federated search tools to improve the discoverability of polar science data.
During the Polar Data Forum III in Helsinki, November 2019, POLDER held two days of workshops to explore the feasibility of using schema.org and its associated technologies to support federated metadata search for polar-relevant metadata catalogues.
During the meeting we agreed the following:
That the tools for developing a community-specific federated search are developing rapidly, so that once a significant number of our data centres have implemented schema.org, it is likely that there will be a clearer path to developing our own federated search.
That we will continue to contribute to global conversations on schema.org in the earth and other natural sciences.
POLDER believes that the recent groundswell in interest in schema.org, driven by the development of Google’s dataset search, offers a rare opportunity to simplify and connect metadata discovery tools. Schema.org is structured header text that is attached to a dataset’s landing page and that can draw metadata elements from existing metadata standards. It is a lightweight way to share the load of aligning metadata standards that does not require a data centre to alter its systems and infrastructure for managing metadata.
However, the open and extensible nature of schema.org poses a danger, in that it would be easy for this community to replicate errors from the past by implementing it in divergent ways. This would undermine the reason for implementing schema.org - the need for a uniform way of sharing basic discovery metadata.
POLDER encourages all publishers of metadata and data in polar regions to implement schema.org in a way that is interoperable with the approaches taken by the science-on-schema, Bioschemas, and Geoschemas communities. The resources listed below will help you ensure that your schema mark-up is interoperable with this broader community.
Resources for metadata providers:
A sample schema.org json-ld file developed at the Polar Data Forum III in Helsinki, November 2019
The Earth Science Information Partners have more resources and hold regular teleconferences to discuss issues. It's quick and easy to join, and participation is encouraged.
Resources for metadata aggregators:
Gleaner and its associated tools for harvesting metadata records
Geocodes are developing prototype federated search tools that you can explore. In particular, you may be interested in exploring their tools that can search on either text or spatial location (though they're not yet integrated).
This is a rapidly developing field and all the groups listed here are interested in your feedback about ways schema.org and its related extensions should evolve to meet the needs of the entire community. We encourage you to post issues on GitHub repositories and to join the various teleconferences and workshops that these groups are organising, to ensure that polar voices are heard.
POLDER will investigate the needs of the polar research community and opportunities for developing metadata aggregation and federated search
POLDER will advise ADC, SCADM, and SOOS on the best approaches to metadata aggregation federated search
POLDER will pursue funding and resource opportunities with other related groups to support metadata aggregation federated search
Once funding/resources are found, POLDER will act as a scientific advisory group for the developers
POLDER will maintain contact with the broader data management community to ensure that polar metadata aggregation and federated search is linked with other global initiatives and minimises duplication of efforts
POLDER will work in as transparent and open a manner as possible, including the open distribution to group materials in publicly accessible resources.
We expect that members of POLDER and the broader data management and polar communities will treat this openly shared information with respect and with due diligence towards citation, attribution, and re-use of the materials (e.g., without “scooping” the community’s work to publish under their own name).
POLDER will adhere to the Polar Information Commons
Planned Products and Outcomes
- Define the core needs of the polar research communities in metadata aggregation and federated search (e.g. functionality, key metadata standards)
- Pursue active involvement from the polar science and data communities to drive the activities of this group
- Document the community’s needs and the available tools in a research/white paper
- Identify key metadata standards used by relevant global, ocean, and polar communities
- Establish the core elements of those metadata standards and cross-walks between key metadata standards used by research communities in the high latitudes
- Investigate existing federated search platforms to identify those that may meet the polar community’s needs and identify benefits/limitations in those platforms
- Investigate available methods and tools currently not widely used by the polar community for developing federated search
- Pursue funding and other resources for developing federated search
- Serve as a venue for discussions around the implementation of metadata aggregation and federated search for high latitude regions through one or more portals
- Report to ADC, SCADM, and SOOS on progress