Federated Search for Polar Regions

Purpose

To provide a single search window to find all published observations collected in the Southern Ocean, regardless of which data repository it is curated in.

Vision

A simple website that allows users to simultaneously search all the major data repositories that host Southern Ocean observing data for large, well-curated datasets that are also available through SOOSmap, as well as new data types and data from small regional projects.

The Need

At present, more than 50 data repositories host data relevant to SOOS, and use different metadata standards to document that data. Creating a single catalogue of records is not possible because of the technical difficulties in translating between metadata standards. A federated search involves interrogating many data catalogues at the same time, accommodating differences in the types of metadata. Similar tools are being developed globally but most efforts have used proprietary tools not easily adopted for use by the SOOS community. The US National Snow and Ice Data Center has built a suitable tool and is willing to share it with SOOS for further development as free and open source software, which will further enable other communities to build on, and contribute to our product. SOOS is working with the Arctic Data Committee and Standing Committee on Antarctic Data Management to design and resource the development of a federated data search tool to support polar scientists.

Planned Data Flows

Federated data searches work by harvesting metadata records from the targeted metadata catalogues and, for each catalogue, mapping key information fields from the metadata records to a central search algorithm. In this way, all of those catalogues are searched simultaneously and a list of relevant metadata records is returned. Clicking an entry in that list will take the searcher to the original data catalogue to read the full metadata record and generally to download the data itself. 

Federated Data Search Work Plan

Task Personnel Timeline Status
Community needs for a federated search tool are identified IPO and DMSC June 2017 Complete
Technical requirements  and workplan for developing a SOOS/polar federated  search tool are defined IPO and international colleagues August 2017 Complete
A scientific manuscript on options for federated search is written IPO and international colleagues April 2018 In progress
Funding and partners to develop a federated search tool are identified IPO and SOOS committees For as long as needed In progress
Basic federated search tool is developed IPO and partners One year after funding is acquired Not yet started
Federated search tool is maintained and improved IPO and partners Ongoing Not yet started

 

Acronyms

DMSC: Data Management Sub-Committee

IPO: International Project Office

Contact

Contact the SOOS Data Officer, Pip Bricher (data [at] soos.aq) if you would like to provide feedback or get involved in the delivery of a federated search tool.

Copyright 2015 Southern Ocean Observing System. All rights reserved.