.. _sec-caps-retrieval: Access Data on a CAPS Server ============================ A range of tools is available to access data and information on a CAPS server. .. csv-table:: :header: "Name", "SW Package", "Description" :widths: 15,15,70 ":ref:`capstool `","caps-tools","A command-line tool for retrieving data and meta information from a CAPS server" ":ref:`rifftool `","caps-tools","A command-line tool for data inspection and extraction from individual CAPS data files (RIFF), e.g., in a CAPS archive" ":ref:`caps_plugin `","seiscomp","The plugin fetches miniSEED and :ref:`RAW ` data from a CAPS server and provides the data to :program:`seedlink`" ":ref:`caps / capss`","seiscomp","The RecordStream implementations for |appname|" ":ref:`cap2caps `","caps-plugins","Automatic or interactive synchronization of two CAPS servers" ":ref:`web interface `","caps-server","The web interface provided by the CAPS server" ":ref:`FDSNWS `","caps-server","FDSNWS dataselect interface provided by the CAPS server" ":ref:`WWS `","caps-server","Winston Waveform Server interface provided by the CAPS server" ":ref:`scardac `","seiscomp","A command-line tool for generating availability information from CAPS archive" .. _sec-caps-recstream: RecordStream: caps/capss ------------------------ |scname| applications access waveform data through the :term:`RecordStream` interface. To fetch data from a CAPS server specific RecordStream implementations may be used: * *caps*: regular RecordStream implementation to access the CAPS server, * *capss*: RecordStream implementation to access the CAPS server secured by SSL, user name and password. Similar to *https*, *capss* will establish a Secure Socket Layer (SSL) communication. .. _sec-caps-rs-config: Configuration ~~~~~~~~~~~~~ In order to make use of the *caps* or the *capss* RecordStream configure the RecordStream URL in :confval:`recordstream`. Let it point to the CAPS server providing the data. Examples for *caps* and *capss*: .. code-block:: properties recordstream = caps://server:18002 recordstream = capss://foo:bar@server:18022 :ref:`Optional parameters ` are available for *caps*/*capss*. .. note:: While the *caps*/*capss* :term:`RecordStream` provides data in real time and from archive, some modules, e.g., :cite:t:`scart`, :cite:t:`fdsnws` or :cite:t:`gis` should be strictly limited to reading from archive only by the option ``arch``: .. code-block:: properties recordstream = caps://server:18002?arch recordstream = capss://foo:bar@server:18022?arch Otherwise requests attempting to fetch missing data may hang forever. .. _sec-caps-opt-params: Optional Parameters ~~~~~~~~~~~~~~~~~~~ Optional RecordStream parameters which can be combined: - ``arch`` - read from CAPS archive only, - ``ooo`` - out of order, data are fetched and provided in the order of their arrival in the CAPS server, useful for analysing if data have arrived out of order, - ``pass`` - password if server requires authentication, - ``request-file`` - file specifying the streams to be requested. One stream per line. Per line: net sta loc stream startTime endTime, - ``timeout`` - timeout of acquisition in seconds. Data acquisition will be restarted, - ``user`` - user name if server requires authentication. .. csv-table:: :header: "URL", "Description" "caps://server:18002?arch","Read data from CAPS archive. Stop data acquisition when all available waveforms are fetched." "caps://server:18002?ooo","Fetch data in the original order of arrival." "caps://server:18002?request-file=request.txt","Request only streams in time intervals given in request.txt" "caps://server:18002?timeout=5","Apply a timeout of 5 seconds." "capss://server:18022?user=foo&pass=bar", "Use secure protocol (SSL) with user name and password. Read the section on :ref:`sec-conf-access-auth` for details on the generation of user names and passwords." Combination with other RecordStream interfaces ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The caps and the capss RecordStream may be combined with other :term:`RecordStream` interfaces. Examples: #. Decimation Use the decimation :term:`RecordStream` :cite:p:`recordstream` interface to fetch data from a CAPS server running on localhost decimated to 1 sample per second. *global configuration:* .. code-block:: sh recordstream = dec://caps/localhost:18002?rate=1 *command line parameter:* .. code-block:: sh -I dec://caps/localhost:18002?rate=1 #. Resample Use the resample :term:`RecordStream` :cite:p:`recordstream` interface to fetch data from a CAPS server running on localhost resampled to 16 samples per second. *global configuration:* .. code-block:: sh recordstream = resample://caps/localhost:18002?rate=16 *command line parameter:* .. code-block:: sh -I resample://caps/localhost:18002?rate=16 .. _sec-caps-file: Direct file access: capstool/rifftool ------------------------------------- The data files in the CAPS archive contain a small additional header describing the data format. Read the :ref:`format documentation ` for more details. In contrast to miniSEED files in :term:`SDS` archives created, e.g., by :cite:t:`slarchive` or :cite:t:`scart`, the original miniSEED files stored in the CAPS archive cannot be directly read by applications. You may access data files directly: * Test and retrieve data files using :ref:`rifftool`. rifftool addresses individual files directly and does not require the CAPS server to be running. * Fetch data streams from a CAPS server based on time windows using the command-line tool :ref:`capstool`. :ref:`capstool` will read the data, strip the header. The data are provide in their original format on stdout or, with :option:`-o` as a single file. :ref:`capstool` can also be used for :ref:`testing the server ` as it provides information on available streams with their time window (:option:`-Q`). .. _sec-caps2caps: Synchronize with another CAPS server: caps2caps ----------------------------------------------- Use :ref:`caps2caps` to synchronize your CAPS server with another one. You may push or pull data on either side. In contrast to the generation of regular :term:`SDS` archives, e.g., by :program:`scart`, the CAPS server will not generate duplicate data records if executing :ref:`caps2caps` multiple times. While synchronizing observe the :ref:`web interface ` for the statistics of received, written or rejected data packages. .. _sec-caps-seedlink: Connect from SeedLink: caps_plugin ---------------------------------- The :ref:`caps_plugin` plugin fetches data from a CAPS server and provides the data to :program:`seedlink`. The plugin can be configured and started as any other plugin for :program:`seedlink` or executed on demand. Select *caps* when choosing the plugin in the seedlink binding configuration. **Examples:** * Fetch data from a remote CAPS server and make them available in your :program:`seedlink` instance: #. configure the :ref:`caps_plugin` plugin in the bindings configuration of your :program:`seedlink` instance pointing to the remote CAPS server #. enable and start :program:`seedlink` .. code-block:: sh seiscomp enable seedlink seiscomp start seedlink * Provide data from a CAPS server by seedlink on the same machine to external clients: #. create another seedlink instance, e.g., :program:`seedlinkP`: .. code-block:: sh seiscomp alias create seedlinkP seedlink #. configure the :ref:`caps_plugin` in the bindings configuration of :program:`seedlinkP` pointing to the local CAPS server. #. enable and start :program:`seedlinkP`: .. code-block:: sh seiscomp enable seedlinkP seiscomp start seedlinkP .. _sec-caps-web: Web interface ------------- The CAPS server ships with a :ref:`web interface `. Beside allowing you to view server statistics, data stored stored on the server can be downloaded for any time if the original format is :term:`miniSEED`. For downloading miniSEED data #. Select the stream(s) of interest. #. Zoom in to the period of interest. Zooming in and out in time works by right-mouse button actions just like in other |scname| GUI applications like :cite:t:`scrttv`. #. Click on the download button to download the miniSEED file. An error message will be printed in case the original format is not miniSEED. .. _fig-web-streams-download: .. figure:: media/web_streams_download.png :width: 18cm Stream perspective of :term:`CAPS` Web interface allowing to download miniSEED data for selected streams. .. _sec-caps-fdsnws: Built-in FDSNWS --------------- |appname| speaks natively FDSN Web Services, FDSNWS :cite:p:`fdsn` providing waveform data via dataselect. Information on events and stations are not delivered. The waveform data will be delivered through the port configured in :confval:`AS.http.port` or the port configured by your Apache server, if available. Contact your system administrator for information on the Apache server. Read the documentation of the :ref:`CAPS server ` for the configuration. .. _sec-caps-wws: Built-in Winston waveform server -------------------------------- |appname| speeks natively Winston Waveform Server protocol, WWS, :cite:p:`wws`, e.g., to :cite:t:`swarm` by USGS. Read the documentation of the :ref:`CAPS server` for the configuration. .. _sec-caps-dataavailability: Data availability information ----------------------------- Web interface ~~~~~~~~~~~~~ The :ref:`Channels perspective of the CAPS web interface ` indicates periods of availability and of gaps on level of network, station, sensor location and channel. The resolution of colors and percentages is linked to the granularity of the data detection which increases with shorter time windows in order to optimize the speed of the calculation. capstool ~~~~~~~~ The CAPS server stores information on received data segments including their start and end times. Information on resulting gaps can be retrieved by :ref:`capstool`. Example: .. code-block:: sh echo "2023,05,01,12,00,00 2023,05,03,00,00,00 NET * * *" | capstool -G --tolerance=0.5 -H localhost scardac ~~~~~~~ The availability of data in the caps archive can be analyzed and written to the |scname| database by the |scname| module :cite:t:`scardac`. For availability analysis add the plugin *daccaps* to the list of :confval:`plugins` and URL of the caps archive to the archive configuration of :cite:t:`scardac`. The *daccaps* plugin ships with the gempa package *caps-server*. Example configuration of :cite:t:`scardac` (:file:`scardac.cfg`): .. code-block:: properties plugins = ${plugins}, daccaps archive = caps:///home/data/archive/caps/ Example call: .. code-block:: sh scardac --plugins="daccaps, dbmysql" -d localhost -a caps:///home/data/archive/caps/ --debug .. note:: As of SeisComP in version 6, scardac has received significant optimization. Instead of scanning the full archive, only files which have changed since the last scan will be examined. This means that when scanning the entire archive, the first run may be more time consuming than subsequent ones when executed within reasonable intervals. The data availability information can be retrieved from the database, e.g., using :cite:t:`fdsnws` or :cite:t:`scxmldump`.