.. highlight:: rst .. _sproc2caps: ########## sproc2caps ########## **Recordstream data acquisition plugin that applies filter and/or a mathematical expression to one or more streams** Description =========== The sproc2caps plugin requests data from a SeisComP record source in real time or based on :ref:`time windows `, :ref:`filters the data` and/or applies :ref:`mathematical expressions ` and sends the processed streams to a CAPS server. Streams can be :ref:`renamed `. Setup ===== Streams ------- The plugin reads the streams to subscribe to from a separate stream map file. The location of the file can be either defined in the plugin configuration or given as a command line argument: .. code-block:: bash streams.map = @DATADIR@/sproc2caps/streams.map Each line of the stream map file defines n input streams and one output stream. By definition at least one input and one output stream must be given. The last argument in a line is the output stream. All other lines are input stream. Lines beginning with a comment are ignored. .. note:: The map file is required even if the stream codes remain the same. Without an entry in the map file the input streams are not treated. Example map file: .. code-block:: bash #Input 1 Input 2 ... Output XX.TEST1..HHZ XX.TEST2..HHZ ... XX.TEST3..HHZ Each stream entry may contain additional stream options e.g. for :ref:`data filtering `. Options are indicated by "?". The following stream options are supported: ====== ============================================= ======== Name Description Example ====== ============================================= ======== filter Filter string filter=BW(4,0.7,2) unit Output unit unit=cm/s expr Expression to be used(Output only) expr=x1+x2 ====== ============================================= ======== Examples of streams with stream options: .. code-block:: bash XX.TEST1..HHZ?filter=BW_HP(4,0.1) XX.TEST2..HHZ?filter=BW_HP(4,0.1) XX.TEST3..HHZ?filter=BW(4,0.7,2)?unit=cm/s For the given example the plugin assigns the following access variables to the streams. The access variables can be used in the mathematical expression string. The *unit* option provides an additional description of the stream. *unit* does not modify the stream. Access variables for N input streams: ========= ============= ============= === ============= map,input input 1 input 2 ... input N ========= ============= ============= === ============= stream XX.TEST1..HHZ XX.TEST2..HHZ ... XX.TESTN..HHZ variable x1 x2 ... xN ========= ============= ============= === ============= When the mathematical expression is evaluated the xi will be replaced with the sample of the corresponding stream at the sample time. The maximum number of input streams is 3. .. _sproc-filter: Filtering --------- Input data can be filtered before :ref:`mathematical expressions ` are applied. All `filters known from the SeisComP framework `_ can be used. By default no filter is applied to the input data. Example for setting the filter in the map file: .. code-block:: bash XX.TEST1..HHZ?filter=BW(4,0.7,2) XX.TEST2..HHZ XX.TEST3..HHZ .. _sproc-expressions: Expressions ----------- The sproc plugin uses the C++ Mathematical Expression Library to evaluate mathematical expressions. The library supports a wide range of mathematical expressions. The complete feature list can be found here_. The number of input variables depends on the number of input streams. The variables are numbered consecutively from 1 to n: x1, x2, ..., xn. Example how to multiply 3 streams: .. code-block:: bash streams.expr = x1 * x2 + x3 .. _here: http://www.partow.net/programming/exprtk/ .. _sproc-rename: Rename Streams -------------- In addition to applying mathematical expressions to streams, the plugin can be also used to rename streams. With the following example we show how to map the streams **GE.APE..BHE** and **GE.BKNI..BHE** to new stream ids and store the output streams in the same CAPS server: 1. Open the plugin configuration and create a clone of the input data stream with: .. code-block:: bash streams.expr = x1 #. Create the mapping file **@DATADIR@/sproc2caps/streams.map** with the following content .. code-block:: bash # Input Output GE.APE..BHE AB.APE..BHE GE.BKNI..BHE GE.BKNI2..BHE .. _sproc-tw: Time windows ------------ Set the time windows using *--begin* and *--end* to set the start and the end times, respectively. When no time window is given, real-time input data are considered. Examples ======== #. To map waveform data for a specific time window reading from a local CAPS server on localhost:18002 and sending to the plugin port of the same CAPS server on localhost:18003 run: .. code-block:: bash seiscomp exec sproc2caps --begin "2019-01-01 00:00:00" --end "2019-01-01 01:00:00" -I "caps://localhost:18002" -a localhost:18003 This will create duplicate data on the CAPS server if the map file renames the streams. To remove the original streams: 1. Configure caps to keep the orignal data for 0 days #. Restart or reload caps #. Read real-time data from an external seedlink server like with `slink2caps `_ but applying the mapping: .. code-block:: bash seiscomp exec sproc2caps -I "slink://host:18000" -a localhost:18003 Module Configuration ==================== | :file:`etc/defaults/global.cfg` | :file:`etc/defaults/sproc2caps.cfg` | :file:`etc/global.cfg` | :file:`etc/sproc2caps.cfg` | :file:`~/.seiscomp/global.cfg` | :file:`~/.seiscomp/sproc2caps.cfg` sproc2caps inherits :ref:`global options`. .. note:: Modules/plugins may require a license file. The default path to license files is :file:`@DATADIR@/licenses/` which can be overridden by global configuration of the parameter :confval:`gempa.licensePath`. Example: :: gempa.licensePath = @CONFIGDIR@/licenses .. confval:: journal.file Type: *string* File to store stream states Default is ``@ROOTDIR@/var/run/sproc2caps/journal``. .. confval:: journal.flush Type: *uint* Unit: *s* Flush stream states to disk every n seconds Default is ``10``. .. confval:: journal.waitForAck Type: *uint* Unit: *s* Wait when a sync has been forced, up to n seconds Default is ``60``. .. confval:: journal.waitForLastAck Type: *uint* Unit: *s* Wait on shutdown to receive acknownledgement messages, up to n seconds Default is ``5``. .. note:: **streams.\*** *Configure operations applied to input streams and the stream mapping.* .. confval:: streams.begin Type: *string* Start time of data time window, default 'GMT' .. confval:: streams.end Type: *string* End time of data time window .. confval:: streams.filter Type: *string* Sets the input filter Default is ``self``. .. confval:: streams.expr Type: *string* Sets the mathematical expression Default is ``x1 + x2``. .. confval:: streams.map Type: *string* Absolute path to the stream map file. Each line holds n input streams and one output stream. Example: CX.PB11..BHZ CX.PB11..BHZ CX.PB11..BHZ CX.PB07..BHZ CX.PB11..BBZ Default is ``@DATADIR@/sproc2caps/streams.map``. .. note:: **output.\*** *Configure the data output.* .. confval:: output.address Type: *string* Data output URL [[caps\|capss]:\/\/][user:pass\@]host[:port]. This parameter superseds the host and port parameter of previous versions and takes precedence. Default is ``localhost:18003``. .. confval:: output.host Type: *string* Data output host. Deprecated: Use output.address instead. Default is ``localhost``. .. confval:: output.port Type: *int* Data output port. Deprecated: Use output.address instead. Default is ``18003``. .. confval:: output.bufferSize Type: *uint* Unit: *B* Size \(bytes\) of the packet buffer Default is ``1048576``. .. confval:: output.backfillingBufferSize Type: *uint* Unit: *s* Length of backfilling buffer. Whenever a gap is detected, records will be held in a buffer and not sent out. Records are flushed from front to back if the buffer size is exceeded. Default is ``180``. .. confval:: output.mseed.enable Type: *boolean* Enable on\-the\-fly MiniSeed encoding. If the encoder does not support the input type of a packet it will be forwarded. Re encoding of MiniSEED packets is not supported. Default is ``true``. .. confval:: output.mseed.encoding Type: *string* MiniSEED encoding to use. \(Uncompressed, Steim1 or Steim2\) Default is ``Steim2``. .. confval:: statusLog.enable Type: *boolean* Log information status information e.g. max bytes buffered Default is ``false``. .. confval:: statusLog.flush Type: *uint* Unit: *s* Flush status every n seconds to disk Default is ``10``. Command-Line Options ==================== Generic ------- .. option:: -h, --help show help message. .. option:: -V, --version show version information .. option:: -D, --daemon Run as daemon. This means the application will fork itself and doesn't need to be started with \&. Verbosity --------- .. option:: --verbosity arg Verbosity level [0..4]. 0:quiet, 1:error, 2:warning, 3:info, 4:debug .. option:: -v, --v Increase verbosity level \(may be repeated, eg. \-vv\) .. option:: -q, --quiet Quiet mode: no logging output .. option:: -s, --syslog Use syslog logging back end. The output usually goes to \/var\/lib\/messages. .. option:: -l, --lockfile arg Path to lock file. .. option:: --console arg Send log output to stdout. .. option:: --debug Debug mode: \-\-verbosity\=4 \-\-console\=1 .. option:: --log-file arg Use alternative log file.