sproc2caps

Recordstream data acquisition plugin that applies filter and/or a mathematical expression to one or more streams

Description

The sproc2caps plugin requests data from a SeisComP record source in real time or based on time windows, filters the data and/or applies mathematical expressions and sends the processed streams to a CAPS server. Streams can be renamed.

Setup

Streams

The plugin reads the streams to subscribe to from a separate stream map file. The location of the file can be either defined in the plugin configuration or given as a command line argument:

streams.map = @DATADIR@/sproc2caps/streams.map

Each line of the stream map file defines n input streams and one output stream. By definition at least one input and one output stream must be given. The last argument in a line is the output stream. All other lines are input stream. Lines beginning with a comment are ignored.

Note

The map file is required even if the stream codes remain the same. Without an entry in the map file the input streams are not treated.

Example map file:

#Input 1      Input 2       ... Output
XX.TEST1..HHZ XX.TEST2..HHZ ... XX.TEST3..HHZ

Each stream entry may contain additional stream options e.g. for data filtering. Options are indicated by “?”.

The following stream options are supported:

Name

Description

Example

filter

Filter string

filter=BW(4,0.7,2)

unit

Output unit

unit=cm/s

expr

Expression to be used(Output only)

expr=x1+x2

Examples of streams with stream options:

XX.TEST1..HHZ?filter=BW_HP(4,0.1)
XX.TEST2..HHZ?filter=BW_HP(4,0.1)
XX.TEST3..HHZ?filter=BW(4,0.7,2)?unit=cm/s

For the given example the plugin assigns the following access variables to the streams. The access variables can be used in the mathematical expression string. The unit option provides an additional description of the stream. unit does not modify the stream.

Access variables for N input streams:

map,input

input 1

input 2

input N

stream

XX.TEST1..HHZ

XX.TEST2..HHZ

XX.TESTN..HHZ

variable

x1

x2

xN

When the mathematical expression is evaluated the xi will be replaced with the sample of the corresponding stream at the sample time. The maximum number of input streams is 3.

Filtering

Input data can be filtered before mathematical expressions are applied. All filters known from the SeisComP framework can be used. By default no filter is applied to the input data.

Example for setting the filter in the map file:

XX.TEST1..HHZ?filter=BW(4,0.7,2) XX.TEST2..HHZ XX.TEST3..HHZ

Expressions

The sproc plugin uses the C++ Mathematical Expression Library to evaluate mathematical expressions. The library supports a wide range of mathematical expressions. The complete feature list can be found here. The number of input variables depends on the number of input streams. The variables are numbered consecutively from 1 to n: x1, x2, …, xn.

Example how to multiply 3 streams:

streams.expr = x1 * x2 + x3

Rename Streams

In addition to applying mathematical expressions to streams, the plugin can be also used to rename streams. With the following example we show how to map the streams GE.APE..BHE and GE.BKNI..BHE to new stream ids and store the output streams in the same CAPS server:

  1. Open the plugin configuration and create a clone of the input data stream with:

    streams.expr = x1
    
  2. Create the mapping file @DATADIR@/sproc2caps/streams.map with the following content

    # Input      Output
    GE.APE..BHE  AB.APE..BHE
    GE.BKNI..BHE GE.BKNI2..BHE
    

Time windows

Set the time windows using –begin and –end to set the start and the end times, respectively. When no time window is given, real-time input data are considered.

Examples

  1. To map waveform data for a specific time window reading from a local CAPS server on localhost:18002 and sending to the plugin port of the same CAPS server on localhost:18003 run:

    seiscomp exec sproc2caps --begin "2019-01-01 00:00:00" --end "2019-01-01 01:00:00" -I "caps://localhost:18002" -a localhost:18003
    

    This will create duplicate data on the CAPS server if the map file renames the streams. To remove the original streams:

    1. Configure caps to keep the orignal data for 0 days

    2. Restart or reload caps

  2. Read real-time data from an external seedlink server like with slink2caps but applying the mapping:

    seiscomp exec sproc2caps -I "slink://host:18000" -a localhost:18003
    

Configuration

etc/defaults/global.cfg
etc/defaults/sproc2caps.cfg
etc/global.cfg
etc/sproc2caps.cfg
~/.seiscomp/global.cfg
~/.seiscomp/sproc2caps.cfg

sproc2caps inherits global options.

Note

Modules/plugins may require a license file. The default path to license files is @DATADIR@/licenses/ which can be overridden by global configuration of the parameter gempa.licensePath. Example:

gempa.licensePath = @CONFIGDIR@/licenses
journal.file

Type: string

File to store stream states Default is @ROOTDIR@/var/run/sproc2caps/journal.

journal.flush

Type: uint

Unit: s

Flush stream states to disk every n seconds Default is 10.

journal.waitForAck

Type: uint

Unit: s

Wait when a sync has been forced, up to n seconds Default is 60.

journal.waitForLastAck

Type: uint

Unit: s

Wait on shutdown to receive acknownledgement messages, up to n seconds Default is 5.

Note

streams.* Configure operations applied to input streams and the stream mapping.

streams.begin

Type: string

Start time of data time window, default ‘GMT’

streams.end

Type: string

End time of data time window

streams.filter

Type: string

Sets the input filter Default is self.

streams.expr

Type: string

Sets the mathematical expression Default is x1 + x2.

streams.map

Type: string

Absolute path to the stream map file. Each line holds n input streams and one output stream. Example:

CX.PB11..BHZ CX.PB11..BHZ

CX.PB11..BHZ CX.PB07..BHZ CX.PB11..BBZ Default is @DATADIR@/sproc2caps/streams.map.

Note

output.* Configure the data output.

output.address

Type: string

Data output URL [[caps|capss]://][user:pass@]host[:port]. This parameter superseds the host and port parameter of previous versions and takes precedence. Default is localhost:18003.

output.host

Type: string

Data output host. Deprecated: Use output.address instead. Default is localhost.

output.port

Type: int

Data output port. Deprecated: Use output.address instead. Default is 18003.

output.bufferSize

Type: uint

Unit: B

Size (bytes) of the packet buffer Default is 1048576.

output.backfillingBufferSize

Type: uint

Unit: s

Length of backfilling buffer. Whenever a gap is detected, records will be held in a buffer and not sent out. Records are flushed from front to back if the buffer size is exceeded. Default is 180.

output.mseed.enable

Type: boolean

Enable on-the-fly MiniSeed encoding. If the encoder does not support the input type of a packet it will be forwarded. Re encoding of MiniSEED packets is not supported. Default is true.

output.mseed.encoding

Type: string

MiniSEED encoding to use. (Uncompressed, Steim1 or Steim2) Default is Steim2.

statusLog.enable

Type: boolean

Log information status information e.g. max bytes buffered Default is false.

statusLog.flush

Type: uint

Unit: s

Flush status every n seconds to disk Default is 10.

Command Line

Generic

-h, --help

show help message.

-V, --version

show version information

-D, --daemon

Run as daemon. This means the application will fork itself and doesn’t need to be started with &.

Verbosity

--verbosity arg

Verbosity level [0..4]. 0:quiet, 1:error, 2:warning, 3:info, 4:debug

-v, --v

Increase verbosity level (may be repeated, eg. -vv)

-q, --quiet

Quiet mode: no logging output

-s, --syslog

Use syslog logging back end. The output usually goes to /var/lib/messages.

-l, --lockfile arg

Path to lock file.

--console arg

Send log output to stdout.

--debug

Debug mode: –verbosity=4 –console=1

--log-file arg

Use alternative log file.