caps2caps

caps2caps synchronizes CAPS servers in real-time

Description

caps2caps can connect two CAPS server instances to synchronize the data in real time. When one server 1 fails and the other one, server 2, continues to operate, the server 1 can back fill the data as soon as it becomes alive again.

caps2caps can run on either side to pull the data from the other server or to push the data to this server:

• For pulling data from a remote to a local server configure the input and the output parameters with the remote and the local server, respectively.

• For pushing data from a local to a remote server configure the input and the output parameters with the local and the remote server, respectively.

caps2caps instances connecting two CAPS servers pulling data from the remote into the local server.

Examples

• Run caps2caps as daemon module.

  1. Configure input and output hosts (input.address, output.address) in caps2caps module configuration, caps2caps.cfg.

  2. Enable and start caps2caps

     seiscomp enable caps2caps
     seiscomp start caps2caps
     

• Run caps2caps on demand in a terminal with specific, explicitly specifying input and output hosts without encryption

  caps2caps -I caps://inputServer:18002 -O caps://outputServer:18003
  

  The same as above but with encrypted data transfer controlled by user name and password

  caps2caps -I capss://user:password@inputServer:18002 -O capss://user:password@inputServer:output:18003
  

• Pull or push data depending on module configuration but ignore the journal file. This allows resending the data

  caps2caps -j ""
  

Module Configuration

etc/defaults/global.cfg

etc/defaults/caps2caps.cfg

etc/global.cfg

etc/caps2caps.cfg

~/.seiscomp/global.cfg

~/.seiscomp/caps2caps.cfg

caps2caps inherits global options.

Note

Modules/plugins may require a license file. The default path to license files is @DATADIR@/licenses/ which can be overridden by global configuration of the parameter gempa.licensePath. Example:

gempa.licensePath = @CONFIGDIR@/licenses

streams

Type: string

Comma separated list of streams. Stream format: NET.STA.LOC.CHA. Streams may contain wildcards

begin

Type: string

Start time of data time window, default is ‘GMT’. Date time format: [YYYY-MM-DD HH:MM:SS].

end

Type: string

End time of data time window. Date time format: [YYYY-MM-DD HH:MM:SS].

maxDays

Default: -1

Unit: day

Type: int

Maximum number of days to acquire regardless if the time window is configured or read from journal. A value of 0 or less disables the check.

days

Default: -1

Unit: day

Type: int

Use to set the start time of data time window n days before the current time.

daysBefore

Default: -1

Unit: day

Type: int

Use to set the end time of data time window n days before the current time.

timeWindowUpdateInterval

Default: -1

Unit: s

Type: int

Sets the interval in seconds at which the relative request time window defined by option days and/or daysBefore is updated. Use a value less or equal zero to disable the update. This feature is supported in archive mode only. A typical use case is when data has to be transmitted continuously with a time delay.

maxRealTimeGap

Default: -1

Unit: s

Type: int

Sets the maximum real-time data gap in seconds. This means, if the start time of the requested time window of a channel is before this value with respect to the current system time then the request is split into a real-time request starting at system time - marginRealTimeGap and a backfill request from requested start time to time - marginRealTimeGap. That prioritizes real-time data and backfills old data in parallel.

marginRealTimeGap

Default: 60

Unit: s

Type: int

The time margin used to request real-time data in combination with maxRealTimeGap with respect to system time.

realtime

Default: true

Type: boolean

Enable real-time mode. Archived data is not fetched.

outOfOrder

Default: false

Type: boolean

Enable out of order mode. Allows transfering data which is not in timely order.

Note

input.* Configuration of data input host.

input.address

Type: string

URL. Format: [[caps|capss]://][user:pass@]host[:port] .

Note

output.* Configuration of data output host.

output.address

Default: localhost:18003

Type: string

URL. Format: [[caps|capss]://][user:pass@]host[:port] .

output.bufferSize

Default: 1048576

Unit: byte

Type: uint

Size of the packet buffer.

output.backfillingBufferSize

Default: 0

Unit: s

Type: uint

Length of backfilling buffer which is a tool to mitigate out-of-order data. Whenever a gap is detected, records will be held in a buffer and not sent out. Records are flushed from front to back if the buffer size is exceeded. A value of 0 disables this feature.

output.mseed

Default: false

Type: boolean

Enable Steim2 encoding for received RAW packets.

output.timeout

Default: 60

Unit: s

Type: int

Timeout when sending a packet. If the timeout expires, the connection will be closed and re-established.

journal.file

Default: @ROOTDIR@/var/run/caps2caps/journal

Type: string

File to store stream states.

journal.flush

Default: 10

Unit: s

Type: uint

Flush stream states to disk in the given interval.

journal.waitForAck

Default: 60

Unit: s

Type: uint

Wait when a sync has been forced, up to given seconds.

journal.waitForLastAck

Default: 5

Unit: s

Type: uint

Wait on shutdown to receive acknownledgement messages, up to the given seconds.

Command-Line Options

Generic

-h, --help

Show help message.

-V, --version

Show version information.

--config-file arg

Use alternative configuration file. When this option is used the loading of all stages is disabled. Only the given configuration file is parsed and used. To use another name for the configuration create a symbolic link of the application or copy it. Example: scautopick -> scautopick2.

--plugins arg

Load given plugins.

-D, --daemon

Run as daemon. This means the application will fork itself and doesn’t need to be started with &.

Verbosity

--verbosity arg

Verbosity level [0..4]. 0:quiet, 1:error, 2:warning, 3:info, 4:debug.

-v, --v

Increase verbosity level (may be repeated, eg. -vv).

-q, --quiet

Quiet mode: no logging output.

--print-component arg

For each log entry print the component right after the log level. By default the component output is enabled for file output but disabled for console output.

--component arg

Limit the logging to a certain component. This option can be given more than once.

-s, --syslog

Use syslog logging backend. The output usually goes to /var/lib/messages.

-l, --lockfile arg

Path to lock file.

--console arg

Send log output to stdout.

--debug

Execute in debug mode. Equivalent to --verbosity=4 --console=1 .

--trace

Execute in trace mode. Equivalent to --verbosity=4 --console=1 --print-component=1 --print-context=1 .

--log-file arg

Use alternative log file.

Input

-I, --input arg

Overrides configuration parameter input.address.

URL of data input host. Format: [[caps|capss]://][user:password@]host[:port] .

--max-real-time-gap

Maximum length of data gap after reconnecting. If exceeded, a real-time stream and backfilling stream will be created in parallel. Setting this value will give highest priority to real-time streams, e.g., for rapid response systems.

Streams

-i, --inventory arg

Inventory XML defining the streams to add.

-A, --add-stream arg

List of streamIDs [NET.STA.LOC.CHA] to add. Wildcards are supported. Use comma-separation without blanks for multiple IDs.

--begin arg

Start time of data request. Applied only on streams not found in the journal. Format: ‘YYYY-MM-DD hh:mm:ss.sss’.

--end arg

End time of data request. Format: ‘YYYY-MM-DD hh:mm:ss.sss’.

--max-days arg

Unit: day

Maximum number of days to acquire regardless if the time window is configured or read from journal. A value of 0 or less disables the check.

--days arg

Unit: day

Begin of data request time window given as days before current time. Applied only on streams not found in the journal.

--days-before arg

Unit: day

End of data request time window given as number of days before current time.

Mode

--archive

Disable real-time mode. Only archived data is fetched and missing records are ignored.

--out-of-order

Use to enable out-of-order mode. Allows transfering data which is not in timely order.

Output

-O, --output arg

Overrides configuration parameter output.address.

URL of data output host. Format: [[caps|capss]://][user:password@]host[:port] .

-b, --buffer-size arg

Size (bytes) of the journal buffer. If exceeded, a sync of the journal is forced.

--mseed

Enables Steim2 encoding for received RAW packets.

Journal

-j, --journal arg

Journal file to store stream states. Use an empty string to ignore the journal file which will transfer the data independent of previous transfers.

--flush arg

Unit: s

Flush stream states to disk every given seconds.

--waitForAck arg

Unit: s

Wait when a sync has been forced, up to the given seconds.

-w, --waitForLastAck arg

Unit: s

Wait on shutdown to receive acknowledgment messages, up to the given seconds.