.. highlight:: rst .. _caps2caps: ######### caps2caps ######### **caps2caps synchronizes CAPS servers in real-time** Description =========== *caps2caps* can connect two |appname| server instances to synchronize the data in real time. When one server 1 fails and the other one, server 2, continues to operate, the server 1 can back fill the data as soon as it becomes alive again. *caps2caps* can run on either side to pull the data from the other server or to push the data to this server: * For **pulling data** from a remote to a local server configure the input and the output parameters with the remote and the local server, respectively. * For **pushing data** from a local to a remote server configure the input and the output parameters with the local and the remote server, respectively. .. _fig-caps2caps: .. figure:: media/caps2caps.png :width: 18cm :align: center caps2caps instances connecting two |appname| servers pulling data from the remote into the local server. Examples ======== * Run caps2caps as daemon module. #. Configure input and output hosts (:confval:`input.address`, :confval:`output.address`) in caps2caps module configuration, :file:`caps2caps.cfg`. #. Enable and start caps2caps .. code-block:: bash seiscomp enable caps2caps seiscomp start caps2caps * Run caps2caps on demand in a terminal with specific, explicitly specifying input and output hosts without encryption .. code-block:: bash caps2caps -I caps://inputServer:18002 -O caps://outputServer:18003 The same as above but with encrypted data transfer controlled by user name and password .. code-block:: bash caps2caps -I capss://user:password@inputServer:18002 -O capss://user:password@inputServer:output:18003 * Pull or push data depending on module configuration but ignore the journal file. This allows resending the data .. code-block:: bash caps2caps -j "" Module Configuration ==================== | :file:`etc/defaults/global.cfg` | :file:`etc/defaults/caps2caps.cfg` | :file:`etc/global.cfg` | :file:`etc/caps2caps.cfg` | :file:`~/.seiscomp/global.cfg` | :file:`~/.seiscomp/caps2caps.cfg` caps2caps inherits :ref:`global options`. .. note:: Modules/plugins may require a license file. The default path to license files is :file:`@DATADIR@/licenses/` which can be overridden by global configuration of the parameter :confval:`gempa.licensePath`. Example: :: gempa.licensePath = @CONFIGDIR@/licenses .. confval:: streams Type: *string* Comma separated list of streams. Stream format: NET.STA.LOC.CHA. Streams may contain wildcards .. confval:: begin Type: *string* Start time of data time window, default is 'GMT'. Date time format: [YYYY\-MM\-DD HH:MM:SS]. .. confval:: end Type: *string* End time of data time window. Date time format: [YYYY\-MM\-DD HH:MM:SS]. .. confval:: maxDays Default: ``-1`` Unit: *day* Type: *int* Maximum number of days to acquire regardless if the time window is configured or read from journal. A value of 0 or less disables the check. .. confval:: days Default: ``-1`` Unit: *day* Type: *int* Use to set the start time of data time window n days before the current time. .. confval:: daysBefore Default: ``-1`` Unit: *day* Type: *int* Use to set the end time of data time window n days before the current time. .. confval:: timeWindowUpdateInterval Default: ``-1`` Unit: *s* Type: *int* Sets the interval in seconds at which the relative request time window defined by option days and\/or daysBefore is updated. Use a value less or equal zero to disable the update. This feature is supported in archive mode only. A typical use case is when data has to be transmitted continuously with a time delay. .. confval:: maxRealTimeGap Default: ``-1`` Unit: *s* Type: *int* Sets the maximum real\-time data gap in seconds. This means, if the start time of the requested time window of a channel is before this value with respect to the current system time then the request is split into a real\-time request starting at system time \- marginRealTimeGap and a backfill request from requested start time to time \- marginRealTimeGap. That prioritizes real\-time data and backfills old data in parallel. .. confval:: marginRealTimeGap Default: ``60`` Unit: *s* Type: *int* The time margin used to request real\-time data in combination with maxRealTimeGap with respect to system time. .. confval:: realtime Default: ``true`` Type: *boolean* Enable real\-time mode. Archived data is not fetched. .. confval:: outOfOrder Default: ``false`` Type: *boolean* Enable out of order mode. Allows transfering data which is not in timely order. .. _input: .. note:: **input.\*** *Configuration of data input host.* .. confval:: input.address Type: *string* URL. Format: [[caps\|capss]:\/\/][user:pass\@]host[:port] . .. _output: .. confval:: output.address Default: ``localhost:18003`` Type: *string* Data output URL [[caps\|capss]:\/\/][user:pass\@]host[:port]. This parameter superseds the host and port parameter of previous versions and takes precedence. .. confval:: output.host Default: ``localhost`` Type: *string* Data output host. Deprecated: Use output.address instead. .. confval:: output.port Default: ``18003`` Type: *int* Data output port. Deprecated: Use output.address instead. .. confval:: output.timeout Default: ``60`` Unit: *s* Type: *int* Timeout when sending a packet. If the timeout expires the connection will be closed and re\-established. .. confval:: output.maxFutureEndTime Default: ``120`` Unit: *s* Type: *int* Maximum allowed relative end time for packets. If the packet end time is greater than the current time plus this value, the packet will be discarded. By default this value is set to 120 seconds. .. confval:: output.bufferSize Default: ``1048576`` Unit: *bytes* Type: *uint* Size \(bytes\) of the packet buffer. .. confval:: output.backfillingBufferSize Default: ``0`` Unit: *s* Type: *int* Length of backfilling buffer. Whenever a gap is detected, records will be held in a buffer and not sent out. Records are flushed from front to back if the buffer size is exceeded. .. confval:: output.maxFutureEndTime Default: ``3600`` Unit: *s* Type: *int* CAPS maximum allowed packet end time. If packet end time is larger than current time plus this value the packet is discarded. .. _output.mseed: .. confval:: output.mseed.enable Default: ``false`` Type: *boolean* Enable on\-the\-fly miniSEED encoding. If the encoder does not support the input type of a packet it will be forwarded. Re\-encoding of miniSEED packets is not supported. .. confval:: output.mseed.encoding Default: ``Steim2`` Type: *string* miniSEED encoding to use. \(Uncompressed, Steim1 or Steim2\) .. confval:: output.mseed.recordLength Default: ``9`` Type: *uint* miniSeed record length. The value represents the exponent in the expression 2\^n, e.g., 2\^9 \= 512 bytes .. _journal: .. confval:: journal.file Default: ``@ROOTDIR@/var/run/caps2caps/journal`` Type: *string* File to store stream states. .. confval:: journal.flush Default: ``10`` Unit: *s* Type: *uint* Flush stream states to disk every n seconds .. confval:: journal.waitForAck Default: ``60`` Unit: *s* Type: *uint* Wait when a sync has been forced, up to n seconds .. confval:: journal.waitForLastAck Default: ``5`` Unit: *s* Type: *uint* Wait on shutdown to receive acknownledgement messages, up to n seconds .. _statusLog: .. confval:: statusLog.enable Default: ``false`` Type: *boolean* Log information status information e.g. max bytes buffered .. confval:: statusLog.flush Default: ``10`` Type: *uint* Flush status every n seconds to disk .. _host: .. confval:: host.storage Default: ``@LOGDIR@`` Type: *string* Path to storage used to determine available disc space and capacity. .. confval:: host.os Type: *string* Set the operating system information, such as Ubuntu 24.04.1 LTS. If this information is not provided, the plugin will attempt to read the value from the PRETTY_NAME variable in the \/etc\/os\-release file. .. confval:: host.agent Type: *string* Set the agent string to send to the CAPS server. By default, the application name is used. .. confval:: host.group Type: *string* Set the group string to send to the CAPS server. Command-Line Options ==================== .. _Generic: Generic ------- .. option:: -h, --help Show help message. .. option:: -V, --version Show version information. .. option:: --config-file file The alternative module configuration file. When this option is used, the module configuration is only read from the given file and no other configuration stage is considered. Therefore, all configuration including the definition of plugins must be contained in that file or given along with other command\-line options such as \-\-plugins. .. option:: --plugins arg Load given plugins. .. option:: -D, --daemon Run as daemon. This means the application will fork itself and doesn't need to be started with \&. .. _Verbosity: Verbosity --------- .. option:: --verbosity arg Verbosity level [0..4]. 0:quiet, 1:error, 2:warning, 3:info, 4:debug. .. option:: -v, --v Increase verbosity level \(may be repeated, e.g., \-vv\). .. option:: -q, --quiet Quiet mode: no logging output. .. option:: --print-component arg For each log entry print the component right after the log level. By default the component output is enabled for file output but disabled for console output. .. option:: --component arg Limit the logging to a certain component. This option can be given more than once. .. option:: -s, --syslog Use syslog logging backend. The output usually goes to \/var\/lib\/messages. .. option:: -l, --lockfile arg Path to lock file. .. option:: --console arg Send log output to stdout. .. option:: --debug Execute in debug mode. Equivalent to \-\-verbosity\=4 \-\-console\=1 . .. option:: --trace Execute in trace mode. Equivalent to \-\-verbosity\=4 \-\-console\=1 \-\-print\-component\=1 \-\-print\-context\=1 . .. option:: --log-file arg Use alternative log file. .. _Input: Input ----- .. option:: -I, --input arg Overrides configuration parameter :confval:`input.address`. URL of data input host. Format: [[caps\|capss]:\/\/][user:password\@]host[:port] . .. option:: --max-real-time-gap Maximum length of data gap after reconnecting. If exceeded, a real\-time stream and backfilling stream will be created in parallel. Setting this value will give highest priority to real\-time streams, e.g., for rapid response systems. .. _Streams: Streams ------- .. option:: -i, --inventory arg Inventory XML defining the streams to add. .. option:: -A, --add-stream arg List of streamIDs [NET.STA.LOC.CHA] to add. Wildcards are supported. Use comma\-separation without blanks for multiple IDs. .. option:: --begin arg Start time of data request. Applied only on streams not found in the journal. Format: 'YYYY\-MM\-DD hh:mm:ss.sss'. .. option:: --end arg End time of data request. Format: 'YYYY\-MM\-DD hh:mm:ss.sss'. .. option:: --max-days arg Unit: *day* Maximum number of days to acquire regardless if the time window is configured or read from journal. A value of 0 or less disables the check. .. option:: --days arg Unit: *day* Begin of data request time window given as days before current time. Applied only on streams not found in the journal. .. option:: --days-before arg Unit: *day* End of data request time window given as number of days before current time. .. _Mode: Mode ---- .. option:: --archive Disable real\-time mode. Only archived data is fetched and missing records are ignored. .. option:: --out-of-order Use to enable out\-of\-order mode. Allows transfering data which is not in timely order. .. _Output: Output ------ .. option:: -O, --output arg Overrides configuration parameter :confval:`output.address`. This is the CAPS server which shall receive the data. .. option:: -b, --buffer-size arg Size \(bytes\) of the journal buffer. If the value ist exceeded, a synchronization of the journal is forced. .. option:: --backfilling arg Default: ``0`` Buffer size in seconds for backfilling gaps. .. option:: --mseed Enable on\-the\-fly miniSEED encoding. If the encoder does not support the input type of a packet, it will be forwarded. Re\-encoding of miniSEED packets is not supported. .. option:: --encoding arg miniSEED encoding to use: Uncompressed, Steim1 or Steim2. .. option:: --rec-len arg miniSEED record length expressed as a power of 2. A 512 byte record would be 9. .. option:: --max-future-endtime arg Maximum allowed relative end time for packets. If the packet end time is greater than the current time plus this value, the packet will be discarded. By default this value is set to 120 seconds. .. option:: --dump-packets Dump packets to stdout. .. option:: --test Disable socket communication. .. option:: --dump Dump all received data to stdout and don't use the input port. .. _Journal: Journal ------- .. option:: -j, --journal arg File to store stream states. Use an empty string to log to stdout. .. option:: -f, --flush arg Flush stream states to disk every n seconds. .. option:: --wait-for-ack arg arg Wait when a sync has been forced, up to n seconds. .. option:: -w, --wait-for-last-ack arg Wait on shutdown to receive acknownledgement messages, up to the given number of seconds. .. _Status: Status ------ .. option:: --status-log Log information status information, e.g., max bytes buffered. .. option:: --status-flush arg Flush status every n seconds to disk. .. _Host: Host ---- .. option:: --host-storage Path to storage used to determine available disc space and capacity. .. option:: --host-os Set the operating system information, such as Ubuntu 24.04.1 LTS to send to the CAPS server. .. option:: --host-agent Set the agent string to send to the CAPS server. .. option:: --host-group Set the group string to send to the CAPS server.