Startup and Configuration

CAPS is developed as a standard SeisComP application. It uses the SeisComP infrastructure for startup, configuration and logging. Please refer to the SeisComP documentation for a comprehensive description.

Figure 12 shows a screen shot of scconfig, which is the central CAPS GUI allowing to configure, start and monitor CAPS.

../_images/scconfig.png

scconfig: SeisComP utility allowing to configure, start and monitor CAPS.

On the command line the following sequence may be used to enable, start and monitor CAPS:

seiscomp enable caps
seoscomp start caps
seiscomp check caps

Dependent on the configured log level CAPS will log to ~/.seiscomp/log/caps. For debugging purposes it is a good practice to stop the CAPS background process and run it in the foreground using the --debug switch:

seiscomp stop caps
seiscomp exec caps --debug

Configuration

etc/defaults/global.cfg
etc/defaults/caps.cfg
etc/global.cfg
etc/caps.cfg
~/.seiscomp3/global.cfg
~/.seiscomp3/caps.cfg

caps inherits global options.

Note

AS.*n CAPS server control parameters

AS.filebase

Type: string

Defines the path to the archive directory. Default is @ROOTDIR@/var/lib/caps/archive.

AS.port

Type: int

Defines the server port for client requests. Default is 18002.

AS.clientBufferSize

Type: int

Unit: B

Size of the client buffer in bytes. In case the client fails to read the buffered data in time (buffer overflow) the connection falls back to archive requests. Default is 16384.

AS.minDelay

Type: int

Unit: s

Limits the retrieval of real-time data. The value specifies the maximum relative end time of the time range to be requested. The maximum absolute end time is now - minDelay. This is only valid for FDSNWS and WWS. Default is -1.

AS.inventory

Type: path

The path to an optional inventory XML file with SeisComP3 schema. This inventory information is used by WWS to populate the channel coordinates. In future possibly more endpoints will make use of it.

AS.logRequests

Type: boolean

Whether to maintain a request log file or not. Each request will be logged and partly traced. Default is false.

AS.logAnonymousIP

Type: boolean

Log only parts of the IP to respect users privacy. Default is false.

AS.logPurge

Type: boolean

Whether to maintain a purge log file or not. Each purge operation will be logged. Default is false.

AS.allow

Type: list:string

List of IPs which are allowed to access the caps(s) port. By default access is unrestricted.

AS.deny

Type: list:string

List of IPs which are not allowed to access the caps(s) port. By default access is unrestricted.

Note

AS.filebase.*n File buffer control parameters

AS.filebase.keep

Type: list:string

Defines the number of days to keep data. Format:

stream ID : days, e.g. AM.*.*.*:365

Default (empty parameter): keep all data forever.

AS.filebase.preallocationSize

Type: int

Unit: B

Preallocation size of data files in bytes. Some file system allow to reserve disk space for files in advance. Especially on spinning disks the read performance will be improved if data can be read sequentially. The speed is traded for disk space consumed by the file since its size will be a multiple of the specified value. Set the value to 0 to disable this feature. Default is 65535.

Note

AS.filebase.cache.*n CAPS does not keep all files of all streams open. It tries to keep open the most frequently used files and closes all others. The more files CAPS can keep open the faster the population of the archive. The limit of open files depends on the security settings of the user under which CAPS is running.

AS.filebase.cache.openFileLimit

Type: int

The maximum number of open files. Because a stream file can have an associated index file this value is half of the physically opened files in worst case. Default is 250.

AS.filebase.cache.unusedFileLimit

Type: int

Limit of cached files in total. This value affects also files that are actually explicitly closed by the application. CAPS will keep them open (respecting the openFileLimit parameter) as long as possible and preserve a file handle to speed up reopening the file later. Default is 1000.

Note

AS.filebase.purge.*n Parameters controlling IO resources occupied by the purge operation. The deletion of many data files at once may have a significant impact on the server performance. E.g. if the server did not run for a while or the keep parameter was reduced significantly, the purge operation may slow down the processing of real-time data.

AS.filebase.purge.idleTime

Type: double

Unit: s

Idle time between two purge runs. Default is 5.

AS.filebase.purge.initIdleTime

Type: double

Unit: s

Idle time before the first purge run starts. Normally after a start the server tries to catch up all data which might be an IO intensive operation. In case of a huge archive the purge operation slow downs the read/write performace of the system too. To reduce the load at start it is a good idea to postpone this operation. Default is 0.

AS.filebase.purge.maxProcessTime

Type: double

Unit: s

Maximum processing time for one purge run. If exceeded the purge task will pause for ‘idleTime’ seconds freeing IO resources. Default is 1.

Note

AS.SSL.*n Parameters for SSL-based data requests

AS.SSL.port

Type: int

Defines the SSL server port for client requests. By default SSL requests are disabled.

AS.SSL.certificate

Type: string

Defines the path to the SSL certificate to use.

AS.SSL.key

Type: string

Defines the path to the private SSL key to use. This key is not shared with clients.

Note

AS.auth.*n Parameters controlling the authentication system for data requests based on user ID, IP addresses, access roles and access control lists.

AS.auth.backend

Type: string

The server provides an authentication plug-in interface. An authentication plugin implements access control checks. It is free where it gets the access information from e.g from a local database/file or a remote server. The option sets which authentication plugin should be used for authentication. Don’t forget to load the plugin in the plugin section. The basic plugin is built-in. Default is basic.

Note

AS.auth.basic.*n Basic authentication parameters. The configuration can be reloaded without restarting the server. Use “kill -SIGUSR1 `pidof caps`” to trigger the reload from outside.

AS.auth.basic.access-list

Type: file

Path to the access control list controlling access based on rules. By default access is unrestricted. Allow rules are evaluated first.

AM.DENY = 127.0.0.1

AM.ALLOW = 127.0.0.1

This example rule set prohibits all AM network stations for localhost because the DENY rule is evaluated after the ALLOW rule.

IP restrictions apply to the guest user only. In addition to IPs the access can be also restricted by user or group. In the latter case the “%” must be placed in front of the group name. Here an example:

AM.ALLOW = %users

AM.R44F5.ALLOW = sysop

Rules are evaluated on the basis of one another. This can lead to misunderstandings. Here an example:

AM.ALLOW = sysop

This rule will allow the AM network for sysop only. But

DENY = %users AM.ALLOW = sysop

will allow the access to the AM network for all users except those are member of the group users. Default is @SYSTEMCONFIGDIR@/caps/access.cfg.

AS.auth.basic.users.shadow

Type: file

Location of the users authentication file. For each user one line of the following format must exist:

username:encrypted_pwd

To encrypt the password mkpasswd can be used. It is recommended to apply a strong algorithm such as sha-256 or sha-512. The command

u=sysop pw=`mkpasswd -m sha-512` && echo $u:$pw

generates one line for user “sysop”. Add the line to the authentication file. Default is @SYSTEMCONFIGDIR@/caps/shadow.cfg.

AS.auth.basic.users.passwd

Type: file

Location of the users access control file. Each line starts with a user ID (uid) or a group ID (gid) and a list of properties in the form:

uid:prop1,prop2

or

%gid:prop1,prop2

“%” indicates a gid instead of a uid. The properties grant access to certain CAPS features. Currently supported property values are:

read

write Default is @SYSTEMCONFIGDIR@/caps/passwd.cfg.

AS.auth.basic.users.group

Type: file

Location of the optional group file. Each line maps a group id to a list of users in format

gid:user1,user2,user3 Default is @SYSTEMCONFIGDIR@/caps/group.cfg.

AS.plugins.port

Type: int

Defines the server port to use for plugin connections. Default is 18003.

AS.plugins.allow

Type: list:string

List of IPs which are allowed to access the plugin port. By default access is unrestricted.

AS.plugins.deny

Type: list:string

List of IPs which are not allowed to access the plugin port. By default access is unrestricted.

AS.plugins.SSL.port

Type: int

Defines the SSL server port to use for plugin SSL connections. The SSL port is disabled by default.

AS.plugins.SSL.certificate

Type: string

Defines the path to the SSL certificate to use.

AS.plugins.SSL.key

Type: string

Defines the path to the private SSL key to use. This key is not shared with clients.

Note

AS.http.*n Web interface control parameters

AS.http.port

Type: int

Defines the server port for HTTP connections. By default the Web interface is disabled.

Typical value: 18081

AS.http.allow

Type: list:string

List of IPs which are allowed to access the http(s) port. By default access is unrestricted.

AS.http.deny

Type: list:string

List of IPs which are not allowed to access the http(s) port. By default access is unrestricted.

AS.http.resolveProxyClient

Type: boolean

Sets if the X-Forwarded-For HTTP header is evaluated to retrieve the real client IP address from a proxy server. This is important if the web frontend is behind a proxy, e.g. Apache. Since data access is configured per IP, the real IP is required to grant access to requested channels. Enabling this opens a possible security hole as clients can then easily spoof their IP if the proxy does not correctly maintain this header or if CAPS does not run behind a proxy. Default is false.

Note

AS.http.SSL.*n Use https instead of http when setting the following parameters

AS.http.SSL.port

Type: int

Defines the server port for HTTPS connections. By default the SSL Web interface is disabled.

AS.http.SSL.certificate

Type: string

Defines the path to the SSL certificate to use.

AS.http.SSL.key

Type: string

Defines the path to the private SSL key to use. This key is not shared with clients.

Note

AS.WS.*n Websocket configuration. The websocket protocol is opened via HTTP requests but different in it’s general handling. The websocket interface implements the standard telnet CAPS protocol but returns JSON data instead of text messages for system requests such as the info request.

AS.WS.port

Type: int

Defines the server port for HTTP connections. By default the Web interface is disabled. Default is -1.

AS.WS.SSL.port

Type: int

Defines the server port for HTTPS connections. By default the SSL Web interface is disabled. Default is -1.

Note

AS.FDSNWS.*n FDSNWS control parameters

AS.FDSNWS.maxTimeWindow

Type: int

Unit: s

Maximum length of time window in seconds per request. A value greater than zero limits the maximum request time window including all data. 0 disables the limit. Default is 0.

AS.FDSNWS.maxRequests

Type: int

Maximum number of requests per post. A value greater than or equal to zero limits the number of request lines per POST request. Default is 1000.

Note

AS.WWS.*n Winston waveform server (WWS) control parameters

AS.WWS.port

Type: int

Server port for WWS connections.

Default (no value): The WWS interface is disabled.

AS.WWS.maxTimeWindow

Type: int

Unit: s

Maximum length of time window in seconds per request. A value greater than zero limits the maximum request time window including all data. 0 disables the limit. Default is 90000.

AS.WWS.maxRequests

Type: int

A value greater than or equal to zero limits the number of request lines per POST request. Default is 100.

AS.WWS.allow

Type: list:string

List of IPs which are allowed to access the WWS port. By default access is unrestricted.

AS.WWS.deny

Type: list:string

List of IPs which are not allowed to access the WWS port. By default access is unrestricted.

Command Line

Generic

-h, --help

show help message.

-V, --version

show version information

--config-file arg

Use alternative configuration file. When this option is used the loading of all stages is disabled. Only the given configuration file is parsed and used. To use another name for the configuration create a symbolic link of the application or copy it, eg scautopick -> scautopick2.

--plugins arg

Load given plugins.

-D, --daemon

Run as daemon. This means the application will fork itself and doesn’t need to be started with &.

Verbosity

--verbosity arg

Verbosity level [0..4]. 0:quiet, 1:error, 2:warning, 3:info, 4:debug

-v, --v

Increase verbosity level (may be repeated, eg. -vv)

-q, --quiet

Quiet mode: no logging output

--print-component arg

For each log entry print the component right after the log level. By default the component output is enabled for file output but disabled for console output.

--component arg

Limits the logging to a certain component. This option can be given more than once.

-s, --syslog

Use syslog logging back end. The output usually goes to /var/lib/messages.

-l, --lockfile arg

Path to lock file.

--console arg

Send log output to stdout.

--debug

Debug mode: –verbosity=4 –console=1

--trace

Trace mode: –verbosity=4 –console=1 –print-component=1 –print-context=1

--log-file arg

Use alternative log file.

Server

-p, --server-port int

Overrides configuration parameter AS.port.

--server-ssl-port int

Overrides configuration parameter AS.SSL.port.

-P, --plugin-port int

Overrides configuration parameter AS.plugins.port.

--http-port int

Overrides configuration parameter AS.http.port.

File System Tuning

Depending on the number of streams CAPS should handle there are some settings which can improve the IO throughput and overall performance. Since channel data are organized in an archive structure where each stream is written into a dedicated file, CAPS needs to open and close a lot of files if thousands of streams are fed into it. In the default configuration CAPS caches up to 250 open files for later reuse. An open file here is not only the data file for the CAPS stream but might also include the index file if records have been received out-of-order. So in the default configuration CAPS need to open 500 file at the same time.

Operating systems control the maximum number of open file descriptors a process might hold. Often a default value is 1024. If the maximum open files in CAPS should be increased to 2000 (assuming CAPS manages 2000 streams) then the limit for the user who runs CAPS should be increased to at least 4000. In many Linux distributions ulimit can be used for that.

Furthermore CAPS requires file descriptors for incoming connections. Each active connection holds a socket descriptor for network communication and a file descriptor (or two if index files are present) for reading data.

Depending on the number of concurrent connections one is expecting, it would be safe to add this number times three to the user limit in the operating system.

Example for 2000 streams:

# The maximum number of open files managed by CAPS.
# 2000 + margin
AS.filebase.cache.openFileLimit = 2100
# Set ulimit to 7500 files: 2100 * 2 + 1000 * 3 (network)
$ ulimit -n 7200

Access Control

CAPS allows to control access to its services through a configuration file. The default file location is ~/.seiscomp/caps/access.cfg. The format of this file is line-based. Each line consist of a key-value pair separated by =. The formal definition of one rule definition is: [DOMAIN.]<ALLOW|DENY> = <IP or network list>

The key part defines the domain and the access decision (allow or deny). The value part contains a list of IP addresses or network masks this rule should be applied to. The default policy is ALLOW. If at least one ALLOW rule is defined for a domain, all not matching IPs are denied. In addition the set of allowed IPs may be further restricted by a DENY rule.

Domains

A domain defines the interface a rule should be applied to. If no domain is specified the rule is assigned to the global domain which affects all interfaces. The following domains exist:

  • STATIONS - Defines access to the client interface on the base of stream ids. The station rules are applied in combination with the global rules. First access to the global domain must be granted, then the fine-grained stream filter is evaluated. Definition: STATIONS[.NET[.STA[.LOC[.CHA]]]].<ALLOW|DENY> = <IP or network list>
  • PLUGINS - Defines access to the plugin interface. If no plugin rule is found the rules of the global domain are applied. Definition: PLUGINS.<ALLOW|DENY> = <IP or network list>
  • WEB - Defines access to the Web interface. If no Web rule is found the rules of the global domain are applied. Definition: WEB.<ALLOW|DENY> = <IP or network list>

IP and network list

The value part of a rule consist of a comma-separated list of IP addresses or network masks. Currently only IPv4 adresses/masks are supported. IP addresses are specified by 4 numbers, e.g. 127.0.0.1. For network masks the short and long representation is supported. E.g. the mask 192.168.0.0/24 (or its long representation 192.168.0.0/255.255.255.0) would match all IPs starting with 192.168.0.X.

Examples

Simple access configuration

# Retrict access to all services to local machine
ALLOW = 127.0.0.1

Advanced access configuration

# Client data access:
# - provide access to all networks
# - limit access of GE network to IPs starting with 139.17.X.X and IP 1.2.3.4
# - deny access to station GE.APE of IP 1.2.3.4
STATIONS.GE.ALLOW = 139.17.0.0/16, 1.2.3.4
STATIONS.GE.APE.DENY = 1.2.3.4

# Accept data from 172.16.1.X but deny data from IP 42
PLUGINS.ALLOW = 172.16.1.0/24
PLUGINS.DENY = 172.16.1.42

# Restrict access to Web interface to local machine
WEB.ALLOW = 127.0.0.1/32

Secure Sockets Layer

The Secure Sockets Layer (SSL) is a standard for establishing a secured communication between applications using insecure networks. Neither client requests nor server responses are readable by communication hubs in between. SSL is based on a public-key infrastructure (PKI) to establish trust about the identity of the communication counterpart. The concept of a PKI is based on public certificates and private keys.

The following example illustrates how to generate a self-signed certificate using the OpenSSL library:

openssl req -new -x509 -sha512 -newkey rsa:4096 -out caps.crt -keyout caps.key -nodes

The last parameter -nodes disables the password protection of the private key. If omitted, a password must be defined which will be requested when accessing the private key. CAPS will request the password on the command line during startup.

To enable SSL in CAPS the AS.SSL.port as well as the location of the AS.SSL.certificate and AS.SSL.key file must be specified. Optionally the unencrypted AS.port may be deactivated by setting a value of -1.