scdb¶
Populates a SQL database from XML files or messages.
Description¶
A major component of the SeisComP system is the database. Almost all applications have only read access to the database, but all the processing results and objects have to be written into the database. This was the task of scdb. In very first versions of SeisComP scdb was the only component that had write access to the database. Its task is to connect to scmaster and populate the database with all received notifier messages. Although it worked it introduced race conditions caused by the latency of the database backend since all other clients received the message at the same time. Accessing the database immediately at this point in time did not guarantee that the object was written already.
In consequence, the scmaster itself gained write access to the database and forwards messages to all clients after they are written to database.
scdb by definition does not check existing objects in the database. It only generates INSERT/UPDATE/DELETE statements based on the data used and sends these statements to the database. E.g. if scdb receives a message to insert a new object into the database and this object exists already, the database will raise an error because scdb hasn’t checked it.
Online mode¶
Now scdb can be used to maintain a backup or archive the database that is not part of the real time processing. When running scdb as database write daemon it can inform a client about the database connection to use. A client sends a DatabaseRequest message and scdb sends back a DatabaseResponse message containing the database connection parameters.
For that it connects to a messaging server and writes all received messages to a configured database, e.g. a backup database.
Note
The database connection received from the messaging server during the handshake is reported to clients requesting a database address. To overwrite the read-only database, just override the application’s database address (with the ‘-d’ option)
Offline mode¶
Another important task of scdb is to populate the database with any SeisComP data model content. In combination with scxmldump it can be used to copy events from one database to another.
For that it does not connect to a messaging server but reads data from XML files and writes it to the database. Offline mode will be used if the ‘–input/-i’ option is provided. Multiple input files can be specified by providing this option multiple times with separate filenames.
Warning
When reading XML files the output database address is not passed with -o but -d. The application’s database address is used.
Examples¶
Connect to a messaging server and write all messages to the output database seiscomp running on the host db-server:
scdb -H [server] -o mysql://sysop:sysop@db-server/seiscomp
As above, but with the read-only database connection using the user sysop and the output database connection using the user writer:
scdb -H [server] -d mysql://sysop:sysop@db-server/seiscomp \ -o mysql://writer:12345@db-server/seiscomp
Import data from the file
data.xml
and write it to the database seiscomp on the host db-server:scdb -i data.xml -d mysql://sysop:sysop@db-server/seiscomp
Import data from three files at once:
scdb -i data1.xml -i data2.xml -i data3.xml \ -d mysql://sysop:sysop@db-server/seiscomp
Configuration¶
etc/defaults/global.cfg
etc/defaults/scdb.cfg
etc/global.cfg
etc/scdb.cfg
~/.seiscomp/global.cfg
~/.seiscomp/scdb.cfg
scdb inherits global options.
-
connection.requestGroup
¶ Type: string
Defines the group on scmaster to subscribe for database requests.
-
connection.provideGroup
¶ Type: string
Defines the group on scmaster to send database response messages to.
-
output.type
¶ Type: string
Defines the output database connection type.
-
output.parameters
¶ Type: string
Defines the output database connection parameters.
Command-line¶
Generic¶
-
-h
,
--help
¶
show help message.
-
-V
,
--version
¶
show version information
-
--config-file
arg
¶ Use alternative configuration file. When this option is used the loading of all stages is disabled. Only the given configuration file is parsed and used. To use another name for the configuration create a symbolic link of the application or copy it, eg scautopick -> scautopick2.
-
--plugins
arg
¶ Load given plugins.
-
-D
,
--daemon
¶
Run as daemon. This means the application will fork itself and doesn’t need to be started with &.
-
--auto-shutdown
arg
¶ Enable/disable self-shutdown because a master module shutdown. This only works when messaging is enabled and the master module sends a shutdown message (enabled with –start-stop-msg for the master module).
-
--shutdown-master-module
arg
¶ Sets the name of the master-module used for auto-shutdown. This is the application name of the module actually started. If symlinks are used then it is the name of the symlinked application.
-
--shutdown-master-username
arg
¶ Sets the name of the master-username of the messaging used for auto-shutdown. If “shutdown-master-module” is given as well this parameter is ignored.
Verbosity¶
-
--verbosity
arg
¶ Verbosity level [0..4]. 0:quiet, 1:error, 2:warning, 3:info, 4:debug
-
-v
,
--v
¶
Increase verbosity level (may be repeated, eg. -vv)
-
-q
,
--quiet
¶
Quiet mode: no logging output
-
--component
arg
¶ Limits the logging to a certain component. This option can be given more than once.
-
-s
,
--syslog
¶
Use syslog logging back end. The output usually goes to /var/lib/messages.
-
-l
,
--lockfile
arg
¶ Path to lock file.
-
--console
arg
¶ Send log output to stdout.
-
--debug
¶
Debug mode: –verbosity=4 –console=1
-
--log-file
arg
¶ Use alternative log file.
Messaging¶
-
-u
,
--user
arg
¶ Overrides configuration parameter
connection.username
.
-
-H
,
--host
arg
¶ Overrides configuration parameter
connection.server
.
-
-t
,
--timeout
arg
¶ Overrides configuration parameter
connection.timeout
.
-
-g
,
--primary-group
arg
¶ Overrides configuration parameter
connection.primaryGroup
.
-
-S
,
--subscribe-group
arg
¶ A group to subscribe to. This option can be given more than once.
-
--content-type
arg
¶ Overrides configuration parameter
connection.contentType
.
-
--start-stop-msg
arg
¶ Sets sending of a start- and a stop message.
-
--m
,
--mode
arg
¶ scdb can either process a XML file and write it to the database or collecting messages from scmaster. If connected to scmaster the mode defines what objects are handled: none (no objects at all), notifier (notifier only) or all (all objects whereas non-notifier objects are INSERTED into the database).
Database¶
-
--db-driver-list
¶
List all supported database drivers.
-
-d
,
--database
arg
¶ The database connection string, format: service://user:pwd@host/database. “service” is the name of the database driver which can be queried with “–db-driver-list”.
-
--config-module
arg
¶ The configmodule to use.
-
--inventory-db
arg
¶ Load the inventory from the given database or file, format: [service://]location
-
--config-db
arg
¶ Load the configuration from the given database or file, format: [service://]location
-
--o
,
--output
arg
¶ If connected to scmaster this flag defines the database connection to use for writing. The configured application database connection (as received from scmaster) is reported to clients as part of a database response messages.