.. highlight:: rst .. _scdbstrip: ######### scdbstrip ######### **Database clean-up of processing results.** Description =========== SeisComP3s processing is continuously writing to the database. This causes the database to grow and to occupy much space on the harddisc. scdbstrip taggles this problem and removed processed objects from the database older than a configurable time span. This clean-up procedure is based on events. scdbstrip will remove all events with an origin time older than specified. It will also remove all associated objects such as picks, origins, arrivals, amplitudes and so on. scdbstrip does not run as a daemon. To remove old objects continuously scdbstrip should be added to the list of cronjobs running every e.g. 30 minutes. The more often it runs the less objects it has to remove and the faster it will unlock the database again. Known issues ============ When running scdbstrip for the first time on a large database it can happen that it aborts in case of MYSQL with the following error message: .. code-block:: sh [ 3%] Delete origin references of old events...08:48:22 [error] execute("delete Object from Object, OriginReference, old_events where Object._oid=OriginReference._oid and OriginReference._parent_oid=old_events._oid") = 1206 (The total number of locks exceeds the lock table size) Exception: ERROR: command 'delete Object from Object, OriginReference, old_events where Object._oid=OriginReference._oid and OriginReference._parent_oid=old_events._oid' failed That means your MYSQL server cannot hold enough data required for deletion. There are two solutions to this: 1. Increase the memory pool used by MYSQL by changing the configuration to: .. code-block:: sh innodb_buffer_pool_size = 64M The size of the new buffer depends on the size of the database that should be cleaned up. 2. Run scdbstrip on smaller batches for the first time: .. code-block:: sh $ scdbstrip -d seis:mypass@localhost/seiscomp3 --days 1000 $ scdbstrip -d seis:mypass@localhost/seiscomp3 --days 900 ... $ scdbstrip -d seis:mypass@localhost/seiscomp3 --days 100 Examples ======== #. Keep the events of the last 30 days .. code-block:: sh scdbstrip --days 30 -d mysql://sysop:sysop@localhost/seiscomp3 .. _scdbstrip_configuration: Configuration ============= | :file:`etc/defaults/global.cfg` | :file:`etc/defaults/scdbstrip.cfg` | :file:`etc/global.cfg` | :file:`etc/scdbstrip.cfg` | :file:`~/.seiscomp3/global.cfg` | :file:`~/.seiscomp3/scdbstrip.cfg` scdbstrip inherits :ref:`global options`. .. confval:: database.cleanup.keep.days Type: *int* The number of days to preserve in the database. This value is just one part of the whole timespan. Hours and minutes are configured separately. Default is ``30``. .. confval:: database.cleanup.keep.hours Type: *int* The number of hours to preserve in the database. This value is just one part of the whole timespan. Days and minutes are configured separately. Default is ``0``. .. confval:: database.cleanup.keep.minutes Type: *int* The number of minutes to preserve in the database. This value is just one part of the whole timespan. Days and hours are configured separately. Default is ``0``. Command-line ============ .. program:: scdbstrip Generic ------- .. option:: -h, --help show help message. .. option:: -V, --version show version information .. option:: --config-file arg Use alternative configuration file. When this option is used the loading of all stages is disabled. Only the given configuration file is parsed and used. To use another name for the configuration create a symbolic link of the application or copy it, eg scautopick \-> scautopick2. .. option:: --plugins arg Load given plugins. .. option:: -D, --daemon Run as daemon. This means the application will fork itself and doesn't need to be started with \&. .. option:: --first-new Overrides configuration parameter :confval:`firstNew`. Verbosity --------- .. option:: --verbosity arg Verbosity level [0..4]. 0:quiet, 1:error, 2:warning, 3:info, 4:debug .. option:: -v, --v Increase verbosity level \(may be repeated, eg. \-vv\) .. option:: -q, --quiet Quiet mode: no logging output .. option:: --component arg Limits the logging to a certain component. This option can be given more than once. .. option:: -s, --syslog Use syslog logging back end. The output usually goes to \/var\/lib\/messages. .. option:: -l, --lockfile arg Path to lock file. .. option:: --console arg Send log output to stdout. .. option:: --debug Debug mode: \-\-verbosity\=4 \-\-console\=1 .. option:: --log-file arg Use alternative log file. Database -------- .. option:: --db-driver-list List all supported database drivers. .. option:: -d, --database arg The database connection string, format: service:\/\/user:pwd\@host\/database. \"service\" is the name of the database driver which can be queried with \"\-\-db\-driver\-list\". .. option:: --config-module arg The configmodule to use. .. option:: --inventory-db arg Load the inventory from the given database or file, format: [service:\/\/]location .. option:: --db-disable Do not use the database at all Settings -------- .. option:: --days arg Overrides configuration parameter :confval:`database.cleanup.keep.days`. The number of days to keep. Hours and minutes are also used the compute the whole time span. .. option:: --hours arg Overrides configuration parameter :confval:`database.cleanup.keep.hours`. The number of hours to keep. Days and minutes are also used the compute the whole time span. .. option:: --minutes arg Overrides configuration parameter :confval:`database.cleanup.keep.minutes`. The number of minutes to keep. Days and hours are also used the compute the whole time span. .. option:: --datetime arg Replaces the days:hours:minutes timespan definition by an arbtrary absolute timestamp in UTC. The format is %Y\-%m\-%d %H:%M:%S. .. option:: -i, --invert Delete all events after the specified time period and not before. .. option:: --keep-events Event\-IDs to keep in the database separated with comma. Mode ---- .. option:: --check .. option:: --clean-unused