.. _concepts_inventory: ********* Inventory ********* Scope ===== This chapter describes the concept of inventories. Overview ======== In |scname| the term :term:`inventory` refers to the meta data of data recording stations. Many |scname| :ref:`modules ` require inventory information to operate correctly and a correct inventory is therefore fundamental to |scname|. The inventory will be read from the :ref:`database ` by default. To insert or update the inventory in the database, this information needs to be created or downloaded and converted. The following sections will describe the process of populating the database and how applications get access to it. The format used for inventory is known as :term:`SCML`. Correct inventory in :term:`SCML` format can be generated by :cite:t:`smp`. Some modules for **converting from other formats** are listed in section :ref:`concepts_inventory_format`. Tools for **processing inventory** are listed in section :ref:`concepts_inventory_tools`. The inventory shall contain all meta data describing the full recording system and the pre-processing of the raw data stream provided to the data processing modules using the :ref:`RecordStream `. Inventories should be always complete w.r.t the processed data streams, correct and up-to-date. Beside network, station, location and stream information they must include the full sensor and datalogger responses. Older version used key files to configure available networks and stations. Because the support of response meta-data was very limited, tools were build to add this functionality. Since the release of Seattle the concept of key files for station meta-data has been completely removed from the system. Now |scname| only handles station meta-data in its own XML format called :term:`SCML`. The task of supporting old key files, dataless SEED and other formats has been out-sourced to external applications. The inventory synchronization is a :ref:`two-stage process `: #. Convert :ref:`external formats ` into :term:`SCML` #. Synchronize inventory pool with the database: merged all inventory files and synchronize with the database using :program:`scinv snyc`. .. _config-fig-inventory-sync: .. figure:: ../media/inventory-sync.png :align: center :width: 12cm Inventory synchronization as a two-stage process All station meta-data are stored in :file:`etc/inventory` and can be organized as needed. Either one file per network, a file containing the complete inventory or one file for all instruments and one file per station. The update script .. code-block:: xml $ scinv sync loads the existing inventory from the database and merges each file in :file:`etc/inventory`. Finally it removes all unreferenced objects and sends all updates to the database. The |scname| :ref:`configuration ` does not deal with station meta-data anymore. It only configures parameters for modules and module-station associations. The management of the inventory can and should be handled by external tools e.g. :cite:t:`smp`. The |scname| documentation describes the :ref:`data model including the inventory `. .. _concepts_inventory_format: Inventory Format ================ A typical inventory file in :term:`SCML` looks like this: .. code-block:: xml ... ... ... The version attribute of the ``seiscomp`` tag represents the schema version of the |scname| data model which is consistent with the database schema version and the version of all other representations. .. note:: All geographic coordinates (latitudes, longitudes, elevation) are assumed in the World Geodetic System 1984 (WGS84) reference system (National Imagery and Mapping Agency 2000). Latitudes, longitudes are provided in degrees but elevations are given in meters. Inventories must be provided to |scname| in XML files in :term:`SCML` format. A convenient way to generate clean and correct inventory files in :term:`SCML` format is :cite:t:`smp`. Tools are provided to convert between other formats: .. csv-table:: :widths: 1 1 :header: Module namespace, Conversion :align: left arclink2inv, Arclink XML to SeisComPML :ref:`dlsv2inv`, dataless SEED to SeisComPML :ref:`inv2dlsv`, SeisComPML to dataless SEED :ref:`fdsnxml2inv`, FDSN StationXML to SeisComPML and back Adding / Updating Inventory =========================== To add inventory information to the |scname| database one either has to write directly to the database with custom script (not recommended) or place :term:`SCML` files in :file:`@SYSTEMCONFIGDIR@/inventory`. The service to import all data in that directory can be called with .. code-block:: sh $ seiscomp update-config inventory This command runs :ref:`scinv` finally. :ref:`scinv` merges all XML files and synchronizes the merged local inventory tree with the database. That is a rather complex process and for more information it is recommended to study the source code of :ref:`scinv`. The bottom line is that inventory data is created from :term:`SCML` files. Because nobody will ever create such XML files by hand, tools are necessary. A quite popular tools is the online station management portal (SMP) :cite:p:`smp`. SMP will output XML in version of :term:`SCML` directly. Another source is the FDSN station webservice (FDSNWS). If the |scname| implementation of FDSNWS is used, the SeisComP XML format can be requested directly as an extension. Otherwise FDSN StationXML will be provided. Inventory in FDSN StationXML needs to be converted with either :ref:`fdsnxml2inv` or :ref:`import_inv`. The latter is a more versatile application that also supports several import formats. .. code-block:: sh $ import_inv fdsnxml download-1234.xml This will automatically place the output XML file in :file:`@SYSTEMCONFIGDIR@/inventory`. Afterwards call .. code-block:: sh $ seiscomp update-config inventory to synchronize the changes with the database. If :program:`scconfig` is used, then either the ``Sync`` button of the `Inventory` panel or the ``Update configuration`` button of the ``System`` panel must pressed. **Summary** In order to populate the database with inventory information, the following steps have to be performed: #. Convert existing station meta data formats to :term:`SCML` #. Place all :term:`SCML` files at :file:`@SYSTEMCONFIGDIR@/inventory` #. Synchronize the inventory files and write the meta data to the database. Run .. code-block:: sh $ scinv sync or .. code-block:: sh $ seiscomp update-config inventory Reading Inventory ================= Applications usually connect to the database and read the necessary inventory information. An application can decide whether it requires full response information including sensor and data logger response functions or just channel information without instrument descriptions. The latter performs faster and some applications do not require full instrument information. An application usually does not require special configuration to read inventory information. A database connection is enough and it comes usually along with the handshake message of the messaging server. If the messaging is not involved, the database can be specified by the command-line option ``-d``: .. code-block:: sh $ myapp -d [type]://[user]:[password]@[host]:[port] There are cases when an application should be run without a database connection but requires inventory information, particularly in combination with the ``--ep`` command line argument. To direct an application to an inventory XML file (again in :term:`SCML` format), ``--inventory-db`` must be used: .. code-block:: sh $ myapp --inventory-db inventory.xml The option ``--inventory-db`` will cause the module to completely bypass the database for reading inventory information even if used for event information. The file :file:`inventory.xml` can be created from the database using :ref:`scxmldump`. .. _concepts_inventory_tools: Related Tools ============= * arclink2inv * :ref:`dlsv2inv` * :ref:`fdsnxml2inv` * :ref:`import_inv` * :ref:`invextr` * :ref:`inv2dlsv` * :ref:`sccnv` * :ref:`scinv` * :ref:`scxmldump` * :ref:`tab2inv` * :ref:`tabinvmodifier`