Research Division EED/Controls Software<P> Release Note 102.1<P> Epicure Distributed Database System<P> EDDS

Research Division EED/Controls Software

Release Note 102.1

Epicure Distributed Database System

EDDS

David M. Kline

Contents

Abstract

The Epicure Distributed Database System is a centralized and distributed system which provides a simple and consistent way of manipulating and accessing devices for data acquisition. There are two different database implementations used by EDDS. The source database which contains a superset of the devices and is implemented using RMS index sequential variable length records. The Optimized Access (OA) and Look-Aside-List (LAL) databases are derived from the source database and are implemented as section files. Both OA and LAL are optimized by using hash tables for device lookup, compressing the source database records into contiguous memory, and sharing information between devices. The EDBServer process maps the OA / LAL into virtual memory space P0. The device information is accessed more efficiently because the VMS paging mechanism is used rather than the RMS subsystem. The source database is accessed when devices need to be added, modified, or deleted. Whereas, the OA/LAL databases are accessed by DAR and other applications that need device information for data acquisition.

The system is centralized by the DBSAServer process receiving and executing all transaction requests which act upon the source database. Since these requests are executed in the order they are received, record locking mechanisms are not required. The system is distributed by placing copies of the OA / LAL databases and the EDBServer process at multiple destinations. Applications may then use the Epicure db_ services to retrieve device information for data acquisition from several sources . Users are not required to reside on the same node as the server processes since both declare themselves as network objects. The db_ and dbsa_ UTIs are responsible for finding and connecting to either server. A backup system was developed to rollback the method of building and distributing databases in the event an unrecoverable condition occurs.

The system is composed of several components which are integrated to provide the services required by operators to add, modify, delete, and report device information; or build and distribute OA / LAL databases to multiple remote destinations. Listed is brief description of these components:

DBConverter
- This application was used for the initial conversion from the DDT/MDT format to the RMS databases.
DBExtractor
- This application would be used as part of the backup system in the event an unrecoverable condition exits. It reads the RMS databases and places the device information back into the MDT/DDT format. The AOA builder will be re-commissioned if this occurs.
RMS Files
- This is the source database where all the devices reside. It contains the superset of device properties and attributes.
Section Files
- These are the OA and LAL databases which are derived from the source database and used for data acquisition.
EDBServer
- The server process which applications use to lookup device information for data acquisition. This server is accessed when the Epicure db_ services are invoked.
DBScanner
- This process is responsible for monitoring the EDBServer processes. It scans all destinations and connects to the EDBServer processes and verifies that the correct OA / LAL serial numbers are present. If the serial numbers differ between the scanner and the EDBServer, the process is restarted automatically. If the EDBServer has been restarted a predetermined number of times, the scanner distributes an OA or LAL automatically. The scanner is also responsible for managing the distribution of the OA / LAL databases to multiple remote destinations.
DBSAServer
- This is the server process which serializes user requests for a device modification in the source database. Additionally, it services requests to construct the OA or LAL databases by creating the SOAL process. Applications access device information and request OA / LAL database builds using the DBSA_ UTI.
SOAL
- This process only exists when a OA / LAL build request has been successfully executed by the DBSAServer. It reads the source database for the devices which compose the OA or LAL databases. It synchronizes with the DBScanner process to distribute the OA / LAL databases and bring the EDBServer processes up to date with the current devices.
DB-Editor
- This interactive application is used by operators who wish to add, modify, or delete devices from the source database and request database builds which are distributed to remote destinations. It uses DECforms as the screen interface and the dbsa_ UTI to manipulate the devices and request builds.

The remainder of this document describes the individual components of the Epicure Distributed Database System (EDDS) with exception of the DBExtractor and DB-Editor. Release or design notes will be distributed as they become available.

RMS Databases

The VMS RMS subsystem implements the file organization and record structure of the source database. The file organization is indexed sequential with a primary and secondary key and contains variable length records. Each variable length record contains all the properties and attributes assigned for a particular device. This minimizes the use of the RMS subsystem and retrieval access times. After a record has been retrieved from the RMS subsystem, the device data is accessed either directly or by offsets. Offsets are used to compact the device information more efficiently and minimize the disk space usage. At the beginning of the device record is a ``device structure'' that contains the minimal properties that can be assigned to the device and offsets to reference property structures. The property structures are located by adding the offset to the beginning address of the device record buffer. These structures contain attributes that may be accessed either directly or by offsets to attribute structures. The attribute structures are located using the same calculation as property structures. The calculation used to locate both types of structures is required only when one views the device record. The dbsa_ UTI eliminates the need for the calculation by returning the device data or a pointer to the data.

The device index and name properties are used as the primary and secondary keys respectively. This is done to maintain consistency with the db_ services and retrieve data quicker. The Device & Synthetic RMS files define the device databases. The device databases & the Template RMS file defines the source database. These files are similar in organization and record structure but differ in content. They are separated to distribute the disk I/O rate by placing them on multiple disk drives. However, presently they're on a single drive. Below describes their content:

Device Databases
- The device databases contain records for every device that is used in the control system. The RMS files are differentiated by the device classes they contain. The synthetic RMS file consists of the SYNTHETIC class whereas the device RMS file consists of classes NORMAL, NONDEVICE, COMPOUND, and SOFTDEVICE.
Template
- The template RMS file consists of the templates which are used to create devices. Because the file organization and record formats are compatible with the device databases, template property and attribute structures are located and accessed similarly.
Status
- The status database contains build information about the RMS files. A record consists of the serial number, build times, and device counts for each one. The DBSAServer process maintains the status as modifications are made to the source database.

Section File Databases

The section files are comprised of the Optimized Access (OA) and Look-Aside-List (LAL) databases and are contained in a global section mapped to a file. Retrieving device data is performed by accessing virtual memory and allowing the VMS paging subsystem to move portions of the database between disk and physical memory. Both databases have the same file organization and record structure but differ in content and build schedule. The OA database contains every device used in the control system and is built once a day providing there has been modifications made to the source database. The LAL database contains the modifications made to the source database since the last OA build. The builds are quicker than the OA since the LAL contains only modified devices. Operators may request LAL builds after device modification from the DB-Editor. The OA & LAL are implemented by hash, shared, and device section files. The following sections describe them:

Hash Organization

The root area is at the beginning of the hash section and contains information about the database. The serial number, build time, format version, hash tables size and offset, and the lowest and highest di values are located in this area. Next are the hash tables, they are used as the initial point to lookup devices. One hash table is organized to access the database using device DI, the other is organized to access it using the device name. The entries in these tables are offsets to hash records. The hash records are the basis for device access and contains the device DI, name, and the offset to where the device property information is located. Also, they contain the offsets to the next hash records.

Device Organization

The device section contains the database serial number at the beginning of the section and is used to avoid mapping different versions of the database. Following the serial number are device records which contain device property data. The data may be accessed directly or by offsets. Property offsets are added to the beginning address of the section to locate the record. The property records contain attributes which may be accessed either directly or by offsets. Attribute offsets are added to the beginning address of the shared section to locate the record.

Shared Organization

This section contains property and attribute records that are shared between devices. The first longword of the section is the database serial number and is used, as the device section, to avoid mapping different versions of the database. The data is retrieved by using the property or attribute offsets in the device section property records and adding them to the beginning address of the shared section. The beamline & template type properties and source, analog, status, and control scaling, function, rate, display, bitnames, and control names attributes are contained in this section.

DB Converter Application

The DBConverter application transforms both the Device & Module Descriptor Template (DDT/MDT) text files into RMS files. There are two parameters that are passed to the converter . The first selects the RMS file build type and the second the operating mode. The build type parameter indicates which RMS file(s) to build. This increases the converter performance and reduces disk space requirements since only the needed RMS files are created. The second parameter specifies the operating mode. The available modes are create or modify and both read the MDT/DDT files for which templates and devices to use. The create mode constructs the RMS files in the directory the converter is executing and will supersede any versions, whereas modify mode uses the existing ones. The converter creates or modifies the status RMS file to reflect the information in the device or template RMS files.

The converter processes devices and templates by translating process table logicals which indicate the MDT/DDT files to read. The files contain a command language interpreter (CLI) that defines the properties and attributes that are assigned to templates and devices. The CLI is comprised of keywords and qualifiers which represent the existence and values of the properties and attributes. Routines are associated with keywords that are called when the CLI recognizes it. The routine extracts the value(s) from the keywords and/or qualifiers and assigns them to the device or template. The converter continues to read the property and attribute definitions until an ``end-of-device'' or ``end-of-template'' is recognized, at which point the information assigned is written to the appropriate RMS file.

The converter begins by processing the input parameters and setting itself up for the selected build type and operating mode. Every MDT/DDT logical is translated and the files are opened. Processing stops when the maximum allowable input files is exceeded or a logical translation fails. The device, synthetic, template, and static RMS files are opened as selected by the operating mode. If the template RMS file is built, the MDT files are read and processed. The converter loops about the MDT files reading in line by line and assigning properties and attributes to a template buffer. When the template definition is complete, the buffer is written to the template RMS file and placed in a cache. The cache is used for fast accesses when devices are created. Template processing is complete when all known MDT files have been read. Devices are created by selecting the appropriate operating mode and DDT files. If only devices are selected, the cache is initialized by placing every record from the template RMS file into dynamic memory. The DDT contains a keyword that indicates which template is used to create the device. The template is read from the cache and placed into a device buffer. The data retrieved from the CLI will overwrite the data in the device buffer for each property and attribute defined. Depending on the property or attribute, the converter will expand or contract the buffer as necessary. For example, the template may contain twenty bitnames however the device may define more or less. If more, the buffer is increased, if less it is decreased. When the device definition is complete, the buffer is written to either the device or template RMS file depending on the class property. If synthetic, it is written to the synthetic RMS file, otherwise it is written to the device RMS file. Device processing is completed when all known DDT files have been read. The status RMS file is updated after each template and device definition to keep track of the highest / lowest di and the template / device counts.

DBSA Server Process

The Epicure Database Serial Access Server (DBSAServer) is a VAX-based process that is responsible for servicing user requests to add, modify, delete, and report devices from the source database and request OA or LAL database builds and distributions. The server processes user requests on a first come first serve basis. The last modification made to a particular device is the current value. The user requests operations on a device, and OA / LAL builds by using the dbsa_ UTI . The server maintains access codes which enable a user to perform operations on the source database and request OA / LAL builds. It will return a status indicating whether the user may perform that operation. The server implements a write-through cache for efficient device reads. As modifications are performed on a particular device, the cache is updated as to maintain a consistent view of the device. There are two queues that are used in the server to keep track of devices. The Device Modification Queue (DMQ) is implemented as a private section mapped to a file and is used to keep track of which devices have been modified since the last OA build. The LAL section file is built from this queue. The Network Request Queue (NRQ) is also implemented as a private section mapped to a file and contains the device modification requests that were received during a OA or LAL build. The device requests are placed in the cache, to maintain a consistent view of the database, and placed into the NRQ. Since the SOAL process is executing, modifications to the source database are deferred until the build completes. After the build completes, the requests are read from the NRQ and placed into both the source database and the DMQ. The server logs all modifications that are performed on the source database into a file. The log indicates the time, device name, requester, and node. Also, it logs who requests OA or LAL builds, and attempted unauthorized requests. A new log file is created at midnight. The server performs OA or LAL build requests by spawning the SOAL process. Communication between the server and SOAL is accomplished by a termination mailbox and a common event flag cluster. A pending read on the termination mailbox is setup and is used to determine the completion status of the build. If the completion status is zero, SOAL was terminated by the DCL stop command, otherwise a VMS condition code will be returned and logged. The common event flag cluster is used to communicate SOAL completion and the type of database to build. When complete, all common event flags are cleared. The server checks the state of these flags when users request builds during a build. If the common event flags indicate a build is either pending or executing, the request is logged and a VMS condition code is returned to the application.

Source To OA/LAL Process

The Source-To-OA/LAL process constructs either OA or LAL databases. The database type is selected by either common event flags or input parameters. Interactively, the user may select the database type (OA or LAL) and the serial number by passing parameters. If ran detached, the selection is made by creating a common event flag cluster with the DBSAServer process and checking which flag is set. The OA database is built by sequentially reading the source database and creating section file records. The LAL is built by reading the DBSAServer DMQ. The device DI is read from the DMQ and then the source database is read for the full device information. The information is used to create section file records. After the OA or LAL is built, the SOAL connects with the DBScanner and requests an OA or LAL distribution. SOAL polls the DBScanner periodically and checks the status of the distribution. If the distribution exceeds a time out period, SOAL sends the DBScanner a request to clear the distribution and to update the EDBServer processes, then terminates with a time out status. Otherwise, SOAL sends the DBScanner a request to update the EDBServer processes then exits with a success status. If any errors are encountered during an OA / LAL build, SOAL will terminate with a fatal status.

DB Scanner Process

The Database Scanner (DBScanner) is a VAX-based process that is responsible for maintaining a consistent view of the OA and LAL database and managing the distribution of them. The scanner directly maps both the OA and LAL database hash section files into virtual memory, extracts the serial numbers and build times, then deletes the virtual memory and unmaps the sections. The OA and LAL build times are compared, and if the OA build time is more recent than the LALs, the LAL is rejected and its serial number becomes zero. The scanner maintains a consistent view of the databases by periodically connecting to the EDBServer process(es) on a particular destination and requesting the OA and LAL serial numbers. The returned serial number values are compared to the scanner mapped serial numbers. If they differ, the EDBServer process is restarted or is sent a message to remap the LAL. The server node name is placed onto the ``OA or LAL error list'' depending which database is in error. The scanner keeps track of how many times it restarted/remapped an EDBServer. If the times exceed a predetermined limit, the destination name is removed from the ``OA or LAL error list'' and is placed onto the ``distribution send list'', and a OA or LAL database is distributed to that destination. During the distribution, that destination is not monitored. The destinations and nodes that are monitored are determined by a set of system table logicals. The destination logical indicates which VAX clusters or nodes are to receive an OA or LAL database, and the monitor logical indicates which nodes to monitor the EDBServer processes on. The scanner monitors the EDBServer processes by translating the ``monitor'' logical and connecting to each EDBServer process on the node that the logical translated to. The OA or LAL database distributions are performed by translating the ``distribution'' logical and connecting to a network object on the VAX cluster or node. After the connection is established, the scanner disconnects and waits for a reply from the destination. When the distribution is complete, the network object executes an application which sends a message to clear off that destination name from the scanner ``distribution send list``. If the network object never responds, the destination remains on the list until it can be removed manually . Otherwise, the destination is enabled for monitoring. An interactive menu driven application is available to authorized users from the EpicureDB account that allows one to communicate with the scanner. The user may request that the status of either the ``OA / LAL error list''; ``distribution send list``; OA / LAL serial numbers and build times be printed at the terminal. The internal lists may be cleared in the event a problem exists. Additionally, the EDBServer processes may be restarted or sent an LAL remap message from the menu.

Epicure Database Server Process

The Epicure DataBase Server (EDBServer) is a VAX-based process that implements the distributed aspect of EDDS by residing on multiple nodes, thus distributing the request load. It receives requests from applications which require device information for data acquisition. The server maps both the OA and or LAL databases hash, device, and shared section files as global sections mapped to a file. Device information is moved between the disk file and physical memory by the VMS paging system thus providing the an efficient retrieval method. The server always maps the OA database for all device information and maps the LAL database only if its build time is later than the OAs. If the LAL build time is before the OAs, the LAL database is unmapped and its serial number is cleared. After mapping the sections into virtual memory, the serial numbers from the hash, device, and shared sections are compared. If they do not compare, the server signals a fatal condition and exits. After initialization has completed, the server waits for requests from applications.

Appendix: DBConverter Example

This appendix provides a description of the available DBConverter parameters and an example of how it may be invoked.

P1 - Database build type.

Valid options: All - Build all databases. Tpl - Build only template database. Dev - Build only device database.

P2 - Operation mode.

Valid options: Cre - Create databases. Mod - Modify databases.

Example: $ dbconverter all cre

Keywords: Epicure, program, database, DBSA, DBSAUTI, editor, OA, LAL, RMS, DBScanner

Distribution: normal

Security, Privacy, Legal

rwest@fsus04.fnal.gov