Research Division EED/Controls Software
Release Note 102.1
Epicure Distributed Database System
EDDS
David M. Kline
Contents
Abstract
The Epicure Distributed Database System is a centralized and
distributed system which provides a simple and consistent
way of manipulating and accessing devices for data acquisition.
There are two different database implementations
used by EDDS. The source database which contains
a superset of the devices and is implemented using RMS
index sequential
variable length records. The Optimized Access (OA) and Look-Aside-List (LAL)
databases are derived from the source database and are implemented as
section files. Both OA and LAL are optimized by using hash tables for
device lookup, compressing the source database records into contiguous
memory, and sharing information between devices. The EDBServer process
maps the OA / LAL into virtual memory space P0. The device information
is accessed more
efficiently because the VMS paging mechanism
is used
rather than the RMS subsystem. The source database is accessed
when devices need to be added, modified, or deleted. Whereas, the OA/LAL
databases are accessed by DAR and other applications that need device
information for data acquisition.
The system is centralized by the DBSAServer process receiving
and executing all transaction requests which act upon the source database. Since
these requests are executed in the order they are received, record locking
mechanisms are not required. The system is distributed by
placing copies of the OA / LAL databases and the EDBServer process at multiple destinations.
Applications may then use the Epicure db_ services
to retrieve device information for data acquisition from several
sources
. Users are not
required to reside on the same node as the server processes since both
declare themselves as network objects. The db_ and dbsa_ UTIs are
responsible for finding and connecting to either server. A backup system was
developed to rollback the method of building and
distributing databases in the event an unrecoverable condition occurs.
The system is composed of several components which are integrated
to provide the services required by operators to add, modify, delete, and
report device information; or build and distribute OA / LAL databases to
multiple remote destinations. Listed is brief description of these components:
- DBConverter
- - This application was used for the initial
conversion from the DDT/MDT
format to the RMS databases.
- DBExtractor
- - This application would be used as part of the backup system
in the event an unrecoverable condition exits. It reads the RMS databases and
places the device information back into the MDT/DDT format. The AOA
builder
will be re-commissioned if this occurs.
- RMS Files
- - This is the source database where all the devices reside. It
contains the superset of device properties and attributes.
- Section Files
- - These are the OA and LAL databases which are derived from
the source database and used for data acquisition.
- EDBServer
- - The server process which applications use to lookup device
information for data acquisition. This server is accessed when the Epicure
db_ services are invoked.
- DBScanner
- - This process is responsible for monitoring the EDBServer
processes. It scans all destinations and connects to the EDBServer processes
and verifies that the correct OA / LAL serial numbers are present. If the
serial numbers differ between the scanner and the EDBServer, the process
is restarted automatically. If the EDBServer has been restarted
a predetermined number of times, the scanner distributes an OA or LAL automatically.
The scanner is also responsible
for managing the distribution of the OA / LAL databases to multiple remote destinations.
- DBSAServer
- - This is the server process which serializes user requests
for a device modification in the source database. Additionally, it services requests to
construct the OA or LAL databases by creating the SOAL process. Applications access device
information and request OA / LAL database builds using the DBSA_ UTI.
- SOAL
- - This process only exists when a OA / LAL build request has been
successfully executed by the DBSAServer. It reads the source database for
the devices which compose the OA or LAL databases. It synchronizes with the
DBScanner process to distribute the OA / LAL databases and bring the
EDBServer processes up to date with the current devices.
- DB-Editor
- - This interactive application is used by
operators who wish to add, modify, or delete devices from the source database and
request database builds which are distributed to remote destinations. It
uses DECforms
as the screen interface and
the dbsa_ UTI
to manipulate the devices and request builds.
The remainder of this document describes the individual
components of the Epicure Distributed Database System (EDDS) with exception
of the DBExtractor and DB-Editor. Release or design notes will be distributed
as they become available.
RMS Databases
The VMS RMS subsystem implements the file organization and record
structure of the source database. The file organization is
indexed sequential with a primary and secondary key and contains
variable length records. Each variable length record contains all
the properties and attributes assigned for a particular device. This
minimizes the use of the RMS subsystem and retrieval access times.
After a record has been retrieved from the RMS subsystem, the device data
is accessed either directly or by offsets. Offsets are used to compact the
device information more efficiently and
minimize the disk space usage. At the beginning of the device record is
a ``device structure'' that contains the minimal properties that can be assigned
to the device and offsets to reference property structures. The property structures are
located by adding the offset to the beginning address of the device record
buffer. These structures contain attributes that may be accessed either directly
or by offsets to attribute structures. The attribute structures
are located using the same calculation as property structures. The calculation used
to locate both types of structures is required only when one views the device record.
The dbsa_ UTI eliminates the need for the calculation by returning the device data
or a pointer to the data.
The device index and name properties are used as the primary
and secondary keys respectively. This is done to maintain consistency with
the db_ services and retrieve data quicker. The Device & Synthetic RMS files define
the device databases. The device databases & the Template RMS file defines
the source database. These files are similar in
organization and record structure but differ in content. They are separated
to distribute the disk I/O rate by placing them on multiple disk drives. However,
presently they're on a single drive. Below describes their content:
- Device Databases
- - The device databases contain records for every
device that is used in the control system. The RMS files are differentiated
by the device classes they contain. The synthetic RMS file consists of the
SYNTHETIC class whereas the device RMS file consists of classes NORMAL,
NONDEVICE, COMPOUND, and SOFTDEVICE.
- Template
- - The template RMS file consists of the templates which are
used to create devices. Because the file organization and record formats
are compatible with the device databases, template property and attribute
structures are located and accessed similarly.
- Status
- - The status database contains build information about the
RMS files. A record consists of the serial number, build
times, and device counts for each one. The DBSAServer process maintains
the status as modifications are made to the source database.
Section File Databases
The section files are comprised of the Optimized Access (OA) and
Look-Aside-List (LAL) databases and are contained in a global section
mapped to a file. Retrieving device data is performed by accessing virtual
memory and allowing the VMS paging subsystem to move portions of
the database between disk and physical memory. Both databases
have the same file organization and record structure but differ
in content and build schedule. The OA database contains every
device used in the control system and is built once a day providing
there has been modifications made to the source database. The LAL
database contains the modifications made to the source database
since the last OA build. The builds are quicker than the OA since
the LAL contains only modified devices. Operators may request LAL
builds after device modification from the DB-Editor. The OA & LAL
are implemented by hash, shared, and device section files. The
following sections describe them:
Hash Organization
The root area is at the beginning of the hash section and
contains information about the database. The serial number, build time,
format version, hash tables size and offset, and the lowest and highest
di values are located in this area. Next are the hash tables, they are used
as the initial point to lookup devices. One hash table is organized to
access the database using device DI, the other is organized to access it using the device
name. The entries in these tables are offsets to hash records. The hash records
are the basis for device access and contains the device DI, name, and the offset
to where the device property information is located. Also, they contain
the offsets to the next hash records.
Device Organization
The device section contains the database serial number at the beginning
of the section and is used to avoid mapping different versions of the database.
Following the serial number are device records which contain device property
data. The data may be accessed directly or by offsets. Property offsets are added
to the beginning address of the section to locate the record. The property
records contain attributes which may be accessed either directly or by offsets.
Attribute offsets are added to the beginning address of the shared section to locate
the record.
Shared Organization
This section contains property and attribute records that are shared
between devices. The first longword of the section is the database serial number
and is used, as the device section, to avoid mapping different versions of the
database. The data is retrieved by using the property or attribute offsets in the
device section property records and adding them to the beginning address of the shared
section. The beamline
& template type properties and source, analog, status, and control scaling,
function, rate, display, bitnames, and control names attributes are contained in this
section.
DB Converter Application
The DBConverter application transforms both the Device & Module
Descriptor Template (DDT/MDT) text files into RMS files. There are two parameters
that are passed to the converter
. The
first selects the RMS file build type and the
second the operating mode. The build type parameter indicates which RMS file(s) to build.
This increases the converter performance and reduces
disk space requirements since only the needed RMS files are created. The second parameter
specifies the operating mode. The available modes are create or modify and both read the
MDT/DDT files for which templates and devices to use. The create mode constructs the RMS files
in the directory the converter is executing and will supersede any versions, whereas modify mode
uses the existing ones. The converter creates or modifies the status RMS file to
reflect the information in the device or template RMS files.
The converter processes devices and templates by translating process table
logicals which indicate the MDT/DDT files to read. The files contain a command
language interpreter (CLI) that defines the properties and attributes that are assigned to templates
and devices. The CLI is comprised of keywords and qualifiers which represent
the existence and values of the properties and attributes. Routines are associated with
keywords that are called when the CLI recognizes it. The routine
extracts the value(s) from the keywords and/or qualifiers and assigns them to the
device or template. The converter continues to read the property
and attribute definitions until an ``end-of-device'' or ``end-of-template'' is recognized,
at which point the information assigned is written to the appropriate RMS file.
The converter begins by processing the input parameters and setting itself
up for the selected build type and operating mode. Every MDT/DDT logical is translated
and the files are opened. Processing stops when the maximum allowable input files
is exceeded or a logical translation fails. The device, synthetic, template, and
static RMS files are opened as selected by the operating mode. If the template RMS
file is built, the MDT files are read and processed. The converter loops about
the MDT files reading in line by line and assigning properties and attributes to a
template buffer. When the template definition is complete, the buffer is written to the template
RMS file and placed in a cache. The cache is used for fast accesses when devices are created.
Template processing is complete when all known MDT files have been read. Devices
are created by selecting the appropriate operating mode and DDT files. If only devices are selected,
the cache is initialized by placing every record from the template RMS file into dynamic memory.
The DDT contains a keyword that indicates which template is used to create the device.
The template is read from the cache and placed into a device buffer. The data retrieved
from the CLI will overwrite the data in the device buffer for each property and attribute defined.
Depending on the property or attribute, the converter will expand or
contract the buffer as necessary. For example, the template may contain twenty bitnames
however the device may define more or less. If more, the buffer is increased, if less
it is decreased. When the device definition is complete, the buffer is written to either
the device or template RMS file depending on the class property.
If synthetic, it is written to the synthetic
RMS file, otherwise it is written to the device RMS file. Device processing is
completed when all known DDT files have been read. The status RMS file is updated after
each template and device definition to keep track of the highest / lowest di and
the template / device counts.
DBSA Server Process
The Epicure Database Serial Access Server (DBSAServer) is a VAX-based process that
is responsible for servicing user requests to add, modify, delete, and report devices from
the source database and request OA or LAL database builds and distributions.
The server processes user requests on a first come first serve basis. The last
modification made to a particular device is the current value. The user requests operations on
a device, and OA / LAL builds by using the dbsa_ UTI
.
The server maintains access codes which enable a user to perform operations on the source
database and request OA / LAL builds. It will return a status indicating whether the user
may perform that operation. The server implements a write-through cache for efficient device
reads. As modifications are performed on a particular device, the cache is updated as to maintain
a consistent view of the device. There are two queues that are used in the server to keep track
of devices. The Device Modification Queue (DMQ) is implemented as a private section mapped
to a file and is used to keep track of which devices have been modified since the last OA build.
The LAL section file is built from this queue. The Network Request Queue (NRQ) is also
implemented as a private section mapped to a file and contains the device modification requests that were
received during a OA or LAL build. The device requests are placed in the cache, to
maintain a consistent view of the database, and placed into the NRQ. Since the SOAL process is executing,
modifications to the source database are deferred until the build completes.
After the build completes, the requests are read from the NRQ and placed into both the source
database and the DMQ. The server logs all modifications that are performed on the source
database into a file. The log indicates the time, device name, requester, and node. Also, it
logs who requests OA or LAL builds, and attempted unauthorized requests.
A new log file is created at midnight. The server performs OA or LAL
build requests by spawning the SOAL
process. Communication between
the server and SOAL is accomplished by a termination mailbox and a common event flag cluster.
A pending read on the
termination mailbox is setup and is used to determine the completion status of the build. If the
completion status is zero, SOAL was terminated by the DCL stop command, otherwise a VMS condition
code will be returned and logged. The common event flag cluster is used to communicate SOAL completion
and the type of database to build. When complete, all common event flags are cleared. The server
checks the state of these flags when users request builds during a build. If the common event flags
indicate a build is either pending or executing, the request is logged and a VMS condition code is
returned to the application.
Source To OA/LAL Process
The Source-To-OA/LAL process constructs either OA or LAL databases. The database type is
selected by either common event flags or input parameters.
Interactively, the user may select the database type (OA or LAL) and the serial number by passing
parameters. If ran detached, the selection is made by creating a common event flag cluster with
the DBSAServer process and checking which flag is set. The OA database is built by sequentially reading
the source database and creating section file records. The LAL is built by reading the DBSAServer DMQ.
The device DI is read from the DMQ and then the source database is read for the full device information.
The information is used to create section file records. After the OA or LAL is built, the SOAL
connects with the DBScanner and requests an OA or LAL distribution. SOAL polls
the DBScanner periodically and checks the status of the distribution. If the distribution
exceeds a time out period, SOAL sends the DBScanner a request to clear the distribution and
to update the EDBServer processes, then terminates with a time out status. Otherwise, SOAL
sends the DBScanner a request to update the EDBServer processes then exits with a success status. If
any errors are encountered during an OA / LAL build, SOAL will terminate with a fatal status.
DB Scanner Process
The Database Scanner (DBScanner) is a VAX-based process that is responsible for maintaining a consistent
view of the OA and LAL database and managing the distribution of them.
The scanner directly maps both the OA and LAL database hash section files into virtual
memory, extracts the serial numbers and build times, then deletes the virtual memory and unmaps the sections.
The OA and LAL build times are compared, and if the OA build time is more recent than the LALs, the LAL is
rejected and its serial number becomes zero. The scanner maintains a consistent view of the databases
by periodically connecting to the EDBServer process(es) on a particular destination and requesting the OA and LAL serial numbers.
The returned serial number values are compared to the scanner mapped serial numbers. If they differ, the
EDBServer process is restarted or is sent a message to remap the LAL. The server
node name is placed onto the ``OA or LAL error list'' depending which database is in error. The scanner
keeps track of how many times it restarted/remapped an EDBServer. If the times exceed a predetermined limit,
the destination name is removed from the ``OA or LAL error list'' and is placed onto the ``distribution send list'',
and a OA or LAL database is distributed to that destination. During the distribution, that destination is
not monitored. The destinations and nodes that are monitored are determined by a set of system table logicals.
The destination logical indicates which VAX clusters or nodes are to receive an OA or LAL database, and the
monitor logical indicates which nodes to monitor the EDBServer processes on. The scanner monitors the EDBServer processes
by translating the ``monitor'' logical and connecting to each EDBServer process on the node that the logical translated
to. The OA or LAL database distributions are performed by translating the ``distribution'' logical and connecting to
a network object on the VAX cluster or node. After the connection is established, the scanner disconnects and waits
for a reply from the destination. When the distribution is complete, the network object executes an application
which sends a message to clear off that destination name from the scanner ``distribution send list``. If the
network object never responds, the destination remains on the list until it can be removed manually
. Otherwise,
the destination is enabled for monitoring. An interactive menu driven application is available to authorized
users from the EpicureDB account that allows one to communicate with the scanner. The user may request that
the status of either the ``OA / LAL error list''; ``distribution send list``; OA / LAL serial numbers and build times
be printed at the terminal. The internal lists may be cleared in the event a problem exists. Additionally, the
EDBServer processes may be restarted or sent an LAL remap message from the menu.
Epicure Database Server Process
The Epicure DataBase Server (EDBServer) is a VAX-based process that implements the distributed aspect
of EDDS by residing on multiple nodes, thus distributing the request load. It receives requests from applications
which require device information for data acquisition. The server maps both the OA and or LAL databases hash, device,
and shared section files as global sections mapped to a file. Device information is moved between the disk file
and physical memory by the VMS paging system thus providing the an efficient retrieval method. The server always maps
the OA database for all device information and maps the LAL database only if its build time is later than the OAs.
If the LAL build time is before the OAs, the LAL database is unmapped and its serial number is cleared. After mapping the
sections into virtual memory, the serial numbers from the hash, device, and shared sections are compared. If they
do not compare, the server signals a fatal condition and exits. After initialization has completed, the server
waits for requests from applications.
Appendix: DBConverter Example
This appendix provides a description of the available
DBConverter parameters and an example of how it may be invoked.
P1 - Database build type.
Valid options:
All - Build all databases.
Tpl - Build only template database.
Dev - Build only device database.
P2 - Operation mode.
Valid options:
Cre - Create databases.
Mod - Modify databases.
Example:
$ dbconverter all cre
Keywords: Epicure, program, database, DBSA, DBSAUTI, editor, OA, LAL, RMS, DBScanner
Distribution: normal
Security, Privacy, Legal
rwest@fsus04.fnal.gov