Research Division EED/Controls Software<P> Design Note 95.1<P> EPICURE OA/LAL Database Integrated Builder System<P> V1.0<P>

Research Division EED/Controls Software

Design Note 95.1

EPICURE OA/LAL Database Integrated Builder System


Cynthia A. Chopp David M. Kline



The purpose of this paper is to propose an implementation strategy for the Epicure database Look-Aside-List (LAL), and to provide descriptions of the components which compose the Optimized Access (OA) View/LAL builds. The LAL is a subset of the OA view and contains only the devices which have been added, modified, or deleted since the last complete database build. The mission of the LAL implementation strategy is to move the Epicure control system closer to providing operations with a simple and consistent way of manipulating devices and device templates transparently without requiring detailed knowledge about the database. By implementing the LAL, faster build times may be realized since the number of devices to process is drastically reduced from the amount that is contained in the OA view. Additionally, the LAL section files are smaller, which have an impact on the distribution time to the remote destinations. An interactive editor will be used to perform the adding, modifying, and deleting of devices and templates using the Epicure ESM routines to produce the required screens.

There are several components which are incorporated into the LAL implementation which interact. The following sections provide a description of the components and suggest how they might perform the required tasks.


The LAL implementation will use the Epicuredb account OA builder as a point of departure. The techniques incorporated to build, and distribute, will be maintained on every destination for both an OA and LAL build. Global variables and parameter passing will be used as switches to indicate which build is being accessed, thus maintaining one group of programs to build either the LAL or OA. The implementation will use a combination of lower priority batch and detached processes to perform the required tasks. A condition handling function will be provided to reduce operator intervention by specifically checking for error conditions which could affect the execution of the OA/LAL build application. The components used to implement the build, distribution, and start up are listed and described below:

- The current device (DDT) and template (MDT) files will require conversion to the RMS format. An application will be written to perform this.

- A DB Editor (DBEditor) will be written, in ESM format, to provide a method of adding, modifying, and deleting devices and templates from the RMS databases. Here authorized users will be able to start up a LAL or OA build. Users are authorized for editor access by utilizing VMS resource identifiers. These identifiers are assigned by the RDCS system management group.
- An application will be written that executes similar to the current AOA Builder implementation to build and distribute the OA or LAL databases. In general, to build and distribute the OA or LAL, the builder requires four subcomponents which interact together. Detailed descriptions of these components is provided in later sections, however, they are briefly described below:

- The current application used to build the OA database will be modified to build a LAL. COA will accept parameters from the command line which direct it to build either an OA or LAL.
- The DB Scanner (DBScanner) is an application that runs as a detached process and scans the nodes which execute an Epicure Database Server process (EDBServer) for valid serial numbers and build times. The application will read an RMS file which contains a list of the valid nodes executing the EDBServer process and send a remote DECnet message to read the current serial number and build time for the OA and LAL. If the serial numbers do not match, the EDBServer process is restarted on that node. Additional communication is required to synchronize the OA or LAL distribution.
- The DB Builder (DBBuilder) application executes as a batch process on one of the core nodes in the Warner cluster, and manages the synchronization of the building and distribution of the OA or LAL databases.
- The DB Extractor (DBExtractor) application is invoked by the DBBuilder application. At this time, COA will only accept devices and templates in the CLI format. The DBExtractor is to convert the devices and templates from an RMS format to the CLI format.

Refer to the attached sheet for a pictorial representation of the Integrated Builder System.

Initial Conversions

The purpose of this application is to automatize the current DDT and MDT files into individual devices and templates, and place them into separate RMS databases. Templates which need to be modified will be done by an authorized user creating a MDT file which represents the template. The MDT file will be presented to the application where it will be transformed and placed into the RMS database.

RMS Databases

The RMS file subsystem will be used to implement the device and template database containing a format that will use one record per device or template. The database will have the device index as the primary key and the name as the alternate, so devices and templates may be accessed either using the DI or NAME property. This database will be accessed through the use of an interactive editing application.

Editor Access

The LAL is created through the use of the DBEditor. Through this application, users can add, delete, or modify a device or template. Once the user is finished they may then start a build of the LAL by choosing a menu option.

Device/Template Alterations

Through the use of the DBEditor menus, users will be able to access the commands of add, delete, or modify to any of the devices or templates.

To add a device or template the user could choose it to be similar to another or create a totally new one. In adding, the device or template is created and stored in a temporary database. Once the user is finished and exits the menu option of adding, this information is inserted into the RMS database in alphabetical order. This information is also placed in a temporary updated device database to be used with the DBExtractor.

In modifying a device, the module is extracted from the RMS database, and the user makes alterations. Once finished, and the user exits the modify menu, this information is inserted back into the RMS database. This information is also placed in the updated device database to be used by the DBExtractor.

In modifying a template, the module is extracted from the RMS database, and stored in a temporary database where the user makes alterations. Once the user has finished and exited the menu, this information is stored in a temporary updated template database. These changes will only be processed with an OA build.

In deleting a device the user is queried for only the device name. Once found, the device will be deleted from the RMS database, along with being placed in the temporary updated device database.

In deleting a template the user is queried for only the template name. This information is stored in a temporary updated templates database, to be processed only with an OA build. Those devices that use the deleted template must also be deleted by the user. The device deletions made by the user will be processed with a LAL or OA build.

Temporary Updated Databases

All altered devices, including deletions, for the day, will be stored in a temporary updated device database. The template which the devices use will also be entered in a temporary database to be used by the DBExtractor. Every time a LAL is built, the information from these files will be used to create the DDT and MDT files for the LAL builder. From these databases, the LAL DBExtractor application will be able to extract only certain modules of devices and their templates used, instead of extracting the entire group of modules like the OA does.

All changes made to templates for the day, will not be processed with the LAL. Any changes made to devices using these changed templates, will be using the old template version until the OA every night process is run or forced. All template changes require a complete rebuild (OA build) to become active. This waiting period is for the user to be sure of the changes made to a template.

OA/LAL Start Up

After the user adds, deletes, or modifies a device or template they have the option to create a LAL. This list is a quicker way to view newly added device information, saving the user the time it takes to do an OA build. It contains only the changes that have been made to the devices and or templates. When the database is accessed, the LAL is viewed first and then the OA. Only at night when a build is run for the OA will the database be complete. Creating a LAL saves time and space since it only saves altered devices and or templates. All changes can be viewed after a LAL build is complete. Changes will not be brought into the OA until the every night job is run, or a forced build is made by an account manager.

Users may create a build of the LAL through the accessing of the DBEditor menus. Users will not be able to create a build of an OA. Only authorized users will have access to initiate an OA build. The only other time an OA will be built is during the every night job.

The system everynite facility will cause an OA build to execute on behalf of the EPICUREDB account. The everynite OA build will read in every RMS record, construct an OA, distribute it, and restart the database servers on WARNER, DISNEY, CRYO, front end, and test stand nodes. The everynite build collapses the OA and LAL into one OA, invalidating any previous LAL builds. If the build was successful mail status messages will be sent to account managers.

DB Builder Application

Each of the components which are used to build an OA or LAL are managed by the DBBuilder application. The application will be triggered by an operator making changes to a device or template and sending a message to submit the DBBuilder as a batch job on one of the faster queues in the Warner cluster. The batch job is passed parameters which identify to build an OA or LAL. In turn the parameters are used to pass the correct ones to the COA and DBExtractor applications. The DBBuilder invokes the DBExtractor application passing the name of the device and template RMS databases to act upon. The DBExtractor will convert the RMS databases into DDT and MDT files which are the input to COA. After the DBExtractor completes successfully, DBBuilder will execute COA. After COA, the OA or LAL will be distributed to the valid destinations as identified by the application boot parameter file. Before the OA or LAL is distributed, a message is sent to the DBScanner process to suspend scanning until the distribution has been completed. Further study will take place to minimize this interval.

DB Extractor Application

The DBExtractor is an application which is invoked by the DBBuilder application to convert the devices and templates into the CLI format that COA accepts. The DBBuilder passes parameters which directs the DBExtractor to read the device and template RMS databases for an OA or LAL build and creates the DDT and MDT files that COA uses to create the OA or LAL section files accordingly. For an OA build, the DBExtractor reads the primary RMS databases which contain the devices and device templates without any further action. For a LAL build, however, the DBExtractor reads the RMS databases setup by the DBEditor which provides the devices which have been modified or deleted.

COA Application

The COA application will be an extension from the current COA implementation. Modifications need to be placed into COA in order for it to build a LAL optionally. The current thinking is to have COA accept parameters which identify a different code path to be executed however, using the same routines which write the section file hash, shared, and device areas. An additional CLI keyword will be added to delete devices to be specified in a future release of this document. The new keyword, upon recognition, will invoke a procedure to null the device hash record OA_device_record field. This will signal the EDBServer process to stop looking for the device and return the NODEVICE status code to the application written.

DB Scanner Application

The DBScanner is a process which executes detached on any of the nodes on the Warner cluster. The DBScanner application is used to restart EDBServer processes when they are running an incorrect version of the OA or LAL. If this condition exists, the server is restarted to the latest version of OA and/or LAL. In a future implementation, the DBScanner application will distribute the OA or LAL to the remote destinations in addition to restarting the EDBServer process. The DBScanner retrieves the most current version of the OA or LAL by mapping the hash section file into it's P0 process space and extracting the OA and LAL serial number and build times directly from the hash root. This eliminates the need for a EDBServer to be present on the local node and placing no overhead on the data acquisition system. After the serial numbers and build times are extracted from the hash roots, the section file is closed and unmapped from the process until the next iteration. This schema provides for the most current version of the OA or LAL to be viewed.

Next the application connects to the remote EDBServer processes and requests their OA and LAL serial numbers. If the serial number is less that the one extracted from the hash root, the EDBServer process is restarted. This iteration continues for all valid destinations as identified by a internal list. The internal list is a list of nodes in which EDBServer processes are present. The list is initialized by the DBScanner reading a non-volatile database implemented as an RMS file and contains the requested node list. A utility program will be present which allows one to modify the RMS file without stopping and restarting the scanner.

Future Enhancements

In future enhancements to the LAL implementation, additional functionality could be added to support the following applications:

Report Application
- A report application could be written to support ``canned'' reports to operators upon invocations. The application would be invoked either from a editor or reporter screen. The RMS databases could be accessed by Datatrieve or possibly DCL or C to generate reports. These reports could be generated either in batch or interactively.

DBSA Server Process
- A DB Serialized Access Server (DBSAServer) process similar to the EDBServer, could be written to serialize access to the RMS databases. RMS currently does not support levels of record locking, therefore, multiple users can not make accesses to the database. The server will be implemented as a network object allowing users to connect and make add, modify, or delete requests. The process additionally provides a means for users to make changes and perform other functions (or edits) without waiting for the application to complete.

- In a future release, COA will be modified to read the RMS database directly. This will eliminate the DBExtractor application thus reducing another step in the OA/LAL build process. Additionally, COA will execute inherently faster since it does not need a parser. We may be able to achieve a drastic speedup because of this.

Security, Privacy, Legal