NATIONAL RADIO ASTRONOMY OBSERVATORY
MEMORANDUM
DATE: July 1995
TO: Addressee
FROM: Ron Maddalena
SUBJECT: Data Base Management System for 140-ft Data Archive
In September 1993 I installed new data archiving software on the 140-ft, part of which writes to computer disk a log summarizing the contents of the archive tapes. (You might want to take a look at the October 1, 1993 NRAO Newsletter for further details.) One of the benefits I saw when installing the new archive system was that we could eventually take the log information and feed it into a data base management system (DBMS) so that staff or observers could query the DBMS and find out when and if certain observations were made. But, until now I couldn't come up with a system that was both cheap and useful.
Other telescopes have DBMS systems for querying about past observations. The VLA, for example (see the NRAO Newsletter for January 1 1995), has software that staff and observers apparently use. The VLA software was home grown, works only on PC's, and has limited query capabilities.
Recently I've been able to get the archive logs for the 140-ft into a DBMS
and, along the way, I think I've come up with improvements on the VLA DBMS
design.
Unlike the VLA system, the data base files I've come up with for the 140-ft
archive don't require home-grown software with limited capabilities that works
on a single architecture. With my system one can choose from many powerful
commercial software systems that run on either Mac's, PC's, or Sun's. Thus, the
user of the 140-ft data base more than likely would be using familiar software
and operating systems and the user's learning curve is reduced to almost zero.
Also, most commercial systems have query capabilities that far exceed those I
could have ever dreamed of recreating. The commercial software systems that can
use the data base are cheap -- if you have a PC or Mac you probably already own
one of the supported packages.
There are a few hidden benefits in my implementation as well. For example,
developing the path from the log files to the data base files was trivial (~6
hours of work); and maintaining the path will cost nothing. The only
disadvantage I know of for the new system is if a prospective user doesn't have
any of the possible commercial software systems.
Thus, what we now have is a choice of cheap, powerful, familiar, useful
systems for querying and keeping track of the 140-ft archive.
The following software systems can have immediate use of the 140-ft DBMS
files:
Paradox
Excel
Lotus 1-2-3
dBase
Quattro and Quattro Pro
You're all set if you have one of these programs or your program knows how to
import data files in any of these formats. Also, I've made it so that you should
be able to use any program that can import "ASCII DELIMITED TEXT"
files (a lot of programs can).
I've tried out the files on the few systems I have access to (e.g., Lotus on
the Suns, and Paradox) but I would appreciate some volunteers to try the files
out on any program, even ones not in the above list.
The data base files are large (up to tens of Mbytes with over 100,000 entries
for 1994 alone), so they cannot be placed on PC/Mac floppies. Instead you'll
need to ftp the files to your PC or Mac, an unavoidable step until we get a real
PC network. Or, depending upon your computer, you may be able to mount a network
file system that contains a copy of the required files.
Our current policies dictate that the files for 1994 cannot be released to
outside users since the information in the logs, like the data itself, is still
proprietary. This restriction doesn't apply to staff nor to 1993 data. New files
for outside users will be created at least one year after the data were taken.
The data base files contain all continuum and auto-correllator observations.
They do not contain most Spectral Processor observations since the archive
software for the majority of Spectral Processor observations is independent of
the standard archiving software.
This is a new capability -- I don't know how often it will be used, who will use it, or the systems people will use on the files. Instead of converting files just in case someone may eventually want to use it I have created only a few file types. If you are interested in using the data base files, please tell me what you want to do and we'll discuss how best to accomplish your goals.