Green Bank Interim Raw Data Archive

It should be borne in mind that this interim archive is purely a disaster recovery archive. It is not meant to be any kind of queryable archive

Directory structure

The interim archive has a fairly straightforward directory structure and appears under /home. It is divided into 3 parts, pre February 2003 data, post Februrary 2003 data and gbtlogs. Data from prior to Feb 2003 appears under early data and is treated differently from later data. (See below) Post Feb 2003 data is divided into 4 sections, science data, testing data, junk data and problem data. The directory names are self explanatory except for the problem data directory. This contains data directories that do not conform to the naming convention sufficiently well to be placed in one of the science, testing or junk directories. At the current time these directories cannot be safely renamed.

The sub directories under science-data, test-data, junk-data and problem-data are tape images sized for AIT-3 tapes (100Gb) in order of date.

The gbtlogs directory contains directories for each year which in turn contain a directory for each month. Under the monthly directories will be found the familiar directory structure that is found under /home/gbtlogs. Not all log directories appear under all months, this means that no log files were written to that directory in the pertinent month. All logfiles in the archive have been gzipped.

Currently the entire archive is available online. As it grows older, data may have to be removed depending on the availability of disc space.

Early Data

These data directories are mostly named in an ad-hoc manner and no effort has been made to seperate them into science, test or junk categories. The problem directory contains duplicate directories that are not a sensbile subset of the associated larger version in the tape-1 or tape-2 directory. Tape-1 contains data from December 5th 2000 through August 16th 2002, tape-2 contains the rest of the data up to January 31st 2003.

Known Issues

  1. Dates: The dates of data directories are not always truly indicative of their creation data. Due to the manner in which some tapes were written and subsequently restored the true creation date has been lost. Where this was known to have happened the creation date was modified to reflect the date/time indicated by the names of the fits files within that directory.
  2. Misplaced files: It is known that occasionally data files are written to the wrong directory by the M&C system. No effort has been made to "rescue" these files.
  3. The files in early-data/tape1/ have a mixture of ownership. Subsequent directories had ownership set to monctrl:monctrl.
  4. Softlinks in xxxxx/Spectrometer/links are pointing to the wrong place.

Tape Listings

The table below provides links to a full listing of all files on a tape and a listing of the directory sizes for each of the tapes in the archive. The full listing link is to a gzipped text file, the expanded file is too large to keep here.

Early data
Full ListDirectory ListNotes
tape 1 tape 1 All data from 12-05-00 to 08-16-02
tape 2 tape 2 All data from 08-16-02 to 01-31-03
problem data problem data problematic directories and files 12-05-00 to 01-31-03
Feb 2003 Onwards
science tape 1 sci tape 1 02/06/03 to 12/14/03
science tape 2 sci tape 2 06/15/03 to 12/14/03
science tape 3 sci tape 3 12/14/03 to 01/23/04
science tape 4 sci tape 4 01/23/04 to 03/03/04
science tape 5 sci tape 5 02/08/04 to 04/18/04
science tape 6 sci tape 6 04/19/04 to 06/01/04
science tape 7 sci tape 7 06/01/04 to 08/21/04
science tape 8 sci tape 8 07/15/04 to 11/01/04
science tape 9 sci tape 9 11/01/04 to 02/07/05
science tape 10 sci tape 10 02/10/05 to 04/07/05
science tape 11 sci tape 11 04/08/05 onwards
test tape 1 test tape 1 02/02/03 to 03/27/04
test tape 2 test tape 2 03/27/04 onwards Not full yet
problem tape 1 problem tape 1 02/03/03 onwards Not full yet
junk tape 1 junk tape 1 02/09/03 onwards Not full yet

Last Updates

Category Last update
Science 07/12/05
Test 07/12/05
Junk 07/12/05
Problem 07/12/05

Modified on Thursday, 14-Jul-2005 11:31:33 EDT