GBT Operations & Commissioning Meeting 12 December 2003 AGENDA 1. Az Track and Telescope Status -- Bob 2. Observing News -- Ron 3. Frontends, IFs, and Spectral Baselines -- Roger 4. Spectrometer Status -- Rich 5. Software Status -- Nicole 6. Scheduling Status -- Carl 7. Project Planning Status -- John 8. Any other Business PRELIMINARY REPORTS 1. Az Track and Telescope Status Track: * SGH has made progress on three of the analyses. They expect to share some results from the plain wearplate on a base plate next week. Progress is encouraging. * Two broken bolts were found this week. * The plate material for the recently ordered wear plates did not pass ultrasonic inspection. A thicker plate is available, and Gadsden is evaluating using it instead. * Tim is reducing the measurement data, and we should be able to look at it today (Friday). Structural defects: * Work continued yesterday on repairs. Should finish next week. * Pete Keating will be sending a preliminary report next week that recommends leaving the elevation shaft as it is. This is contingent upon the assumption that these members are in compression. He is trying to confirm that with former RSI engineers. -- RAA 2. Observing News The number of problems with observing this week has not improved from last week's report. Of the 130 or so hours we were scheduled for observing or tests, about 40 or so was lost because of the weather. Of the remaining 90 hours, 10 were lost to almost exclusively software issues. Of these ten hours, three were given up voluntarily so that the programmers could try to get a handle on why things seem to be falling apart over the last few weeks. As far as I know, the source of the recent rash of problem is still unknown but the problems appear to be new and system/network related, as has been reported in various emails. We have again started Q-band commissioning with everything so far looking good. We'll be doing more Q-band tests today, if the weather holds. And, we've essentially finished the re-commissioning of K-band. For the first time the GBT has been used to observe OH from a comet. Although the project was plagued by weather and software problems, I believe the observers are happy with the quality of their data. -- RJM 3. Frontends, IFs, and Spectral Baselines PF1: Pulled off on December 3 for cryo maintenance and repair of the filterbank in one channel. This work was accomplished, and the receiver reinstalled on Wednesday for observing later in the week. However, Gary found a problem in the 450MHz band that will require opening the dewar for repair. There was not time to get this done, so the receiver will be scheduled down again in late January. Other: The reworked feed defroster is installed and running. Currently the blower is outfitted to the Q, K, Ku, and X band receivers. Routine cryo maintenance was done on the C-band and receiver this week. -- RDN 12/12/2003 4. Spectrometer Status *****Status*****: - No major problems this week. It often likes to be set up twice when changing configurations. *****Development*****: - Finished modifications to high speed samplers for baseline test purposes. - Tested all high speed samplers in the system. In some, it is easy to detect 1-channel wide birdies at n x 100 MHz. Others are clean. - Tested drift in offset in spigot system (mean of zero lag vs time). This was motivated by the fact that the 4-bit mode runs off-scale fairly quickly making it necessary to re-calibrate too often. In 16-bit mode, I measured variations of a few tenths of a per cent, at most, with a stable noise source. In 16-bit mode, the zero lag changes by about 50% for a 1 db change in input power. In the real world, input power changes by 1 db or more. Four-bit mode tests are planned ASAP. We may need to re-think our scaling algorithms... - Continued design of the cross-correlation test fixture. A circuit board is partially laid out. - Continued development of FPGA for LTA re-design. *****Plans*****: - More spigot tests - Continue FPGA development - Continue cross-correlation test fixture development -- Rich and Holly 5. Software Status Single Dish Development IPT #62 - Friday, December 12, 2003 Today ends week 4 of 6 in the current development cycle, which is the 9th and last in 2003. The Plan of Record for the current development cycle is now available from the wiki at Software.PlanOfRecordC92003 along with the most up-to-date status information. Also on the wiki this week, the configuration cases have been uploaded. They can be found at Data.ConfigurationCases. Example scripts for most of the 42 cases are provided. A highlight for this week is that dual-beam/dual-focus capabilities were added to the prototype PTCS data display, tested and accepted. A beta release of the IARDS replacement, which is now called RTFM, will be made available early next week so that everyone can begin to become familiar with the application and provide comments on what additional work is necessary before IARDS can be deprecated. New Linux machines have been put together for the port of the converter rack and analog filter rack. In addition to earth, we now have the machines wind, fire, air, and vodka (water had already been taken by a machine in Tucson). Porting is currently being done for the SIB, and discussions are underway to make sure that known issues with MCB and other components critical to the porting task are resolved before being reimplemented. It is unlikely that this will be completed prior to the end of the cycle, however, as the task is much larger than what was originally thought. Enabling the Archivist to log active surface LVDT temperatures and quadrant detector data was completed and sponsor tested this week. The update will be included in the upcoming release. The Ka-band manager has been completed, and will be used in engineering tests at the end of next week. Since design has completed for the Observing API, the past week has been spent coding and experimenting. A Python wrapper was created for the starlink libraries, which are now being used in tests for coordinate transformation. In operational support this week, the LO1 Coordinator was patched this week to be able to handle zero length scans. Also, because the Python applications were being rebuilt on Redhat 9 in preparation for their release, the M&C system also had to be rebuilt in development. This required the resolution of 472 errors and a handful of warnings, which was taken care of mid-week. The change affected anyone running Python modules in the test environment, as well as Cleo's production environment. When using either of these applications, you must now do so from a Redhat 9 machine. -- NMR 6. Scheduling Status Last Week ========= Observations for proposals GBT03C-041, GBT02A-065, GBT02C-020, GBT02A-069, GBT02C-045, GBT03C-014, GBT02C-048 Completed proposals GBT03C-014 Next Week ========= Observations scheduled for GBT03C-009 [B] PI Jeremy Darling A Direct Measurement of Fine Structure "Constant" Evolution from OH and HI Absorption Lines GBT01A-061 PI Wendy Lane A blind search for redshifted HI 21cm Absorption GBT02A-063 [P] PI Mark Claussen Water Maser Monitoring of Low and Intermediate Mass Young Stellar Objects GBT03C-041 [B] PI Scott Ransom RRS Observations of the Double Binary Pulsar J0737-3039 GBT02A-031 [B] PI Jay Lockman Galactic HI Mapping of X-Ray, UV, and Optical Deep Fields GBT03C-031 PI Bryan Jacoby Timing the pulsars in M62, NGC 6544, and NGC 6624 and Search for Ultra-fast pulsars GBT02A-065 [P] PI Lincoln Greenhill Detection of AGN in Apparently "Normal" Galaxies GBT02C-065 [P] PI James Braatz A Search for Cosmologically Interesting H2O Megamasers GBT02A-046 [P] PI James Braatz Monitoring a Maser Disk in Mrk 1419 GBT03A-015 [B] PI Wendy Lane A Search for 21cm Absorption in High Redshift Damped Lyman-Alpha Absorbers GBT02C-045 [P] PI Paul Kondratko Probing the AGN Environment in Moderate Velocity Active Galactic Nuclei (H2O) GBT03B-026 PI Mallory Roberts Timing of a Millisecond Pulsar Discovered in a Survey of Mid-Latitude EGRET Error Boxes GBT03B-016 [P] PI Andrew Walsh A Complete Picture of Cluster Formation in NGC 1333 GBT02C-048 [P] PI Paul Kondratko A Search for Very-High-Redshift Water Maser Emission December ======== Scheduled hours [backup] Astronomy ~ 324 [21] Maintenance ~ 121 Tests & Comm ~ 258 [16] Shutdown ~ 36 Un-assigned ~ 5 January ======= Scheduled hours [backup] Astronomy ~ 374 [311] Maintenance ~ 18 [124] Tests & Comm ~ 213 [22] Un-assigned ~ 140 February ======== Scheduled hours [backup] Astronomy ~ 221 Maintenance ~ [140] Tests & Comm ~ 119 Un-assigned ~ 357 Proposal Checkouts ================== Awaiting checkout - 0 Active checkout - 29 On Hold - 5 Schedule - 45 Completed - 124 -- RCB 7. Project Planning Status December 8th Planning Meeting Minutes Rm 137 10:00 A.M. [0] Observer comments & operator log items We discussed the problems over the weekend, mostly the ones reported by Karen O'Neil. The problems were: (1) GO/GO_LITE failure: Bogus "Invalid OnBeam" error prevented observing. (2) IF Rack sampler values going to zero (occurred twice) (3) Computer Hardware failure on LO1 500 MHz (4) Servo failure on Azimuth Track (5) Considerable interference seen at the low end of Ku-band (6) Numerous LO1 failures due to gbtaio problems. (7) Spectrometer failure - stopped talking to the network. Problem 1's cause is unknown, and was cured by mysterious incantations by Paul M. Problems 2,3, and 6 are in large measure due to the missing PF1 receiver. It should be noted that we also see these failures occasionally when the config file is *not* wrong. They are due to MCB errors, which are a symptom of an overloaded single-board computer. It's just much worse with the system trying to talk to a non-existent receiver. Problem 7 is due to a software problem. The manager is dying and restarting itself, hence the default configuration reappearing. Problem 5 is under investigation by Jeff Acree. It's probably satellite broadcast downlinks. Action items on these problems are as follows: 1) John will identify, classify, and propose solutions to spurious warnings and indications that are being systematically ignored. If warnings had not been ignored, the PF1 config file problem would have been caught sooner. 2) Bob Anderson will write a memo requiring more aggressive callout decisions. 3) John's group will develop a checklist for receiver/feed changes that will detail all the steps, including software configuration needs and testing. 4) Nicole will schedule a meeting to discuss the needs of the new MCB/SIB driver to be implemented under Linux. This should have the ability to test for device health, and avoid tying up the bus talking to unavailable devices. [1] Resource calendar schedule conflicts and discussions No problems with the schedule as it exists. The cold weather and equipment problems may make today's work spill over into tomorrow and Wednesday. [2] Observing Schedule discussions Backup time exists for the end of December, if people need telescope time for testing. [3] GBT development planning Upcoming meetings, etc. (Richard) Richard will write up the results of our October discussions by the end of this week. This remains the framework for planning the first two cycles (first quarter) of 2004. Project Managers will then be asked to provide specific details for C1 2004 by 18th December, and we will have a planning meeting to discuss on 22nd December. This will allow us to hit the ground running on work for the C1 2004 cycle. Project Managers and Divsion Heads also need to prepare their quarterly report material this month. A combined three and six month forward look will be held in late January, to inform decisions for C2 2004 and beyond. Richard will propose more details later this month. The GBT archive project has been put on hold, as no time is available from Chris Clark or his group to work on it. [4] AOB Carl requested that a study be done to see if the PF1 feed changes could be accomplished quicker. Probably not much gain can be made, but Bob and Harry will look into it. -- JF 8. Any other Business