GBT Operations and Commissioning Meeting 27 June 2003 AGENDA 1. Az Track Status -- Bob A. 2. Spectrometer status -- Rich 3. Observing news -- Ron 4. Spectral Baseline, Front-end, and IF work -- Roger 5. Software status -- Nicole 6. Project scheduling -- John 7. Observing schedule -- Carl 8. Any other business REPORTS & DISCUSSION 1. Az Track Status Work continues on the Finite Element Analysis of the existing track and research for a new track design. No milestones reached on either effort this week. Delivery of the last 4 replacement wear plates is scheduled for the end of July. These will be held as spares. -- RA Discussion. Bob added that another sample of the cracked wear plate and a sample of the new material in the replacement plates are at the metallurgy lab for further testing. We have asked the retrofit contractor for a sequence of work plan and for a cost estimate for the remaining 47 plate joints. GBT structural inspections will probably begin in the second half of July and will extend through August. Definite start dates should be known in a few days. 2. Spectrometer status The spectrometer has been operating fairly reliably lately except for occasional spurts of "DMA errors". The errors have become frequent enough that it seems imperative to find and fix the cause. Most of our spectrometer effort this week has concentrated on this problem. Unfortunately, it's a lot like looking for a needle in a haystack. The following things have been done to try to root out the cause of these problems: - added test points to the VME interface board. - studied the recently upgraded logs for clues and patterns (the new format is great - thanks Mark!) - met with p/o software group to discuss errors that were caught - studied the VME specification and GBT documentation on this topic. - studied waveforms on the oscilloscope We now know a few things that it's not, but there is still significant work to be done. We also gleaned unrelated but useful information from the log files. For instance, we saw one instance where only three our of four quadrants were set up. This may be due to a serial error or software. Three of the four quadrants were programmed to dump every 10 seconds while one of the middle quadrants was left at the default of dumping every 24 seconds!?! This no doubt relates to complaints to the effect that "you have to sometimes set it up twice." A few other things were accomplished this week: - The second system monitor board was upgraded to 32 MHz and tested in the system. - Three scripts were written and tested for the pulsar spigot. They enable control of the serial line and reset line. These address problems which stopped Kaplan's efforts last week. In this process, a firmware bug was uncovered. Plans for next week: - More work on DMA error. - System test of hardware reset (hopefully) - More work on cross-correlation test fixture. -- Rich and Holly Discussion. Rich stated that the DMA error is the main Spectrometer error that is occurring now. 3. Observing news Observing over the last week had a few problems. Most notably, the "SafeHoldMode" problems we had and that were fixed on Wednesday, occurred about six times per 12 hour shift. Each fault resulted in about five to ten minutes of lost observing. Spectrometer failures cost project 2B6 about 3 of its 16 hours. The VLBI runs on Friday through Sunday were the hardest hit with 4 hours lost to winds, a few hours to servo problems, and almost 2 hours to setup problems. On the other hand, the observers I've been interacting with have been very happy. The Galactic Center mapping project is producing some very exciting images. The baselines for the OH mapping observations (12.5 MHz, frequency-switching by ~1 MHz, 1-min per spectra) toward strong continuum supernovae remnants look great. We now have some evidence that the recent changes to the antenna trajectory algorithm have paid off. Before the changes, at the start of an OTF observation, the telescope would rarely be on target on time, as revealed by plots of antenna indicated positions. The positions show that the telescope was being jerked around at the start of an observation as it tried to get back on target at the correct times. The OTF maps we have been making over the last few days show a significant improvement with the telescope on target and on time as best as we can tell. The tools I'm using do not record what the antenna is doing between scans and I cannot say if the antenna's trajectory is as smooth between scans. We also had some issues with the way we are using the Active Surface that I will distribute in a separate e-mail. -- RJM Discussion. Bob and John noted that the SafeHold problem appears to be fixed (there have been no recurrences in the past couple of days, at least). Water was found in one junction box and E-stop box, and there was a bad connector on a power supply. Phil noted that Eric Wilcots and Diedre Hunter were asking about the quality of spectral baselines at the HI frequency (the test spectrum they were sent was a little wavy). Both Ron and Karen thought the baselines at 1420 have been pretty good. 4. Spectral Baseline, Front-end, and IF work Baselines of the X-band receiver using on-off observations of 3C218 were performed on Wednesday evening. The baselines show some structure, but most of the fine-grain structure seen previously with this receiver were absent. A complete report is in preparation. More illumination was shed on the "LO2 Phase" effect mentioned last week. Turns out the observed effect is due not to changes in the LO2 signal phase per se, but rather changes in the length of the cable between the synthesizer and the mixer. Experiments indicate that the IF signal input to the CM first mixer leaks out the mixer LO port, through a 10-20 GHz isolator, reflects off the synthesizer, returns, and interferes with the input signal at the level of about -55dB, producing a ripple in the module gain. This amount is enough to account for the few tenths of a percent ripple level seen. Frankly, we still don't understand what in the system is so sensitive to the Equipment Room temperature - the ripple described above would not be a big deal if it were stable. A disturbing fact is that introduction of a change in the LO2 cable length on one Converter Module perturbs the baselines of all EIGHT channels in that rack. The mechanism for this coupling is, I believe, through the power dividers in the optical receivers and the LO2 synthesizer power splitter. RDN 6/26/03 Discussion. Roger showed on/off spectra taken before and after the X-band feed work. The observations were made against a continuum source with a flux density about 1/3 the system temperature. The small scale structure in the spectrum is dramatically improved, although there is still some large scale structure. Barry noted that for a project he is interested in, it is the large scale structure that is important. Rick noted that the objective of the baseline working group is to get the system as stable as possible through hardware mods, then try take out the rest of the baseline structure by calibration and observing techniques. Roger said that Sri has finished modeling a new thermal transition for the Ku band feed, and that the design had been submitted to the Shop. This design follows the suggestion of Tony Kerr. 5. Software status Single Dish Development IPT #41 - Friday, June 27, 2003 We are ending week 3 of a 5-week development cycle, which will culminate in the release of M&Cv3.15 on 7/9/03. *** The deadline for new requests is this Monday, June 30th - if there are any items that have been submitted to the Project Office site, and are particularly critical to you being able to complete your work, please be sure you let the SDD know by Monday. *** There are 13 commitments being worked on this cycle (details can be perused at http://tryllium.gb.nrao.edu/docs/POR/POR_June03.pdf). 9 are development activities and 4 are analysis activities. Out of 7 MRs required this cycle, 6 have been written and 6 of those have been approved. 4 development tasks have been completed (antenna, PF2, and LRF) and an additional 3 are ready for, or in the process of, being sponsor tested (PTCS data analysis script, spectrometer self test, and EMS ODBC connector). Temperature Monitoring adjustments and the PTCS script for processing focus data are still in progress, in addition to all of the analysis activities. This represents reasonable and expected progress at this stage of the development cycle, which is scheduled to end on 7/9/03. A design review will be held Monday 6/30 for the Antenna Test Range Phase 1 work. A significant amount of time this week has been spent on operational support, particularly troubleshooting the cause of the antenna going into safe hold unexpectedly, and identifying the source of aberrant spectrometer warning messages.                                                  -- NMR Discussion. Nicole stated that the Project Charter for the first phase of data analysis development should be posted on the web by 7 July. She and Ron have been meeting frequently on the project. In response to a question from Dave Hogg, Nicole stated that the near-term priority is still to get the data into a format that is accessible to a variety of packages, and then decide how to move forward to further development. Barry asked if any consideration had been given to reviving Unipops. Ron stated that, in terms of existing packages, he thought there might be a bigger customer base for a path into IDL. Phil agreed that for general spectroscopy, the reduction capabilities and methodology of Unipops were very good and that we should try to achieve similar functionality in whatever approach is taken. (There will be other forums and opportunities to comment on data analysis approaches.) 6. Project scheduling Minutes of Operations & Projects Planning Meeting 23 June 2003 Attendees: Anderson, Bignell, Jewell, Maddalena, Radziwill AGENDA (from J. Ford) 0) Observer comments 1) This week's schedule 2) Next week's schedule 3) July Observing Schedule discussions 4) August Observing Schedule discussions 5) GBT development process and program proposal review Please get further comments to me by the end of the week 6) AOB *** DISCUSSION 0. Observer Comments There were no formal comments submitted in the last week. There was considerable trouble setting up a VLBI run on Friday and a large fraction of the time was lost. This was unfortunate because it was a target of opportunity observation of a gamma ray burster. Part of the trouble was an incorrect frequency in the observe file, but there were also other problems. Ron, Frank, and Glen will conduct a postmortem after Frank returns at the end of this week. F. Camilo also lost some observing time because of the E-stop troubles last week. Carl is looking for a slot to reschedule this project. A couple of the Elevation E-stop junction boxes were very wet, and this may be the cause of the problems. Bob and company will be looking at ways to seal these boxes better against rain and water vapor. 1. This week's schedule The Camilo program might be rescheduled on Thursday, the 26th, in part of the slot for J. Braatz. Carl will discuss this with Jim. 2. Next week's schedule Next Tuesday, 1 July, we have a full day of maintenance scheduled for feed arm trilateration measurements. On Wednesday, 2 July, we have a full maintenance day scheduled for laser interferometry measurements of the track base plate at the retrofit splice. Both groups have said that they probably do not need a full day for their work. Next Monday, when Richard and Tim are back, we will see if this work can be combined into one day. This would allow extra time for painting (painters cannot be on the structure if it is moving). The Minter/Turner observing tests scheduled for the early morning of 4 July are contingent on good results from the X-band receiver baseline tests scheduled for 25 June. Note: Carl will be on vacation for two weeks starting next week (30 June). To the extent possible, we would like to stabilize the schedule before his departure this week. 3. July observing schedule discussions Ron would like to get a couple of hours of GBT time on 11 July for the National Youth Science Campers to get a little data. He is discussing with Richard whether some of the PTCS test time can be used for this. It is not certain whether the NYSC-ers can get transportation here that day, so this has to be confirmed. The GBT structural inspection work has been approved and we are negotiating the start date with the contractor (Modjeski & Masters). 21 July is the present best estimate. This should be confirmed in the next week. The inspections are scheduled for 6 weeks duration, which would take them to the end of August. Bob would like to continue with painting into September, but we would also like to reduce the maintenance time in favor of increased observing time as soon as possible. Over the next couple of weeks, we will determine a plan for maintenance and operations in September. 4. August observing schedule No comments other than those above. 5. GBT development process and program proposal review Everyone should be reviewing the documents and have their comments in by the end of the week. 6. AOB A question arose as to whether we could keep some or all of the prime focus feeds in the prime focus teepee so that the time required for feed changes could be minimized. Bob will look into weight and storage space issues. We deferred discussion of BCPM maintenance issues. PRJ 24 June 2003 7. Observing schedule Last Week ======== Observations for: BB169, BT069, GBT03B-035, GBT02C-034, GBT03A-016, GBT02A-015, GBT02B-006, GBT02C-019, GBT03A-002 Completed: GBT02A-015, GBT03B-035, BB169, BT069 June === Remaining: Astronomy ~ 80 hours Maintenance ~ 12 hours Tests & comm ~ 17 hours July === Astronomy ~ 325 hours Maintenance ~ 219 hours Tests & comm ~ 200 hours August ===== Astronomy ~ 334 hours Maintenance ~ 193 hours Tests & comm ~ 165 hours Un-assigned ~ 51 hours -- RCB Discussion. Phil stated that we have set a goal to drop to 3 maintenance days a week when the summer heavy maintenance period is over in September. A further reduction to 2 days a week during the winter months will be aimed for, although we haven't worked out how to achieve this yet. Dave asked what commissioning work remains this fall: it is mostly Q-band commissioning and PTCS work related to Q-band observing. 8. Any other business None. PRJ 27 June 2003