GBT Commissioning and Operations Meeting 19 September 2003 AGENDA 1. Az Track Status -- Bob A. 2. PTCS Report -- Richard 3. Observing news -- Ron 4. Spectrometer status -- Rich 5. Spectral Baseline, Front-end, and IF work -- Roger 6. Software status -- Nicole 7. Schedule -- Carl 8. Project scheduling -- John 9. Any other business REPORTS 1. Az Track & Elevation Status Bob & Dennis reported that several track segments were re-shimmed over the past week. All but two originally had zinc shims, which do not appear to hold up as well as the bronze-teflon. Two shims were replaced that originally used bronze-teflon: one was the oldest one that had been in for about a year, and the other was the newest one that was probably undershimmed when originally installed. Dennis is working on a "material selection matrix" that is relevant to possible new track designs as well as replacement wear plates for the retrofit option. He is considering strength, fatigue resistance, cost, and availabilty in the size needed. Discussions are underway with Joe Antebi concerning measurements of the track to compare with his finite element model. The weld procedure has been developed and the pre-qualification weld of sample material have been completed for the elevation axle repairs. We have received the final report from the ultrasonic inspection expert at Wiss, Janney, Elstner, and are awaiting the report from Peter Keating and Modjeski & Masters. -- PRJ 2. PTCS Report The observations performed last Thursday/Friday were successful, although there were some intermittent problems with the temperature sensors (which we now believe are cured). These data have been combined with the initial "learning" data-set described last week, and result in an improved prediction; analysis work on this continues. Joe, Kim, Dana and myself met to discuss a revised architecture for the Antenna Manager which will allow us to incorporate the dynamic and static elevation-dependent corrections which we are now measuring. There was good agreement on the route forward, and this will be circulated to a wider audience once it is written up. The experiments planned for this Thursday/Friday have been postponed to avoid the effects of hurricane Isabel. RMP 9/18/03 Discussion: Ron asked if the new focus-tracking model would be installed by the time visitor observing with the K-band system began this fall. Richard replied that the new model will probably be ready by the 15 November project release, but that the improved coefficients that were determined in benign thermal conditions could be installed sooner. 3. Observing news We were hit by a substantial number of problems since last Friday with only a handful happening during the setup time and most coming out of observing time. If I don't count the lost PTCS time last Friday, the operators logs report that observers loss 9 hours of time. The most significant problems were: o The worse week ever for the Spectral Processor which failed twelve times. About half of the time the operator was able to recover in 10 minutes but sometimes it would take 20 minutes. o The Converter Rack had a few problems, as reported in the nightly logs. The hardest hit was Ingrid who lost over two hours due to a combination of Converter and Spectral Processor problems. o GO caused some loss time. In one case, it was switching on its own from pulsar mode to line mode. Restarting GO solved that problem. o The new software for checking on the location of Active Surface FITS files caused a loss of setup time. A patch is now in place. o Robishaw lost 3 hours one night when the Measurements manager was not responsive and when some IF Rack cables had not been reconnected after maintenance. A pulsar observer lost the full hour of setup time when it was discovered that cables to the BCPM were missing. -- RJM Discussion: The faults with the Spectral Processor seem to be all of the same generic type -- missing data. Rich installed the new cables on Wednesday during the day. Data integrity on the cables looked quite good. Statistics on the faults since Wednesday were not known at meeting time. We discussed ways we could trap errors such as missing cables or hung software as part of an end-of-day check. Karen and Ron offered to write a proposal of an automated method to check Tsys for each receiver and to check that the various backends and managers were alive and well. 4. Spectrometer status Spectrometer status 9/18/03 Again there were no complaints about the spectrometer because it was not used this week. There was a fair amount of success testing spigot modes this week. Spigot Modes 1 to 6 and 41 to 46 were successfully tested with an artificial pulsar by Kaplan and Lacasse. These modes provide the following capabilities: Mode Time Resolution BW # IFs # Bits (usec) (MHz) 1 81.92 800 1 16 2 81.92 800 2 summed 16 3 40.96 800 1 8 4 40.96 800 2 summed 8 5 20.48 800 1 4 6 20.48 800 2 summed 4 41 81.92 50 1 16 42 81.92 50 2 summed 16 43 40.96 50 1 8 44 40.96 50 2 summed 8 45 20.48 50 1 4 46 20.48 50 2 summed 4 The use of double-Nyquist sampling to halve the analyzed bandwidth thereby doubling the frequency resolution on the half of the band that is analyzed was successfully tested in spectroscopy mode. This allows for the possibility of kluged 200 MHz and 12.5 MHz modes that were requested by a few pulsar scientists. A few modes that have been requested have not been tested. These will be tried next Monday and Wednesday. A few operational problems showed up during the testing. For example, after sometimes scans did not start when requested. These will be looked into next Wednesday as well. Work has continued on preparing for the redesign of the LTA, Holly has been studying the old LTA Xilinx design and new xilinx technology that can be used in new design. Started to design some xilinx modules that will be used in new LTA design. Two LTA boards that failed to pass self test in the system still do not work well in the system. We have booked some system time, and hopefully, can fix them at the end of this month or early next month. Modifications to the high-speed samplers have hit a snag. The modified samplers are introducing small features into the measured spectra. This will be looked into over the next few days. Software for new reset feature was successfully tested. This should make it unnecessary to power down the hardware to make it sane again. The reset is invoked during initialization and _Reset Parameters._ -- Rich & Holly 5. Spectral Baseline, Front-end, and IF work Testing of the upgraded K-band receiver has gone smoothly this week. Gain balancing to better than 0.25dB at any one frequency is possible using the LNA gain controls. Searches for refrigerator modulation and other gain instabilities were done as well. Thermal cals will continue thru Friday, and barring unforseen problems the receiver will be installed on Monday morning September 22. A thorough inspection and cleaning of the interior of the Q-band receiver is underway, and problems or suspected problems reported last spring are being investigated. It is clear that we have a problem with the reliability of solder joints in stainless coaxial cables used within our receivers. The problem may date to a change in solder and flux products when the vendor used previously went out of business. Search for a solution has started. -- RDN 9/18/03 6. Software status Single Dish Development IPT #51 - Friday, September 19, 2003 This week ends Week 5 of the 6 week development cycle which is the 7th cycle in 2003. Integration testing will be performed next week in preparation for regression tests on the weekend of September 27-28. Pending successful tests, the new version will be made available the evening of September 29. The Plan of Record for the current development cycle is available from the Project Office web site at http://tryllium.gb.nrao.edu/docs/POR/POR_Sept03.pdf. EMS work this week has focused on resolving memory leak issues in preparation for PTCS experiments that were originally to be held on 9/19, and we are pleased to report that as of 9/18 the improvements had been tested to Kim's satisfaction for both the ZIY and EMS components of the solution. PTCS related work will now shift to scoping out quadrant detector upgrades. The root of the performance issues with the PTCS python scripts for pointing and focus data analysis has been isolated as the absence of a bulk retrieve capability in the FQL implementation. This is a reasonable upgrade which may be scheduled during the upcoming cycle. Additionally, an algorithm review was held on Thursday to examine these scripts. Configuration work continues on schedule; Frank and Melinda have conducted many tests with support from Ray and also Paul which will continue next week. Keep in mind this is only the first of three development cycles dedicated to the configuration task; during the next cycle our focus will shift to actively iterating on this functionality as driven by observer support. Development to implement the hard reset functionality in the spectrometer was completed and sponsor tested, and is ready for inclusion in the upcoming release. A new gbtlogview will be ready for sponsor testing next week, also to be available for use at the end of the current cycle. The last of the enhancements to help electronics troubleshoot the DMA issues with the spectrometer was completed this week, and was scheduled to be sponsor tested on Thursday 9/18. The first rounds of critique on an intermediary data format were conducted this week, and tasks identified for the next development cycle to make the format an accurate and sufficient representation of GBT data. This cycle, instructions will be written for how IDL users can now more easily access their data, which will be updated throughout next cycle as the data format is refined. This week, discussions began on K band calibration, in preparation for the upcoming K band commissioning. Next week, discussions will be held regarding data quality, and needs that observers as well as software programs have for real time data quality assessment. Operational support this week involved updating receiver calibration values for PF1 342 and 450 MHz. A utility was also written during this time to make it much easier to load new calibrations. Also, the active surface is no longer one scan behind the other devices in the Scan Coordinator, since this fix was patched in on Monday 9/15. Scans will no longer abort due to this problem. -NMR 7. Schedule Last Week ========= Observations for: GBT02C-060, GBT02C-010, GBT03A-023, GBT02C-009, GBT03A-016, GBT03B-030 September ========= Remaining hours [backup]: Astronomy ~ 102 [30] Maintenance ~ 64 [24] Tests & Comm ~ 140 [6] October ======= Scheduled hours [backup] Astronomy ~ 226 [146] Maintenance ~ 123 Tests & Comm ~ 356 [40] Un assigned ~ 41 November ======== Scheduled hours [backup] Astronomy ~ 42 Maintenance ~ 108 Tests & Comm ~ 6 Shutdown ~ 36 Un assigned ~ 528 -- RCB 8. Project scheduling September 15th Planning Meeting Minutes Rm 241 10:00 A.M. <0> Observer comments None available on the Web Minter's email comments: Summary: The AGBT02A_002_03 setup Tuesday evening did not go very smoothly. This was mostly the result of the observers wishing to observe outside of the nominal X-band receiver frequency range at 11.119 GHz. The setup for the observations and confirmation that everything would work took 2 1/2 hours before real science observations started. Summary: The AGBT02A_002_03 setup was plagued by the power outage Wednesday evening. Consensus of the group is that these problems are no worse than we have been seeing, and it was a concious decision on the part of the planning committee to forge ahead with deveopment and take our lumps with reliability and ease of use. We have taken on the ease of use problem, and since much of the reliability problems seem to be in the user interface, then some of the problems will be solved when new interfaces are introduced. John remarked that we need to separate out our site LAN and telescope LAN dependencies. <1> This week's schedule Moving servo checkout Monday afternoon <2> Next week's schedule No comments on next week's schedule. <3> September Observing Schedule discussions Tentatively plan overnight shutdown for elevation axle repairs w/o Sept 22 Might slip until the next week. Richard has requested that we take the weather and observing into account. We decided that we would go ahead and slip this to the 29th. this is due to uncertainty in the delivery of the final report on repairs, and decision on what avenue of repairs to take. Once we have determined a week, we will rent the equipment to allow a limited juggling of welding days, in order not to waste any good weather we might have. <4> October Observing Schedule discussions Carl is having trouble finding LF backup programs for the last half of October. <5> GBT development planning HF workshop discussions We did not get to this topic. We will take it up next Monday Configuration testing vs Spigot/Spectrometer testing Nicole agreed to ask Melinda if she could spare some time for Rich to test the Spectrometer's spigot modes. <6> AOB K band receiver discussions We decided that it was most prudent to allow another week to fix the remaining problems with the K band receiver, rather than put it up as it is. -- JF Discussion: It was emphasized that the deadline for requests for the next planning cycle are due in on Monday, 22 September. 9. Any other business Dana brought up the issue that Carl Heiles had raised a couple of months ago about the way Tcal tables are applied to the data. Carl prefers that a smoothed, fitted version be used for channel by channel scaling, as oppposed to an interpolated value. Ron said this was, in fact, the plan. We need to make sure this project is on the list of projects to be considered and addressed. PRJ 19 Sep 2003