GBT Operations & Commissioning Meeting 17 September 2004 AGENDA 1. Az track, antenna Report -- Bob 2. Observing News -- Ron 3. Scheduling Status -- Carl 4. Frontends, IFs, and Spectral Baselines -- Roger 5. Spectrometer Status -- Rich 6. Software Status -- Nicole 7. Project Planning Status -- John 8. Any other Business REPORTS 1. Az track, antenna Report We replaced plate 19 today due to a crack and high tilt values. We used 4340 material. We received 4 additional 4140 plates this week. We currently have 5 nominal thickness plates, and one 2-1/8" plate on hand. The 3D run at SGH finished this past Wednesday night. ABAQUS also ran the model and sent them results today. They have not processed them fully as of this writing. The consensus (I think) of opinion on the metallurgical results is that the original material likely had a low fatigue resistance compared to AISI 4140 material in general. We also believe that fatigue then reduces the remaining margin. While not an exact chemical match for plate 44, there is nothing that stands way out of specification. Telescope news: All bearings have been converted to oil, and all of the servo motor blowers have been modified - just in time for the next big rain! -- RAA 2. Observing News We supported six observing proposals in eleven observing sessions, a moderately light week, plus two receiver checkouts. About 55 hrs went to maintenance, seven to tests, and the remaining 105 hrs to observing. Average time to setup an experiment was 45 minutes. Lost time amounted to about 5% to 6% of the time scheduled for observing or tests. The major causes were: Failure of the Spectrometer to setup for 4-bank, 8-sampler, 800 MHz: 4 hrs Spigot Hardware/Software : 1.3 hrs Spigot setup : 0.5 hrs Other : 0.4 hrs Problems with dynamic corrections and a failed temperature sensor added one to two hours of extra overhead to some observing projects. I anticipate the support for next week should be about the same as this. In other news, we've put together the observing support scheduled for October. -- RJM 3. Scheduling Status Last Week ========= Observations for proposals GBT04B-021, GBT04A-045, GBT04B-022, GBT02A-044, GBT04C-012, GBT04B-037, GBT04C-018, GBT01A-029 Completed proposals GBT02A-044, GBT04B-037 Next Week ========= Observations scheduled for GBT04B-021 [P] PI Thomas Troland The 13 GHz SO Zeeman effect in an outflow region GBT04B-022 [B] PI Thomas Troland The magnetic field in a compact high velocity cloud GBT04C-012 [B] PI Jennifer Donovan Deep Searches for Young Pulsars in ``Shell'' Supernova Remnants GBT04B-036 PI Paul Demorest Precision Timing of Binary and Millisecond Pulsars GBT04B-026 [B] PI Michael Kramer Timing the First Double Pulsar System GBT01A-004 [P] PI Barry Turner A Search for the Fine Structure of the Hydrogen Atom GBT04C-008 [B] PI Yurii Pidopryhora Mapping the Galactic Halo HI: Evidence of Outflow from the Galactic Plane? GBT01A-029 [P] PI Steve Eales A First Investigation of the Origin of Galaxies with the GBT Scheduled hours [backup] ======================== Category/Month-> September October November December Astronomy ~ 347 [116] 335 [420] 425 [294] 425 [194] Maintenance ~ 210 91 [051] 21 [077] 28 [077] Test & Comm ~ 163 [038] 275 [078] 225 [063] 187 [044] Shutdown ~ 0 0 36 36 Un-assigned ~ 0 44 14 69 Proposal Checkouts ================== Awaiting checkout - 0 Active checkout - 16 On Hold - 5 Schedule - 67 Completed - 222 Current backlog [hours prior to 04B*] = 1091 Total time to discharge [hours] = 2782 * Includes projects that are on hold for trimester 04B -- RCB 4. Frontends, IFs, and Spectral Baselines Operational issues: The refrigerator in S-band was exchanged. Baselines: The 60MHz ripple has been significantly improved in Converter Rack A. Several temperature sensitive components were found, and the module airflow was modified. Rack B will be addressed starting next week. Q-band: Gary and Jonah have confirmed that the mixers exhibit highly variable noise characteristics when they are cooled. We would like to rearrange the receiver topology, moving the mixers to room temperature and adding a room temperature preamplifier. Planning to accomplish this in the time available is underway. Ka Receiver: Software: The linux based manager has been tested and has shown no problems. Hardware: Testing of stability, receiver and cal temperatures is ongoing. Receiver temperatures of 12-27K have been measured in the continuum channels, temperatures depending on the phase switch settings. The video amps are showing non-linearity, which is strange because they seemed fine last spring. That problem is under investigation. Testing of spectral response and Trx over the spectral channels with the millimeter converters begins next week. Independent RF testing of the mmConverter modules is completed. R. Norrod 9/17/2004 5. Spectrometer Status Spectrometer Report 2 September - 9 September 2004 *****Operations***** There were two failures related to the spectrometer in the past week. The first had to do with the spigot raids. Something caused two of the four raids (one on each 3ware card) to go into an initialization state. Since they are under hardware control,it was impossible to take data to disk. Luckily the observer was able to attach an external LaCie disk and write directly to that. Ransom will try to place the raids under more direct software control next week so that it is easier to recover from these types of problems. The second problem had to do with the spectrometer having trouble loading certain combinations of Xilinx personalities into the Memory Cards over last weekend. A work-around was found and used until Tuesday. A microprocessor on one of the Correlator Control Cards lost communication when this happened. It was replaced and the problem has not recurred. The microprocessor failure is the third instance of a serial port failure on this type of device in the past two months. The microprocessors can be "revived" by prolonged erasure and reprogramming. However, past experience has shown that they almost surely fail again at some point. Also the only way to reliably test them is to run them in the system under real observing conditions. The best solution seems to be to throw away devices that fail. However, this is a bit painful at $35 a chip. *****Development***** LTA replacement: Work on LTA PCB design is continuing. 70% finished. Spigot Mode Testing: Successfully tested modes 81 to 9A (all "true" 200 MHz spigot modes with 1024 and 2048 lags.) Cross-Correlation Test Fixture: The filter board is presently under construction. The box for it is in drafting. The output module is 80% designed. A new estimate of completion date for this project was made. The test fixture should be operational in mid-November. Spigot true-200 MHz modes: Tests were conducted Thursday and Friday of last week. Modes 81 to 9A (all 1024 lag and some 2048 lag) passed the engineering tests. Additional testing should validate the rest of the modes during the last week of September (9/28 and 9/29). 1.31 msec interference: Ransom is still analyzing data taken by O'Neil. No conclusive results from there yet. During testing of the 200 MHz spigot modes by Escoffier and Lacasse a mechanism for this type of interference was discovered. Unfortunately, it does not seem to apply to modes that have problems, but does apply to modes that do not!! *****Plans***** - Continue LTA replacement and Cross-Correlation Text Fixture development. - Follow up on 1.31 ms "interference". -- Rich and Holly 6. Software Status Single Dish Development IPT #98 - Friday, September 17, 2004 This is week 5 of 6 in the sixth development cycle (C6) in 2004. The Plan of Record is available from the wiki at Software.PlanOfRecordC62004. Work on the Data Handling project continued this week. Jim, Bob, and Paul spent most of their week working on IDL development for the GBT. Documentation is available on the wiki at Data.IDLMain. A significant amount of coding was completed this week on data I/O and a gbtidl module was created within the GBT python program collection (sparrow) to facilitate distributed development. Memory leak issues still exist. Gregorian receivers have been ported and Melinda has put together a detailed plan for testing during integration week. Engineering tests for the Common MM Converter kick off Friday morning with software team participation. Minor Ka receiver parameters changes are not being completed due to dependencies in Cleo. The key activity for this week was the start of coding changes for frequency conversions for the LO, as per Frank and Brian's specifications. Frank's example scripts were just received this week. Although it will be challenging to complete this work in time for this release, Melinda is prepared for the extra effort to make this happen. Observing API work also continued this week, focused on improving the SameHA and OffOn scans, adding a "wait" statement to support observing runs which require start and stop times, and making the graphical Scheduling Block builder operational. There are some minor yet substantial adjustments that must be made to the graphical scheduling block builder before Nicole is comfortable with the release, and the documentation must still be compiled, so the release will occur at the regular cycle boundary (~Sept 30th). Work on GFM will, however, commence as planned in the upcoming week as work on the observing system tapers. Operational support this week continued to resolve subreflector issues, and significant time was spent on this activity. Patches were put in place on Thursday. Joe explains the situation as follows: The GBT servo controller code (from PCD) has a bug which causes velocity feedforward processing to act differently, based on the time-of-arrival of the command. If a command arrives too early to be processed, the rate commands are set to zero until the time of timetag in the command. In this situation, commands were repeated coming too early, and rates were essentially zeroed out. A change to the antenna code during the last cycle affected the time at which the subreflector receives commands by about 100ms. Normally, AzEl command time of arrival is checked before each release, to verify timing, but not for the subreflector. Tests are now in place to verify both AzEl and Subreflector timings, and have corrected the timing of the subreflector commands. No additional operational support was reported. -- NMR 7. Project Planning Status September 13, 2004 Planning Meeting Rm 137 10:45 A.M. [0] Observer comments & operator log items resulting in lost time: Observer's comments received from: Jeremy Darling Summary: The system worked well for most of his observations, but several well-known problems were encountered. The RFI environment at PF2 was busy, but quiet windows exist. Operator's log comments Significant failures the past week stemmed from: Computing: Network failure (at remote observers' site!), prospero failed, spigot disks, earth insane, titania locked up Software: Dynamic corrections, Spectrometer manager, Accessor, antenna manager abort, active surface Electronics: Spectrometer setup, optical receiver 4 missing, DCR bad channel 1 Telescope/Servo: Other Problems from Observing Reports: T. Minter: GFM, spectral processor balance messages, white space in config file causes errors G. Langston: Bad spectrometer lags J. Lockman: No instructions for config tool usage in documents. Carl will ensure that the new summary page contains a link to these instructions. K. O'neil: GFM doesn't understand PF scans, GO hang [1] Resource calendar schedule conflicts and discussions Spectrometer repairs needed John will grab some time between Roger and Karen's time today. Antenna software checkout time needed Joe will test this on the sky Thursday evening. Failed temperature sensor replacement needed This will be replaced tomorrow morning if Kim agrees to it. [2] Observing Schedule discussions Nearly finished scheduling from October to January. Check the schedules after Wednesday to see the results. Network outage is now the weekend of October 7th. Downtime will be about 5 hours for the first chunk, followed by other unspecified outages. Connectivity may still be possible through Socorro. [3] GBT development planning Time to have our quarterly project review, along with the quarterly report, 6 week cycle meeting, 6 month meeting, and yearly planning meeting. Last cycle, we stated our plans for the next 6 months or so were to shift to doing more maintenance, in addition to finishing several current quarterly report, 6 week cycle meeting, 6 month meeting, and projects. We need to meet to plan the support necessary for the Penn Array receiver, the science center support, and the possible 140 ft restoration project. Richard will propose times and dates for these activities. We will need to nail down required commissioning/test time for February through April before November 15th. [4] AOB Possible visitors on September 30th. They will need a tour of the GBT in early afternoon. -- JMF 8. Any other Business