GBT Operations and Commissioning Meeting 30 April 2004 AGENDA 1. Az Track and Telescope Status -- Bob 2. Observing News -- Ron 3. Scheduling Status -- Carl 4. Frontends, IFs, and Spectral Baselines -- Roger 5. Spectrometer Status -- Rich 6. Software Status -- Nicole 7. Project Planning Status -- John 8. Any other Business PRELIMINARY REPORTS 1. Az Track and Telescope Status Az track * Discussions continue with SGH on next analysis steps. * A replacement plate for the modified wear plate has been received and will be installed when time permits (short maintenance days next week). Elevation shaft * Preparation activities are under way for the repair of the elevation shaft. -- RAA 2. Observing News After the elevation drives were restored last Friday, we serviced 7 proposals, one of which was a VLBI run, in 15 observing sessions, plus a couple of PTCS tests. About 160 hours were scheduled for tests and observing. After the problem with the telescope drives were fixed, we had 20 hours for tests and 125 hours for observing. Other than the drives, the remaining lost time was rather reasonable: DCR (start tick) : 3 hours Weather (Winds) : 2.25 hrs Hardware (Antenna Transducer; I.F. Power) : 2 hrs Other Software and Network : 2 hrs This is a lost time of 16% or 6%, depending on whether or not you count the lost time for the elevation motor. Because of the large number of Spigot observations, almost all the setups lasted the full hour. PF2 program checkouts and Ka-band commissioning are on the agenda for next week. -- RJM 3. Scheduling Status Last Week ========= Observations for proposals GBT02A-063, GBT04A-022, GBT01A-064, GBT04A-045, GBT02C-054, GBT02C-043, GBT02A-046, BK110 , GBT04A-008 Next Week ========= Observations scheduled for GBT04A-041 PI Jean-Luc Margot The interiors of Mercury and Venus from their spin dynamics GBT04A-022 [P] PI Toby Moore Variations in star-formation efficiency in the W3 GMC GBT01A-064 [B] PI James Cordes Search for Pulsars from Point Sources in the Galactic Center GBT03B-012 [P] PI Crystal Brogan Search for "Kilojansky" Water Masers in Four Starburst Galaxies GBT02A-046 PI James Braatz Monitoring a Maser Disk in Mrk 1419 April ===== Scheduled hours [backup] Astronomy ~ 422 Maintenance ~ 79 Tests & Comm ~ 215 Un-assigned ~ 5 May === Scheduled hours [backup] Astronomy ~ 400 [117] Maintenance ~ 88 [64] Tests & Comm ~ 220 [20] Shutdown ~ 26 Un-assigned ~ 11 June ==== Scheduled hours [backup] Astronomy ~ 308 [11] Maintenance ~ 220 Tests & Comm ~ 121 [15] Proposal Checkouts ================== Awaiting checkout - 0 Active checkout - 19 On Hold - 7 Schedule - 63 Completed - 168 Current backlog [hours prior to 04A*] = 1681 Total time to discharge [hours] = 3011 * Includes projects that are on hold for trimester 04A -- RCB 4. Frontends, IFs, and Spectral Baselines Ka-band and mmConverter: Receiver testing over the weekend went well; see separate report by Brian Mason. The receiver and mmConverters will be installed on the GBT next week. Other this week: PF1 was removed on Tuesday; planned installation of PF2 was weathered out, but should occur on Friday. -- RDN 4/29/2004 5. Spectrometer Status *****Operations***** The only rough sailing this week has been occasional crashes by the spigot computer. The spigot computer has been freezing recently during attempts to copy (rsync) the data from the spigot raid disks to external disks. Wolfgang has installed a patched version of the kernel, which seems (at least right now) to have fixed the problem.Wolfgang has been trying new kernels to try to improve this. *****Development***** Progress continues on the Cross-Correlation Test Fixture. Boards for the voltage regulators have been completely tested. The 800 to 1600 MHz filter has been worked on considerably; a manufacturable design has been found. More tweaking will be done to reduce cost and size. Kaplan and Lacasse conducted additional tests of the spigot 8-bit and 4-bit summed-polarizaion modes. The modes are still not working correctly and it is not clear where the problem lies. A telesconference was held on Thursday to discuss the problems and bring Escoffier up to speed on them. Additional tests are planned. Escoffier has been working on the 200 MHz spigot modes. He is still working on a problem with address generation in the memory boards. Chen continued working on LTA schematic design. The new spigot computer is now ready for testing. *****Plans***** Cross-Correlation Test Fixture: - tweak the 800 MHz filter design - design the 200 MHz filter - layout the filter multiplexing board. Spigot: - Additional work is planned on the spigot summed polarization modes and on testing new modes. - Continue work on true 200 MHz modes. - Test new spigot computer. LTA replacement: - Continue schematic design -- Rich, Holly, and Karen 6. Software Status Single Dish Development IPT #78 - Friday, April 30, 2004 Today ends week 4 of 6 in development cycle C3 in 2004. The Plan of Record is available from the wiki at Software.PlanOfRecordC32004. Release documentation for our latest release and other previous cycles can be found at Software.SoftwareReportCentral on the wiki. Lots of improvements have been made this week to GFM in its test environment, which is accessed using "gfm-test". In test, GFM correctly handles all bad scans, is much faster, reliably fits all good data, handles Q-band data correctly, and reports the correct Tsys for bean switched data at K and Q bands. These updates will be released at the end of the cycle, once integration and regression testing has been completed. GFM is also being updated to handle data from the new Ka-band receiver, and this will be tested by Brian next week during his commissioning runs. Several MRs were approved for PTCS work this week, and development has started (updating the antenna refraction model, updating the antenna pointing & focusing model, inclinometers). MRs for the accelerometer manager and two active surface items from last year's Project Charter are under review. One active surface item (active surface GUI does not display actuators as faulted when the actuators are disconnected in the actuator room) has been approved, coded and is ready for testing. For the Penn Array Receiver, a JNI interface to GPIB is being written. The MR has been completed and approved, however, we have been notified that there are some strategy changes that have been made by NASA, so no development work will be started on this item this cycle. CCB code generation continues and the manager is to be ready for lab testing by the end of this cycle. Manager connections to receive data and send configurations to the ccb demo server have been successful. Changing CCB manager parameters using device explorer have been tested. The SDFITS generator was released Thursday PM, and the CLASS generator is being released today. The release notes will be posted at SoftwareReportCentral, and the documentation is on the wiki at Data.Sdfits and Data.ClassBeta respectively. The CLASS generator is being released for the first time, hence the beta period. Please encourage the CLASS users you know to provide feedback on the wiki. By reading the release notes and documentation, you will have all the information you need to determine whether each converter will work for your specific observing case. PLEASE READ THE DOCUMENTATION before reporting errors Operational support over the past two weeks has included: + GO was not picking up LPCs from GFM, causing GFM to stall. Paul added a one-line fix to GO on 4/28 that resolved this issue completely, and also yielded a perceptible GFM performance boost. + DCR was missing start ticks again on 4/28, so a quick fix was patched in. Additional investigation is planned for Friday 4/30. --NMR 7. Project Planning Status April 26, 2004 Planning Meeting Minutes Rm 137 10:45 A.M. [0] Observer comments & operator log items to discuss Operator's log comments Significant failures this week stemmed from: Computing: Prospero reboots, spigot missing ticks Wolfgang reports that this is caused by a kernel problem due to the 8GB/8processor server. No solution is known. Software: Converter rack, Analog Filter Rack, go_lite, GFM, sd_fits, presto/tempo corrupted Nicole reported that fixes for the converter rack and AF rack are already in place to be released at the end of the cycle. Likewise the GFM problems. The sd_fits problem has been confirmed, and will be worked on. The presto/tempo problem will be investigated. It appears that the binaries are all of an old vintage, and that they have not changed. Electronics: Optical driver/rcvr #2 noisy We will be looking into this on a maintenance day. Telescope/Servo: weather, elevation motor #3/4 Elevation motor #4 melted the solder out of the commutator ring. It was removed to be taken to a motor shop for evaluation, and this triggered a problem with the elevation drive system, causing about 18 hours of downtime. Observing Summaries G. Langston: AGBT03C_030_03: Nissim Kanekar's 855 MHz observations of quasars Summary: Running smoothly. Details: There was strong RFI near 883 MHz and some RFI near the 912 MHz line. Nissim wanted to confirm the 912 MHz observations so we configured for simultaneous ACS and SP observations of the 912 MHz line. Observation setup went smoothly except that go_lite crashed twice. The third try we were able to start running scans with go_lite. K. O'Neil: 04A-08 Summary: The observations had a number of small problems, each of which lost 5-10 minutes. Additionally, approximately 2 hours were lost to high winds. Also, I'd like to add in another plea for the new observing api, as the type of observing i'm doing - moving to a source, balancing the IF, firing the cals, and then doing an off-on, is remarkably painful to do in the current system, even with GO tables. Problems: (1) Antenna hung in activating on the scan coordinator. Scan coordinator had to be restarted. (2) GO_LITE crashed twice. Once after the above problem, once while tryingto end observing from a go table (3) IF rack CR1 appeared to lose power. I switched to CR2 for the reminder of observing (but later re-checked CR1 and it seemed fine) (4) Antenna elevation motor was not engaged after wind shut down (5) the config_tool found an number of problems with my set-up, but it took awhile to determine what was wrong. some suggestions on this are below. (6) config_tool set did not set the beam to "c" as is needed at l-band, but left it at "1" (from the last observer) F. Ghigo: GBT04A_046 Checkout Summary: Az/El ephemeris tables did not work. Other tests were successful using drift scans across GPS satellites. K. O'Neil: AGBT04A_008 Summary: Once prospero was back up and everything restored, the observations went extremely smoothly, bar one crash of GO in the middle of observing. Problems: (1) prospero was down at the start of the run. See operator logs, or talk to Wolkgang for more info (2) gbt GO daemon died as a result of (1) called Paul M. and he got the daemon up and running for us (3) GO_LITE crashed once this happened because the observer (me) ws inpatient and click on two GO buttons at once. F. Ghigo: 2C_043 (Finkbeiner) Problems: 1) "sdfits" did not run on data where the IF Noise source was being used. 2) Several software hangups starting up on Saturday (April 24) about 30 minutes lost after the setup period. 3) The weather prevented any good data being taken in the Friday session, but some useful test data was taken with the noise source. The weather was good on Saturday afternoon, and some good maps were done at X-band, but the weather clouded up when starting the K-band mapping, so that was only partly successful. K. O'Neil: 02C-43 (Finkbeiner), 04A-8 (O'Neil), 04A-45 (Ransom), Ka-band test (Mason) Problems Summary: (1) "sdfits" did not run on data where the IF Noise source was being used (2C43) (2) Config API doesn't have DCR/ACS option (2C43) (3) Optical Drive/Receiver 2 less stable than OD/OR 1 (2C43) (4) Config API "set_manager_on('acs')" option failed (4A45) (5) BCPM manager dead on startup (4A45) Observing Summaries: 2C43: Once I took over from Frank, the observing went well for this project, bar the fact that the weather was poor. 4A8 : No problems encountered 4A45: This project ran remotely both the BCPM (with the operator running the experiment) and the spigot (with the observer controlling the spigot). Bar a few minor problems, the remote observing went extremely well. Ka-band test: I have no info on this, except I didn't get called K. O'Neil 2C43 (Finkbeiner), 1A64 (Cordes) Problems Summary: (1) Problems with the automatic spigot calibration in 16-bit mode (1A64) (2) PRESTO/TEMPO programs in ~pulsar area corrupted (1A64) Observing Summary: 2C43: Due to evening clouds, observed at C-band and X-band for evening. no problems encountered 1A64: Slow start-up due to both the complication of running the spigot at higher frequencies, and this being the observer's first time using the spigot. Spigot computer data acquisition unreliable since Saturday run. [1] Resource calendar schedule conflicts and discussions Day #1 Conflict with IF Manager testing and spectrometer testing The Spectrometer spigot testing will take precedence over IF Manager testing. Mark Clark can co-exist for some of the tests, and so he will attempt to do the tests that he can do. Day #2 OK. The Ka band receiver will be installed next week for observations the week after. [2] Observing Schedule discussions Test time requests have been received from most projects. The awaiting checkout queue has several Q and K band proposals in it that are awaiting technical improvements. It is unlikely that we will be able to immprove things anytime soon. Some hope is being held out via different data processing schemes. There was discussion about what to do with these proposals. [3] GBT development planning We discussed adding a milestone to instrument development projects that would send them through the equipment room for system integration tests before scheduling them on the telescope. It was noted that this increases the lag time for scheduling the new system, but increases the predictability of the schedule. Spectral baseline project status and plans IF system The IF system work will end with the testing and implementation of the LO buffer amplifiers at the converter modules, and with characterizing the nonlinearities that are being investigated. Q band receiver The Q band test dewar is yielding some interesting results that we wish to fully understand before embarking on the next phase of the baseline project. The Q band receiver will not be modified this summer, so these test results will not carry over to improved Q band performance next winter, but should yield results by the following winter. Other receivers Other receivers have been characterized, and no further baseline improvement work is planned. [4] AOB The GBT will be moving to a 4 10 hour day maintenance schedule as of May 17th. Support personnel will likewise change their schedules to allow maximum use of these days. Phil will issue a letter to that effect next week. -- JMF 8. Any other Business