GBT Operations & Commissioning Meeting 25 June 2004 AGENDA 1. Az Track and Telescope Status -- Bob 2. Observing News -- Ron 3. Scheduling Status -- Carl 4. Frontends, IFs, and Spectral Baselines -- Roger 5. Spectrometer Status -- Rich 6. Software Status -- Nicole 7. Project Planning Status -- John 8. Any other Business PRELIMINARY REPORTS 1. Az Track and Telescope Status Track work: We shimmed 4 joints this week. Two plates (previously unidentified) have some pretty bad cracks on the bottom side. Since we have only one spare plate, we are monitoring them for now. Material for new plates is due in Gadsden around July 4, and we probably won't get them until the end of July. One is a thin plate, which doesn't help us with the two in question. 4 additional plates were ordered this week, with delivery around Labor Day. Art continues with wear plate static analyses, and is getting interesting results. Elevation level walkway: Art also continues with a model of this walkway to determine the forces that led to the buckling of the brace. We also performed strain gauge tests this week, and the loads induced by tipping the structure were put into Art's model. The model then showed buckling in the manner experienced. A new gusset has been designed and is in fabrication. It will be installed on the beam, and then the cracked one will be repaired. -- RAA 2. Observing News We supported five different observing proposals in ten observing sessions, a much lighter load than typical. Of the 110 hours scheduled, 12 went to PTCS and API tests, the remainder to observing. Lost time was under 4% (4.3 hrs), about half of which were reported by Karen and involved Spigot and BCPM observing over last weekend (2.3 hrs). About 1.5 hrs were loss to three problems were the system was in various states of confusion and which required callouts to fix. I anticipate that next week will be about the same since we have only four scheduled proposals and a good number of hours scheduled for various system tests. -- RJM 3. Scheduling Status Last Week ========= Observations for proposals GBT02A-041, GBT04B-035, GBT02A-046, GBT02A-069, GBT02B-005, GBT04B-007 Completed proposals GBT02B-005 Next Week ========= Observations scheduled for GBT02A-041 PI Robert Rood Stalking the Cosmic 3-Helium Abundance GBT03C-031 PI Bryan Jacoby Timing the pulsars in M62, NGC 6544, and NGC 6624 and Search for Ultra-fast pulsars GBT04B-029 PI Ingrid Stairs Timing New Binary and Millisecond Pulsars from the Parkes Multibeam Survey GBT02A-017 PI Dana Balser The Galactic Evolution of 4He: The dy/dz Relation BU027 PI James Ulvestad Monitoring Young Supernovae in Arp 299 Scheduled hours [backup] ======================== Category/Month-> June July August September Astronomy ~ 332 332 [032] 301 [014] 330 [152] Maintenance ~ 220 208 214 210 Test & Comm ~ 167 187 [008] 145 [023] 162 [053] Un-assigned ~ 0 17 84 18 Proposal Checkouts ================== Awaiting checkout - 0 Active checkout - 5 On Hold - 6 Schedule - 65 Completed - 188 Current backlog [hours prior to 04B*] = 1649 Total time to discharge [hours] = 2608 * Includes projects that are on hold for trimester 04B -- RCB 4. Frontends, IFs, and Spectral Baselines There were no significant operational issues this week. Baselines: We found that the 4-way power dividers in the Optical Receiver module use lumped resonant circuits to help provide broadband performance. This may be the source of the bothersome temperature sensitivity. A possible replacement was identified, and a demo unit is being shipped here for evaluation. Q-band: Gary took two LO multiplier units to CV to test using a 40GHz spectrum analyzer. More steps were taken to eliminate contamination from the waveguide assemblies, which continues to be a problem. The VNA is also being used to test the polarizer and other receiver components. Ka-band/mmConverter: Wiring of the five mmConverter modules continues. All semi-rigid cabling is completed. The power supply/LO chassis is ready to be assembled and wired when a technician is available to do the work. R. Norrod 6/24/2004 Comments: Roger reported that replacing the power dividers would cost 9 x $1400. The L.O. power requirements for the Q-band multiplier will require more entries in the receiver's L.O. power table and the nominal power levels will not be the same 5V we've been accustomed to seeing. 5. Spectrometer Status *****Operations***** EDT has evaluated the failed EDT card from the Spectral Processor. Due to extensive heat damage on the board, they recommended, and we agreed, to replace the board. Heat damage was noted over a year ago and a fan was installed to minimize further damage. It is not clear whether the recent failure is due to heat damage. The spigot2 computer had problems writing to its /data3 drives. The problem was fixed in the short term by switching to another disk. Further tests of /data3 are planned. *****Development***** The uploading of scaling of offset coefficients to the spigot cards from the spigot computer has not been 100% reliable. In an effort to improve this, firmware and software were written to perform a CRC on the coefficients. The CRC of the data received by the spigot can be compared to that sent by the spigot computer. The first round of testing of this software occurred on 6/24. More work is required. A small amount of effort went into the testing of spigot timing. A conversation with Escoffier yielded some ideas of other things to try as well as a commitment to check some of the documentation. A simple circuit was designed and built. It will delay the burst of data used for testing sufficiently to allow us to see the beginning of the data. This week have been trying to find out the cause of big step in ACF. Three projects had reported big step problem in ACF, they all used low speed 9-level mode. So far, not sure that bit step found in 9-level is because only 9-level has problem , or only 9-level have been used recently. Need to do some test in the system to verify. If only 9-level has problem then: >From the location of the step, trace to the cards and chips on correlator card, but it doesn't look like chip's problem, because in other modes these cards and chips also had been used but there was not problems reported. LTA processes 3-level and 9-level in the same way, if 3-level has no problem, 9-level shouldn't has problem as well. There are some place could cause the big steps, such timing and data processing ( 9-level processes data is different with 3- level) and so on: Looking into the log files of these project, find that computer acquiring data took from 190 msecs to 741 msecs, processing data took from 3 msecs to 29 msecs, and writing data took from 92 msecs to 14234 msecs. So if integration time is 10 seconds, and the worst case happened, then before the quadrant complete dumping to computer, new integration starts. The result is that the computer won't get all the data of that quadrant (negative big step?) and the memory of that DRAM won't be cleared completely, so the integration after current integration will have same amount of positive big step ( the lags didn't cleared accumulated with current lags). this explanation seems match the step pattern found. But need some system time to test, prove it and to answer some other questions. Basicly, still need a lot of work to find out the cause of the step. *****Plans***** Spigot: - Continue work on spigot card firmware modification. - Continue work on absolute timing of spigot data vs mode. Lag drop-outs: - Continue work on finding the cause of 1024-lag-long pulses in acfs. Rich and Holly 6. Software Status Single Dish Development IPT #86 - Friday, June 25, 2004 Today ends Week 6 of 6 in the fourth development cycle (C4) in 2004. The Plan of Record which shows what you can expect in these releases is available from the wiki at Software.PlanOfRecordC42004. Documentation showing results from our integration and regression testing, and other previous cycles release information, can be found at Software.SoftwareReportCentral on the wiki. Mark and Melinda were both at the SPIE conference this week. They both made presentations on Tuesday, and reports from Richard and Melinda indicate that their talks were well received. The releases of M&Cv4.4 and GFM4.4 are scheduled for Wednesday, June 30th. Two sessions of integration testing were conducted this week in preparation for regression testing on Monday night (the 28th), and several errors were found and fixed (as they should be). Observing API tests continued for a few hours each day this past week. An Execution Block GUI is being used to interactively run the tests. The data is successfully being written, and continuum results (including dynamic corrections) are viewable with GFM. Documentation is now being written to show staff astronomers how to participate in tests after the beta release next week. Operational support over the past week has included: + Because the DCR was not rebooted properly after a test session, errors were present during the setup for an observation immediately following maintenance. Reset box clicks were made too fast in the post maintenance software procedure, so the instructions have been updated. + Last night, there were questions about the active surface but no software changes were made. --NMR 7. Project Planning Status June 21, 2004 Planning Meeting Minutes Rm 137 10:45 A.M. 0) Observer comments & operator log items resulting in lost time: Operator's log comments Significant failures this week stemmed from: Computing: Unneccessary ackground jobs running on spigot2 John will talk to Chris about making sure these don't keep coming back. Possible RAID array failure on spigot2 Chris says that the RAID appears OK BCPM2 comatose The system was brought back to life, and was OK. the M&C manager failed a couple of times. Amy will investigate the procedures used to reanimate it. Software: Spectral Processor, Archivist, Antenna, GO, Grail, GFM There was much discussion about the spectral processor problems of last week and the fact that the M&C code seemed to change without notice. It was pointed out that the SP is not tested during the regression tests. Nicole will make sure this is remedied. Electronics: S and X band receivers warmed up due to a compressor fault Telescope/Servo: None Other Problems from Observing Reports: K. O'neil: (1) A memory intensive background program was being run on spigot2 by Chris Clark which interfered with observing [Saturday morning] (2) BCPM manager failure [Saturday evening] (3) GO failed to set the phase table in the scan coordinator [Sunday morning] (4) GFM failed to catch the "bad phase table" error and instead gave a less useful "can't divide by zero" error [Sunday morning] (5) BCPM "Set power levels" did not set power levels correctly [Sunday morning] (6) BCPM2 computer lost communication with everything [Sunday morning] (7) No monitor is readily available in the control room to connect to the BCPM2 computer and diagnose problems [Sunday morning] BCPM2 is supposed to be connected to a serial port like Gemini for this purpose. No monitor is needed. (8) BCPM manager became confused after BCPM2 computer lost and regained communication [Sunday morning] (9) Spigot failed in data taking, most likely due to a failed raid disk [Sunday afternoon] (10) Computer hardware failure on antenna 1) Resource calendar schedule conflicts and discussions OK, except for Harry's got both B and D stuff scheduled for the same day. 2) Observing Schedule discussions 3) GBT development planning Meetings Schedule Cycle 5 planning meeting, June 22nd, 10-12 4) AOB --JMF Comments: From the planning meeting on Wednsday, the major conclusions were that projects like Ease of Use, PTCS will receive full support; we'll be going light on fixing bugs; we'll be starting the I.F. improvement project. We still need to find someone with the time to be the project scientist for the baseline improvement project. 8. Any other Business