GBT Commissioning Meeting 4 April 2003 AGENDA 1. Az Track Status -- Bob A. 2. Spectral Baseline, Front-end, and IF work -- Roger 3. Q-band and other commissioning news -- Ron 4. Software status -- Nicole 5. Spectrometer status -- Rich 6. Project scheduling -- John 7. Observing schedule -- Carl 8. AIPS++ report -- Joe M. 9. Any other business REPORTS & DISCUSSION 1. Az Track Status * The 2nd and 3rd replacement wear plates are due here Sunday afternoon.  They will be installed next week, with a target of Wednesday afternoon for plate 27 and Thursday afternoon for plate 31. There has been no appearance of a crack on the surface of either plate. * A work scope and requisition for the track  analysis is in the hands of Procurement. * The shop weld demonstration (proof of concept) is scheduled to begin April 29th. * The field weld of the track, pending a successful demonstration and satisfactory metallurgy results, is scheduled to begin June 2.  A preliminary work scope is being sent to Continental Field Services to begin estimating and scheduling. * The weighing continues.  Two trucks have been re-weighed, a decision will be made on whether to accept another or redo it today (Thursday).  * The bolt strain gauge experiment was reperformed this week, and the data is being reduced. -- RA Discussion. Bob reported that the wear plates are still on track for delivery on Sunday. He expects to have one of them installed by Tuesday afternoon, and the other Thursday afternoon. We are considering re-weighing two of the trucks using the improved measurement method. Bob noted that we need to plan for optimal use of the period between 2-9 June when the track retrofit work is underway -- we should do as many parallel maintenance activities as possible. This will be discussed in the Monday planning meeting. 2. Spectral Baseline, Front-end, and IF work Lab testing of the revamped C-band receiver was completed. While work was proceeding on the receiver, we installed a layer of ferrite absorber on the perimeter of the thermal transition gap. Evaluation of the system on the telescope with this absorber will provide additional information related to gap leakage. The receiver is scheduled for installation next week. Test and calibration of the S-band receiver is now beginning. Research into cable phase temperature coefficients produced encouraging measurements on a type of Andrews heliax which is usable in several places within our system. The temperature coefficient measures less than +/-5ppm/C, compared with typically +75ppm/C for some existing cable types, over our operating temperature range. The heliax has been installed at two critical locations for evaluation, but systematic tests with the spectrometer have not yet been done. We anticipate three useful observational tests coming up as the GBT schedule allows: -- Evaluation of the Q-band receiver baselines. This will be quite weather dependent. -- Evaluation of the Andrews heliax in the Optical Receiver-to-Converter Module path, which tends to generate a 63MHz baseline ripple. -- Evaluation of the C-band receiver with new LNAs and ferrite absorber as described above. -- RN Discussion. Rick added that he and Dana were beginning to write the baseline investigation report. Roger noted that the Andrews cables had solder-on connectors that must be made up at the factory, so the cables must all be ordered to length. The solder-on connectors are superior to clamp on connectors, which are used with some of the competing cables. Richard noted that there is a desire to re-start conventional holography at the end of April, and that we should discuss this at the planning meeting on Monday. 3. Q-band and other commissioning news Commissioning activates on the telescope this week consisted of two nights of Q-band tests, one of which was unproductive because of the weather. The other was very productive and were aimed at investigating the beam shape and efficiency of the telescope. Most of the data remains unreduced. Paraphrasing my e-mail from Wed. night: We now have efficiency measurements of a few sources at various elevations and a total-power beam map using the strong source 3C273. The maps should have a dynamic range of better than 25 dB. There is no evidence of a scattering plateau surrounding the main beam and I think we will need to use the moon instead. At last week's commissioning meeting, I showed plots of cuts through the beam that appeared to have bumps on the wings of the beam profile. To explore the cause, I made cuts through the beams at various axial focus settings. Even a cursory examination of the data reveals that the focus settings that maximizes the gain, minimizes the beam width in the azimuth directions, and minimizes the beam width in the elevation direction are all different. The gain vs. focus curve has a 10-mm wide top that is flatter than one would expect. Although I cannot eliminate classical astigmatism, to me the profile bumps suggest that the FEM model may need to be scaled by about 10%, which is not unreasonable. The out of focus cuts show asymmetries in both Az and El that are reminiscent of coma. If it is coma, its magnitude is also rather small. The 'configuration' document that Frank and I have been working on is almost complete. We're hoping to distribute copies late this afternoon. Karen is beginning to look at the data she took last week for Spectrometer mode checkouts. Next week we'll be spending time on the PTCS review, observer support, and spectrometer checkouts. The Q-band scheduled for next week may not happen because of the rearrangement of schedules due to this week's Spectrometer failure. -- RJM Discussion. Everyone was impressed and very encouraged by the good results that Ron got at Q-band. This seems to indicate that the telescope is performing well and the dish and subreflector settings done on the basis of photogrammetry, the FEM, and lower frequency observations were quite good. Ron noted that the astigmatism correction could probably be done through emperical trials. 4. Software status Software Development Division #31 - Friday, April 4, 2003 We are completing Week 4 of a 6-week development cycle, to culminate in the release of M&C v3.13 on 4/23/03 +/- 1 day. Our key activity this week has been operational support for the ailing spectrometer. The SDD has responded to several requests made by Electronics that will aid in troubleshooting for this device, including the ability to capture debugging information even in the event of a shutdown. Joint efforts between SDD and Electronics also uncovered logic errors in the firmware which are being addressed. These incremental enhancements will be patched into the current M&C version as they become ready, so as not to hold up progress by Electronics. The plan for porting the spectrometer software to the Linux platform outlined last summer has been accelerated as well. Porting the software to Linux will not solve the challenges that we are experiencing now, but it will significantly enhance the ability of Electronics to detect and solve problems more rapidly now and in the future. Although the porting activity has been fast-tracked, this is still a risky update to the system which is prone to unexpected complications. Here is how we plan on mitigating the risk: efforts over the upcoming days to a week will be focused on getting the spectrometer online on the new platform. However, this means that full testing will not be done at this time. Instead, we expect that it will take the remainder of our development cycle (another two weeks until approximately April 23rd), to ensure that the spectrometer on the new platform is tested and reliable. If we are not able to complete the port within the upcoming three weeks, it will very likely take one to two more months due to the types of problems that could be uncovered and require work. M&C development work continues on schedule with the exception of some active surface MRs, which have been moved from the April commitment list to background activities to accommodate the extra work required to help on the spectrometer. The activities that have been sacrified include modifying the messages from the active surface, and improving error handling, which has the greatest impact on Electronics' ability to move forward with active surface hardware improvements. We can recover from this over the next one to two cycles and still stay on track for delivery of PTCS-driven active surface work this fall. Out of the remaining 8 MRs approved by sponsors, fixing the zero blanking time has already been patched into the spectrometer, spectrometer quick tests and fixes for Xilinx mesages are ready for sponsor testing, active surface slaves operating in series is ready for software testing but not sponsor testing yet, adding new samplers to the active surface has been successfully integation tested but not sponsor tested yet, adding feedback parameters and improving sampler selection capabilities are under development, and leaving spectrometer hardware in a pseudo random test mode has been rejected. There are 2 additional MRs for GO; the one that was required for out of focus beam map work was completed this week and awaits sponsor testing. Preparation for the arrival of Claire Chandler next week to work on out of focus beam maps has included corresponding with parts of that team in Europe and arranging for software utilities to be downloaded and installed in GB. EMS work continues on schedule to support the PTCS design review next week. Finishing touches are being put on a live demonstration of the system's capabilities, which Kim is using to identify the characteristics of the next EMS iteration. High level assessments have been compiled on all 137 requests by the end of March deadline required to support the newly launched "Ease of Use" campaign. SDD will review these results with Ron during the upcoming week to determine the best way to phase the activities. The SDD requests that you continue to submit all new software requests through the software Project Office at http://projectoffice.gb.nrao.edu and provide us with comments to help us make our system work better for you. -- NMR Discussion. Jim asked about the mechanism for getting additional feedback from the observer when a Request was submitted to ProjectOffice. Nicole replied that requests for more information would be done manually. 5. Spectrometer status As everyone already knows, we have experienced a completely disabling failure in the spectrometer. The interface computer (a VME based Sparc computer) for the GBT Spectrometer failed, and the spare for it did not work when installed. Efforts to swap daughter boards (trying to make one good computer out of parts from two bad ones) were also unsuccessful. Hopefully, this is the culmination of the recent deterioration in reliablity noted by observers. It is certainly the culmination of several long days of trouble-shooting. We are doing all that we can to alleviate the situation, but it will be Monday at the earliest before we can complete repairs. We have sent the failed parts to the vendor for expedited repairs, and in parallel with this are proceeding along an upgrade path outlined previously to port the system to a PC running Linux, using a PCI to VME adapter. A system on which to prototype this developement has been cobbled together in Brandt's office. As part of an effort to improve system reliability, the system monitor firmware been modified. The modifications should reduce the chance of spectrometer crashing or hanging. The code has compiled successfully. The next step is to simulate it and make sure it works well as it is supposed to. An effort to produce trouble-shooting documents that will make it possible for several engineers or techinicians to fix routine failures in the spectrometer is continuing. This week, a draft procedure for trouble shooting in response to the error message 'Load Xilinx Personality failed" was written. Documentation produced by Escoffier for the Pulsar Spigot Card was reviewed and comments were forwarded to Ray. The design of the VME Interface board is in the process of being studied. Thus far the design seems very solid and in agreement with the VME specification. Plans for next week are very dependent on the arrival of repaired Themis Sparc computers. As soon as they show up, we will install and configure them. Then we will run several tests to assure that things basically work. This should be followed by an extended robustness test, or an observing or commissioning session by a very friendly observer. If we get over this hump and still have some time left, we will get back to testing LTA S/N 8 which has a slot dependency problem. -- JF, HC, RL Discussion. John stated that the "Plan A" fix to the Spectrometer was to finish the port to Linux, possibly this weekend. Plan B is to reinstall the repaired Themis boards when they come back (probably not before Tuesday). We discussed whether to try to bring the Spectrometer back on line this weekend if the port to Linux was finished. After the meeting, we decided that we would continue working to bring the Spectrometer back to life asap (including working this weekend), but that we would define a test plan that lays out the minimum set of tests that must be passed before the Spectrometer will be released for use. John has since written this plan, which may require some overnight engineering tests. 6. Project scheduling March 31st Planning Meeting Minutes 10:00 A.M. 1) This week's schedule Weighing impacts on maintenance and access Weighing scrubbed for Monday due to weather. 4 more days needed to complete the weighing task. As last week, access will be allowed when not actually jacking the telescope. Bolt strain test will be worked in this week sometime when not weighing. 2) Next week's schedule Wear plate replacement We hope to get 2 more cracked plates done by Thursday, April 10th. This is contingent upon receiving the 2 plates by Monday morning, April 7th. Richard mentioned a possible tour of the GBT by the PTCS review committee Tuesday early afternoon, or Wednesday late afternoon. 3) April Observing Schedule Track Weld schedule and effects The track welding will not take place in April. We will find out this week a probable date for it to occur, and then set the schedule from that. 4) May Observing Schedule Institution of long maintenance days Personnel schedule changes (9/40?, 8/10?) The contract for Structural Inspections cannot be let until sometime in May, due to need to be approved by NSF. This will delay the start of long maintenance days until June. At that point, a decision will be made on personnel schedules. Track weld shutdown may take place late in May, or more probably in June. Bob will try and get probably dates from the vendor. It should be noted that we are trying to get this done ASAP, and this is the reason that we are having trouble getting a hard date scheduled. 5) AOB Spectrometer Discussion It was agreed that the spectrometer reliability problems must be solved. The Spectrometer software and hardware people will meet Tuesday at 11:00 to discuss the memo circulated by John on Friday. Spigot card tests will be delayed until further progress on the reliability problems can occur. Ron wanted opinions of scheduling the telescope for tonight. 5 parties wanted the telescope: 1) Spectrometer checkouts 2) Q band efficiency measurements 3) Q band baseline observations 4) Jim Braatz K band observing 5) Roger's Ku band tests John stated that if the weather is suitable for Q band, then that's what we should do. It is essential to get some baseline data before we quit Q band for the summer, in case we need to do some work on the receiver before next winter. Spectrometer checkouts and the Ku band tests should be scheduled as a backup program, so that good high-frequency weather is used for Q or K band operations. Q band commissioning will replace Spectrometer Checkouts on the schedule, and Spectrometer Checkouts will be put on the schedule as backup projects. PTCS time on April 7th was also given up for Q band commissioning or K band observing. -- JF 7. Observing schedule For 4/4/ commissioning meeting: Last week ======= Observations for: GBT02B-019, GBT02A-046, GBT02C-054, GBT02B-021, GBT02A-031, GBT02C-023, GBT01A-014, GBT02B-002 Spectrometer failure has resulted in cancelling two projects. April ==== Astronomy scheduled ~ 398 hours (256 hours backup) Maintenance ~ 157 hours Tests & commissioning ~ 165 hours May === Astronomy ~ 241 hours Maintenance ~ 134 hours Tests & commissioning ~ 292 hours Will schedule ~ 80 hours more Astronomy -- RCB 8. AIPS++ report o Stable Snapshot. For details see: In final week of five week cycle, ending April 4th (extended by one week). Current Stable Snapshot available: http://aips2.nrao.edu/docs/reference/updates.html o Project Office The address is: http://projectoffice.aips2.nrao.edu o Key Targets: Bob: gbtmsfiller: fill multi-bank data to single MS Joe: average and decimate design/implementation Jim: Observing support Bob and Joe's remaining targets are not complete. The MS change is substantial and additional time for testing will be required prior to release. It will be available next stable with testing beginning shortly. The MS decimation routines will have the design complete but will not be implemented. o Key Support Calibration problem in K-band Substantial time has gone into this. There is one final test on the reduction end to be done before it can be concluded that it is a problem in the data. Any comments on benchmarks? The velocity offset widget has been checked in (really) to the daily version of the plotter - any comments on this? Key Defect work: AOCso04429 cannot access all integrations AOCso04377 imager failed and asked that this bug be submitted AOCso04370 the units in the header for the freq columns are wrong AOCso04344 Saved with KEEP but could not retrieve AOCso04048 does not return a record AOCso04001 naic.import hangs during execution AOCso03939 FITS file made from an image has incorrect int units AOCso03872 gauss fitting routine needs way to restrict channel o Other GBT coordination meeting will be next week when Joe and Steve are in GB. There is a candidate for the GB position interviewing on Friday the 11th. Calendar: NRAO VC: 4/14-4/15 (4/8- 4/16: Joe, Steve, Tim: CV, GB) George Moellenbrock in GB for support 4/21-4/28 -- JMcM Discussion. Steve added that George Moellenbrock will be in GB from 21-28 April to aid in observer support. Jim stated that he had written a tool to do some Fourier filtering of spectral data that could be used to improve baselines. He is checking this into Dish. He also has written a tool to browse the FITS files to look for occasions in which the telescope quits tracking. 9. Any other business None. PRJ 4 April 2003