Subject: 2.5m obslog 20060619 (MJD 53906) Date: Tue, 20 Jun 2006 05:24:37 -0600 (MDT) From: jcb@apo.nmsu.edu APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Monday June 19, 2006 (MJD 53906) ---=== OBSERVING TEAM ===--- Days: Dan Long Night: Elena Malanushenko John Barentine Also Starring: French Leger (site support) With: Jon Brinkmann (phone support) And: Steph Snedden (phone support) Music By: Henry Mancini and his Orchestra ---=== OBSERVING PLAN ===--- Science! ---=== OBSERVING SUMMARY ===--- All in all a mixed bag tonight, with alternating periods of clouds, bad seeing, and clouds with bad seeing. A failed power supply on sp1 caused us to lose about two hours of time tonight in diagnosing and solving the problem. Fighting low efficiency on the guider, we obtained partial signal on two plates (2209 and 2547) and may have enough signal on a third (2537) but await word from FNAL as to whether to call it done. Brian, could you please check ?? The VNC server on the DIMM computer barfed around 04:30Z. The display in the control room stopped updating at this time and a Windows error on the computer at the DIMM indicated a system problem that caused the application to crash. The process, however, was still alive and we couldn't kill it to start a new VNC server. Updates were being made to the APO WWW pages so we could still get to the seeing information. Integrated dust count for the night is about 16,500 dust-hours. ---=== OBSERVING LOG ===--- Afternoon: ---------- No problems with the afternoon check. Spectro focus was checked but not changed, best focus about 17 C. 01:45Z Doors Open, Fans On. 02:15Z Enclosure Open. Night: ------ Skies were partly cloudy during twilight and the east wind portended bad seeing. Also during this time Elena had a problem starting the guider monitor from her SOP session. The culprit was a guide star too close to the edge of the frame for SOP to cope with, and the canonical solution was to do a startMon -simple. 03:41Z Cart 5 Plate 2547 (tile 8217, seguefaint147). GSOGTF. At 04:06Z we got a critical SOP error: no reply (CAMNAG: 2006-06-20 03:40:04Z) We also noted that the preCalib, started according to Elena's SOPGUI at 03:36Z, but never read out to the display screens in the control room nor did it ever appear on disk (the last files to appear on disk were a bias, expnum=39856, at 03:01Z). Yet the 1st science exposure, started at 03:42Z, continued running, at least according to SOPGUI. There was no readout and a second exposure was underway before we caught the problem with the critical error. We tried to stop the 2nd exposure at 04:15Z, but got an "Error in Tcl Script" window that said: "Error: mechanicals did not respond correctly; response was P failed: {no exposure to pause}" We tried forcing a readout to see if any data were there and got: sp1> endStare -force appending END EXPOSURE TIME is: 2.1 2006-06-20 04:23:11Z: beginData has failed: Malformed ID string: Error Went to get French and we had a look down at the scope. All the specmech stuff was in order, no indications of problems. French also checked the connections to the mux box on the rotator, saying that it had been recently moved and so bad connections might be to blame. No evident problems there either. We phoned Jon, who logged in and immediately noted that SoS was slamming sdsshost2 with connection attempts on a second-by-second basis and assumed that some zombie process somewhere was responsible. It turns out that there were, among other things going on, seven simultaneous IDL sessions on SoS that were unneccesary, which we killed. Also Jon found a number of ssh servers running. At 05:07Z we simultaneously got the following critical errors on sp1: ln2_empty 2ndary_dewar_press exec_boot followed by exec_boot on sp2. Jon suspected a power supply on sp1; French found a red LED lit on the +30V CCD power supply on sp1 and swapped it out. Immediately the critical errors stopped, and we were able to get responses back from the camera with cc camcheck commands. After doing iacks (because of the power supply swap and subsequent reboot), we started getting information back from the cameras. We took a bias which read out, although it was very pink -- perhaps the cameras warmed a bit during all of this? The second had normal levels. We then started again on this plate from scratch and by 05:50Z we were taking science data. We did 900 s exposures on this plate for the rest of the time the plate was available, although the last two had to be declared bad because we barely missed the (S/N)^2 = 2 cutoff in one camera, b2. 07:27Z Cart 1 Plate 2209 (tile 1594, chunk116). No GSOGTF, however an fk5InFiber number 10 landed our quarry. We paused here to do a focus sweep, unsure if the low efficiency on the last plate was due to bad focus or bad seeing (or both). Did three exposures of 1200 s and 2 x 900 s, respectively before this plate expired but did not collect enough signal to call it done. 08:49Z Cart 2, Plate 2537 (tile 8207, segue146). Brian has requested that we try different exposure times on this plate from about 300 s to 600 s in the hopes they can better understand the signal needs of these particular plates in the future. The conditions under which we took the data, however, were (1) moon, (2) non-photometric with moderately thick cirrus and (3) seeing > 2 arcsec. On the GTF we got a rotator error of 13234943 -- ever too slightly large to correct. We figured this out after we had no GSOGTF, no fk5InFiber number 10, etc. Forced the fiducial, did a short test slew in rotation only to check the result, and tried again. This time we landed a single guide star in fiber 9 after going to an fk5InFiber number 10 and making offsets sticky. From there, we did a rotateAboutFiber number 9 making an initial offset guess of +100 arcsec. That gained us some more guide stars, so we think the guess was reasonably correct. However, clouds were thickening during this entire setup time, to the point that we were losing guide stars and the efficiency varied from 70% from guider frame to guider frame. Also note that guide fiber 11 has a total of three objects within one fiber diameter of its center, so in order to keep the guider from being thoroughly confused, we disabled this fiber for guiding. The same problem happened last night and this seemed to be the solution. At this point, we decided to wait. The clouds would equally affect the other two available plates at this time, both SEGUE, so the chances of having to toss science exposures for lack of sufficient signal seemed good. Also, the conditions were far from optimal for bracketing exposure times within the requested range as variations due to the clouds would overwhelm the effect of changing the exposure time. By 09:30Z it was clear enough toward the direction on the sky of this plate that we had guide stars in almost every fiber. While waiting on the clouds we preCalib'd so we were ready to rock n' roll once the sky cleared. We took the following exposures: Expnum Time (s) b1 r1 b2 r2 ------ -------- --- --- --- --- 39874 600 0.9 2.4 2.6 0.4 39875 450 0.7 1.4 1.3 0.3 39876 450 0.6 1.3 1.4 0.3 39877 450 0.4 1.0 1.2 0.2 39878 300 0.4 1.3 1.0 0.2 39879 300 0.4 1.5 0.9 0.2 SoS did in fact reduce the exposures of caget tpm_TM_PMSSAvg tpm_TM_TelairAvg.VALA tpm_TM_PMSSAvg 16.9464 tpm_TM_TelairAvg.VALA 17.2025 Mean offset (pix) Collimator Position sp1 sp2 b1 r1 b2 r2 A B C A B C .25 -.02 .07 -.01 18020 18040 17600 6050 3380 3420 Still set for approximately 17 C The startMon Bomb ----------------- On attempting to do a startMon (with no -simple) tonight after starting up SOP, loadCartridge, etc., Elena got this error: SOPGUI: startMonitor Star 6 won't fit in an 450*450 region, scaled 150 Star 6 won't fit in an 450*450 region, scaled 150 while executing "error "Star $i won't fit in an $ysize*$xsize region, scaled $scale"" invoked from within "if [catch { set sreg [subRegNew $reg $star_ysize $star_xsize [expr $yc - $star_ysi ze/2] [expr $xc - $star_xsize/2]] }] { regD ..." ("foreach" body line 61) invoked from within "foreach i {10 11 9 8 7 6 5 4 3 2 1} { if {![info exists guiderData($i,reg)] || $guiderData($i,reg) == "NULL" || ![info exists gFiber( ..." (procedure "mosaic_rotated_fibers" line 10) invoked from within "mosaic_rotated_fibers $reg $scale 0 0 4" invoked from within "return [mosaic_rotated_fibers $reg $scale 0 0 4]..." (procedure "show_rotated_fibers" line 7) invoked from within "show_rotated_fibers $scale $size $size $bkgd" invoked from within "set reg [show_rotated_fibers $scale $size $size $bkgd]..." (procedure "showGuideStars" line 43) invoked from within "showGuideStars 150 450 75 1 1" invoked from within "if {![info exists guiderInfo(simple)]} { showSimpleFrame } elseif {$guiderInfo(simple) == 1} { showSimpleFrame } else { showGu ..." (procedure "showGuiderFrame" line 4) invoked from within "showGuiderFrame" invoked from within "if $display { showGuiderFrame }" invoked from within "if {$imageName == $guiderInfo(lastImage)} { ;# old filename? return "" } else { guiderFrameDel; # clean up set guiderInfo(las ..." invoked from within "if [catch {set imageName [lastGuiderFile]} msg] { echo "$msg" return "" } elseif {[regexp {([^/]*gimg)([0-9]+)(.fits$)} $imageName]} { ..." (procedure "checkForNewImage" line 11) invoked from within "checkForNewImage 1 0" ("eval" body line 1) invoked from within "eval $procname $args" checkForNewImage is scheduled, further errors sent to murmur