Subject: 11May00 51675 SDSS nightlog Date: Thu, 11 May 2000 02:35:08 -0600 (MDT) From: sjnk@apo.nmsu.edu (Scot Kleinman) Thursday, May 10, 2000 Apache Point Observatory SDSS 2.5m Observing Log (MJD 51675) OBSERVING TEAM: =============== Scot Kleinman (APO) Atsuko Nitta Kleinman (APO) John Brinkman (APO) Eric Neilsen (JHU) PT VISITORS: ========= Jim Annis (FNAL) - phone Bill Boroski (FNAL) Richard Kron (UC) Ron Rechenmacher (FNAL) -phone Safety Crew THE PLAN: ========= Shakeup. Test the new versions of iop/sop, TCC, astroDa. SYNOPSIS FOR NIGHT: =================== DA:14_28 - requested tests passed. One new one we made up did not. TCC:2.2.8 - seemed to work, but we did not do a particularly long track. It bailed out of a telrun session at some point by itself, but didn't seem to have any other effect. We tracked (and imaged) two FASTT fields and a short stretch of 32N. (Ron was online during the FASTT equatorial scan.) IOP:3_15_0 - no new problems SOP:3_15_0 - loadCart bug we couldn't figure out prevented further tests also small error with instChange. Imager: - we have out of focus images for JEG in /data/photo/optics/focus51675. we did two FASTT fields on tapes which are to be shipped. StephTest: - we didn't get to this since we could not loadCart in SOP. (This test was a gotoField, note offset, then re-fiducialize, gotoField again, and note new offsets.) TrumplerAward - Awarded to Scott Burles. Congratulations, Scott. Does this mean you won't have time for us anymore? Great job! When can I get a copy of your thesis? TELESCOPE STATUS: ================= Stored at 30deg with cart 7, plate 322 loaded. DIMM: ----- not logged tonight INTERLOCKS: =========== none bypassed. SOFTWARE USED: ============== IOP/SOP: v3_15_0 Watcher: v2_8_2 MCP: v4_2_4 TPM: v1_0_8 AstroDa: 14_28 TCC: 2.2.8 WEATHER SYNOPSIS (see below for hourly data): ============================================= Clear, a little windy (about 20-30mph). GENERAL COMMENTS (see Problems below for details): ================================================== John B. cut the new version of IOP/SOP. We then did the preflight checkout by taking biases on both the imager and the spectrographs, and then running endNight. While taking imager biases, both id2 drives failed. We used the archiver cheat sheet to recover both drives. id2-2 recovered, but we still had problems with id2-1. We decided to swap tapes for id2-1 and resumed the drives. They both seemed to be happy BUT the watcher kept on complaining that id2-1 was NOT happy. This was strange as we could not see any arch Errors in the murmurlog. Within a few seconds after clearing the watcher id2-1 error message, the same error continually reappeared. We then did the 10-reboot test (rebooted the crates 10 times) and checked things with the "iccSoundOff" command as Ron instructed. The crates came back online fine. Details of DA tests are in the Problem section below, even though they aren't really problems. (Well, one of them is...) The TPM log wasn't running tonight- see problems section below. We could not initially get a guider image from the spectro guider, but traced it down to the fibers not being re-connected after last night's engcam use and imaging. We tested the TCC by slewing a bunch of places in mount and fk5 coordinates. It all seemed to go ok. We interspersed a few Menu commands and all these worked too as long as we did a track/stop. If we didn't, the telescope would go to an incorrect position. Ex. track 121,30 mount in TCC use Menu to move alt to 45 telescope goes DOWN in alt If you do a track/stop first, it's OK, but this is easy to forget. I know such behavior is a feature of the TCC, but it's dangerous and really should be fixed without relying on a track/stop command. Since we decided to do a FASTT filed as a test scan, we stopped our bias drift, but because of earlier tape problems (pausing and resuming), we decided to pause archiving, flush the queue, initialize the pool, and put in new tapes. We realize we should probably have done poolInit last, but instead we: archPause initNight -poolInit archFlushQ archResume initNight -tapeLoad (with new tapes) and it seemed to work. We started imaging FASTT field H at lambda=-29 (field starts at-22). We did a multiLskip -5 and it seemed to show up on the scrolling displays, but we also got this message from IOP: im> multiLskip -n 5 42 0 lskip 17 (cmr off by one, really 16 rows) 42 1 lskip 17 (cmr off by one, really 16 rows) 42 2 lskip 17 (cmr off by one, really 16 rows) 42 3 lskip 17 (cmr off by one, really 16 rows) 42 4 lskip 17 (cmr off by one, really 16 rows) Is this normal? We got Ron who got Jim on the phone while we were in the FASTT field so they could look at the dec problem. We went to 32N to do an off-eq scan, but then Ron called and wanted to go back to the equator- so we went to do a new FASTT field (I). multiLskip on 32N had an error. See problem section. ~23:30 I noted my TCC telrun session had stopped. The TCC process was running, but it had bailed out somehow. I did a new telrun and it worked fine. Off to spectro with Cart 7 plate 322. instChange in SOP worked, but it reported (multiple times): Failed! The TCC won't tell me the LST Failed! The TCC won't tell me the LST Even after we got there, it kept saying it, so we CTRL-C'd out and SOP seemed to recover all right. We switched instruments for the safety crew and tried to loadCart and got error messages. See problems below. We tried to find the problem ourselves, but couldn't figure it out and the developers were unavailable. Consequently, SOP didn't get much testing. The safety crew convinced me to use the mouse with my left hand. Atsuko was told to get a foot stool. We closed around 1:30 and left around 2:45. DATA ACQUISITION (see IOP/SOP output below): =========================================== IMAGING: -------- Run Type --- ---- 1479 bias 1480 bias 1481 FASTT field H starting at lambda=-29 frame 25 done with autoLskipping frame 44 done with multiLskip frame 96 at lambda=-15 (end of FASTT field) lskips: -0.014, +0.002, +0.019, +0.008, -0.015. skygang: raBoreErr: -2.4 +- 6.4", decBoreErr: 21471.0 +-64420.0" azErr: 5960.6 +-17875.2", elErr: 20898.5 +-62704.3" muErr: -2.4 +- 6.4", nuErr: 21471.0 +-64420.0" Correct idReport boresight values: xBore = -1293.74 mm, yBore = 0.15 mm. 1482 test off-eq scan of 32N startLambda 0 Note- we've done this scan before, so these data may be useful for something, although we have moon now, of course. frame 20 - end (Ron wants to go back to eq.) 1483 FASTT field I starting at lambda=-3 ended frame 58 at Lambda=5.5 (should be enough for the astrometrists) skygang: raBoreErr: -5.5 +- 1.3", decBoreErr: -3.0 +- 7.3" azErr: 8.8 +- 2.0", elErr: -4.1 +- 7.4" muErr: -5.5 +- 1.3", nuErr: -3.0 +- 7.3" Correct idReport boresight values: xBore = 0.18 mm, yBore = 0.33 mm. 1484 test off-eq scan of 32N startLambda 0 Frame 54 -300 focus Frame 60 +300 focus Frame 61 end SPECTROSCOPY: ------------- Exposure Type -------- ---- 3720 bias 3721 bias 3722 bias 3723 bias TAPES ----- JL2779 JL2780 JL2781 JL2782 JL2783 JL2784 JL2785 are to be shipped. (They have 2 FASTT fields which are good.) MIRRORS: ======== PRIMARY: -------- MIGS SHOULD BE Axial A= 1.9284 1.9284 Axial B= 0.5589 0.5594 Axial C= 0.5574 0.5579 Transverse D= 0.3729 0.3744 Lateral E= 1.9065 1.9065 Lateral F= 0.4074 0.4074 GALILS Commanded: 5883.,-3086.,-197.,4201.,7343.,7041. Actual: 5900., -3100., -200., 4200., 7350., 7050. SETMIR VALUES primDesOrient: =0.00, -19.50, 7.80, 427.00, 1067.00 primOrient: SECONDARY: ---------- Focus: -548 (TCC) -794 (abs) Temp: 16 Altitude: 55 MIGS Axial A= 0.7169 Axial B= 0.7520 Axial C= 0.7399 Transverse D= 0.0160 GALILS Commanded: -985688.,-998567.,-1008845.,-1169.,-6002. Actual: -985786., -998504., -1008797., -1150., -6000. SETMIR VALUES secDesOrient: -791.95, 5.00, -5.00, 50.00, -42.09 secOrient: -791.95, 4.99, -4.95, 50.22, -42.31 FOCUS LOG: ========== M1 Piston M2 Piston Temp Wind Time Inst setmir TCC setmir Az El C Dir MPH filt FWHM " not logged tonight. LN2 SCHEDULE: ============= Spectro filled 19:45 and 01:40. PROBLEMS: ========= ID2 --- drvState id6 use=1 dsc_archStNotMount ?= dsc_archStNotMount (0x8000C1B3) drive=1 Wed May 10 16:29:34 MDT 2000: tapeWaitFor reports success Waiting for command response from id2 (crate 1, ICC 1), archiver server (client 2) Waiting for command response from id2 (crate 1, ICC 1), archiver server (client 2) Waiting for command response from id2 (crate 1, ICC 1), archiver server (client 2) Wed May 10 16:30:11 MDT 2000: IOP Not ready (blew up on pad): tapeMount: Request sense failure Failed to receive exabyte command completion status invoked from within "iccExec $tapeList archReadSense 2" invoked from within "set sense2 [iccExec $tapeList archReadSense 2]..." (procedure "tapeRewind" line 12) invoked from within "tapeRewind $tapeList " invoked from within "if {$doRewind} { tapeRewind $tapeList }" (procedure "tapeLoad" line 5) invoked from within "tapeLoad $tapeList $doRewind $doUnload" im> archReadSense 1 0x06280080 --> what does 06280800 mean? im> archReadSense 2 0x00000080 Watcher complained about id2-1 (archUseDrv 0 and archDrvSts Error) even after it resumed archiving and the archiver status was "busy". I didn't see any id2 problems in the murmur log while I was continuously getting the "beeps" and red lights on id2-1. TPM --- It does not appear that the TPM is starting itself like it is supposed to. Every night I've been on since last week, we've had to restart the TPM by hand. Tonight when we tried it, we got the following: startTpmLog restart warning: no nodes are configure to boot system v14_24 will attempt use of system: System is not properly configured MURMUR server information is not available Configure it with iccConfig Presumably, this is because of the astroDa change, but we have no instructions on what to do about this. WE NEED THEM and we NEED THE AUTO-TPM to be re-implemented. TCC/GuideCam ------------ Our initial attempt at getting a spectro guider image failed, although the error messages were not particularly helpful: sp1> Modu="prt_ChangeDev"; Text="the guide camera TCC_GCAM1 will not be usable until you fix the problem" Modu="prt_ChangeDev"; Text="could not initialize new device: TCC_GCAM1" Modu="exe_Collimate"; EText="altitude unknown; could not adjust collimation and focus" We fixed things by plugging in the guider fibers and doing a new set inst in the TCC. multiLskip ---------- The 32N multiLskip resulted in: im> multiLskip 16 0 lskip 17 (cmr off by one, really 16 rows) CAMERA: FAILURE on ftelnet: connection error in ftelnet: connection refused Error: Telnet to camera failed: connection error in ftelnet: connection refused while executing "ftelnet $cameraConnectTime $retryDelay $tries $latNode $portNum" invoked from within "set camera($cameraName) [ftelnet $cameraConnectTime $retryDelay $tries $latNode $portNum]..." Sorry, we didn't do a tb since we were stopping the run and going back to an eq. scan. (But on second thought, it looks like a normal error which would have gone away had we done it a second time.) loadCart -------- sp1> loadCart 7 322 /p/iop/v3_15_0/params/cartridgeInfo.par Error: Syntax error: cartridgeId is not a member of GPROBE Error: h6.cartridgeId Error: ^ Error sp1> tb while executing "exprGet $gp.cartridgeId" invoked from within "if {[exprGet $gp.cartridgeId] == $cartridge} { set n [exprGet $gp.gProbeId] lappend fiberList $n foreach field "exists xc ..." ("loop" body line 4) invoked from within "loop i 0 [chainSize $vals] { set gp [chainElementGetByPos $vals $i] if {[exprGet $gp.cartridgeId] == $cartridge} { set n [exprGe ..." (procedure "loadCartridge" line 94) invoked from within "loadCartridge 7 322" ("uplevel" body line 1) invoked from within "uplevel 1 [tclx_unknown2 $args]" invoked from within "return [uplevel 1 [tclx_unknown2 $args]]..." (procedure "unknown" line 8) invoked from within "loadCart 7 322" DA Tests -------- test1: One "archPause 1" was issued and then another "archResume". It took about 24 seconds from the time I issued the archPause 1 command (breathe) until it took effect. But archResume took effect immediately. test2: We issued 3 archPause commands in a row. After making sure that archiver was paused, we issued one archResume command. The archiver resumed for id1, id2, id4, id5 but did not resume for id3 and id6. In the murmurlog, it did say that archResume for all nodes. We issued archResume for id3 and id6 individually and they resumed. We tried again (three archPause, one archResume). it did not seem like any of the archivers resumed this time. Another archResume was issued and it was paused again. The third archResume got the archivers to resume. WEATHER DATA: ============= Wind Time Temp F Dewp F MPH Direction Dust Sky Precip. not logged tonight. [pmt]cl = partly, mostly, totally cloudy DATA LOGS FROM IOP/SOP: =======================