Date: Sat, 13 Jan 2001 06:49:52 -0700 From: "s. a. snedden" Subject: 2.5-m obslog 20010112 (MJD 51922) Friday Jan 12 2001 Apache Point Observatory SDSS 2.5m Observing Log (MJD 51922) OBSERVING TEAM: =============== Atsuko Nitta Kleinman (APO) Scot Kleinman (APO) Steph Snedden (APO) part PT part 2.5m VISITORS: ========= Larry Carey (UW) Mike Carr (PU) Jim Gunn (PU) Steve Kent (FNAL) Craig Loomis (APO) Robert Lupton esq. (PU) Connie Rockosi (UC) THE PLAN: ========= Shake Down #2. MCP and Imager/IOP shake are the high priority shakes (we have Connie and Robert only until Sunday). SYNOPSIS FOR NIGHT: =================== MCP, Watcher and IOP were tested. Some PRs were closed, some new ones were opened. TELESCOPE STATUS: ================= Never opened Stowed at 30 deg. with no instruments on. Counterweights at 20. Spectro fill lines on in auto mode. INTERLOCKS: =========== none bypassed SOFTWARE USED: ============== IOP/SOP: v3_25_2 Watcher: v2_8_12 MCP: v4_4_1 TPM: v2_2_1 AstroDa: 14_34 TCC: 2.4.1 WEATHER SYNOPSIS (see below for hourly data): ============================================= Clouds. Snow. (partly cloudy from sunset to evening twilight). GENERAL COMMENTS (see Problems below for details): ================================================== Bing is held up in Paris, France due to a visa problem. Therefore until Jan. 15, Steph will run PT and Scot will take the 2.5m shift on clear nights photometric nights when PT needs to be run. There are high priority patches which must be observed during the first half of the night. We expect Bing at APO on Jan 15 and hopefully start observing on Jan. 16. Steph started out tonight on the PT, but when clouds closed in, shut it down, and switched back to 2.5-m shakedown. Imager: Connie, Jim, Mike Carr's work continued. We had a scare that the telescope was moving in AZ inside the enclosure in the afternoon and the doghouse hit a toolbox drawer. New Rule: Never Ever Forever Move The Telescope In Az Inside the Enclosure Unattended. I'm sure there's an I Ching hexagram that says "The wise man attends well his enclosed moving telescope". When we were building the fiducial tables last night, Steph was in the enclosure moving the telescope. This is OK. But don't even think about moving the scope in Az if you are not at the telescope inside the enclosure. There is a knob on the back of the telescope that can get caught by MANY different tools etc. hanging on the wall and the toolbox drawer need only slightly open before it hits the dog house. Some of these things are much higher than the doghouse paddle and the doghouse will have no protection against them. Mike Carr has modified the cart so that we can do inst change at ROT=0:00:00. Mike, Connie, Jim tested it and it works. Thank you Mike! Communication: We are back talking to the mirror with TCC. TPM shows we are getting 0.0000 MIGS for M1 F (Lateral F). In TPM, 0.0000 often means there is no communication. We rebooted the MIGS, but it did not affect the M1 F readings. Larry will take a look at it tomorrow during the day --- when we discovered this, there was on-going work at the telescope on the imager and hence it was not possible to check on the M1 F MIG. Our highest priority of the night is to shake MCP and Imager/IOP changes and therefore we did not think it was worth asking Larry to stay up and look into the M1 F MIG. Spectro Mechanicals: Craig re-established connection to the mechanicals. We had no simple way to make sure we were talking to the right mechanicals, i.e. when we asked for info on the mechanicals in sp1 were we getting sp1 info and not sp2 info? We tested this in a primitive way by unplugging the cable at the xyplex and checking to see to which mechanicals we can talk to. We found out that they were plugged in backwards and connected so that we are talking to what we think we are. MCP Shaking: ============ Redo the rotator fiducial table. Atsuko ran some more rotator sweeps, and a new table was generated. This was needed because the rotator fiducials were renumbered today. Fiducials 1-5 from last night were not valid. The new table was checked into v4_4 (branch: 1.6.4). Tagging etc. will wait until other MCP changes are finished tonight. We ran a NOCVS version of the MCP while we got this data, to access the new fiducial numbers. We continued to run this MCP version until the end of the night. Note that we never saw fiducials 41 and 117 (in the newest numbering scheme). We did not see them last night, either. We believe that there may be a physical reason for this. Perhaps the read head should be looked at? Closed PR MCP/1652: ------------------- Atsuko created inhibits on some axes, and checked the status bits from axis status. The bits agreed with the inhibit conditions. We believe this is fixed, hence closed the PR. Observers: the axis status bits can be gotten from "axis status" in a Telrun session, and interpreted with the aid of the following page: http://sdsshost.apo.nmsu.edu/mcp/status.html. This was also added to Folklore. Closed PR MCP/1265: ------------------- "s" Menu key used on rotator will cause MCP hang. Repeated trials with the "s" key failed to confuse the MCP, so the PR was closed. Watcher: ======== Closed PR Watcher/424: ---------------------- When running a "none" server, gang data is not available to the watcher. A "start_servers -K -n -i" was issued, a bias drift was started and the watcher astroline display updated the gang information. The PR was closed. IOP Shaking: ============ setImagerAmpsOn(Off): --------------------- Connie has written setImagerAmpsOn and setImagerAmpsOff. setImagerAmpsOff appears to set the voltages properly. Once the voltages were set to the "Off" values (15.4V), we started a bias drift. We should have gotten an error message telling us that the values are wrong. Indeed we did get the following: im> goDrift -bias Sat Jan 13 03:16:05 MST 2001: Beginning data taking leap seconds = -46.9224 Sat Jan 13 03:16:14 MST 2001: beginData has failed: im: Amplifier voltages are not at their operating values!!!! Execute setvddnom before taking data!! Error: goDrift ends with error in beginData We then started another drift. We should get information on how to recover the DA, but we did not. Instead, we got the following: im> goDrift -bias statusEntry daqBusy on node id1 is not 0 but 1 Error: Aborting: system is already busy im> readoutAbort im> goDrift -bias statusEntry daqBusy on node id1 is not 0 but 1 Error: Aborting: system is already busy We applied the following to get the DA happy: im> iccExec $active readoutAbort {id1 {}} {id2 {}} {id3 {}} {id4 {}} {id5 {}} {id6 {}} {ad1 {}} {ad2 {}} {ad3 {}} {ad4 {}} im> iccExec $active statusGet daqBusy {id1 0} {id2 0} {id3 0} {id4 0} {id5 0} {id6 0} {ad1 0} {ad2 0} {ad3 1} {ad4 0} Then issued another drift and got the promised error message: im> goDrift -bias Sat Jan 13 03:41:11 MST 2001: Beginning data taking leap seconds = -46.9224 There already is a teamster, PID 5419028, owned by snedden Sat Jan 13 03:41:21 MST 2001: beginData has failed: im: Amplifier voltages are not at their operating values!!!! Execute setvddnom before taking data!! Error: goDrift ends with error in beginData Atsuko noted that the error should say to execute setImagerAmpsOn instead of setvddnom, and should mention to do a readoutAbort. It should also tell you to detach the camera connection. A PR was filed. setImagerAmpsOn fails, but the voltages are at least set to the proper values before failing. Here is the tb from the failure that Steph got: no files matched glob pattern "/tmp/mrlReport*" while executing "glob /tmp/mrlReport*" invoked from within "set reportList [glob /tmp/mrlReport*]..." Here is the tb that Atsuko got when issuing the same command in the same IOP version: im> setImagerAmpsOn Error: can't read "flag": no such variable im> tb can't read "flag": no such variable while executing "if {$flag == 0} { set state "nom" } else { if {$flag == 1} { set state "low" } else { error "getVddState returned invalid response" ..." (procedure "getVddState" line 4) invoked from within "getVddState" invoked from within "set vddState [getVddState]..." (procedure "vddnomPatch" line 2) invoked from within "vddnomPatch" ("uplevel" body line 1) invoked from within "uplevel 1 $name" invoked from within "if {[info command $name] != ""} { uplevel 1 $name }" (procedure "call_proc" line 3) invoked from within "call_proc vddnomPatch" (procedure "setImagerAmpsOn" line 8) invoked from within "setImagerAmpsOn" imagerCCDErrors: ---------------- Seems to work. We did "cc camcheck" to see that the usual message on CCD61 comes up. Then we execute the above command, reissue "cc camcheck". This time, there is no CCD 61 message which is expected. To revert back to where camcheck gets all error info, we did, cc allcamact. imagerCCDErrors has no help string. im> cc camcheck CCD61_I+TRIM2 158 im> noImagerCCDErrors im> cc camcheck im> cc allcamact im> cc camcheck CCD61_I+TRIM2 158 -------------------------------------- endNight -doTpm ran successfully. DATA ACQUISITION (see IOP/SOP output below): =========================================== IMAGING: -------- All of the following were taken with the imager *off* the telescope. Run Stripe Strip Flavor Lambda Last Frame # Begin End ------------------------------------------------------------------ 2010 100 O ignore -179.41 165.31 26 2011 100 O ignore -179.41 -179.41 24 2018 100 O bias -178.42 -178.42 55 2019 100 O ignore -178.42 -37.25 62 2020 100 O engineering -178.42 -178.42 24 2023 100 O bias -178.42 1.61 40 SPECTROSCOPY: ------------- none MIRRORS: ======== PRIMARY: -------- MIGS SHOULD BE Axial A= 0.0410 0.0160 Axial B= -.0469 -0.0109 Axial C= .0480 0.0270 Transverse D= 3.7460 0.2269 Lateral E= 1.9090 1.8984 Lateral F= n/a 0.4054 GALILS Commanded: Actual: SETMIR VALUES primDesOrient: primOrient: SECONDARY: ---------- Focus: Temp: Altitude: MIGS Axial A= Axial B= Axial C= Transverse D= GALILS Commanded: Actual: SETMIR VALUES secDesOrient: secOrient: FOCUS LOG: ========== M1 Piston M2 Piston Temp Wind Time Inst setmir TCC setmir Az Alt C Dir MPH filt FWHM " (rel) (abs) N/A LN2 SCHEDULE: ============= Spectro filled at 06:00. Dewar weight 88 lb. Imager dewar weght at 06:00: 163 lb. PROBLEMS: ========= DA: --- During a bias drift, all photo nodes suddenly showed "no Tape". There were no watcher warnings. After ending the drift, an initNight -tapeLoad was done. We immediately got the following error on all photo nodes: dsc I archFailMsg Rimfire failure ID = 2 Status = 0x70040202 0x82 (dsc_I_archFailMsg Rimfire failure ID = 2 Status = 0x70040202 0x82 : Jan 13 01:31:47 MST ) The tapeLoad appeared to complete normally. and unarchived frames were sent to tape. Some gangs appear to have not made it to the Unix side, however there were also no indications of lost gangs on the Unix side. Here is the result of a check of the gangs and the pool. im> checkGangsFrames 2018 gangs untransferred gangs? checking runs 2018 none im> iccExec [photoList] poolGroupInfo {id1 {Usage: poolGroupInfo group}} {id2 {Usage: poolGroupInfo group}} {id3 {Usage: poolGroupInfo group}} {id4 {Usage: poolGroupInfo group}} {id5 {Usage: poolGroupInfo group}} {id6 {Usage: poolGroupInfo group}} im> iccExec [photoList] poolGroupInfo 2018 {id1 {{ELEM0 {{NAME {2018}} {FRAMECNT 240} {GANGCNT 40} {FRAMECNTA 240} {GANGCNTA 40} {USAGE {{TOTAL 3554760} {PERMANENT 0} {ARCHIVED 3554760}}}}}}} {id2 {{ELEM0 {{NAME {2018}} {FRAMECNT 240} {GANGCNT 40} {FRAMECNTA 240} {GANGCNTA 40} {USAGE {{TOTAL 3554760} {PERMANENT 0} {ARCHIVED 3554760}}}}}}} {id3 {{ELEM0 {{NAME {2018}} {FRAMECNT 240} {GANGCNT 40} {FRAMECNTA 240} {GANGCNTA 40} {USAGE {{TOTAL 3554760} {PERMANENT 0} {ARCHIVED 3554760}}}}}}} {id4 {{ELEM0 {{NAME {2018}} {FRAMECNT 240} {GANGCNT 40} {FRAMECNTA 240} {GANGCNTA 40} {USAGE {{TOTAL 3554760} {PERMANENT 0} {ARCHIVED 3554760}}}}}}} {id5 {{ELEM0 {{NAME {2018}} {FRAMECNT 240} {GANGCNT 40} {FRAMECNTA 240} {GANGCNTA 40} {USAGE {{TOTAL 3554760} {PERMANENT 0} {ARCHIVED 3554760}}}}}}} {id6 {{ELEM0 {{NAME {2018}} {FRAMECNT 240} {GANGCNT 40} {FRAMECNTA 240} {GANGCNTA 40} {USAGE {{TOTAL 3554760} {PERMANENT 0} {ARCHIVED 3554760}}}}}}} Here was the state of the Teamster during this: > 0 __sginap(0x1770, 0x1770, 0x0, 0xa, 0xeff68, 0x3e8, 0x7ffefad4, 0x7ffefad8) ["/xlv44/6.5.6m/work/irix/lib/libc/libc_n32_M3/sys/sginap.s":13, 0xfafbc04] 1 _sginap(0x1770, 0x1770, 0x0, 0xa, 0xeff68, 0x3e8, 0x7ffefad4, 0x7ffefad8) ["/xlv44/6.5.6m/work/irix/lib/libc/libc_n32_M3/sys/sginapSCI.c":34, 0xfb04390] 2 dscUtil_Sleep(tmo = 60000) ["/usrdevel/ron/astroda/src/dscUtil.c":1131, 0x10129d28] 3 dscCLI_CLISleep(clientData = (nil), interp = 0x10a72308, argc = 2, argv = 0x7fff2100) ["/usrdevel/ron/astroda/src/dscCLI.c":2449, 0x10136158] 4 dscCLI_Dispatch(cdata = 0x7, interp = 0x10a72308, argc = 2, argv = 0x7fff2100) ["/usrdevel/ron/astroda/src/dscCLI.c":393, 0x101343b8] We had earlier issued a couple of archPauses when tapes were loaded and hundreds of biase frames done earlier in the day started archiving. Perhaps one of the archPauses got queued and issued during the bias drift? The next bias drift transferred frames & gangs normally, so ...? A PR was filed. WEATHER DATA: ============= Wind Time Temp F Dewp F MPH Direction Dust DIMM Sky Precip. 16:01 38 19 26 208 (SSW) 245 - pcl none 17:01 35 20 18 189 (S) 299 - pcl none 18:01 34 20 20 205 (SSW) 344 - pcl none 19:01 34 21 27 191 (S) 382 - mcl none 20:01 32 22 23 198 (SSW) 430 - mcl none 21:01 25 25 20 261 (W) 864 - cloudy snow 22:01 22 21 20 286 (WNW) 20 - cloudy snow 23:01 21 20 10 269 (W) 47 - cloudy snow 00:01 20 18 23 260 (W) 108 - cloudy flurries 01:01 20 18 25 260 (W) 35 - cloudy flurries 02:01 20 17 15 262 (W) 1 - cloudy flurries 03:01 19 17 21 294 (WNW) 41 - cloudy flurries 04:01 18 16 8 287 (WNW) 56 - cloudy flurries 05:01 17 15 14 313 (NW) 32 - cloudy flurries 06:01 17 14 13 290 (WNW) 52 - cloudy flurries [pmt]cl = partly, mostly, totally cloudy