From: ank@apo.nmsu.edu Subject: 2.5mobslog 20010628 (MJD 52089) Date: Fri, 29 Jun 2001 02:42:50 -0600 (MDT) Thursday Jun 28 2001 Apache Point Observatory SDSS 2.5m Observing Log (MJD 52089) OBSERVING TEAM: =============== Dan Long (APO) Atsuko Nitta Kleinman (APO) SUPPORT: ========= Jon Brinkmann (APO) French Leger (FNAL/APO) via phone THE PLAN: ========= Shakeup according to the shakeup plan SYNOPSIS FOR NIGHT: =================== It seems that AZ encoder is not reading the position correctly. We did as many software tests we could without going to the sky( weather would have kept us closed anyway ). -Eric's log archive script worked and we archived 50 days worth of logs/par files on JL4222. The tape is in the file cabinet in the computer room and should be shipped to Eric when he checks remotely that the script did indeed run successfully. -All the chips can now be viewed on the scd ( 9x9 or 16x16 ). -goStare -bias -plateId -9999 doesn't ask the plate id question. -Limited TCC testing was done ( a few small slews in the enclosure ) -grabInstruments complains if it can't get ports or relay server. TELESCOPE STATUS: ================= We did not open. DIMM: ===== not functional INTERLOCKS: =========== none bypassed SOFTWARE USED: ============== IOP/SOP: v3_51_0 Watcher: v2_9_0 MCP: v5_2_1 AstroDa: 14_36 TCC: 2.5.4 TPM: v2_3_7 AstroDa: 14_36 tccmon: v1_6 sdssProcedures: v1_21 WEATHER SYNOPSIS (see below for hourly data): ============================================= There were thunderstorms around during the late afternoon and early evening, with light rain persisting until about 21:00. The clouds started to clear around 22:30, but moved back in about an hour later. It was cloudy the rest of the night. GENERAL COMMENTS (see Problems below for details): ================================================== Telescope ---------- Tonight when setting the fiducials we got large azimuth errors each time we crossed the 120 fiducial. These errors were in the range of 3000 to 5000 counts ( far too large to correct ). The error increased by this 3000 to 5000 counts regardless of the direction of the slew. The magnitude of the slew was only a few degrees ( typically 119 to 122 or vice versa ). Thinking that fiducial might be giving us trouble, we tried another fiducial. The 105 fiducial exhibited the same behavior. When slewing back to the 120 fiducial the error was 40000 counts or so, again always in the same direction. To determine if the fiducial system was having trouble or the telescope was really getting further off with each move, we measured the distance from the stow scribe mark on the azimuth drive disk to part of the mounting for the azimuth drive when the telescope was at 121 ( by the encoders ). The telescope was then slewed back and forth between 119 and 122 three times, then returned to 121 ( by the encoders ). The scribe mark was measured again and found to be about 3mm to the right ( increasing azimuth angle ). This would seem to indicate that the fiducials are working properly and that there is something wrong with the encoders. With input from French, we tried a couple of things. First we rebooted the mcp. No change in behavior. Second, we tried swapping the cables on the two azimuth encoders. Once again, the behavior was the same. The cables were swapped back. What ever is affecting the encoders is affecting both in the same way. Since both lines go into the slip detection box, and testing was done on that unit during the day, that seems a reasonable place to start looking. The telescope has been left in this state for the day crew to examine. With the poor weather and no loss of science time, it was decided not to try swapping to the spare slip detection module. TCC (v2_5_4) ------------- We slewed the telescope a little bit in the enclosure. There was no noted changes for the small amount of slewing we did. IOP (v3_51_0) -------------- initNight, cleanupMurmur, bias taking, endNight -doTpm all worked. 16x16 binning with SCD: ************************ We can now bin up to 16x16 using the scd display. We tried the new 9x9 and 16x16 binning while taking bias. We can now see all of all chips by 9x9 binning. When we changed the scd to 16x16 binning, the display did change, but we also got error messages. The messages are, Error:can't set "sed_id1_bin":{id1{Specified value out of range}} The traceback gave: can't set "scd_id3_bin": {id3 { Specified value out of range}} while executing ".scd.displays.inodes.id3.bin.menu invoke active" ("uplevel" body line 1) invoked from within "uplevel #0 [list $w invoke active]" invoked from within "if {[$w type active] == "cascade"} { $w postcascade active set menu [$w entrycget active -menu] tkMenuFirstEntry $menu } elseif {[$w type activ ..." (procedure "tkMenuInvoke" line 14) invoked from within "tkMenuInvoke .scd.displays.inodes.id3.bin.menu 1" (command bound to event) Under "display" menu on the scd display, one can use "All of All Chips" to get 9X9 binning of all chips at one click. Monthly Log Archiving *********************** Eric forgot to link his instruction page to the sdsshost procedure page, but he emailed us where we can find the page so we can try running his archive scripts. We decided to archive from MJD=52039 to 52089 (16G of data according to archiveSize command). This is because in /skycat/tapes/scheduled, we found archiTarTape-52039.par so we knew the last time this script was run on MJD52039. The script started ~21:00 and finished ~23:40 successfully. Tape JL4222 contains the log archive data and currently sits on the self with the other imager/spectro/TPM tapes in the computer room. Once Eric remotely checks all is well, the tape should be shipped. With now his archive scripts working, "endNight -doTpm" which we run on cloudy nights should only do the bookkeeping part, but not write the TPM logs to tapes. Eric requested the messages we see on the screen to be recorded, but since there are so many messages, I saved one day worth of archiving messages. Here is a "day"worth of message while tarring. : Writing 41 TPM 52053 /astrolog/52089/idTarTape-JL4222.par exists, reading device and VSN information deviceList is: {idTarTape-JL4222 /dev/mt/tps4d3nrnsvc} fileMarkList is {/dev/mt/tps4d3nrnsvc 43} Will write directory: /mcptpm/52053 Tarring with: exec tar cf /dev/mt/tps4d3nrnsvc cfg interlocks.cfg ab.cfg mount.cfg pmss.cfg 20010524-000000 20010523-000000 mcpFiducials-52053.dat admin.cfg temperatures.cfg mirrors.cfg mcpCmdLog-52053.dat tpm.cfg freq_directory ampslip.cfg 2> /dev/null finished writing to /dev/mt/tps4d3nrnsvc Recording with :appendToTarTapePar /astrolog/52089/idTarTape-JL4222.par TPM 52053 43 nextFileMarkList is: {/dev/mt/tps4d3nrnsvc 44} Writing 42 LOG 52053 /astrolog/52089/idTarTape-JL4222.par exists, reading device and VSN information deviceList is: {idTarTape-JL4222 /dev/mt/tps4d3nrnsvc} fileMarkList is {/dev/mt/tps4d3nrnsvc 44} Will write directory: /astrolog/52053 Tarring with: exec tar cf /dev/mt/tps4d3nrnsvc oldlogfiles.archlog runlog-2330.txt endNight.log idFrameLog-002330-03.par idFrameLog-002331-03.par idFrameLog-002332-03.par idFrameLog-002333-03.par idFrameLog-002334-03.par idFrameLog-002335-03.par idFrameLog-002336-03.par idTapeLog-JL4071.par runlog-2331.txt idTarTape-JL4077.par idFrameLog-002330-02.par idFrameLog-002331-02.par idFrameLog-002332-02.par idFrameLog-002333-02.par idFrameLog-002334-02.par idFrameLog-002335-02.par idFrameLog-002336-02.par tapeStat.par iop.v3_41_5a.log idTapeLog-JL4072.par iop.v3_45_8.log runlog-2332.txt ioptape.archlog idSysLog-52053.par idFrameLog-002330-01.par idFrameLog-002331-01.par idFrameLog-002332-01.par idFrameLog-002333-01.par idFrameLog-002334-01.par idFrameLog-002335-01.par idFrameLog-002336-01.par idReport-52053.par idTapeLog-JL4073.par runlog-2333.txt 2332.photo.archlog 2333.photo.archlog 2330.photo.archlog 2331.photo.archlog 2336.photo.archlog 2334.photo.archlog 2335.photo.archlog idFrameLog-002330-10.par idFrameLog-002331-10.par idFrameLog-002332-10.par idFrameLog-002333-10.par idFrameLog-002334-10.par idFrameLog-002335-10.par idFrameLog-002336-10.par 2332.astro.archlog 2333.astro.archlog 2330.astro.archlog 2331.astro.archlog 2336.astro.archlog 2334.astro.archlog 2335.astro.archlog sdReport-52053.par idTapeLog-JL4074.par idTapeLog-JL4064.par runlog-2334.txt 2ndln2.dat idFrameLog-002330-07.par idFrameLog-002331-07.par idFrameLog-002332-07.par idFrameLog-002333-07.par idFrameLog-002334-07.par idFrameLog-002335-07.par idFrameLog-002336-07.par timer.log 0.tpm.archlog idWeather-52053.par idTapeLog-JL4075.par idTapeLog-JL4065.par runlog-2335.txt idCCDLog-71-52053.par idCCDLog-73-52053.par idCCDLog-72-52053.par idCCDLog-75-52053.par idCCDLog-74-52053.par idCCDLog-76-52053.par idCCDLog-61-52053.par idCCDLog-60-52053.par idCCDLog-63-52053.par idCCDLog-62-52053.par idCCDLog-65-52053.par idCCDLog-64-52053.par idCCDLog-51-52053.par idCCDLog-53-52053.par idCCDLog-52-52053.par idCCDLog-55-52053.par idCCDLog-54-52053.par idCCDLog-56-52053.par idCCDLog-41-52053.par idCCDLog-43-52053.par idCCDLog-42-52053.par idCCDLog-45-52053.par idCCDLog-44-52053.par idCCDLog-46-52053.par idCCDLog-31-52053.par idCCDLog-33-52053.par idCCDLog-32-52053.par idCCDLog-35-52053.par idCCDLog-34-52053.par idCCDLog-36-52053.par idCCDLog-21-52053.par idCCDLog-23-52053.par idCCDLog-22-52053.par idCCDLog-25-52053.par idCCDLog-24-52053.par idCCDLog-26-52053.par idCCDLog-11-52053.par idCCDLog-13-52053.par idCCDLog-12-52053.par idCCDLog-15-52053.par idCCDLog-14-52053.par idCCDLog-16-52053.par idCCDLog-01-52053.par idCCDLog-03-52053.par idCCDLog-02-52053.par idCCDLog-04-52053.par idCCDLog-91-52053.par idCCDLog-90-52053.par idCCDLog-93-52053.par idCCDLog-92-52053.par idCCDLog-95-52053.par idCCDLog-94-52053.par idCCDLog-81-52053.par idCCDLog-83-52053.par idCCDLog-82-52053.par idCCDLog-85-52053.par idCCDLog-84-52053.par idCCDLog-86-52053.par idFrameLog-002330-06.par idFrameLog-002331-06.par idFrameLog-002332-06.par idFrameLog-002333-06.par idFrameLog-002334-06.par idFrameLog-002335-06.par idFrameLog-002336-06.par hinv.out idTapeLog-JL4076.par idTapeLog-JL4066.par runlog-2336.txt idTarTape-JL4070.par idFrameLog-002330-05.par idFrameLog-002331-05.par idFrameLog-002332-05.par idFrameLog-002333-05.par idFrameLog-002334-05.par idFrameLog-002335-05.par idFrameLog-002336-05.par iop.neilsen.log idTapeLog-JL4067.par idDaParam-sd1-52053.par idDaParam-ad3-52053.par idDaParam-ad2-52053.par idDaParam-ad1-52053.par idDaParam-ad4-52053.par idDaParam-id3-52053.par idDaParam-id2-52053.par idDaParam-id1-52053.par idDaParam-id6-52053.par idDaParam-id5-52053.par idDaParam-id4-52053.par idTapeSummary-52053.par idFrameLog-002330-04.par idFrameLog-002331-04.par idFrameLog-002332-04.par idFrameLog-002333-04.par idFrameLog-002334-04.par idFrameLog-002335-04.par idFrameLog-002336-04.par skippy.log idTapeLog-JL4078.par idTapeLog-JL4068.par iop.v3_45_2.log idAstroline-c1-002332.par idAstroline-c2-002332.par idAstroline-c3-002332.par idAstroline-c4-002332.par idAstroline-c5-002332.par idAstroline-c6-002332.par idAstroline-r6-002332.par idAstroline-r7-002332.par idAstroline-r8-002332.par idAstroline-r9-002332.par idTapeLog-JL4069.par idAstroline-c1-002333.par idAstroline-c2-002333.par idAstroline-c3-002333.par idAstroline-c4-002333.par idAstroline-c5-002333.par idAstroline-c6-002333.par idAstroline-r6-002333.par idAstroline-r7-002333.par idAstroline-r8-002333.par idAstroline-r9-002333.par exposureLog-52053.par murmurHeartBeat.error iop.v3_42_0.log iop.v3_43_0.log iop.v3_44_0.log idAstroline-c1-002330.par idAstroline-c2-002330.par idAstroline-c3-002330.par idAstroline-c4-002330.par idAstroline-c5-002330.par idAstroline-c6-002330.par idFrameLog-052053-11.par idAstroline-r6-002330.par idAstroline-r7-002330.par idAstroline-r8-002330.par idAstroline-r9-002330.par idFrameLog-002330-09.par idFrameLog-002331-09.par idFrameLog-002332-09.par idFrameLog-002333-09.par idFrameLog-002334-09.par idFrameLog-002335-09.par idFrameLog-002336-09.par idAstroline-c1-002331.par idAstroline-c2-002331.par idAstroline-c3-002331.par idAstroline-c4-002331.par idAstroline-c5-002331.par idAstroline-c6-002331.par idAstroline-r6-002331.par idAstroline-r7-002331.par idAstroline-r8-002331.par idAstroline-r9-002331.par idFrameLog-002330-08.par idFrameLog-002331-08.par idFrameLog-002332-08.par idFrameLog-002333-08.par idFrameLog-002334-08.par idFrameLog-002335-08.par idFrameLog-002336-08.par server.log opConfig-52053.par iop.v3_45_6.log idAstroline-c1-002336.par idAstroline-c2-002336.par idAstroline-c3-002336.par idAstroline-c4-002336.par idAstroline-c5-002336.par idAstroline-c6-002336.par idAstroline-r6-002336.par idAstroline-r7-002336.par idAstroline-r8-002336.par idAstroline-r9-002336.par nomurmur.log 0.spectro.archlog oldMurmur.tar opBC-51791.par iop.v3_45_4.log opCamera-51960.par idAstroline-c1-002334.par idAstroline-c2-002334.par idAstroline-c3-002334.par idAstroline-c4-002334.par idAstroline-c5-002334.par idAstroline-c6-002334.par idAstroline-r6-002334.par idAstroline-r7-002334.par idAstroline-r8-002334.par idAstroline-r9-002334.par scanpars logfiles.archlog iop.v3_45_5.log opECalib-51627.par idAstroline-c1-002335.par idAstroline-c2-002335.par idAstroline-c3-002335.par idAstroline-c4-002335.par idAstroline-c5-002335.par idAstroline-c6-002335.par idAstroline-r6-002335.par idAstroline-r7-002335.par idAstroline-r8-002335.par idAstroline-r9-002335.par focus-52053.par 2> /dev/null finished writing to /dev/mt/tps4d3nrnsvc Recording with :appendToTarTapePar /astrolog/52089/idTarTape-JL4222.par LOG 52053 44 nextFileMarkList is: {/dev/mt/tps4d3nrnsvc 45} Writing 43 MURMUR 52053 /astrolog/52089/idTarTape-JL4222.par exists, reading device and VSN information deviceList is: {idTarTape-JL4222 /dev/mt/tps4d3nrnsvc} fileMarkList is {/dev/mt/tps4d3nrnsvc 45} ill write directory: /skycat/oplogs/murmurlog/52053 Tarring with: exec tar cf /dev/mt/tps4d3nrnsvc astroda.v14_36.log 990661397.iop.v3_45_8.log 2> /dev/null finished writing to /dev/mt/tps4d3nrnsvc Recording with :appendToTarTapePar /astrolog/52089/idTarTape-JL4222.par MURMUR 52053 45 nextFileMarkList is: {/dev/mt/tps4d3nrnsvc 46} Writing 44 TPM 52054 /astrolog/52089/idTarTape-JL4222.par exists, reading device and VSN information deviceList is: {idTarTape-JL4222 /dev/mt/tps4d3nrnsvc} fileMarkList is {/dev/mt/tps4d3nrnsvc 46} HELP STRINGS ************* We tried several "help iop_commands". For the ones we tried, they all worked and we got help page. From the shakeup plan, I thought "help -search=drift" will give me list of commands with a word "drift" in them. Instead, I got a help string as shown below: im> help -search=drift Typing help with NO arguments, a list of known facilities is displayed. Available options: help * List ALL commands and full messages associated . help cmd Show the message text associated to cmd help cmd -full Show the message text associated to cmd and the facility help -facil List all available commands. help -facil=facil1,facil2 List commands with facility = facil1 and facil2 help -search=pattern List all commands whose messages contain pattern help -search=pattern -full List all commands whose messages contain pattern; the full text message will be displayed. help -sort List all commands sorted by command name. Existing facilities: iop SOP --- goStare -bias does what has been advertised ******************************************** sp1>goStare -bias asks for plateId sp1>goStare -bias -plateId -9999 does not ask plateID SOPGUI:goStare 0 bias -init does not ask plateID grabInstruments *************** The grabInstruments do complain if it cannot grab the instrument and/or if it cannot talk to the watcher. Here is what we did: Atsuko did a grab instruments which was successful. Dan then tried and got the following response. Is the traceback supposed to be there? sp1> grabInstruments -handlers 120 CAMERA sp1: FAILURE on ftelnet: connection error in ftelnet: connection refus ed ********************************************* ***FAILED TO ATTACH TO CAMERA**************** ********************************************* MECH: FAILURE on ftelnet 1000000 1500000 2 t-g-sdss-2.apo.nmsu.edu 2300 CAMERA sp2: FAILURE on ftelnet: connection error in ftelnet: connection refus ed ********************************************* ***FAILED TO ATTACH TO CAMERA**************** ********************************************* MECH: FAILURE on ftelnet 1000000 1500000 2 t-g-sdss-2.apo.nmsu.edu 2400 grabCAMERA sp1: FAILURE on ftelnet: connection error in ftelnet: connection refused ********************************************* ***FAILED TO ATTACH TO CAMERA**************** ********************************************* InstrumeMECH: FAILURE on ftelnet 1000000 1500000 2 t-g-sdss-2.apo.nmsu.edu 23 00 nts -drop CAMERA sp2: FAILURE on ftelnet: connection error in ftelnet: connection refus ed ********************************************* ***FAILED TO ATTACH TO CAMERA**************** ********************************************* MECH: FAILURE on ftelnet 1000000 1500000 2 t-g-sdss-2.apo.nmsu.edu 2400 ************************************************** ====>>>>>>RELAY SERVER FAILED TO START!!!!>>>>>RELAY SERVER FAILE ..." invoked from within "if {![info exists packeteerPort] && $dt > 0} { set serverFailedToStart [catch { startServer -backup -relay [cameraType] -listen 2 } serverSt ..." invoked from within "if {$drop || $stop_handlers} { if 1 { requestListen -[cameraType] 1; # tell server we're quitting } if [info exists packeteerPort ..." (procedure "grabInstruments" line 78) invoked from within "grabInstruments -handlers 120 -reconnect" ("eval" body line 1) invoked from within "eval $procname $args" grabInstruments is scheduled, further errors sent to murmur Next Atsuko logged out the ports but didn't drop the instruments. This means Dan should be able to connect to the cameras but not to the relay server. The response indicates that and indeed complains loudly. The same traceback occurs as above. Do we need this as a part of the complaint? sp1> grabInstruments -handlers 120 ************************************************** ====>>>>>>RELAY SERVER FAILED TO START!!!!>>>>>RELAY SERVER FAILE ..." invoked from within "if {![info exists packeteerPort] && $dt > 0} { set serverFailedToStart [catch { startServer -backup -relay [cameraType] -listen 2 } serverSt ..." invoked from within "if {$drop || $stop_handlers} { if 1 { requestListen -[cameraType] 1; # tell server we're quitting } if [info exists packeteerPort ..." (procedure "grabInstruments" line 78) invoked from within "grabInstruments -handlers 120 -reconnect" ("eval" body line 1) invoked from within "eval $procname $args" grabInstruments is scheduled, further errors sent to murmur DOCS ---- All the new documents can be found on our sdsshost page but Eric's monthlyLogging.html. SoS ---- PR 1725 "SoS choked on blue spectra" was closed. DATA ACQUISITION (see IOP/SOP output below): =========================================== IMAGING: -------- Run Stripe Strip Flavor Lambda Last Frame # Begin End ------------------------------------------------------------------ 2423 100 O ignore -8.06 -1.79 54 2424 100 O ignore 19.78 24.83 48 SPECTROSCOPY: ------------- UT Exp Time flavor comment =============================================== ------ sequence 10363, plate -9999 ------- 00:19 10363 0.0 bias {{}} 00:29 10364 0.0 bias {{}} 00:33 10365 0.0 bias {{}} 00:41 10366 0.0 bias {{}} MIRRORS: ======== PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A: 0.0740 0.0710 Axial B: -0.0970 -0.0980 Axial C: 0.0590 0.0610 Trans D: 4.1790 4.1800 Lateral E: 1.9360 1.9365 Lateral F: 0.4370 0.4370 GALILS Commanded: 5400. -3700. 900. -200. 31550. 30650. Actual: 5400. -3700. 900. -200. 31550. 30650. SETMIR VALUES primDesOrient: 0.00 -11.80 23.00 1256.90 642.10 primOrient: 0.00 -12.16 22.81 1257.41 642.19 SECONDARY: ---------- Focus: 0.0 Air temp: 13.7 C Altitude: 30.03 MIGS TONIGHT NOMINAL Axial A: -1.2810 -1.2900 Axial B: -1.3010 -1.3150 Axial C: -1.2880 -1.3060 Trans D: 0.1690 0.1890 GALILS Commanded: 1583850. 1583850. 1583850. 2600. 3800. Actual: 1584852. 1582118. 1582591. 2600. 3800. SETMIR VALUES secDesOrient: 1257.00 0.00 0.00 0.00 -175.00 secOrient: 1256.48 0.15 -0.88 -0.58 -175.22 FOCUS LOG: ========== LN2 SCHEDULE: ============= at 2:42am imager@182 spectr@70 PROBLEMS: ========= WEATHER DATA: ============= Wind Time Temp F Dewp F MPH Direction Dust DATA LOGS FROM IOP/SOP: =======================