Subject: 2.5m obslog 20000331 (mjd 51635) Date: Sat, 1 Apr 2000 05:05:10 -0700 (MST) From: hayes@apo.nmsu.edu (Jeffrey Hayes) [mail to: obs-report-25m@apo.nmsu.edu include MJD in subject] Friday Mar 31 2000 Apache Point Observatory SDSS 2.5m Observing Log (MJD 51635) OBSERVING TEAM: =============== Jeffrey Hayes (APO) shift B Steph Snedden (APO) shift B Scot Kleinman (APO) shift the first Atsuko Nitta Kleinman (APO) shift the first Bing Chen (JHU) PT VISITORS: ========= Larry Carey (UW) Don Holmgren (FNAL) Steve Kent (FNAL) THE PLAN: ========= Spectro if clouds remain, imaging if conditions warrant. If we remain in the enclosure, try doing an open shutter goDrift with lights out in the dome and rotate the rotator all the way around to look for light leaks. SYNOPSIS FOR NIGHT: =================== Clouds, humidity, and fun with crates and weird camera problems. Snow. Most folks bailed by 01:30, with the two dedicated and industrious Observers staying by their posts until the bitter end (i.e. twilight). TELESCOPE STATUS: ================= We left the imager on. DIMM: ----- n/a as we have had snow, clouds and other joy. INTERLOCKS: =========== none bypassed. SOFTWARE USED: ============== IOP/SOP: v3_12_2 Watcher: v2_8_2 MCP: v4_2_1 WEATHER SYNOPSIS (see below for hourly data): ============================================= Cloudy at start with predictions of more clouds and a change of showers. At midnight, we still had glorious weather -- clouds, snow and wind. At 02:00, high hummidity. GENERAL COMMENTS (see Problems below for details): ================================================== During the preflight check, we ran an imager bias only to discover the ad crates weren't playing. We stopped the drift, rebooted the crate and did a ptvmeReset in iop. The next drift still didn't work. The murmur log was reporting: dArchSrv TEXTONLY archCmd: archNoAutoDAQ and dLine dsc_W_trcWarn msgQReceive err, TMO assumed, errno=0x3d0004 (850 interrupts We left messages for Ron and Don and Connie. See problem area below. We finally fixed it by rebooting the camera. Why this worked, who knows? Anyone? At 24:00 we put the imager on the telescope so that we could perform slewing tests using the rotator, and try and see if the reported light leak has been found. While slewing the rotator to 180 degrees from 0, we had an abort, with the actual positions and error being equal but of opposite sign in the MCP menu. The same happened when we went from 0 to -180, as well. We saw nothing in the way of stray light in our experiments. All the gang files are in /data/astro/rotator_test. The frames are in the usual place on sdsshost under this MJD and the appropriate run numbers (see below). We ran endNight at 04:40, but did not ship the tapes as we have only light-leak data. Also we had a problem with endNight: vis just after the endNight copy: (it copied everything okay)... "endNight mail imager run summeries Error: can't read "runArray(1318,end)" no such element in array" DATA ACQUISITION (see IOP/SOP output below): =========================================== IMAGING: -------- run type ---- ---- 1310 bias (noticed astro crates not behaving- rebooted and ptvmeReset) 1311 bias still no ad crates online. 1312 attempted bias 1313 attempted bias 1314 attempted bias 1315 attempted bias 1316 attempted bias 1317 real bias 1319 dark -- slew from 0 deg to 180 deg on rotator: started at frame 12 ended at frame 14 from TCC. Nothing obvious light-leak-wise. 1320 dark -- slew from 0 to 180 via MCP at vel=10000. Started at frame 11 and ended frame 96. Again, nothing obvious light-leak-wise. 1321 dark -- slew from 0 to -180 via MCP at vel=10000. Started at frame 11 and ended at frame 96. Still nothing obvious light-leak-wise. 1322 dark w/ shutters closed. Slew from 0 to 180 via MCP at vel=10000. Started at frame 9 and ended frame 95. 1323 dark w/ shutters closed. Slew from 0 to -180 via MCP at vel=10000. Started at frame 9 and ended frame 94. All gangs for these latter 5 runs are in /data/astro/rotator_tests. We did not archive the frames as we saw nothing. SPECTROSCOPY ------------ exposure type -------- ---- 3286 -bias -init MIRRORS: ======== PRIMARY: -------- MIGS SHOULD BE Axial A= 1.9314 1.9315 Axial B= 0.5564 0.5570 Axial C= 0.5564 0.5570 Transverse D= 0.4084 0.4075 Lateral E= 1.8985 1.8980 Lateral F= 0.3989 0.3990 GALILS Commanded: Actual: SETMIR VALUES primDesOrient: primOrient: SECONDARY: ---------- Focus: -257.2rel (camera) -377.47 absolute Temp: Altitude: MIGS Axial A= 0.3389 Axial B= 0.3869 Axial C= 0.3789 Transverse D= -0.0800 GALILS Commanded: Actual: SETMIR VALUES secDesOrient: secOrient: FOCUS LOG: ========== Time Inst Focus Az El Temp Dir MPH filt FWHM " LN2 SCHEDULE: ============= Imager dewar replaces around 16:30. Spectro dewars were filled around 22:00. and 04:40. Latter weight 205 lbs PROBLEMS: ========= astro crates ------------ Following the sdsshost WWW procedures: First, we did a ptvmeReset, then: ~ Host>ptvmeTest -n 3 -a 0x18000000 PID: 561458 IOPB size: 64 bytes ping: no operation: read loopCnt: 1 (defaulted) 100 loops per dot LWordCnt: 0x00000001 (defaulted) controller: 0 (defaulted) remNode: 3 --> /dev/ptvme0n3 remAddr: 0x18000000 (defaulted) IxferMode: 0x00 (defaulted) TxferMode: 0x00 (defaulted) errno = 0: open ("/dev/ptvme0n3", O_RDWR) = 3 errno = 0: ioctl (fd=3, PT_IOCTL_GET_PRIORITY, &ptPriority) = 0 ptPriority = 0x03 errno = 0: ioctl (fd=3, PT_IOCTL_GET_VECTOR, &ptVector) = 0 ptVector = 0x01 errno = 0: ioctl (fd=3, PT_IOCTL_GET_TIMEOUT, &ptTimeoutSave) = 0 ptTimeout = 900 ticks errno = 0: ioctl (fd=3, PT_IOCTL_GET_OP_MODE, &ptOpMode) = 0 ptOpMode = 1 SINGLE_IO errno = 0: ioctl (fd=3, PT_IOCTL_GET_DEBUG_MODE, &ptDebugMode) = 0 ptDebugMode = 0 NORMAL errno = 0: ioctl (fd=3, PT_IOCTL_GET_DEBUG, &ptDebugSave) = 0 ptDebug = 0x00000000 errno = 0: ioctl (fd=3, PT_IOCTL_GET_NODE_ID, 0) = 0x00 . 1 3 ms 18:14 errno = 0: lseek (fd=3, 0x00000000, SEEK_CUR) = 0x00000000 00000000: 01 B4 03 08 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 00000010: 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F 00000020: 20 21 22 23 24 25 26 27 28 29 2A 2B 2C 2D 2E 2F 00000030: 30 31 32 33 34 35 36 37 38 39 3A 3B 3C 3D 3E 3F ?> A subsequent goDrift resulted in: im> goDrift -bias statusEntry daqBusy on node ad1 is not 0 but 1 Error: Aborting: system is already busy im> iccExec [astroList] readoutEnd {ad1 {}} {ad2 {}} {ad3 {}} {ad4 {}} im> goDrift -bias statusEntry daqBusy on node ad1 is not 0 but 1 Error: Aborting: system is already busy im> iccExec [astroList] readoutAbort {ad1 {}} {ad2 {}} {ad3 {}} {ad4 {}} im> goDrift -bias statusEntry daqBusy on node ad1 is not 0 but 1 Error: Aborting: system is already busy im> iccExec [astroList] readoutAbort {ad1 {}} {ad2 {}} {ad3 {}} {ad4 {}} im> iccExec [astroList] statusGet daqBusy {ad1 0} {ad2 0} {ad3 0} {ad4 0} This time, the drift started, but none of the astro chips were showing on the monitors and we got the same murmur messages as before. m> endDrift Fri Mar 31 18:23:26 MST 2000: stopping drift with: iccExec id1 id2 id3 id4 id5 id6 ad1 ad2 ad3 ad4 readoutEnd Fri Mar 31 18:24:36 MST 2000: waited too long for the daqBusy...giving up Fri Mar 31 18:24:39 MST 2000: closed shutters Fri Mar 31 18:24:39 MST 2000: endData completed, DA is not daqBusy=0 Flavor of this run (science, bias, calibration, engineering, ignore, help) [bias]: Mountain estimate of the overall data quality (unknown, bad, acceptable, good) [unknown]: bad leap seconds = 32 im> iccExec [astroList] statusGet daqBusy {ad1 1} {ad2 1} {ad3 1} {ad4 1} im> iccExec [astroList] statusGet daqBusy {ad1 1} {ad2 1} {ad3 1} {ad4 1} im> iccExec [astroList] readoutEnd {ad1 {}} {ad2 {}} {ad3 {}} {ad4 {}} im> iccExec [astroList] statusGet daqBusy {ad1 1} {ad2 1} {ad3 1} {ad4 1} im> iccExec [astroList] readoutAbort {ad1 {}} {ad2 {}} {ad3 {}} {ad4 {}} im> iccExec [astroList] statusGet daqBusy {ad1 1} {ad2 0} {ad3 1} {ad4 0} im> iccExec [astroList] statusGet daqBusy {ad1 1} {ad2 0} {ad3 1} {ad4 0} im> iccExec [astroList] statusGet daqBusy {ad1 0} {ad2 0} {ad3 0} {ad4 0} All the astro boards' lights were reading only PWR and SYS. We rebooted a few more times and this is what the murmur log -grep ad1 showed during another bias attempt: Mar 31 19:05:04 sdssad1 dAssem dsc_I_trcInfo assemTask: initialization ptFd=9 Mar 31 19:05:07 sdssad1 dInit dsc_I_iccReady ICC version v14_20 is ready, built Mar 16 2000 15:19:27 Mar 31 19:05:29 sdsshost IDA 551700 TEXTONLY Murmur Heartbeat to id1 id2 id3 id4 id5 id6 ad1 ad2 ad3 ad4 in init4Data... Mar 31 19:05:30 sdssad1 dDataSrv dsc_S_success Normal successful completion Mar 31 19:05:35 sdsshost IDA 551700 TEXTONLY Heartbeat found from id1 id2 id3 id4 id5 id6 ad1 ad2 ad3 ad4 Mar 31 19:05:40 sdssad1 dArchSrv TEXTONLY archCmd: archNoAutoDAQ Mar 31 19:06:09 sdsshost IDA 551700 TEXTONLY Synchronizing sequence numbers. \n\t\tFor id1 id2 id3 id4 id5 id6, asmSeqNum=-1, for ad1 ad2 ad3 ad4, asmSeqNum=7 Mar 31 19:06:13 sdssad1 dLine dsc_I_trcInfo readoutBegin rcvd: ccdCh status entries have 6 CCDs; frame s Mar 31 19:06:13 sdssad1 dLine dsc_I_trcInfo readoutBegin: destin.: daqPoolWrite=1 daqRegnWrite=-1 Mar 31 19:06:13 sdsshost IDA 551700 TEXTONLY finished with iccExec -riskyOn data -async id1 id2 id3 id4 id5 id6 ad1 ad2 ad3 ad4 readoutBegin Mar 31 19:06:33 sdssad1 dLine dsc_W_trcWarn msgQReceive err, TMO assumed, errno=0x3d0004 (850 interrupts Mar 31 19:07:01 sdsshost IDA 551700 TEXTONLY stopping drift with: iccExec id1 id2 id3 id4 id5 id6 ad1 ad2 ad3 ad4 readoutEnd Mar 31 19:07:01 sdssad1 dLine dsc_I_trcInfo readoutEnd msg rcvd Don called back and while he was calling, we were trying new biases which ended up complaining about communications to the id* crates. We then exited iop, rebooted everything and tried again. Other things started failing at this point, so we considered th system messed up and reset everything. The ids came back, but the ads were still not playing. Each endDrift had to be followed by readoutAbort commands as before. Later, id1's drives took themselves offline - who knows why? Fun with crates! After much fiddling, Don could find nothing wrong. The symptom on his end is the astrometrics aren't getting any pixels. Time to start debugging the camera, so we called Connie and left a message. We were also getting: Mar 31 20:04:07 sdssad1 dLine dsc_E_trcErr WRONG num of pix?? for (sr<<16)|ccd:0x00000005 rowIdx:0 Mar 31 20:04:07 sdssad1 dLine dsc_E_trcErr WRONG num of pix?? 0 should be 2128 Mar 31 20:04:07 sdssad1 dLine dsc_E_trcErr WRONG num of pix?? for (sr<<16)|ccd:0x00000004 rowIdx:0 Mar 31 20:04:07 sdssad1 dLine dsc_E_trcErr WRONG num of pix?? 0 should be 2128 Mar 31 20:04:07 sdssad1 dLine dsc_E_trcErr WRONG num of pix?? for (sr<<16)|ccd:0x00000003 rowIdx:0 Mar 31 20:04:07 sdssad1 dLine dsc_E_trcErr WRONG num of pix?? 0 should be 2128 Mar 31 20:04:07 sdssad1 dLine dsc_E_trcErr WRONG num of pix?? for (sr<<16)|ccd:0x00000002 rowIdx:0 Mar 31 20:04:07 sdssad1 dLine dsc_E_trcErr WRONG num of pix?? 0 should be 2128 Our next attempt was to power cycle the camera and try again. This worked! Sorry, Don. Misc 1 early moring 20000401 ------ It appears that the lava light is suffering from a severe stochastic excess. We noted that at 140VAC, the number, sphericalness, and colour of the wax balls that are created do not have any predictability. We think that this could be due to the obviously cloudy fluid in which they are contained, and hypothesize that, when the fluid is changed, that the properties of the balls will return to a predicatable behaviour. We think that a PR on the fluid change is not needed, as this is essentially a monitoring programme, and not amenable to PR tracking. Snedden points out that the colour is a predictable quantity, and what Hayes thought was colour change is, in fact, his inability to distinguish colours in the early morning. We feel that an NSF grant is required in order to track down the stochastic excess of the number and sphericalness of the wax balls, so that these may be expressed in closed form. We will also be e-mailing a request under the SDSS PoO for outside collaborators. Our study will have general applicability to various inflationary cosmologies and will appear in an ApJ Letter at a later date. Misc 2 later that day... ------ At about 02:30 we were called by the SPRR from El Paso. It appears that one of their North bound trains reported a "big piece of glass" laying near the track between Alamo and El Paso near Dog Canyon, and wondered if we might know anything about it. Being prudent and not wishing to cause either alarm or extra expense, we told them that it probably belongs to Sac Peak. WEATHER DATA: ============= Wind Time Temp F Dewp F MPH Direction Dust Sky Precip. 17:00 45 10 16 236 (SW) 2471 mcl 18:01 41 9 20 250 (WSW) 1751 mcl 19:01 34 26 8 51 (NE) 1274 mcl 20:01 30 28 10 73 (ENE) 298 tcl 21:01 29 28 13 88 (E) 259 tcl snow 22:01 29 27 16 68 (ENE) 142 tcl snow 23:01 27 24 13 78 (ENE) 51 tcl 00:01 26 24 12 86 (E) 157 tcl 01:01 27 24 12 109 (ESE) 948 tcl 02:01 30 27 8 142 (SE) 2685 pcl 03:01 30 29 8 175 (S) 3112 pcl 04:01 26 25 8 87 (E) 877 pcl mcl=mostly cloudy, tcl=totally cloudy, DATA LOGS FROM IOP/SOP: ======================= not currently available