Date: Sun, 23 Dec 2001 06:20:04 -0700 (MST) From: prn@apo.nmsu.edu (Pete Newman) Subject: 2.5m obslog 20011222 (MJD 52266) APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Saturday December 22, 2001 (MJD 52266) ---=== OBSERVING TEAM ===--- Swing: Steph Snedden (APO) Night: Jurek Krzesinski (APO), Pete Newman (APO) Support: Carlos Gonzales (FNAL), Wendell Jordan (APO), French Leger (FNAL), all working after hours on both 2.5m and PT - Thanks, guys! ---=== OBSERVING PLAN ===--- Science: spectroscopy until moonset (08:26 UT), then imaging if conditions allow. ---=== OBSERVING SUMMARY ===--- Seeing and a flakey secondary actuator prompted spectrography all night. Slow progress while moon up, then faster. Three plates done (761, 570, 555) plus good data on a faint-galaxies plate (804). ---=== OBSERVING LOG ===--- Northerly winds kept the seeing above the limit for imaging until 10:20UT (about 2hr before morning twilight), so we did spectrography all night. At sunset, we were waiting for the secondary A axis actuator to be replaced. Telescope returned to observers at 02:00 UT (about 30 min after end of twilight) with the actuator running fine after the stepper motor was replaced. However, during plate 804 (around 04:15 UT - see below), actuator A started running slowly again while making focus and scale changes with relative focus=+150, and the Heidenhain positions about +3000000. We called French to discuss. We homed the axial actuators. Actuator A ran slow for the first 6-minute time-out, then ran faster on the second. This is curious, but true and repeatable. After homing, all three actuators ran together back to our focus position, and focus and scale changes worked ok. We are all somewhat confused by why homing should free-up actuator A. One guess is that the lead screw on actuator A is sticky near the focus positions (dead moth?). Another is that the Galil amplifier is failing. In any case, we decided to use homing as a workaround if it fails again tonight. It ran fine until ~06:40 UT when it again slowed during a scale change, then again a few times later. We used the folklore cheat (nudge using 'proc enable collimate') to finish the tasks in progress (with a pauseStare), then homed again after each plate was finished. We logged time during each excursion to telescope hardware problems. PR2789 updated. We again saw large fiducial errors on all drive axes around 04:00 UT and on most (all?) slews to instrument change. See problem section #1 for examples. After switching to cartridge 8, we saw the same 'read timed out on port LTA19' error in response to axis init's as seen on MJD 52264. It seemed that an axis stop followed by an axis init cured the problem, or it may have just been that last axis init. About 08:00 UT, Pete's Xterm misbehaved so he could not type into most of his windows. This required logging off and back on. In clearing up orphan processes, he mistakenly killed the murmur server. This was restarted with a cleanupMurmur command by 08:10 UT, so there is probably a gap of about 10 minutes in the murmur log around then. All offsets seem to be to the east again! Observing sequence: UT 02:30 Plate 687, cartridge 5. No data. Stars found on gotoField with offsets cleared, but the field is very close to the moon. This was a useful test of the secondary actuator but no good for science. 03:37 Plate 799, cartridge 9. No data. Stellar template plate. We could not find either guide stars or and FK5 star even with spiral searches of 2 circuits of 7 arcsec steps. What was wrong? We discovered the problem on the next plate. 04:13 Plate 804, cartridge 3. Not finished in 4 exposures (ran past design HA), but good data, with smear. Faint galaxies plate. Again no guide or FK5 stars on the gotoField. We eventually found a double-whammy: we had huge fiducial errors on all three axes (see problems section #1 for the murmur messages) AND the secondary axis A actuator was running slow. We fixed the fiducials and found guide stars, then lost them when we tried to focus, as described above. 07:19 Plate 761, cartridge 1. Done in 3 exposures. We had (S/N)^2=2.9 on r1 (and much more on the other cameras) from MJD 52265. No stars on gotoField, but found quickly with 7 arcsec east offset. The secondary actuator work-round worked, and we re-homed the secondary again after the plate was done. 09:32 Plate 570, cartridge 8. Done in 3 exposures. Large fiducial errors from the slew to instrument change. Once corrected, found guide stars on gotoField. Secondary actuiator needed homing again at end of this plate. 11:07 Plate 555, cartridge 7. Done in 3 exposures. Large fiducial errors from the slew to instrument change. Large east offset (~12 arcsec) needed to find guide stars. ---=== IMAGING RUN SUMMARY ===--- Run Stripe Flavor Lambda Last Begin End Frame Comments ------------------------------------------------------------------ 2839 100 O ignore 149.54 163.31 47 doghouse bias ---=== IMAGING RUN DETAILS ===--- ---=== SKIPPY RESULTS ===--- Run Frame nFrames stars muErr muRms nuErr nuRms rot --------------------------------------------------------------- ---=== LTMATCH RESULTS ===--- Run Field nFields nGood rowMean rowSig colMean colSig rot --------------------------------------------------------------- ---=== SPECTROSCOPY DATA SUMMARY ===--- UT Exp Time flavor comment (S/N)^2 totals ========================================== b1 r1 b2 r2 ----- sequence 12373, plate -9999 ------- 23:03 12373 0.0 bias ----- sequence 12374, plate 687 ------- NIL 02:48 12374 10.0 flat 02:50 12375 2.0 arc 03:14 12376 901.0 target USELESS - IGNORE ----- sequence 12377, plate -9999 ------- 04:07 12377 0.0 bias ----- sequence 12378, plate 804 ------- 26.4 25.2 22.0 31.9 NOT DONE 05:01 12378 10.0 flat 05:04 12379 2.0 arc 05:33 12380 1501.0 target 06:00 12381 1501.0 target 06:18 12382 901.0 target 06:35 12383 901.0 target 06:50 12384 240.0 smear 06:53 12385 10.0 flat 06:55 12386 2.0 arc ----- sequence 12387, plate 761 ------- 17.2 13.4 26.3 13.8 DONE 07:28 12387 10.0 flat 07:31 12388 2.0 arc 08:06 12389 1803.0 target 08:41 12390 1502.0 target 09:02 12391 1013.0 target 09:09 12392 240.0 smear 09:11 12393 10.0 flat 09:13 12394 2.0 arc ----- sequence 12395, plate 570 ------- 15.4 14.2 16.4 16.2 DONE 09:51 12395 10.0 flat 09:53 12396 2.0 arc 10:12 12397 900.0 target 10:26 12398 703.0 target 10:45 12399 707.0 target 10:51 12400 240.0 smear ----- sequence 12401, plate 555 ------- 21.3 14.9 21.4 20.7 DONE 11:17 12401 10.0 flat 11:19 12402 2.0 arc 11:39 12403 901.0 target 11:53 12404 701.0 target 12:16 12405 1204.0 target 12:23 12406 240.0 smear 12:26 12407 10.0 flat 12:28 12408 2.0 arc ---=== TELESCOPE OFFSETS AND SCALE I ===--- Time Instrument getclock Az Alt Rot Scale ------------------------------------------------------------------------------ 20:02 5 687 1009076572 0.001546 -0.000100 -0.023996 0.999810 23:15 3 804 1009088107 0.003514 -0.000601 0.000547 0.999860 00:44 1 761 1009093471 -0.005408 -0.002466 -0.004289 0.999910 04:24 7 555 1009106647 -0.006605 0.002000 -0.016175 0.999950 ---=== TELESCOPE OFFSETS AND SCALE II ===--- ---=== DATA TAPE SUMMARY ===--- Goes: JL4837 Stays: JL4838 ---=== FOCUS LOG ===--- setmir piston Temp Wind Time Inst scale M1 M2 Foc Az Alt (C) MPH Dir filt fwhm ------------------------------------------------------------------------------ 20:03 5 687 0.99981 1875 2940 200 -44 48.1 -5.2 12 337 -- 1.9 22:06 3 804 0.99986 1381 2360 50 -23 55.0 -5.2 14 355 -- 1.7 00:47 1 761 0.99991 888 1919 50 123 69.0 -5.4 18 9 -- 2.2 03:17 8 570 0.99998 198 1295 -20 11 60.5 -4.9 18 3 -- 1.5 04:23 7 555 0.99995 493 1528 -20 200 65.6 -4.6 19 14 -- 1.7 ---=== WEATHER LOG ===--- Wind Time Temp F Dewp F MPH Direction Dust DIMM Sky 15:58 24 4 3 293 (WNW) 112 - Clear 16:30 23 7 5 270 (W) 162 - " 17:02 22 7 0 202 (SSW) 165 - " 17:32 22 5 13 339 (NNW) 132 - " 18:02 22 5 8 327 (NNW) 103 - " 18:35 22 5 12 342 (NNW) 96 - " 19:07 23 5 9 349 (NNW) 104 - " 19:37 22 6 14 307 (NW) 111 - " 20:40 22 6 10 356 (NNW) 106 - " 21:12 22 6 15 342 (NNW) 113 - " 21:44 22 5 13 349 (NNW) 91 - " 22:16 22 5 8 339 (NNW) 99 - " 22:49 21 6 6 19 (NNE) 78 - " 23:19 21 5 16 3 (N) 103 - " 23:51 21 4 12 351 (NNW) 85 - " 00:21 21 0 18 354 (NNW) 91 - " 01:01 22 -7 16 350 (N) 65 - " 02:01 23 -20 15 6 (N) 37 - " 03:01 23 -18 24 20 (NNE) 43 - " 03:22 22 -17 19 1 (N) 75 - " 03:52 22 -17 23 20 (NNE) 64 - " 04:23 23 -16 18 13 (NNE) 75 - " 04:54 24 -13 18 9 (N) 90 - " 05:27 25 -13 14 22 (NNE) 91 - " ---=== TELESCOPE STATUS ===--- UT 02:00 Fans on, doors opened. 02:30 Enclosure off. 12:40 Enclosure on, fans off. Status at end of night 05:45 MST / 12:45 UT: Telescope stowed at: 30 deg Instrument mounted: Engineering camera Counterweights at: 130 (but balance not checked) Autofill systems: On 180L LN2 dewar weights: Spectro 140 lb Imager 109 lb Interlocks bypassed: Slip detection, building motion, roll-up door kick-back. IR Sky Camera LN2 dewar was filled at ~ 00:30, 05:00 and 10:00 UT. ---=== SOFTWARE USED ===--- IOP/SOP: v3_64_3a Watcher: v2_13_2 MCP: v5_8_5 TPM: v2_4_5 AstroDa: v14_40 TCC: 2.5.5 sdssProcedures: v1_34 ---=== MIRROR NUMBERS ===--- NB: MIGs ok, Galils were still updating. PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A 0.0720 0.0750 Axial B 0.8010 0.8020 Axial C 0.0700 0.0710 Trans D -1.4920 -1.4860 Lateral E 1.8590 1.8540 Lateral F 1.4290 1.4240 GALILS Commanded: -24050. -33150. -28500. -200. 31650. 30700. Actual: 5400. -3700. 900. -200. 31550. 30650. SETMIR VALUES PriDesOrient: 0.00 -11.80 23.00 1256.90 642.10 PriOrient: 0.00 -12.16 22.81 1257.41 642.19 SECONDARY: ---------- Focus: 0.00 Air Temp.: -5.2 Alt.: 29.999996 MIGS TONIGHT NOMINAL Axial A 1.5470 1.5500 Axial B 1.2330 1.2260 Axial C 1.2490 1.2480 Trans D 0.2410 0.2320 GALILS Commanded: 3721281. 3688695. 3700915. -6150. -5500. Actual: 1599620. 1565254. 1586350. -10395. -9766. SETMIR VALUES SecDesOrient: 1257.00 -3.40 -10.00 0.00 133.82 SecOrient: 1257.01 -3.36 -9.98 -0.23 133.67 ---=== PROBLEMS IN DETAIL ===--- Problem #1 - Large fiducial errors All fiducial corrcetions before about 20:39 (murmur time) were within bounds, then at the end of a slew to instrument change, we saw first a large altitude error, then errors on rot and az. This pattern - big errors after a slew, especially to instrument change - continued all night. Below are sample messages from the watcher: altitude (correction for altitude 3000 is too large (Dec 22 20:33:12): Dec 22 20:33:24 MST ) azimuth (correction for azimuth -2433 is too large (Dec 22 21:09:51): Dec 22 21:10:02 MST ) rotator (correction for rotator 800288 is too large (Dec 22 20:29:18): Dec 22 20:29:30 MST ) This last is not spurious, as several more errors with similar values were recorded. Below are the last good correction and first (real) bad correction from the tLatch section of logViewer: Altitude: 20:18:37 dsc_I_trcInfo alt fiducial 45.77 deg 20:18:37 dsc_I_trcInfo err = 153 poserr = 153 ticks 20:26:43 dsc_I_trcInfo Applying correction -44 to azimuth ... 20:33:12 dsc_I_trcInfo alt fiducial 45.76 deg 20:33:12 dsc_I_trcInfo err = -3000 poserr = -3000 ticks 20:33:12 dsc_I_trcInfo Applying correction 3000 to altitude 20:33:12 dsc_E_trcErr correction for altitude 3000 is too large Rotator: (the rotator fiducal errors are quite confused in the murmur log around 20:58) 20:58:20 dsc_I_trcInfo rotator MS.ON 20:58:20 dsc_I_trcInfo Applying correction 46 to rotator ... 21:09:52 dsc_I_trcInfo rot fiducial 33 (214.01) deg 21:09:52 dsc_I_trcInfo err = 1063 poserr = 1102 ticks 21:09:52 dsc_I_trcInfo Applying correction -1102 to rotator Azimuth: 21:06:54 dsc_I_trcInfo az fiducial 45.71 deg 21:06:54 dsc_I_trcInfo err = -411 poserr = 42 ticks 21:09:43 dsc_I_trcInfo Applying correction -42 to azimuth ... 21:09:51 dsc_I_trcInfo az fiducial 45.72 deg 21:09:51 dsc_I_trcInfo err = 2433 poserr = 2433 ticks 21:09:51 dsc_I_trcInfo Applying correction -2433 to azimuth 21:09:51 dsc_E_trcErr correction for azimuth -2433 is too large