From: Russet McMillan
Subject: 3.5m Night Log 20071018
Date: Fri, 19 Oct 2007 06:36:57 -0600 (MDT)
                        Apache Point Observatory
                        3.5m Telescope Night Log
                       Thursday, October 18, 2007
                                                         ACTUAL
 ASTRONOMER     OBSERVER(S)              INSTRUMENT    START  FINISH
 -------------------------------------------------------------------
 T.Murphy       R.McMillan               APOLLO        18:30  19:35
 R.Walterbos    R.Walterbos,C.Wu,        DIS           19:40  06:35
                S.Cisneros
   INST     SCHED(TWILIGHT) WEATH     EQUIP      OBS     DAY  NOT USED
 I/ROW/DN   START  FINISH   D    B    D    B    D    B   USE   D    B
 ---------------------------------------------------------------------
 UW15/O     18:35  19:35    0    0    0   0.1   0   0.3  0.7   0    0
 NM04/O     19:35  06:17    0    0    0    0   6.4  4.3   0    0    0
  note: scheduled times listed include instrument change time
                ------------- ACTIVITY LOG --------------
 Observing Specialist: R.McMillan
 A nice clear night with poor and variable seeing first half,
 moderate seeing second half.
 I had trouble calibrating gratings during checkout because the SN
 program backup was still running, and hven was sometimes very slow
 to respond so it would take more than a minute to display an image.
 The backups began at 09:00 and didn't finish until 21:20, so it's
 really not good that they're slowing down our servers and network so
 much for >12 hours a day.
 During telescope checkout, I had multiple axis inits fail because of
 BadAxisDTime on various axes.  Even after successful inits, there
 were intermittent BadAxisDTime errors showing up on all axes at
 alarmingly high rates.
 While APOLLO was running, the telescope spontaneously stopped
 tracking twice (00:42 UT and 01:06 UT) with "read timed out on LTA3"
 -- apparently the azimuth wasn't responding to status queries.  This
 happened during normal tracking with no slew and (I think) no
 offsets, once with and once without the laser running.  Both of
 these errors apparently involved the azimuth, but we were still
 seeing the many BadAxisDTime errors on all axes, so I was suspecting
 that either communications in general were messed up, or else the
 TCC computer was failing.  Jon Brinkmann and I discussed whether to
 do a hot swap of the TCC computer, but decided not to since we
 couldn't reach Russell.
 When the SN backup finished, the frequent BadAxisDTime errors also
 stopped.  There were no repeats of the tracking stops either, and it
 is plausible that the network load from the backup was somehow
 slowing communications between the axis controllers and the TCC.  In
 future, we should require that backups end or be stopped by start of
 observing.  (I did consider this tonight, but I wasn't sure it would
 make a difference -- now we know).
 Once the frantic first few hours were over, we had no real problems
 for the remainder of the night.  The NA2 guider did freeze up once,
 but it restarted without difficulty and no time was lost.
                       ------- FAILURE LOG -------
  PROG   INST   FAILURE MODE     TIME
       (SEDFNVOG)   TI/SHU    START  FINISH  DESCRIPTION
 ----------------------------------------------------------------------
 UW15     A          T/S      18:42 18:46    read timed out on LTA3
 UW15     A          T/S      19:06 19:08    read timed out on LTA3
               ---------------- FOCUS LOG ---------------
 Time  Instrument    Focus    Az  El  Temp Strc Prim  Sec  Air  filt FWHM
 -----------------------------------------------------------------------
               ------------- WEATHER LOG ----------------
                                   Wind                      Precip.or
     Time   Temp F Dewp F  MPH  Direction    Dust    Sky    Conditions
    -------------------------------------------------------------------
     16:00    44    21      6   326   (NW)   1219    clear     none
     17:00    45    20      3    15  (NNE)   1356    clear     none
     18:00    45    17      7   336  (NNW)   1045    clear     none
     19:00    45    16     16     9    (N)    944    clear     none
     20:00    46    15     13    13  (NNE)    886    clear     none
     21:00    46    15     12    16  (NNE)    926    clear     none
     22:00    45    12     21    15  (NNE)    745    clear     none
     23:00    44    12     10     4    (N)    602    clear     none
     00:00    44    11     13    26  (NNE)    477    clear     none
     01:00    44    11      2    94    (E)    423    clear     none
     02:00    44    11      7     7    (N)    396    clear     none
     03:00    48    18     14    31  (NNE)    135    clear     none
     04:00    49    15     25    22  (NNE)    106    clear     none
     05:00    48    18     18    15  (NNE)    111    clear     none
     06:00    47    19     17    38   (NE)    110    clear     none
     Note: the wind was coming from the azimuth listed.
     The convention used is north=0 degrees, east=90 degrees.
     The dust count is particles > 1u per 0.1 cubic feet.
                              TELESCOPE STATUS
      Telescope drives operational. Current TCC version: 2.9.1
      Current TUI version: TUI 1.4.1