Subject: obs log 20000304 (mjd 51608)
Date: Sun,  5 Mar 2000 06:22:38 -0700 (MST)
From: hayes@apo.nmsu.edu (Jeffrey Hayes)
[mail to: obs-report-25m@apo.nmsu.edu
          include MJD in subject]
                         Saturday Mar 04 2000
                 Apache Point Observatory SDSS 2.5m
                            Observing Log
                             (MJD 51608)
OBSERVING TEAM:
===============
Jeffrey Hayes (APO)  shift 2
Ellyne Kinney (APO)  shift 2
Scot Kleinman (APO)  shift 1
Atsuko Nitta Kleinman (APO) shift 1
Bing Chen (JHU) PT
VISITORS:
=========
Andrew Connolly (UP)
French Leger (hon. APO)
Steve Kent (FNAL)
Ron Rechenmacher (FNAL) - phone
THE PLAN:
=========
Spectroscopy all night unless a miracle occurs. Start of night had
some high clouds with more on the way  according to weather maps
and reports.
NIGHT IN A NUTSHELL:
====================
Spectroscopy: do as may spectro plates as we can as in the real survey.
We obtained 2 plates tonight -- 267 and 279.  Both were declared done, so
we only have 1997 left to do.  At 02:30 the wind exceeded our closure limits
and at 03:30, the clouds came in.  We called it a night at astronomical
twilight (roughly 05:10).
SYNOPSIS FOR NIGHT:
===================
Starting in the late afternoon and continuing into the early
evening, French and Atsuko put every spec. cartridge on the
telescope and adjusted the slitheads. We took guider flats of each
one and confirmed good focus in all of them.  A few were bad to
begin with.  Cartridges 6, and 9, which are currently disabled
(mechanically challenged?) were not adjusted in this way.
cables as instructed after the cartridge mount).  After initial setup
and an attempt to take a bias, we got the following critical
errors:
 vme,sd1,dsc_E_trcErr fatal - _PutFrameData status=0x8000c63c sp1Da
 vme,sd1,sd1,dscE_trrErr fatal - no more dq buffers - ccd=1 sp1Da
Since we have no idea what these mean other than something is wrong
with sd1, we rebooted it and tried again. Repeated attempts at this
(and downgrading to sop v3_9_0 which ran last night, and plugging
in the sensor cables) failed, so we called in Ron.  Eventually
he found a workaround although we don't understand why the system
is in the state where it's needed.  Although, the crate thinks the
spectro images are gangs, it was writing them as if they were
frames. The sd1 pool is configured for 100byte frames and 9.5MB
gangs, hence it would die after writing 100bytes of each image.
The work around is to do a manual poolInit (either after or instead
of the initNight -poolInit) of:
poolInit 1 1 9500000 9500000
By 21:00, we had the problem fixed and wentToField.
We found a problem with startGuider, fixed it and filed a PR.
We will have to cut a new version of sop before future sop operations
(Observers: or source ~sjnk/guiderMon.tcl in the current version of
sop.) Andy cvs checked in the corrected version.
We started our first science exposure at 22:01. FWHMs started out
around 2.00 (nothing to get too excited about, but better than 3-4
certainly!).
We declared the plate (267) done at 23:21. (Our focus was at 1500.)
At 23:35, we started slewing for cartridge 7, plate 279. The new focus
was 1300.  At 01:25 we were reading out the 3rd integration when sd1 had
problems.  We did a "ptvmeTest" on node 4 (sd1) and found the following
message:
errno =  5: read  (fd=3, 0x7F7F2BD0, 0x00000004) = -1 (I/O error)
      CSB status: 0x80430010 AUX = (Timeout - LAN idle > 150 msec - target
inited? or VMEbus error)
                             CMP = (Command Completed With Error Status)
                             INNA = 0
We tried a "ptvmeReset" which seemed to work, but then when we tried to
do a "goStare 0 -bias -init" we still had the message:
sp1> goStare 0 -bias -init
Sun Mar 05 01:44:29 MST 2000: statusEntry daqBusy on node sd1  is not 0 but
1
sp1 cameraDetach {} sdss-ts-spectro.apo.nmsu.edu 2600
sp2 cameraDetach {} sdss-ts-spectro.apo.nmsu.edu 2500
Error: Aborting: system is already busy
As second "ptvmeTest" gave the same message as before.  Ellyne then
rebooted sd1.
We retried the "goStare 0 -bias -init" after the reboot and got:
sp1> goStare 0 -bias -init
CAMERA: FAILURE on ftelnet: connection error in ftelnet: connection refused
CAMERA: FAILURE on ftelnet: connection error in ftelnet: connection refused
sp1 cameraDetach {} sdss-ts-spectro.apo.nmsu.edu 2600
sp2 cameraDetach {} sdss-ts-spectro.apo.nmsu.edu 2500
Error: cannot take control of sp1 camera line
We then logged into sdss-ts-spectro and logged out all the ports.  The
"goStare 0 -bias -init" then worked fine.
We continued with a new 3rd integration as previous one was lost when
sd1 was rebooted.
We declared plate 279 done at 02:20 with a focus of 1300.
The next cartridge used was number 8/plate 292.  However, by the time we
slewed to the field, the wind started gusting to 41 mph, and so we had to
close at 03:00.
We ran endNight 51608 (S Kent predicted failure), and to be sure, he was
right.  endNight complained about there being no idReport-51608.par file, and
indded there was none, as was to be expected as we did no imaging.  There
as an sdReport-51608.par file, and when we tried an "endNight 51608 -doSpectro"
we had the same error message.  A PR was filed.  Both the current and
the latest versions of sop/iop (v3_10_0 and V3_8_8 respectively)
have the same problem.
TELESCOPE STATUS:
=================
Telescope closed at 03:00, and finally put to bed at 05:30.
DIMM:
-----
At 00:15, seeing was 1.4 arc-secs.
At 01:15, seeing was 2.5 arc-secs.
At 03:30, seeing was 1.5 arc-secs.
INTERLOCKS:
===========
None bypassed.
SOFTWARE USED:
==============
IOP/SOP: v3_10_0
Watcher: v2_6_7
MCP: v4_0_1
WEATHER SYNOPSIS (see below for hourly data):
=============================================
Generally fair (i.e. partly cloudy), with gusty winds to 25 mph.
After about 02:30, gusts over the closure limits.  At 04:00 thick
cirrus, re-enforcing the closure.  Winds and clouds continued till
day-break.
GENERAL COMMENTS (see Problems below for details):
==================================================
DATA ACQUISITION (see IOP/SOP output below):
===========================================
IMAGING:
--------
SPECTROSCOPY:
-------------
See above general comments for the description of the night's activities.
MIRRORS:
========
PRIMARY:
--------
MIGS                   SHOULD BE
Axial A= 1.9315          1.9315
Axial B= 0.5559          0.5570
Axial C= 0.5559          0.5570
Transverse D= 0.4089     0.4075
Lateral E= 1.8985        1.8980
Lateral F= 0.3984        0.3990
GALILS
Commanded: 4850 -2350 100 -9500 50 -100
Actual:  4850 -2350 100 -9500 50 -100
SECONDARY:
----------
Focus: 2250 ?
Temp: 5.3C
Altitude: 30  (before any TCC slews)
MIGS
Axial A= -0.0609
Axial B= -0.0459
Axial C= -0.0649
Transverse D= 0.1040
GALILS
Commanded: 2807200 2794350 2784050 -6750 -11250
Actual:  2830889 2821403 2805491 -6750 -11250
LN2 SCHEDULE:
=============
Spectrographs filled at 19:00.
Imager dewar replaced around 20:00. The old one was about half full,
but showed pressure of only 10 or so- hence the imager was complaining.
The old dewar is sitting outside the dewar house along with one other
low pressure dewar that was in my way in getting to the full ones.
Spectrographs filled at 06:00.
PROBLEMS:
=========
SOP:DiamondPoint:
-----------------
sp1> diamondPoint 1.5 60 -az_alt 1
Sat Mar 04 23:09:50 MST 2000: Set frames to Rows=(512+1) and
Cols/2=(20+1024+20)  Binning=(1,4)
sp1 camstat {} sdss-ts-spectro.apo.nmsu.edu 2500
sp2 camstat {} sdss-ts-spectro.apo.nmsu.edu 2500
setupCamera {SPEC1 31JAN00 $Name: v4_5 $ \n \n}
sdss-ts-spectro.apo.nmsu.edu 2600
sp1 setupCamera {} sdss-ts-spectro.apo.nmsu.edu 2600
setupCamera {SPEC2 31JAN00 $Name: v4_5 $ \n \n}
sdss-ts-spectro.apo.nmsu.edu 2500
sp2 setupCamera {} sdss-ts-spectro.apo.nmsu.edu 2500
Sat Mar 04 23:10:06 MST 2000: Setting sp1 up for data taking
Sat Mar 04 23:10:06 MST 2000: Setting sp2 up for data taking
sp1 camstat {} sdss-ts-spectro.apo.nmsu.edu 2600
sp2 camstat {} sdss-ts-spectro.apo.nmsu.edu 2500
Error: called "murmur" with too many arguments
sp1> Modu="mir_Adjust"; Text="Cannot compute PrimOrient"
sp1> tb
called "murmur" with too many arguments
while executing
"murmur "Requested binned dimensions of camera" $camName  "does not
match camstat values" $toScreen"
    invoked from within
"if {[expr $cameraData(binned_lines)] != [expr [cameraRowsBin]] ||
[expr $cameraData(binned_pixels)] != [expr [cameraPixelsBin]] } {
          murmur  ..."
    ("foreach" body line 2)
    invoked from within
"foreach camName [activeInstruments] {
       if {[expr $cameraData(binned_lines)] != [expr
[cameraRowsBin]] ||  [expr $cameraData(binned_pixels)] != [ ..."
    (procedure "init4Bin" line 55)
    invoked from within
"init4Bin $binit"
    (procedure "diamondPoint" line 40)
    invoked from within
"diamondPoint 1.5 60 -az_alt 1"
WEATHER DATA:
=============
                               Wind
     Time   Temp F Dewp F  MPH   Direction    Dust      Sky Precip.
     16:01    51    17     14   213  (SSW)    509
     17:01    46    16     17   203  (SSW)    459
     18:01    44    11     14   218  (SW)     465
     19:01    43    11      9   208  (SSW)    556
     20:01    44     9     12   187  (S)      471
     21:01    42     5     11   179  (S)      516
     22:01    42    15     12   148  (SSE)    569
     23:01    41    17     19   127  (SE)     536
     00:01    43    12     19   144  (SE)     500
     01:01    43    14     21   154  (SSE)    525
     02:01    40    17     29   144  (SE)     562
     03:01    43    14     21   175  (S)      612
     04:01    41    13     24   194  (SSW)    482
     05:01    41    14     34   195  (SSW)    578
DATA LOGS FROM IOP/SOP:
=======================