Date: Wed, 21 Sep 2005 12:08:24 GMT From: observer@apo.nmsu.edu Subject: 2.5m obslog 20050920 (MJD 53634) APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Tuesday September 20, 2005 (MJD 53634) ---=== OBSERVING TEAM ===--- Swing: John Barentine Night: Mike Harvanek Support: Hubert Lampeitl (FNAL) Steph Snedden Kaike Pan Eric Neilsen (by phone) ---=== OBSERVING PLAN ===--- SN Science! ---=== OBSERVING SUMMARY ===--- We imaged 82 S from lambda +110 to -122 (including ramps) under mostly clear (but not photometric) skies and in mostly bad seeing with lots of moon. ---=== OBSERVING LOG ===--- Afternoon --------- No issues during checkout. Checked telescope motion via TCC slews as a final test of the new iop version (v4_4_0). Everything looks fine. Went to check spectro focus and ran intro trouble with the spectro DA. Details in the Problems section. Night ----- 01:45Z Run 5618: On-telescope bias in twilight for 37 frames. 02:10Z Run 5619: 82 S. Pointing good from the beginning (lambda +110). Seeing greater than 2" and varying substantially. Focus loop having a hard time keeping up due to seeing variations. In focus around frame 31 (lambda +114) but focus loop is struggling and we are going in and out of focus. Twilight ended around the same time (frame 31, lambda +114). The sky was clear but the cloud camera indicated that it was not photometric. The seeing became consistently . % Execution halted at: REMOVE2REDO 257 /local/ups/prd/Linux/idlspec2d/v4_10_7/pro/apo2d/remove2redo.pro % $MAIN$ None of the files from the postCalib or goSpecFocus appeared on disk. SOP hung with a message about waiting for the read to finish. I killed the SOP Session and started it up again, but could not grab the instruments. Steph came to help and we found that the master server had died. Restarted the servers. We also manually logged out the spectro camera ports via telnet. Restarted SOP again and took a bias, which read out but never made it to disk. We then rebooted the spectro crate. After it came back up, we tried another initNight and a bias, at which point we were warned that the numbering would be wrong (which we knew) and the master server died again. Restarted that and carried on, although frames kept appearing in the pool but not making it to disk. Seeing how tonight was designated for SN, we checked to make sure imager bias frames were being archived and written to disk properly. During a test bias (Run 5617) we noted a discrepancy between the date/time stamps on gangs and frames. Example: 5615 18.0 -gcra-- 388800 20-Sep-2005 17:48:58.40 TAI 5610 426.21 --c-a-- 5792416 15-Sep-2005 06:11:00.94 TAI 5610 426.31 --c-a-- 5792416 15-Sep-2005 06:11:00.94 TAI 5610 426.41 --c-a-- 5792416 15-Sep-2005 06:11:00.94 TAI 5610 426.51 --c-a-- 5792416 15-Sep-2005 06:11:00.94 TAI 5610 427.11 --c-a-- 5792416 15-Sep-2005 06:11:36.77 TAI 5615 19.0 -gcra-- 388800 20-Sep-2005 17:49:34.22 TAI 5610 427.21 --c-a-- 5792416 15-Sep-2005 06:11:36.77 TAI 5610 427.31 --c-a-- 5792416 15-Sep-2005 06:11:36.77 TAI 5610 427.41 --c-a-- 5792416 15-Sep-2005 06:11:36.77 TAI 5610 427.51 --c-a-- 5792416 15-Sep-2005 06:11:36.77 TAI 5610 428.11 --c-a-- 5792416 15-Sep-2005 06:12:12.59 TAI 5615 20.0 -gcra-- 388800 20-Sep-2005 17:50:10.05 TAI i.e., the gangs have the correct date and the frames have an incorrect date. We tried forcing a new teamster, since the spectro biases were clearly in the pool, but no dice. At last, we issued a frames2Unix which *did* start moving frames out of the pool, but this confounds us because starting a new teamster is supposed to call frames2Unix.