Date: Thu, 13 Jun 2024 08:53:38 -0600
Subject: Re: [sloan-25mlog 7291] Re: 2.5m obslog 20240612 (SJD 60474)
From: John Downey
Camera problem is fixed.  It was the serial connection.  The big
giveaway on that was that the camera controller would not connect and
commanded fill sequences would not execute, but the camera temperature
regulation was still working.  If the cameras are at -100C and -125C
respectively instead of roughly -190C then you know the camera
controller is alive and regulating the CCD temperatures.
On 6/13/24 06:26, John Downey via sloan-25mlog wrote:
> I expect the spectro problem is also heat related.
>
> We can look at the option of increasing chiller flow through the
> spectro heat exchanger since it's plumbing is large enough for that to
> work.  We can also look at the option of leaning out the glycol
> mixture in the summer to improve it's heat capacity.
>
> On 6/13/24 06:11, kpan--- via sloan-25mlog wrote:
>>                         APACHE POINT OBSERVATORY SDSS 2.5M
>>
>>                                   OBSERVING LOG
>>                            Wednesday June 12, 2024
>>
>>                                    (SJD 60474)
>>
>> ----  Observing Team ----
>>
>> Kaike Pan
>>
>> John Downey (on the phone)
>>
>>
>> ---- Observing Plan ----
>>
>> Wed 12 Jun 2024    60474   0.41   03:15   03:34  10:32    13:58
>> 20:58   ---     bright until 06:58 ( 4.4 h), then dark ( 3.6 h)
>>
>>
>> ---- Observing Summary ----
>>
>> The night started with a clear sky but poor seeing (2.0-3.0", partly
>> due to low altitudes of observed fields, from 30 to 45 degrees).
>> Clouds rolled in as soon as dark time started at 07:00Z, and seeing
>> improved to 1.2-1.8 (dark fields observed at higher altitudes too).
>> It is completely clouded out by 07:18Z after completed the 1st dark
>> design. stars came back at 7:45Z with good seeing (1.0-1.5").
>>
>> Notes:
>> 1: Communication to SP1Cam was lost at 8:05Z. After some efforts (see
>> problems in detail section), we decided to manually fill both r1 and
>> b1 camera every two hours, and observed bright designs with only
>> apogee data for the rest of the night. 1 hours was lost to the problem.
>>
>> Without communication to sp1cam, autofill will not work, manual fill
>> was done at 3 AM, 5 AM. I plan to do another fill at 6:30 AM before I
>> go to bed.
>>
>> 2. Robot air temperature raised above 40 C at 9:30Z. I powered off
>> FPS at 9:40Z after complete the ongoing design (temperature was at
>> 40.2C), turned it back on at 10:15C when the temperature dropped
>> below 30 C. 35 minutes was lost to the issue.
>>
>> 3. One collision was reported on robot 995 tonight.
>>
>> 4. LUI automatical summary fails to get design type info tonight,
>> fails to get info for designs having only APOGEE data.
>>
>> ---- Automatic Summary ----
>>
>> Time     Field-Design-Conf  DesignType BOSS/APOGEE Seeing Overhead
>> Complete? Comment
>> 02:36:27 101204-615687-14845  ........... 1x730/1xAB  2.1" 0:00  Done
>> 03:00:47 101204-615688-14846  ........... 1x730/1xAB  2.4" 4:24. Done
>> 03:17:32 101204-615689-14847  ........... 1x730/1xAB  2.4" 4:47. Done
>> 03:34:36 101204-615690-14848  ........... 1x730/1xAB  1.9" 5:20  Done
>> 03:52:05 101775-617607-14849  ........... 1x730/1xAB  2.4" 6:51. Done
>> 04:11:08 101775-617608-14850  ........... 1x730/1xAB  2.3" 5:10. Done
>> 04:28:29 101775-617609-14851  ........... 1x730/1xAB  1.5" 4:53. Done
>> 04:45:31 101775-617610-14852  ........... 1x730/1xAB  2.1" 6:01. Done
>> 05:03:45 103239-621558-14853  ........... 1x730/1xAB  2.3" 8:42 Done
>> added hartmann
>> 05:24:42 103240-621630-14854  ........... 1x730/1xAB  1.9" 6:09 Done
>> 05:42:57 100552-612958-14855  ........... 1x730/1xAB  2.1" 5:55 Done
>> 06:01:04 100570-613604-14856  ........... 2x730/2xAB  3.4" 5:59 Done
>> 06:51:24 100107-610515-14858  ........... 1x900/1xAB  1.5" 7:21 Done
>> added hartmann
>> 06:51:24 100107-610515-14858  ........... 1x900/1xAB  1.5" 7:21 Done
>> 07:13:44 100117-610523-14859  ........... 1x900/1xAB  1.1" 34:41 Done
>> wait for Clouds moving out
>> 09:06:41 103235-621410-14861. bright.     1XAB 70:00 Done SP1cam lost
>> communication
>> 10:15:02 100477-611683-14862. bright      1XAB 45:00 Done powered off
>> FPS due to high temp
>> 10:37:16 100498-612008-14863  bright. 1XAB                    Done
>> 10:54:58 100538-612625-14864. bright. 1XAB                    Done
>>
>> Notes: LUI automatical summary fails to get design type info, fails
>> to get info for designs having only APOGEE data.
>>
>> ---- Engineering Issues ----
>>
>> 1. Communication to sp1cam was lost at 8:00Z, could not re-establish
>> it. Without the communication, autofill to cameras fails (as tested
>> by startfillSeg). After discussed with John, we decided to manually
>> fill cameras every two hours until day staff come in. More
>> information in problems in details section.sp1MicroNotTalking and
>> camCheck.stale alerts are currently disabled.
>>
>> 2. The ambient temperature was at 23.5 C in the evening, the robot
>> temperature reaches 40.1C at 9:30Z, after 7.5 hours operations. The
>> ambient air temperature is at 19.7 C at 9:30Z. I decided to power off
>> FPS to let robots cool down a little. I recalled that we removed the
>> IEB cover last year during June and July which helped cool down
>> robots. Should we do the same?
>>
>>
>> ---- Observing Log ----
>>
>> Afternoon:
>> BOSS collimation was checked, no adjustment made.
>> 00:31:13  Hartmann collimate output
>>     sp1: offset: r= -989 (In Focus);  b= -0.2 (In Focus)
>>     sp1: pred. move: spAverageMove= -462
>>     sp1: pred. resid: r= -526, b= -1.7, txt= OK, spTemp = 10.8
>> motorPosition
>>       sp1:    6334,    6333,    6322;
>> ----------
>> Median spectrographs temperatures:
>>       Tsp1= 10.9
>>       Outside air temperature= 23.7;
>> Both APOGEE evening cal and BOSS cal were taken, checked. They look
>> good.
>> There is no bright spot in fvc dark.
>> exercised fvc loop and robots, they behavior normally.
>> 0:46:22 OBSERVER.kpan 75 jaeger d
>> fvc_filename=/data/fcam/60474/fimg-fvc1n-0001.fits
>> 00:46:29 OBSERVER.kpan 75 jaeger i fvc_centroid_method=sep
>> 00:46:29 OBSERVER.kpan 75 jaeger i fvc_rms=41.61
>> 00:46:29 OBSERVER.kpan 75 jaeger i fvc_perc_90=60.3
>> 00:46:29 OBSERVER.kpan 75 jaeger i fvc_percent_reached=3.2
>>
>> Started ventilation 01:05Z.
>> Night:
>> =====
>> 02:12Z: open.
>> Per John Donor's email, observing queue was regenerated right before
>> the first field at 02:25Z.
>>
>> 00:26:20 apogee exposure  file= apRaw-49120001 3  B Dark
>> 00:26:41 boss exposure  0 sec  file=0370147 bias
>> 00:28:01 boss exposure  20 sec  file=0370148 dark
>> 00:28:33 apogee exposure  file= apRaw-49120002 60  B Dark
>> 00:29:11 boss exposure  4 sec  file=0370149 arc
>> 00:29:53 boss exposure  4 sec  file=0370150 arc
>>   00:31:10 Hartmann: offset r= -989 (In Focus) b = -0.2 (In Focus)
>> pred_move = -462 pred_resid r= -526 b= -1.7 cmnt=OK
>> 00:34:32 boss exposure  25 sec  file=0370151 flat
>> 00:39:25 apogee exposure  file= apRaw-49120003 10  B QuartzFlat
>> 00:41:17 apogee exposure  file= apRaw-49120004 12  B ArcLamp
>> 00:43:30 apogee exposure  file= apRaw-49120005 40  B ArcLamp
>> 00:43:35 boss exposure  4 sec  file=0370152 arc
>>
>>
>> =======================================================
>> =====>>>>> 00:45:19 Config Loaded: -999--999-14844
>> =======================================================
>>
>> 00:46:36 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0001.fits
>> 00:46:29  fvc_perc_90= 60.3 fvc_percent_reached= 3.2 fvc_rms= 41.61
>> 00:50:47 apogee exposure  file= apRaw-49120006 12  A ArcLamp
>> 00:52:59 apogee exposure  file= apRaw-49120007 40  A ArcLamp
>> 01:00:25 apogee exposure  file= apRaw-49120008 60  A Dark
>> 01:11:17 apogee exposure  file= apRaw-49120009 30  A ArcLamp
>> 01:16:40 apogee exposure  file= apRaw-49120010 30  B ArcLamp
>> 02:21:39 apogee exposure  file= apRaw-49120011 5  B DomeFlat
>> 02:23:10 apogee exposure  file= apRaw-49120012 5  B DomeFlat
>> 02:24:30 apogee exposure  file= apRaw-49120013 5  B DomeFlat
>> 02:25:26 boss exposure  4 sec  file=0370153 arc
>> 02:26:06 boss exposure  4 sec  file=0370154 arc
>>   02:27:23 Hartmann: offset r= -659 (In Focus) b = -2.2 (In Focus)
>> pred_move = 24 pred_resid r= -683 b= -2.1 cmnt=OK
>> 02:28:01 boss exposure  25 sec  file=0370155 flat
>> 02:29:48 boss exposure  25 sec  file=0370156 flat
>> 02:31:36 boss exposure  25 sec  file=0370157 flat
>>
>>
>> =======================================================
>> =====>>>>> 02:36:27 Config Loaded: 101204-615687-14845
>> =======================================================
>>
>> 02:38:42 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0003.fits
>> 02:38:23  fvc_perc_90= 80.8 fvc_percent_reached= 0.6 fvc_rms= 59.02
>> 02:39:11 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0004.fits
>> 02:39:02  fvc_perc_90= 11.9 fvc_percent_reached= 51.8 fvc_rms= 17.49
>> 02:39:54 boss exposure  4 sec  file=0370158 arc
>> 02:48:10 apogee exposure  file= apRaw-49120014 35  B Object
>> 02:48:34 boss exposure  730 sec  file=0370159 science
>> 02:54:31 apogee exposure  file= apRaw-49120015 35  A Object
>>
>>
>> =======================================================
>> =====>>>>> 03:00:47 Config Loaded: 101204-615688-14846
>> =======================================================
>>
>> 03:03:24 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0005.fits
>> 03:03:06  fvc_perc_90= 79.1 fvc_percent_reached= 0.6 fvc_rms= 57.49
>> 03:03:54 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0006.fits
>> 03:03:45  fvc_perc_90= 11.2 fvc_percent_reached= 51.6 fvc_rms= 15.8
>> 03:04:44 apogee exposure  file= apRaw-49120016 35  A Object
>> 03:05:08 boss exposure  730 sec  file=0370160 science
>> 03:11:00 apogee exposure  file= apRaw-49120017 35  B Object
>>
>>
>> =======================================================
>> =====>>>>> 03:17:32 Config Loaded: 101204-615689-14847
>> =======================================================
>>
>> 03:20:23 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0007.fits
>> 03:20:05  fvc_perc_90= 80.5 fvc_percent_reached= 0.6 fvc_rms= 58.07
>> 03:20:52 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0008.fits
>> 03:20:43  fvc_perc_90= 11.6 fvc_percent_reached= 53.2 fvc_rms= 29.49
>> 03:21:51 apogee exposure  file= apRaw-49120018 35  A Object
>> 03:22:06 boss exposure  730 sec  file=0370161 science
>> 03:28:07 apogee exposure  file= apRaw-49120019 35  B Object
>>
>>
>> =======================================================
>> =====>>>>> 03:34:36 Config Loaded: 101204-615690-14848
>> =======================================================
>>
>> 03:37:27 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0009.fits
>> 03:37:09  fvc_perc_90= 76.2 fvc_percent_reached= 1.0 fvc_rms= 56.76
>> 03:37:56 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0010.fits
>> 03:37:47  fvc_perc_90= 11.5 fvc_percent_reached= 46.4 fvc_rms= 25.18
>> 03:39:22 apogee exposure  file= apRaw-49120020 35  A Object
>> 03:39:37 boss exposure  730 sec  file=0370162 science
>> 03:45:37 apogee exposure  file= apRaw-49120021 35  B Object
>>
>>
>> =======================================================
>> =====>>>>> 03:52:05 Config Loaded: 101775-617607-14849
>> =======================================================
>>
>>
>> 03:54:45 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0011.fits
>> 03:54:29  fvc_perc_90= 73.8 fvc_percent_reached= 0.8 fvc_rms= 51.91
>> 03:55:14 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0012.fits
>> 03:55:05  fvc_perc_90= 12.0 fvc_percent_reached= 46.7 fvc_rms= 34.0
>> 03:55:57 boss exposure  4 sec  file=0370163 arc
>> 03:58:24 apogee exposure  file= apRaw-49120022 35  A Object
>> 03:58:39 boss exposure  730 sec  file=0370164 science
>> 04:04:39 apogee exposure  file= apRaw-49120023 35  B Object
>>
>>
>> =======================================================
>> =====>>>>> 04:11:08 Config Loaded: 101775-617608-14850
>> =======================================================
>>
>> 04:13:29  fvc_perc_90= 74.2 fvc_percent_reached= 1.2 fvc_rms= 50.65
>> 04:13:49 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0013.fits
>> 04:14:16 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0014.fits
>> 04:14:07  fvc_perc_90= 11.5 fvc_percent_reached= 49.9 fvc_rms= 13.89
>> 04:15:44 apogee exposure  file= apRaw-49120024 35  A Object
>>
>> 04:16:00 boss exposure  730 sec  file=0370165 science
>> 04:22:00 apogee exposure  file= apRaw-49120025 35  B Object
>>
>>
>> =======================================================
>> =====>>>>> 04:28:29 Config Loaded: 101775-617609-14851
>> =======================================================
>>
>> 04:31:19 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0015.fits
>> 04:31:01  fvc_perc_90= 76.7 fvc_percent_reached= 0.6 fvc_rms= 52.77
>> 04:31:49 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0016.fits
>> 04:31:40  fvc_perc_90= 10.3 fvc_percent_reached= 50.3 fvc_rms= 8.02
>> 04:32:47 apogee exposure  file= apRaw-49120026 35  A Object
>> 04:33:05 boss exposure  730 sec  file=0370166 science
>> 04:39:03 apogee exposure  file= apRaw-49120027 35  B Object
>>
>>
>> =======================================================
>> =====>>>>> 04:45:31 Config Loaded: 101775-617610-14852
>> =======================================================
>>
>> 04:48:36 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0017.fits
>> 04:48:20  fvc_perc_90= 76.3 fvc_percent_reached= 0.8 fvc_rms= 53.95
>> 04:49:06 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0018.fits
>> 04:48:56  fvc_perc_90= 11.4 fvc_percent_reached= 47.5 fvc_rms= 26.05
>> 04:51:01 apogee exposure  file= apRaw-49120028 35  A Object
>> 04:51:16 boss exposure  730 sec  file=0370167 science
>> 04:57:17 apogee exposure  file= apRaw-49120029 35  B Object
>>
>>
>> =======================================================
>> =====>>>>> 05:03:45 Config Loaded: 103239-621558-14853
>> =======================================================
>>
>> 05:07:15 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0019.fits
>> 05:06:56  fvc_perc_90= 84.3 fvc_percent_reached= 0.2 fvc_rms= 60.43
>> 05:07:43 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0020.fits
>> 05:07:25 boss exposure  4 sec  file=0370168 arc
>> 05:07:34  fvc_perc_90= 11.2 fvc_percent_reached= 46.2 fvc_rms= 8.43
>> 05:08:06 boss exposure  4 sec  file=0370169 arc
>>   05:09:22 Hartmann: offset r= 2638 (Out of focus) b = -3.2 (In
>> Focus) pred_move = 1833 pred_resid r= 805 b= 2.5 cmnt=OK
>> 05:09:53 boss exposure  4 sec  file=0370170 arc
>> 05:11:54 apogee exposure  file= apRaw-49120030 35  A Object
>> 05:12:09 boss exposure  730 sec  file=0370171 science
>> 05:18:10 apogee exposure  file= apRaw-49120031 35  A Object
>>
>>
>> =======================================================
>> =====>>>>> 05:24:42 Config Loaded: 103240-621630-14854
>> =======================================================
>>
>> 05:27:26 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0021.fits
>> 05:27:08  fvc_perc_90= 82.8 fvc_percent_reached= 0.4 fvc_rms= 57.91
>> 05:27:55 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0022.fits
>> 05:27:46  fvc_perc_90= 12.0 fvc_percent_reached= 48.4 fvc_rms= 8.09
>> 05:28:38 boss exposure  4 sec  file=0370172 arc
>> 05:30:13 apogee exposure  file= apRaw-49120032 35  A Object
>> 05:30:29 boss exposure  730 sec  file=0370173 science
>> 05:36:29 apogee exposure  file= apRaw-49120033 35  B Object
>>
>>
>> =======================================================
>> =====>>>>> 05:42:57 Config Loaded: 100552-612958-14855
>> =======================================================
>>
>> 05:45:33 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0023.fits
>> 05:45:14  fvc_perc_90= 85.7 fvc_percent_reached= 1.4 fvc_rms= 61.01
>> 05:46:01 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0024.fits
>> 05:45:52  fvc_perc_90= 11.3 fvc_percent_reached= 52.6 fvc_rms= 41.13
>> 05:46:44 boss exposure  4 sec  file=0370174 arc
>> 05:48:21 apogee exposure  file= apRaw-49120034 35  A Object
>> 05:48:35 boss exposure  730 sec  file=0370175 science
>> 05:54:36 apogee exposure  file= apRaw-49120035 35  B Object
>>
>>
>> =======================================================
>> =====>>>>> 06:01:04 Config Loaded: 100570-613604-14856
>> =======================================================
>>
>> 06:03:40 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0025.fits
>> 06:03:22  fvc_perc_90= 84.0 fvc_percent_reached= 0.4 fvc_rms= 57.93
>> 06:04:10 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0026.fits
>> 06:04:01  fvc_perc_90= 12.1 fvc_percent_reached= 45.4 fvc_rms= 9.19
>>
>> 06:04:53 boss exposure  4 sec  file=0370176 arc
>> 06:06:30 apogee exposure  file= apRaw-49120036 35  A Object
>> 06:06:45 boss exposure  730 sec  file=0370177 science
>> 06:12:45 apogee exposure  file= apRaw-49120037 35  B Object
>> 06:20:16 apogee exposure  file= apRaw-49120038 35  B Object
>> 06:20:40 boss exposure  730 sec  file=0370178 science
>> 06:26:37 apogee exposure  file= apRaw-49120039 35  A Object
>>
>>
>> =======================================================
>> =====>>>>> 06:32:55 Config Loaded: 100568-613481-14857
>> =======================================================
>>
>> 06:35:42 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0027.fits
>> 06:35:23  fvc_perc_90= 81.9 fvc_percent_reached= 0.4 fvc_rms= 59.74
>> 06:36:10 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0028.fits
>> 06:36:01  fvc_perc_90= 11.9 fvc_percent_reached= 37.3 fvc_rms= 26.11
>> 06:36:53 boss exposure  4 sec  file=0370179 arc
>> 06:38:39 apogee exposure  file= apRaw-49120040 35  A Object
>> 06:39:05 boss exposure  730 sec  file=0370180 science
>> 06:44:56 apogee exposure  file= apRaw-49120041 35  B Object
>>
>>
>> =======================================================
>> =====>>>>> 06:51:24 Config Loaded: 100107-610515-14858
>> =======================================================
>>
>> 06:54:04 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0029.fits
>> 06:53:46  fvc_perc_90= 78.9 fvc_percent_reached= 0.0 fvc_rms= 58.29
>> 06:54:33 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0030.fits
>> 06:54:20 boss exposure  4 sec  file=0370181 arc
>> 06:54:24  fvc_perc_90= 11.0 fvc_percent_reached= 49.2 fvc_rms= 15.51
>> 06:55:01 boss exposure  4 sec  file=0370182 arc
>>   06:56:17 Hartmann: offset r= 329 (In Focus) b = 1.8 (In Focus)
>> pred_move = -124 pred_resid r= 453 b= 1.4 cmnt=OK
>> 06:56:45 boss exposure  4 sec  file=0370183 arc
>> 06:58:13 apogee exposure  file= apRaw-49120042 43  B Object
>> 06:58:38 boss exposure  900 sec  file=0370184 science
>> 07:05:59 apogee exposure  file= apRaw-49120043 43  A Object
>>
>>
>> =======================================================
>> =====>>>>> 07:13:44 Config Loaded: 100117-610523-14859
>> =======================================================
>>
>> 07:16:18 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0031.fits
>> 07:15:59  fvc_perc_90= 81.1 fvc_percent_reached= 0.4 fvc_rms= 61.81
>> 07:16:48 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0032.fits
>> 07:16:39  fvc_perc_90= 13.1 fvc_percent_reached= 44.9 fvc_rms= 32.96
>> 07:17:33 boss exposure  4 sec  file=0370185 arc
>>
>> 07:47:55 apogee exposure  file= apRaw-49120044 43  A Object
>> 07:48:19 boss exposure  900 sec  file=0370186 science
>> 07:55:36 apogee exposure  file= apRaw-49120045 43  B Object
>>
>>
>> =======================================================
>> =====>>>>> 08:03:24 Config Loaded: 104486-624979-14860
>> =======================================================
>>
>> 08:06:11 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0033.fits
>> 08:05:53  fvc_perc_90= 70.7 fvc_percent_reached= 0.6 fvc_rms= 47.78
>> 08:06:40 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0034.fits
>> 08:06:31  fvc_perc_90= 9.7 fvc_percent_reached= 57.6 fvc_rms= 7.08
>>
>>
>> =======================================================
>> =====>>>>> 09:06:41 Config Loaded: 103235-621410-14861
>> =======================================================
>>
>> 09:09:31 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0035.fits
>> 09:09:13  fvc_perc_90= 81.0 fvc_percent_reached= 1.2 fvc_rms= 58.5
>> 09:09:59 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0036.fits
>> 09:09:50  fvc_perc_90= 10.3 fvc_percent_reached= 49.5 fvc_rms= 10.13
>> 09:12:34 apogee exposure  file= apRaw-49120046 46  B Object
>> 09:20:52 apogee exposure  file= apRaw-49120047 46  A Object
>>
>>
>>
>> =======================================================
>> =====>>>>> 10:15:02 Config Loaded: 100477-611683-14862
>> =======================================================
>>
>> 10:22:33 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0037.fits
>> 10:22:15  fvc_perc_90= 76.7 fvc_percent_reached= 0.2 fvc_rms= 55.2
>> 10:23:01 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0038.fits
>> 10:22:52  fvc_perc_90= 11.4 fvc_percent_reached= 50.8 fvc_rms= 15.43
>> 10:24:44 apogee exposure  file= apRaw-49120048 35  A Object
>> 10:31:01 apogee exposure  file= apRaw-49120049 35  B Object
>>
>>
>>
>> =======================================================
>> =====>>>>> 10:37:16 Config Loaded: 100498-612008-14863
>> =======================================================
>>
>> 10:39:46  fvc_perc_90= 77.0 fvc_percent_reached= 0.6 fvc_rms= 54.78
>> 10:40:06 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0039.fits
>> 10:40:35 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0040.fits
>> 10:40:26  fvc_perc_90= 11.3 fvc_percent_reached= 52.2 fvc_rms= 72.37
>> 10:41:53 apogee exposure  file= apRaw-49120050 35  B Object
>> 10:48:14 apogee exposure  file= apRaw-49120051 35  B Object
>>
>>
>> =======================================================
>> =====>>>>> 10:54:58 Config Loaded: 100538-612625-14864
>> =======================================================
>>
>> 10:57:34 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0041.fits
>> 10:57:16  fvc_perc_90= 79.4 fvc_percent_reached= 0.6 fvc_rms= 53.68
>> 10:58:03 FVC exposure /data/fcam/60474/proc-fimg-fvc1n-0042.fits
>> 10:57:54  fvc_perc_90= 10.9 fvc_percent_reached= 52.9 fvc_rms= 9.16
>> 10:59:21 apogee exposure  file= apRaw-49120052 35  A Object
>> 11:05:38 apogee exposure  file= apRaw-49120053 35  B Object
>> 11:12:44 apogee exposure  file= apRaw-49120054 5  B DomeFlat
>> 11:14:03 apogee exposure  file= apRaw-49120055 5  B DomeFlat
>> 11:15:23 apogee exposure  file= apRaw-49120056 5  B DomeFlat
>> 11:24:39 apogee exposure  file= apRaw-49120057 60  B Dark
>> 11:35:17 apogee exposure  file= apRaw-49120058 60  B Dark
>> 11:45:55 apogee exposure  file= apRaw-49120059 60  B Dark
>>
>>
>> ---- Mirror Numbers ----
>>
>>
>> PRIMARY:
>> --------
>> Scale: 1.000000
>>
>> MIGS        TONIGHT    NOMINAL
>> Axial A:     0.2000     0.1970
>> Axial B:    -0.0010    -0.0012
>> Axial C:     0.0650     0.0530
>> Trans D:     0.4210     0.4100
>> Lateral E:     0.3380     0.3380
>> Lateral F:     0.5455     0.5455
>>
>> GALILS
>> Commanded:  9800 -300 6750 19150 -3550 -3250
>> Actual:     9800 -300 6750 19150 -3550 -3250
>>
>> SETMIR VALUES
>> PrimDesOrient: 0.0 -18.5 -7.8 -427.0 -998.0
>> PrimOrient:    0.0 -18.23 -7.9 -427.17 -999.0
>>
>> SECONDARY:
>> ----------
>>
>> Focus:      0.0000
>> Air temp:     0.00
>> Altitude:   30.0
>>
>> MIGS        TONIGHT    NOMINAL
>> Axial A:    -1.0400    -1.0380
>> Axial B:     0.2200     0.2180
>> Axial C:    -1.1100    -1.1050
>> Trans D:    -0.0240    -0.0330
>>
>> GALILS
>> Commanded: 5886150 5811550 5946950 -1500 -7250
>> Actual:    5883807 5811189 5906087 -1500 -7250
>>
>> SETMIR VALUES
>> SecDesOrient: 4657.0 -24.5 -17.5 18.0 125.41
>> SecOrient:    4656.89 -24.58 -17.7 17.56 125.31
>>
>>
>>
>> ---- Telescope Offsets and Scale ----
>>
>> Time      Field-Design     Az  Alt    Rot RA Off Dec Off Rot Off
>> Guide RMS (um)
>> ================================================================================
>>
>> 03:04:44 101204-615688  144.5 35.0  -55.3  0.005   0.008
>> 0.099          0.544
>> 03:21:51 101204-615689  144.3 37.1  -51.7  0.004   0.008
>> 0.096          0.452
>> 03:39:22 101204-615690  144.3 39.3  -47.8  0.004   0.009
>> 0.091          0.407
>> 03:58:24 101775-617607  155.0 32.6  -39.8  0.007   0.007
>> 0.172          0.591
>> 04:15:00 101775-617608  154.8 34.0  -36.7  0.007   0.008
>> 0.170          0.735
>> 04:32:47 101775-617609  154.7 35.6  -32.0  0.006   0.008
>> 0.164          0.383
>> 04:51:01 101775-617610  154.8 37.3  -27.5  0.005   0.009
>> 0.157          0.769
>> 05:11:54 103239-621558   52.1 32.2  249.9 -0.006   0.012
>> -0.077         36.137
>> 05:30:00 103240-621630   45.3 34.1  245.9 -0.006   0.013
>> -0.080          0.363
>> 05:45:00 100552-612958   55.5 38.7  253.0 -0.006   0.014
>> -0.079          0.646
>> 06:00:48 100552-612958   51.7 41.4  250.3 -0.006   0.013
>> -0.075          0.519
>> 06:18:58 100570-613604   43.9 40.7  243.8 -0.006   0.014
>> -0.079          0.486
>> 06:38:39 100568-613481   32.0 45.8  233.6 -0.005   0.016
>> -0.078          0.856
>> 06:58:13 100107-610515   30.0 76.1  238.0 -0.005   0.016
>> -0.068          0.598
>> 07:13:37 100107-610515   15.6 77.3  225.3 -0.004   0.015
>> -0.069          0.347
>> 07:47:55 100117-610523  -30.3 76.1  183.2 -0.003   0.013
>> -0.078          5.079
>> 08:03:14 100117-610523  -42.6 74.1  172.7 -0.002   0.012
>> -0.075          0.219
>> 09:12:34 103235-621410  -26.3 40.3  185.4 -0.000   0.011
>> -0.074          0.345
>> 09:29:01 103235-621410  -31.2 38.6  181.3  0.000   0.010
>> -0.076          0.342
>> 10:24:44 100477-611683  -49.8 59.0  170.3 -0.003   0.011
>> -0.082         40.424
>> 10:41:53 100498-612008  -53.3 51.7  161.3 -0.003   0.009
>> -0.085          3.489
>> 10:59:21 100538-612625  -57.6 40.1  160.1 -0.002   0.008
>> -0.077          0.353
>> 11:15:00 100538-612625  -61.0 37.4  158.0 -0.002   0.007
>> -0.078          0.483
>>
>>
>>
>>
>> ---- Focus Log ----
>>
>> Time      Field-Design      M1      M2 Focus     Az   Alt  Temp Wind Dir
>> ================================================================================
>>
>> 03:04:44 101204-615688    0.00 4425.92  -210  144.5  35.0 0.0    1 136
>> 03:21:51 101204-615689    0.00 4412.98  -215  144.3  37.1 0.0    6  92
>> 03:39:22 101204-615690    0.00 4426.08  -193  144.3  39.3 0.0    1 206
>> 03:58:24 101775-617607    0.00 4408.50  -238  155.0  32.6 0.0    2 229
>> 04:15:00 101775-617608    0.00 4371.00  -269  154.8  34.0 0.0    1 164
>> 04:32:47 101775-617609    0.00 4391.50  -242  154.7  35.6 0.0    2 178
>> 04:51:01 101775-617610    0.00 4361.45  -265  154.8  37.3 0.0    3 194
>> 05:11:54 103239-621558    0.00 4360.39  -287   52.1  32.2 0.0    2 171
>> 05:30:00 103240-621630    0.00 4373.82  -266   45.3  34.1 0.0    5 183
>> 05:45:00 100552-612958    0.00 4366.51  -255   55.5  38.7 0.0    2 196
>> 06:00:48 100552-612958    0.00 4355.89  -255   51.7  41.4 0.0    1 162
>> 06:18:58 100570-613604    0.00 4395.52  -218   43.9  40.7 0.0    3 141
>> 06:38:39 100568-613481    0.00 4433.91  -181   32.0  45.8 0.0    2 126
>> 06:58:13 100107-610515    0.00 4353.43  -169   30.0  76.1 0.0    2 173
>> 07:13:37 100107-610515    0.00 4327.24  -194   15.6  77.3 0.0    4 196
>> 07:47:55 100117-610523    0.00 4268.75  -254  -30.3  76.1 0.0    7 217
>> 08:03:14 100117-610523    0.00 4206.68  -318  -42.6  74.1 0.0    5 196
>> 09:12:34 103235-621410    0.00 4296.52  -318  -26.3  40.3 0.0    9 214
>> 09:29:01 103235-621410    0.00 4336.02  -285  -31.2  38.6 0.0    6 199
>> 10:24:44 100477-611683    0.00 4292.17  -263  -49.8  59.0 0.0    6 194
>> 10:41:53 100498-612008    0.00 4324.12  -239  -53.3  51.7 0.0   10 216
>> 10:59:21 100538-612625    0.00 4388.69  -227  -57.6  40.1 0.0    9 225
>> 11:15:00 100538-612625    0.00 4420.88  -205  -61.0  37.4 0.0    5 257
>>
>>
>>
>>
>> ---- Telescope Status ----
>>
>>
>>   01:10Z    Started ventilation
>>   02:15Z    encl25m : open
>>   11:20Z    encl25m : closed
>>
>> No interlocks bypassed
>>
>> Status at: 12:10Z
>> Telescope stowed at:    121.0, 30.0,   0.0  mount
>> Instrument mounted:  FPS
>> Counterweights at:  691.7, 694.2, 711.4, 711.4
>> LN2 autofill systems: Disconnected
>> 180L LN2 dewar scale:  SP1  132.0 lbs,    0.1 psi
>> Chiller Flow: 7.21 L/min to FPS, 2.18 L/min to GFAs, 5.0/6.0 gpm total
>> Chiller Temp: 16.4C
>> No chiller alarms
>>
>>
>>
>> ---- Software Used ----
>>
>>    -- STUI software:
>> STUI          : 1.7.18b1
>> alerts        : 2.3.0
>> boss          : 6.0.2
>> -boss.specMec : 3.0.4
>> apo           : 4.0.1a0
>> fliswarm      : 0.5.1a0
>> jaeger        : 1.7.4
>> kaiju         : 1.4.0
>> coordio       : 1.11.2
>> fps_calibrati : 2024.04.01+91998f8
>> cherno        : 1.0.0
>> astrometrynet : 0.93
>> hal           : 1.3.4
>> hartmann      : 2.0.2
>> hub           : 5.0.1a0
>> mcp           : 2.1-1-ge461
>> -mcp.plc      : 35
>> -mcp.azFiduci : v1_149
>> -mcp.altFiduc : v1_149
>> -mcp.rotFiduc : v1_149
>> tcc           : 1.4.0
>> apogee        : not availble
>> apogeecal     : not availble
>> apogeeql      : v1_7_0dev
>> STUIAdditions : FAILED
>> Boson         : 0.4.9
>> sdss-obstools       : 3.9.3
>> TPM                 : 1.2-31-gd7eb
>> MCP                 : 2.1-1-ge461
>> idlspec2d           : v6_1_3
>> Kronos              : 1.7.0
>> roboscheduler       : 1.4.0
>>
>> /dev/sdb1                2.0T  461G  1.6T  23% /home
>> /dev/sdd1                 22T   20T  1.2T  95% /data
>>
>>
>>
>> ---- APOGEE Data Summary ----
>>
>> Using nfowler=1 for DOMEFLAT
>> Using nfowler=1 for DOMEFLAT
>> Using nfowler=1 for DOMEFLAT
>> Using nfowler=1 for DOMEFLAT
>> Using nfowler=1 for DOMEFLAT
>> Using nfowler=1 for DOMEFLAT
>>    num    field  design config   exptype    dithpix nread relflux
>> okay    dx      dy    sigx   sigy  fpi1 fpi2
>> -------- ------ ------ ------ ------------ ------- ----- ------- ----
>> ------- ------- ------ ------ ---- ----
>> 49120001 100570 592571  14843         DARK    13.5     3      33
>> True      --      --     --     --   --   --
>> 49120002 100570 592571  14843         DARK    13.5    60     -33
>> True      --      --     --     --   --   --
>> 49120003 100570 592571  14843   QUARTZFLAT    13.5    10     106
>> True      --   -0.26     --   0.86   --   --
>> 49120004 100570 592571  14843 ARCLAMP-THAR    13.5    12      99
>> True    1.10    0.55   1.06   0.89   --   --
>> 49120005 100570 592571  14843  ARCLAMP-UNE    13.5    40      94
>> True    0.69    0.39   1.08   0.88   --   --
>> 49120006   -999   -999  14844 ARCLAMP-THAR    13.0    12      92
>> True    0.53    0.59   1.07   0.87   --   --
>> 49120007   -999   -999  14844  ARCLAMP-UNE    13.0    40      94
>> True    0.14    0.37   1.11   0.89   --   --
>> 49120008   -999   -999  14844         DARK    13.0    60       3
>> True      --      --     --     --   --   --
>> 49120009   -999   -999  14844          FPI    13.0    30     135
>> True   11.99    0.33   1.16   0.88   --   --
>> 49120010   -999   -999  14844          FPI    13.5    30     136
>> True  -12.08    0.33   1.17   0.88   --   --
>> 49120011   -999   -999  14844     DOMEFLAT    13.5     5     114
>> True      --   -0.26     --   0.86   --   --
>> 49120012   -999   -999  14844     DOMEFLAT    13.5     5     112
>> True      --   -0.24     --   0.86   --   --
>> 49120013   -999   -999  14844     DOMEFLAT    13.5     5     109
>> True      --   -0.24     --   0.86   --   --
>> 49120014 101204 615687  14845       OBJECT    13.5    35     111
>> True    1.09   -0.26   1.75   0.87  101  100
>> 49120015 101204 615687  14845       OBJECT    13.0    35     118
>> True    0.55   -0.26   1.75   0.90  106  101
>> 49120016 101204 615688  14846       OBJECT    13.0    35     123
>> True    0.55   -0.26   1.76   0.90  106  102
>> 49120017 101204 615688  14846       OBJECT    13.5    35     121
>> True    1.09   -0.26   1.75   0.87  102   99
>> 49120018 101204 615689  14847       OBJECT    13.0    35     116
>> True    0.55   -0.26   1.76   0.90  104  102
>> 49120019 101204 615689  14847       OBJECT    13.5    35     111
>> True    1.07   -0.24   1.75   0.86  100  103
>> 49120020 101204 615690  14848       OBJECT    13.0    35     114
>> True    0.55   -0.26   1.76   0.91  104  101
>> 49120021 101204 615690  14848       OBJECT    13.5    35     114
>> True    1.09   -0.24   1.74   0.86   99  100
>> 49120022 101775 617607  14849       OBJECT    13.0    35     136
>> True    0.55   -0.25   1.72   0.90  105  101
>> 49120023 101775 617607  14849       OBJECT    13.5    35     127
>> True    1.08   -0.25   1.71   0.87  100  100
>> 49120024 101775 617608  14850       OBJECT    13.0    35     130
>> True    0.55   -0.25   1.71   0.90  103  102
>> 49120025 101775 617608  14850       OBJECT    13.5    35     130
>> True    1.08   -0.25   1.74   0.87  100  101
>> 49120026 101775 617609  14851       OBJECT    13.0    35     130
>> True    0.56   -0.25   1.72   0.91  103  100
>> 49120027 101775 617609  14851       OBJECT    13.5    35     134
>> True    1.08   -0.24   1.71   0.88  101   98
>> 49120028 101775 617610  14852       OBJECT    13.0    35     141
>> True    0.56   -0.25   1.72   0.91  103   99
>> 49120029 101775 617610  14852       OBJECT    13.5    35     138
>> True    1.08   -0.24   1.72   0.88   98   99
>> 49120030 103239 621558  14853       OBJECT    13.0    35     144
>> True    0.55   -0.25   1.76   0.91  100  102
>> 49120031 103239 621558  14853       OBJECT    13.5    35     132
>> True    1.10   -0.24   1.74   0.88  100  109
>> 49120032 103240 621630  14854       OBJECT    13.0    35     133
>> True    0.56   -0.25   1.72   0.91  100  102
>> 49120033 103240 621630  14854       OBJECT    13.5    35     124
>> True    1.09   -0.24   1.71   0.88   98  103
>> 49120034 100552 612958  14855       OBJECT    13.0    35     119
>> True    0.56   -0.25   1.75   0.90  103  101
>> 49120035 100552 612958  14855       OBJECT    13.5    35     109
>> True    1.08   -0.23   1.71   0.87   99  105
>> 49120036 100570 613604  14856       OBJECT    13.0    35     100
>> True    0.54   -0.24   1.76   0.90  102  102
>> 49120037 100570 613604  14856       OBJECT    13.5    35      95
>> True    1.07   -0.23   1.75   0.87  101  101
>> 49120038 100570 613604  14856       OBJECT    13.5    35      92
>> True    1.07   -0.23   1.75   0.86  101  104
>> 49120039 100570 613604  14856       OBJECT    13.0    35      90
>> True    0.54   -0.24   1.75   0.90  101  102
>> 49120040 100568 613481  14857       OBJECT    13.0    35      83
>> True    0.54   -0.24   1.75   0.90  102  103
>> 49120041 100568 613481  14857       OBJECT    13.5    35      76
>> True    1.07   -0.22   1.75   0.86  101  102
>> 49120042 100107 610515  14858       OBJECT    13.5    43      49
>> True    1.06   -0.22   1.74   0.86  102  102
>> 49120043 100107 610515  14858       OBJECT    13.0    43      45
>> True    0.54   -0.22   1.75   0.89  102  105
>> 49120044 100117 610523  14859       OBJECT    13.0    43      40
>> True    0.52   -0.22   1.75   0.85  104  105
>> 49120045 100117 610523  14859       OBJECT    13.5    43      39
>> True    1.07   -0.21   1.74   0.85  105  104
>> 49120046 103235 621410  14861       OBJECT    13.5    46      48
>> True    1.05   -0.21   1.72   0.87  104  102
>> 49120047 103235 621410  14861       OBJECT    13.0    46      42
>> True    0.56   -0.21   1.72   0.89  106  101
>> 49120048 100477 611683  14862       OBJECT    13.0    35      35
>> True    0.53   -0.20   1.68   0.88  107   41
>> 49120049 100477 611683  14862       OBJECT    13.5    35      43
>> True    1.08   -0.20   1.70   0.86  104   41
>> 49120050 100498 612008  14863       OBJECT    13.5    35      41
>> True    1.07   -0.16   1.75   0.87  104   99
>> 49120051 100498 612008  14863       OBJECT    13.0    35      39
>> True    0.55   -0.20   1.74   0.90  104  102
>> 49120052 100538 612625  14864       OBJECT    13.0    35      51
>> True    0.58   -0.20   1.69   0.89  103   99
>> 49120053 100538 612625  14864       OBJECT    13.5    35      52
>> True    1.09   -0.16   1.76   0.88  103   98
>> 49120054 100538 612625  14864     DOMEFLAT    13.5     5     113
>> True      --   -0.18     --   0.86   --   --
>> 49120055 100538 612625  14864     DOMEFLAT    13.5     5     110
>> True      --   -0.18     --   0.85   --   --
>> 49120056 100538 612625  14864     DOMEFLAT    13.5     5     110
>> True      --   -0.19     --   0.84   --   --
>> 49120057 100538 612625  14864         DARK    13.5    60      13
>> True      --      --     --     --   --   --
>>    num     exptype   nread mtp1 mtp2 mtp3 mtp4 mtp5 mtp6 mtp7 mtp8
>> mtp9 mtp10
>> -------- ---------- ----- ---- ---- ---- ---- ---- ---- ---- ----
>> ---- -----
>> 49120003 QUARTZFLAT    10   69   74   83   92   93   88   88 91  
>> 91    83
>> 49120011   DOMEFLAT     5   80   90   98  109  116  103  108 114 
>> 109   103
>> 49120012   DOMEFLAT     5   80   90   98  107  114  101  107 112 
>> 107   102
>> 49120013   DOMEFLAT     5   80   90   96  106  113  100  105 110 
>> 106   101
>> 49120054   DOMEFLAT     5   80   89   96  105  113   99  104 110 
>> 106   101
>> 49120055   DOMEFLAT     5   80   89   95  104  113   98  103 109 
>> 105   100
>> 49120056   DOMEFLAT     5   81   90   95  105  113   98  104 110 
>> 106   102
>>
>> MJD   UTC       Field-Design-Config Exposure Type         Dith Reads
>> Arch
>> --------------------------------------------------------------------------------
>>
>> 60474 00:26:20 100570-592571-14843  49120001 Dark B        3 a-b-c
>> 60474 00:28:33 100570-592571-14843  49120002 Dark B       60 a-b-c
>> 60474 00:39:25 100570-592571-14843  49120003 Quartzflat B       10 a-b-c
>> 60474 00:41:17 100570-592571-14843  49120004 ThAr Arc B       12 a-b-c
>> 60474 00:43:30 100570-592571-14843  49120005 UNe Arc B       40 a-b-c
>> 60474 00:50:47   -999-  -999-14844  49120006 ThAr Arc A       12 a-b-c
>> 60474 00:52:59   -999-  -999-14844  49120007 UNe Arc A       40 a-b-c
>> 60474 01:00:25   -999-  -999-14844  49120008 Dark A       60 a-b-c
>> 60474 01:11:17   -999-  -999-14844  49120009 FPI Lamp A       30 a-b-c
>> 60474 01:16:40   -999-  -999-14844  49120010 FPI Lamp B       30 a-b-c
>> 60474 02:21:39   -999-  -999-14844  49120011 Domeflat B        5 a-b-c
>> 60474 02:23:10   -999-  -999-14844  49120012 Domeflat B        5 a-b-c
>> 60474 02:24:30   -999-  -999-14844  49120013 Domeflat B        5 a-b-c
>> 60474 02:48:10 101204-615687-14845  49120014 Object B       35 a-b-c
>> 60474 02:54:31 101204-615687-14845  49120015 Object A       35 a-b-c
>> 60474 03:04:44 101204-615688-14846  49120016 Object A       35 a-b-c
>> 60474 03:11:00 101204-615688-14846  49120017 Object B       35 a-b-c
>> 60474 03:21:51 101204-615689-14847  49120018 Object A       35 a-b-c
>> 60474 03:28:06 101204-615689-14847  49120019 Object B       35 a-b-c
>> 60474 03:39:21 101204-615690-14848  49120020 Object A       35 a-b-c
>> 60474 03:45:37 101204-615690-14848  49120021 Object B       35 a-b-c
>> 60474 03:58:24 101775-617607-14849  49120022 Object A       35 a-b-c
>> 60474 04:04:40 101775-617607-14849  49120023 Object B       35 a-b-c
>> 60474 04:15:44 101775-617608-14850  49120024 Object A       35 a-b-c
>> 60474 04:22:00 101775-617608-14850  49120025 Object B       35 a-b-c
>> 60474 04:32:47 101775-617609-14851  49120026 Object A       35 a-b-c
>> 60474 04:39:03 101775-617609-14851  49120027 Object B       35 a-b-c
>> 60474 04:51:01 101775-617610-14852  49120028 Object A       35 a-b-c
>> 60474 04:57:16 101775-617610-14852  49120029 Object B       35 a-b-c
>> 60474 05:11:54 103239-621558-14853  49120030 Object A       35 a-b-c
>> 60474 05:18:10 103239-621558-14853  49120031 Object B       35 a-b-c
>> 60474 05:30:13 103240-621630-14854  49120032 Object A       35 a-b-c
>> 60474 05:36:29 103240-621630-14854  49120033 Object B       35 a-b-c
>> 60474 05:48:20 100552-612958-14855  49120034 Object A       35 a-b-c
>> 60474 05:54:36 100552-612958-14855  49120035 Object B       35 a-b-c
>> 60474 06:06:29 100570-613604-14856  49120036 Object A       35 a-b-c
>> 60474 06:12:45 100570-613604-14856  49120037 Object B       35 a-b-c
>> 60474 06:20:16 100570-613604-14856  49120038 Object B       35 a-b-c
>> 60474 06:26:37 100570-613604-14856  49120039 Object A       35 a-b-c
>> 60474 06:38:39 100568-613481-14857  49120040 Object A       35 a-b-c
>> 60474 06:44:56 100568-613481-14857  49120041 Object B       35 a-b-c
>> 60474 06:58:13 100107-610515-14858  49120042 Object B       43 a-b-c
>> 60474 07:05:59 100107-610515-14858  49120043 Object A       43 a-b-c
>> 60474 07:47:55 100117-610523-14859  49120044 Object A       43 a-b-c
>> 60474 07:55:36 100117-610523-14859  49120045 Object B       43 a-b-c
>> 60474 09:12:34 103235-621410-14861  49120046 Object B       46 a-b-c
>> 60474 09:20:51 103235-621410-14861  49120047 Object A       46 a-b-c
>> 60474 10:24:44 100477-611683-14862  49120048 Object A       35 a-b-c
>> 60474 10:31:01 100477-611683-14862  49120049 Object B       35 a-b-c
>> 60474 10:41:53 100498-612008-14863  49120050 Object B       35 a-b-c
>> 60474 10:48:14 100498-612008-14863  49120051 Object A       35 a-b-c
>> 60474 10:59:21 100538-612625-14864  49120052 Object A       35 a-b-c
>> 60474 11:05:38 100538-612625-14864  49120053 Object B       35 a-b-c
>> 60474 11:12:44 100538-612625-14864  49120054 Domeflat B        5 a-b-c
>> ThAr Offsets: ['-0.56']
>> UNe Offsets: ['-0.56']
>> Object Offsets: Max: 0.57, Min: 0.49, Mean: 0.53
>>
>>
>> ---- BOSS Data Summary ----
>>
>> MJD   UTC      Field-Design-Config  Exposure Type    SOS   ETime Hart
>> --------------------------------------------------------------------------------
>>
>> 60474 00:26:44 100570-592571-14843  00370147 Bias    r1-b1     0 Closed
>> 60474 00:28:03 100570-592571-14843  00370148 Dark    r1-b1    20 Closed
>> 60474 00:29:50 100570-592571-14843  00370149 Hart    r1-b1     4 Left
>> 60474 00:30:29 100570-592571-14843  00370150 Hart    r1-b1     4 Right
>> 60474 00:34:30 100570-592571-14843  00370151 Flat    r1-b1    25 Open
>> 60474 00:43:34 100570-592571-14843  00370152 Arc     r1-b1     4 Open
>> 60474 02:26:04   -999-  -999-14844  00370153 Hart    r1-b1     4 Left
>> 60474 02:26:44   -999-  -999-14844  00370154 Hart    r1-b1     4 Right
>> 60474 02:27:59   -999-  -999-14844  00370155 Flat    r1-b1    25 Open
>> 60474 02:29:46   -999-  -999-14844  00370156 Flat    r1-b1    25 Open
>> 60474 02:31:34   -999-  -999-14844  00370157 Flat    r1-b1    25 Open
>> 60474 02:39:48 101204-615687-14845  00370158 Arc     r1-b1     4 Open
>> 60474 02:48:27 101204-615687-14845  00370159 Science r1-b1   730 Open
>> 60474 03:05:02 101204-615688-14846  00370160 Science r1-b1   730 Open
>> 60474 03:22:00 101204-615689-14847  00370161 Science r1-b1   730 Open
>> 60474 03:39:31 101204-615690-14848  00370162 Science r1-b1   730 Open
>> 60474 03:55:51 101775-617607-14849  00370163 Arc     r1-b1     4 Open
>> 60474 03:58:33 101775-617607-14849  00370164 Science r1-b1   730 Open
>> 60474 04:15:54 101775-617608-14850  00370165 Science r1-b1   730 Open
>> 60474 04:32:58 101775-617609-14851  00370166 Science r1-b1   730 Open
>> 60474 04:51:10 101775-617610-14852  00370167 Science r1-b1   730 Open
>> 60474 05:08:03 103239-621558-14853  00370168 Hart    r1-b1     4 Left
>> 60474 05:08:44 103239-621558-14853  00370169 Hart    r1-b1     4 Right
>> 60474 05:09:47 103239-621558-14853  00370170 Arc     r1-b1     4 Open
>> 60474 05:12:03 103239-621558-14853  00370171 Science r1-b1   730 Open
>> 60474 05:28:32 103240-621630-14854  00370172 Arc     r1-b1     4 Open
>> 60474 05:30:23 103240-621630-14854  00370173 Science r1-b1   730 Open
>> 60474 05:46:38 100552-612958-14855  00370174 Arc     r1-b1     4 Open
>> 60474 05:48:29 100552-612958-14855  00370175 Science r1-b1   730 Open
>> 60474 06:04:47 100570-613604-14856  00370176 Arc     r1-b1     4 Open
>> 60474 06:06:39 100570-613604-14856  00370177 Science r1-b1   730 Open
>> 60474 06:20:34 100570-613604-14856  00370178 Science r1-b1   730 Open
>> 60474 06:36:47 100568-613481-14857  00370179 Arc     r1-b1     4 Open
>> 60474 06:38:59 100568-613481-14857  00370180 Science r1-b1   730 Open
>> 60474 06:54:59 100107-610515-14858  00370181 Hart    r1-b1     4 Left
>> 60474 06:55:39 100107-610515-14858  00370182 Hart    r1-b1     4 Right
>> 60474 06:56:39 100107-610515-14858  00370183 Arc     r1-b1     4 Open
>> 60474 06:58:31 100107-610515-14858  00370184 Science r1-b1   900 Open
>> 60474 07:17:25 100117-610523-14859  00370185 Arc     r1-b1     4 Open
>> 60474 07:48:13 100117-610523-14859  00370186 Science r1-b1   900 Open
>>
>>
>>
>>
>> ---- BOSS Spectrographs Collimation ----
>>
>> Time      Field-Design   Temp  R off  B off   Move  Resid
>> ================================================================================
>>
>> 00:31:10    nan-nan      10.8   -989   -0.2   -462   -1.7
>> 02:27:23   -999--999     11.4   -659   -2.2     24   -2.1
>> 05:09:22 103239-621558   11.2   2638   -3.2   1833    2.5
>> 06:56:17 100107-610515   10.7    329    1.8   -124    1.4
>>
>>
>> ---- Time Tracking ----
>>
>> *=*=*= Observatory: APO
>> *=*=*= MJD: 60474
>>         Fill night time in Hours below
>> *=*=*= Total bright time                     : 4.4
>> *=*=*= Bright time, queued observations      : 5.0
>> *=*=*= Bright time, commissioning/engineering: 0
>> *=*=*= Bright time lost to weather           : 0
>> *=*=*= Bright time lost to technical issues  : 0
>>
>> *=*=*= Total dark time                       : 3.6
>> *=*=*= Dark time, queued observations        : 0.8
>> *=*=*= Dark time, commissioning/engineering  : 0
>> *=*=*= Dark time lost to weather             : 0.4
>> *=*=*= Dark time lost to technical issues    : 1.8
>>
>>
>>
>>
>> ---- Problems in Detail ----
>>
>> Lost communication to sp1Cam
>>
>> Communication to sp1Cam was lost at 08:05Z. Here are some relevant
>> log messages:
>> 8:07:08 .boss 4429 boss d text="Performing camcheck on sp1cam."
>> 08:07:08 .alerts 0 alerts w alert=boss.camCheck.stale,warn,"at
>> 2024-06-13 08:06:08; no change for 179 sec, with None",enabled,noack,-1
>> 08:07:16 .alerts 0 alerts w alert=boss.camCheck.stale,warn,"at
>> 2024-06-13 08:06:08; no change for 188 sec, with None",enabled,noack,-1
>> 08:07:26 .apo 0 apo w diskspace=sdss5-hub,/data,22438,1099
>> 08:07:28 .alerts 0 alerts w alert=boss.camCheck.stale,warn,"at
>> 2024-06-13 08:06:08; no change for 199 sec, with None",enabled,noack,-1
>> 08:07:47 .tcc 0 tcc W Text="number keys does not match number of values"
>> 08:07:50 .alerts 0 alerts w alert=boss.camCheck.stale,warn,"at
>> 2024-06-13 08:06:08; no change for 221 sec, with None",enabled,noack,-1
>> 08:07:53 OBSERVER.kpan_2 32 hal e error="Command 'boss exposure arc
>> itime=4 noreadout' timed out."
>> 08:08:08 .boss 4431 boss d text="Performing camcheck on sp1cam."
>> 08:08:08 .alerts 0 alerts w alert=boss.camCheck.stale,warn,"at
>> 2024-06-13 08:06:08; no change for 239 sec, with None",enabled,noack,-1
>> 08:08:16 .alerts 0 alerts w alert=boss.camCheck.stale,warn,"at
>> 2024-06-13 08:06:08; no change for 248 sec, with None",enabled,noack,-1
>> 08:08:26 .apo 0 apo w diskspace=sdss5-hub,/data,22438,1100
>> 08:08:28 .alerts 0 alerts w alert=boss.camCheck.stale,warn,"at
>> 2024-06-13 08:06:08; no change for 259 sec, with None",enabled,noack,-1
>> 08:08:41 CmdStarted: OBSERVER.kpan 103 boss ping
>> 08:08:41 OBSERVER.kpan 103 boss d text="Sending sa to sp1mech"
>> 08:08:41 OBSERVER.kpan 103 boss i text="sp1daq: OK"
>> 08:08:42 OBSERVER.kpan 103 boss d text="Completed message from sp1mech"
>> 08:08:42 OBSERVER.kpan 103 boss i text="Response parsed."
>> 08:08:42 OBSERVER.kpan 103 boss i text="sp1mech: OK"
>> 08:08:47 .tcc 0 tcc W Text="number keys does not match number of values"
>> 08:08:50 .alerts 0 alerts w alert=boss.camCheck.stale,warn,"at
>> 2024-06-13 08:06:08; no change for 281 sec, with None",enabled,noack,-1
>> 08:08:55 .alerts 0 alerts w alert=boss.camCheck.stale,warn,"at
>> 2024-06-13 08:06:08; no change for 287 sec, with None",enabled,noack,-1
>> 08:08:59 CmdStarted: OBSERVER.kpan 104 boss reloadControllers
>> 08:08:59 OBSERVER.kpan 104 boss i text="Disconnecting sp1cam..."
>> 08:08:59 OBSERVER.kpan 104 boss i text="Disconnecting sp1mech..."
>> 08:08:59 OBSERVER.kpan 104 boss i text="Disconnecting sp1daq..."
>> 08:08:59 OBSERVER.kpan 104 boss i text="pausing to let the
>> controllers disconnect themselves...."
>> 08:08:59 .boss 0 boss w text="Controller sp1cam disconnected."
>> 08:08:59 .boss 0 boss w text="sp1cam removed from available hardware."
>> 08:08:59 .boss 0 boss w text="Controller sp1mech disconnected."
>> 08:08:59 .boss 0 boss w text="sp1mech removed from available hardware."
>> 08:09:01 OBSERVER.kpan 104 boss i text="Reloading ['sp1cam',
>> 'sp1mech', 'sp1daq'] controllers..."
>> 08:09:01 OBSERVER.kpan 104 boss i text="Attaching sp1cam..."
>> 08:09:01 OBSERVER.kpan 104 boss i text="Attaching sp1mech..."
>> 08:09:01 OBSERVER.kpan 104 boss i text="Attaching sp1daq..."
>> 08:09:01 OBSERVER.kpan 104 boss :
>> 08:09:01 CmdDone: OBSERVER.kpan 104 boss reloadControllers
>> 08:09:03 .boss 0 boss i text="Controller sp1cam connected."
>> 08:09:03 .boss 0 boss d text="Sending CAMCHECK to sp1cam"
>> 08:09:03 .boss 0 boss i text="Controller sp1mech connected."
>> 08:09:08 .boss 0 boss i aliveAt=1718266148
>> 08:09:08 .boss 4433 boss d text="Performing camcheck on sp1cam."
>> 08:09:08 .alerts 0 alerts w alert=boss.camCheck.stale,warn,"at
>> 2024-06-13 08:06:08; no change for 299 sec, with None",enabled,noack,-1
>> 08:09:08 .boss 0 boss w text="Command CAMCHECK to sp1cam timed out"
>> 08:09:16 .alerts 0 alerts w alert=boss.camCheck.stale,warn,"at
>> 2024-06-13 08:06:08; no change for 308 sec, with None",enabled,noack,-1
>> 08:09:20 CmdStarted: OBSERVER.kpan 105 boss ping
>> 08:09:20 OBSERVER.kpan 105 boss d text="Sending sa to sp1mech"
>> 08:09:20 OBSERVER.kpan 105 boss i text="sp1daq: OK"
>> 08:09:21 OBSERVER.kpan 105 boss d text="Completed message from sp1mech"
>> 08:09:21 OBSERVER.kpan 105 boss i text="Response parsed."
>> 08:09:21 OBSERVER.kpan 105 boss i text="sp1mech: OK"
>> 08:09:26 .apo 0 apo w diskspace=sdss5-hub,/data,22438,1100
>> 08:09:28 .alerts 0 alerts w alert=boss.camCheck.stale,warn,"at
>> 2024-06-13 08:06:08; no change for 319 sec, with None",enabled,noack,-1
>> 08:09:47 .tcc 0 tcc W Text="number keys does not match number of values"
>> 08:09:50 CmdStarted: OBSERVER.kpan 106 boss reloadControllers
>> 08:09:50 OBSERVER.kpan 106 boss i text="Disconnecting sp1cam..."
>> 08:09:50 OBSERVER.kpan 106 boss i text="Disconnecting sp1mech..."
>> 08:09:50 OBSERVER.kpan 106 boss i text="Disconnecting sp1daq..."
>> 08:09:50 OBSERVER.kpan 106 boss i text="pausing to let the
>> controllers disconnect themselves...."
>> 08:09:50 .boss 0 boss w text="Controller sp1cam disconnected."
>> 08:09:50 .boss 0 boss w text="sp1cam removed from available hardware."
>> 08:09:50 .boss 0 boss w text="Controller sp1mech disconnected."
>> 08:09:50 .boss 0 boss w text="sp1mech removed from available hardware."
>> 08:09:50 .alerts 0 alerts w alert=boss.camCheck.stale,warn,"at
>> 2024-06-13 08:06:08; no change for 341 sec, with None",enabled,noack,-1
>> 08:09:52 OBSERVER.kpan 106 boss i text="Reloading ['sp1cam',
>> 'sp1mech', 'sp1daq'] controllers..."
>> 08:09:52 OBSERVER.kpan 106 boss i text="Attaching sp1cam..."
>> 08:09:52 OBSERVER.kpan 106 boss i text="Attaching sp1mech..."
>> 08:09:52 OBSERVER.kpan 106 boss i text="Attaching sp1daq..."
>> 08:09:52 OBSERVER.kpan 106 boss :
>> 08:09:52 CmdDone: OBSERVER.kpan 106 boss reloadControllers
>> 08:09:54 .boss 0 boss i text="Controller sp1cam connected."
>> 08:09:54 .boss 0 boss d text="Sending CAMCHECK to sp1cam"
>> 08:09:54 .boss 0 boss i text="Controller sp1mech connected."
>> 08:09:55 .alerts 0 alerts w alert=boss.camCheck.stale,warn,"at
>> 2024-06-13 08:06:08; no change for 347 sec, with None",enabled,noack,-1
>> 08:09:59 .boss 0 boss w text="Command CAMCHECK to sp1cam timed out"
>> 08:10:08 .boss 0 boss i aliveAt=1718266208
>> 08:10:08 .boss 4435 boss d text="Performing camcheck on sp1cam."
>> 08:10:08 .alerts 0 alerts w alert=boss.camCheck.stale,warn,"at
>> 2024-06-13 08:06:08; no change for 359 sec, with None",enabled,noack,-1
>> 08:10:16 .alerts 0 alerts w alert=boss.camCheck.stale,warn,"at
>> 2024-06-13 08:06:08; no change for 368 sec, with None",enabled,noack,-1
>> 08:10:26 .apo 0 apo w diskspace=sdss5-hub,/data,22438,1100
>> 08:10:28 .alerts 0 alerts w alert=boss.camCheck.stale,warn,"at
>> 2024-06-13 08:06:08; no change for 379 sec, with None",enabled,noack,-1
>> 08:10:31 CmdStarted: OBSERVER.kpan 107 boss exposure abort
>> 08:10:31 hal.hal 3793 boss i exposureState=ABORTED,0.0,0.0
>> 08:10:31 hal.hal 3793 boss f text="exposure aborted"
>> 08:10:31 CmdDone: hal.hal 3793 boss exposure arc itime=4 noreadout
>> 08:10:31 OBSERVER.kpan 107 boss :
>> 08:10:32 CmdDone: OBSERVER.kpan 107 boss exposure abort
>> 08:10:32 .boss 0 boss d text="Sending eS to sp1mech"
>> 08:10:33 .boss 0 boss d text="Completed message from sp1mech"
>> 08:10:42 CmdStarted: OBSERVER.kpan 108 boss reloadControllers
>> 08:10:42 OBSERVER.kpan 108 boss i text="Disconnecting sp1cam..."
>> 08:10:42 OBSERVER.kpan 108 boss i text="Disconnecting sp1mech..."
>> 08:10:42 OBSERVER.kpan 108 boss i text="Disconnecting sp1daq..."
>> 08:10:42 OBSERVER.kpan 108 boss i text="pausing to let the
>> controllers disconnect themselves...."
>> 08:10:42 .boss 0 boss w text="Controller sp1cam disconnected."
>> 08:10:42 .boss 0 boss w text="sp1cam removed from available hardware."
>> 08:10:42 .boss 0 boss w text="Controller sp1mech disconnected."
>> 08:10:42 .boss 0 boss w text="sp1mech removed from available hardware."
>> 08:10:44 OBSERVER.kpan 108 boss i text="Reloading ['sp1cam',
>> 'sp1mech', 'sp1daq'] controllers..."
>> 08:10:44 OBSERVER.kpan 108 boss i text="Attaching sp1cam..."
>> 08:10:44 OBSERVER.kpan 108 boss i text="Attaching sp1mech..."
>> 08:10:44 OBSERVER.kpan 108 boss i text="Attaching sp1daq..."
>> 08:10:44 OBSERVER.kpan 108 boss :
>> 08:10:44 CmdDone: OBSERVER.kpan 108 boss reloadControllers
>> 08:10:45 .mcp 0 mcp w rotBadFiducial=-5,10.60
>> 08:10:46 .boss 0 boss i text="Controller sp1cam connected."
>> 08:10:46 .boss 0 boss d text="Sending CAMCHECK to sp1cam"
>> 08:10:46 .boss 0 boss i text="Controller sp1mech connected."
>> 08:10:47 .tcc 0 tcc W Text="number keys does not match number of values"
>> 08:10:50 .alerts 0 alerts w alert=boss.camCheck.stale,warn,"at
>> 2024-06-13 08:06:08; no change for 401 sec, with None",enabled,noack,-1
>> 08:10:51 .boss 0 boss w text="Command CAMCHECK to sp1cam timed out"
>> 08:10:55 .alerts 0 alerts w alert=boss.camCheck.stale,warn,"at
>> 2024-06-13 08:06:08; no change for 407 sec, with None",enabled,noack,-1
>> 08:11:08 .boss 0 boss i aliveAt=1718266268
>> 08:11:08 .boss 4437 boss d text="Performing camcheck on sp1cam."
>> 08:11:08 .alerts 0 alerts w alert=boss.camCheck.stale,warn,"at
>> 2024-06-13 08:06:08; no change for 419 sec, with None",enabled,noack,-1
>> 08:11:16 .alerts 0 alerts w alert=boss.camCheck.stale,warn,"at
>> 2024-06-13 08:06:08; no change for 428 sec, with None",enabled,noack,-1
>> 08:11:24 CmdStarted: OBSERVER.kpan 109 boss reloadControllers
>> 08:11:24 OBSERVER.kpan 109 boss i text="Disconnecting sp1cam..."
>> 08:11:24 OBSERVER.kpan 109 boss i text="Disconnecting sp1mech..."
>> 08:11:24 OBSERVER.kpan 109 boss i text="Disconnecting sp1daq..."
>> 08:11:24 OBSERVER.kpan 109 boss i text="pausing to let the
>> controllers disconnect themselves...."
>> 08:11:24 .boss 0 boss w text="Controller sp1cam disconnected."
>> 08:11:24 .boss 0 boss w text="sp1cam removed from available hardware."
>> 08:11:24 .boss 0 boss w text="Controller sp1mech disconnected."
>> 08:11:24 .boss 0 boss w text="sp1mech removed from available hardware."
>> 08:11:26 .apo 0 apo w diskspace=sdss5-hub,/data,22438,1100
>> 08:11:26 OBSERVER.kpan 109 boss i text="Reloading ['sp1cam',
>> 'sp1mech', 'sp1daq'] controllers..."
>> 08:11:26 OBSERVER.kpan 109 boss i text="Attaching sp1cam..."
>> 08:11:26 OBSERVER.kpan 109 boss i text="Attaching sp1mech..."
>> 08:11:26 OBSERVER.kpan 109 boss i text="Attaching sp1daq..."
>> 08:11:26 OBSERVER.kpan 109 boss :
>> 08:11:26 CmdDone: OBSERVER.kpan 109 boss reloadControllers
>> 08:11:28 .alerts 0 alerts w alert=boss.camCheck.stale,warn,"at
>> 2024-06-13 08:06:08; no change for 439 sec, with None",enabled,noack,-1
>> 08:11:28 .boss 0 boss i text="Controller sp1cam connected."
>> 08:11:28 .boss 0 boss d text="Sending CAMCHECK to sp1cam"
>> 08:11:28 .boss 0 boss i text="Controller sp1mech connected."
>> 08:11:33 .boss 0 boss w text="Command CAMCHECK to sp1cam timed out"
>> 08:11:36 CmdStarted: OBSERVER.kpan 110 boss clearExposure
>> 08:11:37 .boss 248 boss w text="this command has already been
>> finished!!!! (client.v3906 248): exposure arc itime=4 noreadout"
>> 08:11:37 .boss 248 boss i exposureState=IDLE,0.0,0.0
>> 08:11:37 .boss 248 boss w text="this command has already been
>> finished!!!! (client.v3906 248): exposure arc itime=4 noreadout"
>> 08:11:37 .boss 248 boss :
>> 08:11:37 .boss 0 boss i exposureState=IDLE,0.0,0.0
>> 08:11:37 OBSERVER.kpan 110 boss :
>> 08:11:37 CmdDone: OBSERVER.kpan 110 boss clearExposure
>> 08:11:37 .boss 0 boss d text="Sending eS to sp1mech"
>> 08:11:38 .boss 0 boss d text="Completed message from sp1mech"
>> 08:11:45 CmdStarted: OBSERVER.kpan 111 boss reloadControllers
>> 08:11:45 OBSERVER.kpan 111 boss i text="Disconnecting sp1cam..."
>> 08:11:45 OBSERVER.kpan 111 boss i text="Disconnecting sp1mech..."
>> 08:11:45 OBSERVER.kpan 111 boss i text="Disconnecting sp1daq..."
>> 08:11:45 OBSERVER.kpan 111 boss i text="pausing to let the
>> controllers disconnect themselves...."
>> 08:11:45 .boss 0 boss w text="Controller sp1cam disconnected."
>> 08:11:45 .boss 0 boss w text="sp1cam removed from available hardware."
>> 08:11:45 .boss 0 boss w text="Controller sp1mech disconnected."
>> 08:11:45 .boss 0 boss w text="sp1mech removed from available hardware."
>> 08:11:47 .tcc 0 tcc W Text="number keys does not match number of values"
>> 08:11:47 OBSERVER.kpan 111 boss i text="Reloading ['sp1cam',
>> 'sp1mech', 'sp1daq'] controllers..."
>> 08:11:47 OBSERVER.kpan 111 boss i text="Attaching sp1cam..."
>> 08:11:47 OBSERVER.kpan 111 boss i text="Attaching sp1mech..."
>> 08:11:47 OBSERVER.kpan 111 boss i text="Attaching sp1daq..."
>> 08:11:47 OBSERVER.kpan 111 boss :
>> 08:11:47 CmdDone: OBSERVER.kpan 111 boss reloadControllers
>> 08:11:49 .boss 0 boss i text="Controller sp1cam connected."
>> 08:11:49 .boss 0 boss d text="Sending CAMCHECK to sp1cam"
>> 08:11:49 .boss 0 boss i text="Controller sp1mech connected."
>>
>> tried reloadControllers, boss cold start, restart bossICC, none of
>> them work. Went down to the telescope, check voltages, cables, media
>> converter, didn't find obvious wrong items. I reset ethernet cables,
>> re-tried reloadControllers, cold start, restart bossICC, still don't
>> work.
>> Called John to discuss what is next. We decided to check whether
>> startfillseq works, it doesn't. Then I decided to manually fill both
>> r1 and b1 camera every two hours, and observe bright designs with
>> only apogee data.
>>
>> alerts on sp1MicroTalk and sp1Cam are currently disabled.
>>
>> ____________________________________________________________
>> Logs of Nightly APO Activities
>> https://trac.sdss.org/tracmailman/browser/private/sloan-25mlog/index.html
>>
> ____________________________________________________________
> Logs of Nightly APO Activities
> https://trac.sdss.org/tracmailman/browser/private/sloan-25mlog/index.html