From: Kevin Covey Subject: Re: [sloan-25mlog 6024] Re: 2.5m obslog 20220420 (SJD 59690) Date: Thu, 21 Apr 2022 22:34:11 +0000 --_000_MW4PR02MB745922A772444ADA2CB6BB37D8F49MW4PR02MB7459namp_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi John, & Jon, This seems like the right call -- Jon, please do consult MWM-exec & let us = know if we should change course! Thanks, -kevin ________________________________ From: Donor, John Sent: Thursday, April 21, 2022 12:18 PM To: Kevin Covey ; dmbiz@apo.nmsu.edu ; = obs-report-25m@apo.nmsu.edu Cc: Jon Bird ; Drew Chojnowski Subject: Re: [sloan-25mlog 6024] Re: 2.5m obslog 20220420 (SJD 59690) Hey y'all, Since this is time sensitive (the entire epoch will be automatically marked= not started soon because of the short max_length for bright time) I went a= head and marked that design manually complete. If that is a problem let me = know and we'll mark the whole epoch not started so it can be rescheduled. -John ________________________________ From: sloan-25mlog on behalf of Donor, John= Sent: Thursday, April 21, 2022 11:14:53 AM To: Kevin Covey; dmbiz@apo.nmsu.edu; obs-report-25m@apo.nmsu.edu Cc: Jon Bird; Drew Chojnowski Subject: [sloan-25mlog 6024] Re: 2.5m obslog 20220420 (SJD 59690) I realized shortly after sending that email that coming back would mean rep= eating all 4 designs, because bright time requires all observations occur w= ithin a few hours. So I think we should definitely just hit "complete" and = deal with the one design's worth of no dither? But someone should sign off = on that. -John ________________________________ From: Kevin Covey Sent: Thursday, April 21, 2022 11:09:43 AM To: Donor, John; dmbiz@apo.nmsu.edu; obs-report-25m@apo.nmsu.edu Cc: Jon Bird; Drew Chojnowski; Nidever,David Subject: Re: [sloan-25mlog 6021] 2.5m obslog 20220420 (SJD 59690) [EXTERNAL EMAIL WARNING] DO NOT CLICK LINKS or open attachments unless you = recognize the sender and know the content is safe. Hi John, I am AOK with not coming back, but I'm also cc'ing Drew & David so they can= assess if that will cause issues for the APOGEE pipeline... -kevin ________________________________ From: Donor, John Sent: Thursday, April 21, 2022 8:12 AM To: dmbiz@apo.nmsu.edu ; obs-report-25m@apo.nmsu.edu Cc: Kevin Covey ; Jon Bird Subject: Re: [sloan-25mlog 6021] 2.5m obslog 20220420 (SJD 59690) Hi Dmitry, 46830, the last design on field 101806 that you commented on, has two expos= ures taken at the same dither position (looks like "B", 13.499). Is there a= reason you didn't dither on the last exposure? The automatic check won't allow a design to finish without a quality A and = B exposure. But I can manually mark this design finished if we don't want t= o come back for a single dither position (CC Kevin and Jon B.) -John ________________________________ From: sloan-25mlog on behalf of dmbiz@apo.n= msu.edu Sent: Thursday, April 21, 2022 6:53:43 AM To: obs-report-25m@apo.nmsu.edu Subject: [sloan-25mlog 6021] 2.5m obslog 20220420 (SJD 59690) [EXTERNAL EMAIL WARNING] DO NOT CLICK LINKS or open attachments unless you = recognize the sender and know the content is safe. APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Wednesday April 20, 2022 (SJD 59690) ---- Observing Team ---- Viktor Malanushenko Dmitry Bizyaev ---- Observing Plan ---- Monday-Friday, April 16-20 If clear, queue, then observe one dither field each night. Select from desi= gns 35987 (APOGEE), 35988 (BOSS), 35989 (APOGEE), 35990 (BOSS), until you g= et one successful complete dither on each. If there are clouds, then just observe from the queue. Observability windows should be good for second half of the night: put 3598= 7, 35988, 35989, 35990 into Kronos VizPreview for observability windows ---- Observing Summary ---- Clear, productive night with decent seeing. Observed: Science fields from the queue 02:06:22 Config Loaded: 102959-50361-4961 1x900s 1xAB DONE 02:37:20 Config Loaded: 101240-44085-4962 2x900s 2xAB done design 03:23:01 Config Loaded: 101240-44086-4963 1x900s 1xAB done design 03:49:49 Config Loaded: 101240-44087-4964 2x900s 2xAB done design 04:29:30 Config Loaded: 101240-44088-4965 2x900s 2xAB DONE FIELD 08:45:36 Config Loaded: 102300-48449-4980 1x900s 1xAB DONE 09:09:29 Config Loaded: 101806-46827-4981 1x900s 1xAB DONE 09:35:25 Config Loaded: 101806-46828-4982 1x900s 1xAB DONE 09:59:26 Config Loaded: 101806-46829-4983 1x900s 1xAB DONE 10:21:24 Config Loaded: 101806-46830-4984 1x900s 1xAB not done, although t= he APOGEE signal is more than sufficient. 11:09:15 Config Loaded: 100569-40386-4986 1x900s 1xAB DONE 10:45:28 Config Loaded: 103229-51302-4985 Jaeger collision: 902 - many times, see log. We were unable to load and execute the design because of the positioner 902 got locked every time. Ski= pped this field. Telescope dither test with design 35987 (APOGEE). 12pt : see log from 05:20Z to 08:41Z on the clear sky and 1.6"->1.3" seeing. Notes ----- -- dust accumulated: 13.9K -- in the afternoon it took us 3 attempts to make "fliswarm reconnect" suc= cessful. See "Problems in Detail" for errors. -- We've got a line in the Kronos display right after we reloaded 'Obs Queu= e ': ERROR: ['could not schedule field at 59690.489695835284'] But in 10 minutes the error disappeared. -- We didn't have problems with the robot 1340 tonight. -- We finished all 4 designs for Field 101806, which is bright_8x4_v1. Last design 46830 has got a very good signal, but was not turned green in Kronos. -- Not unexpected, but the back/open motion of the enclosure is still running on slow speed ---- Engineering Issues ---- None ---- Observing Log ---- 00:33ZAfternoon: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --Test Actors, 23:56:44 -- APOGEE Short Dark - 23:57:34 -- 23:59:37 -- APOGEE: Evening Cals -- 00:48:08-- done -- boss bias, dark: files=3D340585,340586 00:05:42 Hartmann collimate output sp1: offset: r=3D -1649 (Out of focus); b=3D 0.8 (In Focus) sp1: pred. move: spAverageMove=3D -953 sp1: pred. resid: r=3D -696, b=3D -2.2, txt=3D OK, spTemp =3D 8.2 BOSS collimation was checked. No adjustment was made. ---------- motorPosition sp1: 626, 776, -621; ---------- Median spectrographs temperatures: Tsp1=3D 8.1 Outside air temperature=3D 16.3; ---------- boss arc, flat: files=3D340589,340590 jaeger power on --gfas fliswarm reconnect -- run 3 times : first and second was errors (in section= "Problems in Detail") 00:17:20 CmdDone: OBSERVER.viktorm 77 fliswarm reconnect 00:19:07 CmdStarted: OBSERVER.viktorm 78 fliswarm talk -c gfa temperature -= 20 (00:26:13 CmdDone: OBSERVER.viktorm 78 fliswarm talk -c gfa temperature -20= ) 00:20:05 CmdDone: OBSERVER.viktorm 79 jaeger unwind 00:22:16 CmdDone: OBSERVER.viktorm 84 jaeger configuration load --from-posi= tions Checked and confirmed all robots are folded. Robot position checked with "jaeger fvc loop --no-apply". 00:23:32 OBSERVER.viktorm 86 jaeger i fvc_rms=3D43.42 00:23:32 OBSERVER.viktorm 86 jaeger i fvc_perc_90=3D68.9 00:23:32 OBSERVER.viktorm 86 jaeger i fvc_percent_reached=3D1.8 00:50Z start ventilation 01:00Z on the telescope all checked and ok. Night: =3D=3D=3D=3D=3D=3D 01:46:29 encl25m : open; =3D=3D=3D=3D=3D=3D=3D=3D 02:06:22 Config Loaded: 102959-50361-4961 02:10:09 Hartmann collimate output sp1: offset: r=3D 329 (In Focus); b=3D -8.3 (Out of focus) sp1: pred. move: spAverageMove=3D 1482 sp1: pred. resid: r=3D -1153, b=3D -3.6, txt=3D OK, spTemp =3D 7.8 02:11:19 APO.viktorm.jaeger 6756 jaeger d text=3D"Saving processed image /d= ata/fcam/59690/proc-fimg-fvc1n-0003.fits" 02:11:17 APO.viktorm.jaeger 6756 jaeger i text=3D"[FVC]: RMS full fit 15.29= 0 um." 02:11:17 APO.viktorm.jaeger 6756 jaeger i fvc_perc_90=3D23.5 02:20:44 apogee.expose=3DapRaw-41280013, 43, A, Object 02:20:50 boss exposure 900.0, file=3D340595 02:28:24 apogee.expose=3DapRaw-41280014, 43, B, Object =3D=3D=3D=3D=3D=3D=3D 02:37:20 Config Loaded: 101240-44085-4962 02:41:53 Hartmann collimate output sp1: offset: r=3D 329 (In Focus); b=3D -1.2 (In Focus) sp1: pred. move: spAverageMove=3D 357 sp1: pred. resid: r=3D -28, b=3D -0.1, txt=3D OK, spTemp =3D 7.7 02:43:03 APO.viktorm.jaeger 6797 jaeger d text=3D"Saving processed image /d= ata/fcam/59690/proc-fimg-fvc1n-0005.fits" 02:43:02 APO.viktorm.jaeger 6797 jaeger i text=3D"[FVC]: RMS full fit 14.59= 0 um." 02:43:02 APO.viktorm.jaeger 6797 jaeger i fvc_perc_90=3D24.7 02:49:29 apogee.expose=3DapRaw-41280015, 43, B, Object 02:49:35 boss exposure 900.0, file=3D340600 02:57:14 apogee.expose=3DapRaw-41280016, 43, A, Object 03:06:30 apogee.expose=3DapRaw-41280017, 43, A, Object 03:06:36 boss exposure 900.0, file=3D340601 03:14:11 apogee.expose=3DapRaw-41280018, 43, B, Object =3D=3D=3D=3D=3D=3D=3D=3D 03:23:01 Config Loaded: 101240-44086-4963 03:26:40 APO.viktorm.jaeger 6831 jaeger d text=3D"Saving processed image /d= ata/fcam/59690/proc-fimg-fvc1n-0007.fits" 03:26:39 APO.viktorm.jaeger 6831 jaeger i text=3D"[FVC]: RMS full fit 45.86= 0 um." 03:26:39 APO.viktorm.jaeger 6831 jaeger i fvc_perc_90=3D24.6 03:32:00 apogee.expose=3DapRaw-41280019, 43, B, Object 03:32:06 boss exposure 900.0, file=3D340602 03:39:45 apogee.expose=3DapRaw-41280020, 43, A, Object =3D=3D=3D=3D=3D=3D=3D=3D 03:49:49 Config Loaded: 101240-44087-4964 03:53:11 APO.viktorm.jaeger 6861 jaeger d text=3D"Saving processed image /d= ata/fcam/59690/proc-fimg-fvc1n-0009.fits" 03:53:10 APO.viktorm.jaeger 6861 jaeger i text=3D"[FVC]: RMS full fit 31.62= 0 um." 03:53:10 APO.viktorm.jaeger 6861 jaeger i fvc_perc_90=3D27.4 03:57:47 apogee.expose=3DapRaw-41280021, 46, A, Object 03:57:54 boss exposure 900.0, file=3D340603 04:06:01 apogee.expose=3DapRaw-41280022, 46, B, Object 04:14:10 apogee.expose=3DapRaw-41280023, 43, B, Object 04:14:17 boss exposure 900.0, file=3D340604 04:21:55 apogee.expose=3DapRaw-41280024, 43, A, Object =3D=3D=3D=3D=3D=3D=3D=3D 04:29:30 Config Loaded: 101240-44088-4965 04:33:03 APO.viktorm.jaeger 6885 jaeger d text=3D"Saving processed image /d= ata/fcam/59690/proc-fimg-fvc1n-0011.fits" 04:33:02 APO.viktorm.jaeger 6885 jaeger i text=3D"[FVC]: RMS full fit 386.1= 40 um." We decided to re-do the FVC loop again: 04:37:27 APO.viktorm.jaeger 6891 jaeger i text=3D"[FVC]: RMS full fit 385.6= 60 um." 04:37:28 APO.viktorm.jaeger 6891 jaeger d text=3D"Saving processed image /d= ata/fcam/59690/proc-fimg-fvc1n-0013.fits" 04:37:27 APO.viktorm.jaeger 6891 jaeger i fvc_perc_90=3D25.8 04:41:09 apogee.expose=3DapRaw-41280025, 46, A, Object 04:41:15 boss exposure 900.0, file=3D340605 04:49:21 apogee.expose=3DapRaw-41280026, 46, B, Object 04:57:31 apogee.expose=3DapRaw-41280027, 43, B, Object 04:57:38 boss exposure 900.0, file=3D340606 05:05:17 apogee.expose=3DapRaw-41280028, 43, A, Object (observers39) [observer@sdss5-hub: ~]$ field 101240 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Design/Field S/N^2=3D b1 r1 MJDs =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Design 44085 2.90 8.99 59690 Design 44086 1.59 4.89 59690 Design 44087 2.91 8.57 59690 Design 44088 2.78 7.20 59690 This Field 101240 10.17 29.65 =3D=3D=3D=3D=3D=3D=3D=3D Telescope dither test with design 35987 (APOGEE) = =3D=3D=3D=3D=3D=3D=3D=3D 05:20Z started. The sky is clear. Seeing ~ 1.6" at the beginning, 1.3 at the end. 05:22:43 CmdStarted: OBSERVER.viktorm 119 jaeger configuration load --no-cl= one 35987 05:26:14 CmdStarted: OBSERVER.viktorm 120 jaeger fvc loop --max-iterations = 4 --target-90-percentile 10 05:26:52 OBSERVER.viktorm 120 jaeger d text=3D"Saving processed image /data= /fcam/59690/proc-fimg-fvc1n-0014.fits" 05:27:30 OBSERVER.viktorm 120 jaeger d text=3D"Saving processed image /data= /fcam/59690/proc-fimg-fvc1n-0015.fits" 05:28:08 OBSERVER.viktorm 120 jaeger d text=3D"Saving processed image /data= /fcam/59690/proc-fimg-fvc1n-0016.fits" 05:28:36 OBSERVER.viktorm 120 jaeger d text=3D"Saving processed image /data= /fcam/59690/proc-fimg-fvc1n-0017.fits" 05:26:39 OBSERVER.viktorm 120 jaeger i fvc_perc_90=3D67.9 05:27:19 OBSERVER.viktorm 120 jaeger i fvc_perc_90=3D25.8 05:27:57 OBSERVER.viktorm 120 jaeger i fvc_perc_90=3D25.2 05:28:35 OBSERVER.viktorm 120 jaeger i fvc_perc_90=3D25.4 05:26:39 OBSERVER.viktorm 120 jaeger i text=3D"[FVC]: RMS full fit 54.450 u= m." 05:27:19 OBSERVER.viktorm 120 jaeger i text=3D"[FVC]: RMS full fit 19.020 u= m." 05:27:57 OBSERVER.viktorm 120 jaeger i text=3D"[FVC]: RMS full fit 19.070 u= m." 05:28:35 OBSERVER.viktorm 120 jaeger i text=3D"[FVC]: RMS full fit 18.880 u= m." 05:29:45 CmdStarted: OBSERVER.viktorm 122 jaeger configuration slew 05:23:46 Config Loaded: 20956-35987-4966 05:35:18 apogee.expose=3DapRaw-41280029, 30, A, Object 05:35:24 boss exposure 300.0, file=3D340607 05:44:29 CmdStarted: OBSERVER.viktorm 131 jaeger configuration load --no-cl= one 35987 05:48:01 OBSERVER.viktorm 132 jaeger d text=3D"Saving processed image /data= /fcam/59690/proc-fimg-fvc1n-0018.fits" 05:48:39 OBSERVER.viktorm 132 jaeger d text=3D"Saving processed image /data= /fcam/59690/proc-fimg-fvc1n-0019.fits" 05:49:17 OBSERVER.viktorm 132 jaeger d text=3D"Saving processed image /data= /fcam/59690/proc-fimg-fvc1n-0020.fits" 05:49:45 OBSERVER.viktorm 132 jaeger d text=3D"Saving processed image /data= /fcam/59690/proc-fimg-fvc1n-0021.fits" 05:47:48 OBSERVER.viktorm 132 jaeger i text=3D"[FVC]: RMS full fit 54.600 u= m." 05:48:28 OBSERVER.viktorm 132 jaeger i text=3D"[FVC]: RMS full fit 40.050 u= m." 05:49:06 OBSERVER.viktorm 132 jaeger i text=3D"[FVC]: RMS full fit 18.590 u= m." 05:49:44 OBSERVER.viktorm 132 jaeger i text=3D"[FVC]: RMS full fit 17.980 u= m." 05:47:48 OBSERVER.viktorm 132 jaeger i fvc_perc_90=3D69.7 05:48:28 OBSERVER.viktorm 132 jaeger i fvc_perc_90=3D24.7 05:49:06 OBSERVER.viktorm 132 jaeger i fvc_perc_90=3D23.3 05:49:44 OBSERVER.viktorm 132 jaeger i fvc_perc_90=3D15.3 05:52:34 CmdDone: OBSERVER.viktorm 133 jaeger configuration slew 05:45:28 Config Loaded: 20956-35987-4969 05:55:31 apogee.expose=3DapRaw-41280030, 30, A, Object 05:55:38 boss exposure 300.0, file=3D340608 06:08:29 OBSERVER.viktorm 140 jaeger i text=3D"[FVC]: RMS full fit 54.300 u= m." 06:09:09 OBSERVER.viktorm 140 jaeger i text=3D"[FVC]: RMS full fit 33.100 u= m." 06:09:47 OBSERVER.viktorm 140 jaeger i text=3D"[FVC]: RMS full fit 19.130 u= m." 06:10:25 OBSERVER.viktorm 140 jaeger i text=3D"[FVC]: RMS full fit 19.430 u= m." 06:08:42 OBSERVER.viktorm 140 jaeger d text=3D"Saving processed image /data= /fcam/59690/proc-fimg-fvc1n-0022.fits" 06:09:20 OBSERVER.viktorm 140 jaeger d text=3D"Saving processed image /data= /fcam/59690/proc-fimg-fvc1n-0023.fits" 06:09:58 OBSERVER.viktorm 140 jaeger d text=3D"Saving processed image /data= /fcam/59690/proc-fimg-fvc1n-0024.fits" 06:10:26 OBSERVER.viktorm 140 jaeger d text=3D"Saving processed image /data= /fcam/59690/proc-fimg-fvc1n-0025.fits" 06:08:29 OBSERVER.viktorm 140 jaeger i fvc_perc_90=3D70.1 06:09:09 OBSERVER.viktorm 140 jaeger i fvc_perc_90=3D26.4 06:09:47 OBSERVER.viktorm 140 jaeger i fvc_perc_90=3D25.1 06:10:25 OBSERVER.viktorm 140 jaeger i fvc_perc_90=3D27.6 06:13:08 CmdDone: OBSERVER.viktorm 141 jaeger configuration slew 06:05:36 Config Loaded: 20956-35987-4972 06:16:06 apogee.expose=3DapRaw-41280031, 30, A, Object 06:16:13 boss exposure 300.0, file=3D340609 06:30:23 CmdDone: OBSERVER.viktorm 152 jaeger configuration reverse 06:30:50 CmdStarted: OBSERVER.viktorm 153 jaeger configuration load --no-cl= one --epoch-delay 4800 35987 06:32:17 .jaeger 0 jaeger w text=3D"Positioner 437 replied to SEND_TRAJECTO= RY_DATA UID=3D4379 with 'VALUE_OUT_OF_RANGE'. (JaegerUserWarning)" 06:32:17 OBSERVER.viktorm 154 jaeger f error=3D"Trajectory failed with erro= r: Something went wrong sending the trajectory: At least one SEND_TRAJECTOR= Y_COMMAND failed" 06:32:17 CmdDone: OBSERVER.viktorm 154 jaeger configuration execute Trying again with different epochdelay: 06:35:36 CmdStarted: OBSERVER.viktorm 157 jaeger configuration load --no-cl= one --epoch-delay 3600 35987 06:38:56 OBSERVER.viktorm 159 jaeger d text=3D"Saving processed image /data= /fcam/59690/proc-fimg-fvc1n-0026.fits" 06:39:34 OBSERVER.viktorm 159 jaeger d text=3D"Saving processed image /data= /fcam/59690/proc-fimg-fvc1n-0027.fits" 06:40:12 OBSERVER.viktorm 159 jaeger d text=3D"Saving processed image /data= /fcam/59690/proc-fimg-fvc1n-0028.fits" 06:40:40 OBSERVER.viktorm 159 jaeger d text=3D"Saving processed image /data= /fcam/59690/proc-fimg-fvc1n-0029.fits" 06:38:45 OBSERVER.viktorm 159 jaeger i fvc_perc_90=3D71.2 06:39:23 OBSERVER.viktorm 159 jaeger i fvc_perc_90=3D25.4 06:40:01 OBSERVER.viktorm 159 jaeger i fvc_perc_90=3D24.6 06:40:39 OBSERVER.viktorm 159 jaeger i fvc_perc_90=3D22.8 06:38:45 OBSERVER.viktorm 159 jaeger i text=3D"[FVC]: RMS full fit 54.690 u= m." 06:39:23 OBSERVER.viktorm 159 jaeger i text=3D"[FVC]: RMS full fit 32.500 u= m." 06:40:01 OBSERVER.viktorm 159 jaeger i text=3D"[FVC]: RMS full fit 17.500 u= m." 06:40:39 OBSERVER.viktorm 159 jaeger i text=3D"[FVC]: RMS full fit 16.420 u= m." ** Execute the following command 5 times: jaeger fvc loop --no-apply --no-w= rite-summary. Record these 5 exposure numbers in the log 06:42:05 OBSERVER.viktorm 160 jaeger d text=3D"Saving processed image /data= /fcam/59690/proc-fimg-fvc1n-0030.fits" 06:42:55 OBSERVER.viktorm 161 jaeger d text=3D"Saving processed image /data= /fcam/59690/proc-fimg-fvc1n-0031.fits" 06:43:44 OBSERVER.viktorm 162 jaeger d text=3D"Saving processed image /data= /fcam/59690/proc-fimg-fvc1n-0032.fits" 06:44:30 OBSERVER.viktorm 163 jaeger d text=3D"Saving processed image /data= /fcam/59690/proc-fimg-fvc1n-0033.fits" 06:45:12 OBSERVER.viktorm 164 jaeger d text=3D"Saving processed image /data= /fcam/59690/proc-fimg-fvc1n-0034.fits" 06:42:03 OBSERVER.viktorm 160 jaeger i text=3D"[FVC]: RMS full fit 17.230 u= m." 06:42:54 OBSERVER.viktorm 161 jaeger i text=3D"[FVC]: RMS full fit 18.850 u= m." 06:43:42 OBSERVER.viktorm 162 jaeger i text=3D"[FVC]: RMS full fit 16.820 u= m." 06:44:28 OBSERVER.viktorm 163 jaeger i text=3D"[FVC]: RMS full fit 14.790 u= m." 06:45:11 OBSERVER.viktorm 164 jaeger i text=3D"[FVC]: RMS full fit 18.200 u= m." 06:42:03 OBSERVER.viktorm 160 jaeger i fvc_perc_90=3D23.1 06:42:54 OBSERVER.viktorm 161 jaeger i fvc_perc_90=3D26.0 06:43:42 OBSERVER.viktorm 162 jaeger i fvc_perc_90=3D23.5 06:44:28 OBSERVER.viktorm 163 jaeger i fvc_perc_90=3D19.2 06:45:11 OBSERVER.viktorm 164 jaeger i fvc_perc_90=3D26.2 06:45:46 CmdStarted: OBSERVER.viktorm 165 jaeger configuration slew ** base offset 06:54:27 apogee.expose=3DapRaw-41280032, 30, A, Object 06:54:34 boss exposure 300.0, file=3D340610 07:02:13 CmdStarted: OBSERVER.viktorm 168 cherno offset -1.20 0.58 -432 07:05:22 apogee.expose=3DapRaw-41280033, 30, A, Object 07:10:45 CmdStarted: OBSERVER.viktorm 169 cherno offset -0.43 0.34 -432 07:13:16 apogee.expose=3DapRaw-41280034, 30, A, Object 07:18:27 CmdStarted: OBSERVER.viktorm 170 cherno offset -0.35 -0.21 -432 07:19:52 apogee.expose=3DapRaw-41280035, 30, A, Object 07:25:23 CmdStarted: OBSERVER.viktorm 171 cherno offset 0.70 0.33 -432 07:27:46 apogee.expose=3DapRaw-41280036, 30, A, Object 07:33:00 CmdStarted: OBSERVER.dmbiz 36 cherno offset -0.41 -1.05 -432 07:35:10 apogee.expose=3DapRaw-41280037, 30, A, Object 07:40:37 CmdDone: OBSERVER.viktorm 174 cherno offset 0.31 -0.08 -432 07:43:25 apogee.expose=3DapRaw-41280038, 30, A, Object 07:48:56 CmdStarted: OBSERVER.viktorm 175 cherno offset 0 0 -432 07:50:24 apogee.expose=3DapRaw-41280039, 30, A, Object 07:55:47 CmdStarted: OBSERVER.viktorm 176 cherno offset 1.97 0.35 -432 07:57:26 apogee.expose=3DapRaw-41280040, 30, A, Object 08:02:41 CmdStarted: OBSERVER.viktorm 177 cherno offset -2.07 -0.02 -432 08:04:49 apogee.expose=3DapRaw-41280041, 30, A, Object 08:10:19 CmdStarted: OBSERVER.viktorm 180 cherno offset -0.12 0.92 -432 08:11:47 apogee.expose=3DapRaw-41280042, 30, A, Object 08:17:03 CmdStarted: OBSERVER.viktorm 181 cherno offset 0.71 0.36 -432 08:18:10 apogee.expose=3DapRaw-41280043, 30, A, Object 08:23:21 CmdStarted: OBSERVER.viktorm 182 cherno offset -0.54 -0.40 -432 08:24:37 apogee.expose=3DapRaw-41280044, 30, A, Object 08:29:47 CmdStarted: OBSERVER.viktorm 183 cherno offset 0.77 0.11 -432 08:30:44 apogee.expose=3DapRaw-41280045, 30, A, Object 08:35:54 CmdStarted: OBSERVER.viktorm 184 cherno offset 0 0 -432 08:37:00 apogee.expose=3DapRaw-41280046, 30, A, Object =3D=3D=3D=3D=3D=3D=3D=3D Returning back to the queue observations =3D=3D=3D= =3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D 08:45:36 Config Loaded: 102300-48449-4980 08:49:03 sp1.motor.move=3D -1430, -1430, -1429 08:49:03 Hartmann collimate output sp1: offset: r=3D -1319 (Out of focus); b=3D 4.8 (Out of focus) sp1: pred. move: spAverageMove=3D -1430 sp1: pred. resid: r=3D 111, b=3D 0.3, txt=3D OK, spTemp =3D 6.4 08:50:13 APO.viktorm.jaeger 6999 jaeger d text=3D"Saving processed image /d= ata/fcam/59690/proc-fimg-fvc1n-0036.fits" 08:50:12 APO.viktorm.jaeger 6999 jaeger i text=3D"[FVC]: RMS full fit 363.6= 60 um." 08:50:12 APO.viktorm.jaeger 6999 jaeger i fvc_perc_90=3D22.9 08:54:16 apogee.expose=3DapRaw-41280047, 43, A, Object 08:54:22 boss exposure 900.0, file=3D340615 09:01:57 apogee.expose=3DapRaw-41280048, 43, B, Object =3D=3D=3D=3D=3D=3D=3D=3D 09:09:29 Config Loaded: 101806-46827-4981 09:13:30 APO.viktorm.jaeger 7031 jaeger d text=3D"Saving processed image /d= ata/fcam/59690/proc-fimg-fvc1n-0038.fits" 09:13:29 APO.viktorm.jaeger 7031 jaeger i fvc_perc_90=3D24.9 09:13:29 APO.viktorm.jaeger 7031 jaeger i text=3D"[FVC]: RMS full fit 41.86= 0 um." 09:13:30 APO.viktorm.jaeger 7031 jaeger d text=3D"Saving processed image /d= ata/fcam/59690/proc-fimg-fvc1n-0038.fits" 09:13:29 APO.viktorm.jaeger 7031 jaeger i text=3D"[FVC]: RMS full fit 41.86= 0 um." 09:13:29 APO.viktorm.jaeger 7031 jaeger i fvc_perc_90=3D24.9 09:20:10 apogee.expose=3DapRaw-41280049, 43, B, Object 09:20:16 boss exposure 900.0, file=3D340618 09:27:55 apogee.expose=3DapRaw-41280050, 43, A, Object =3D=3D=3D=3D=3D=3D=3D=3D 09:35:25 Config Loaded: 101806-46828-4982 09:38:15 APO.viktorm.jaeger 7060 jaeger d text=3D"Saving processed image /d= ata/fcam/59690/proc-fimg-fvc1n-0040.fits" 09:38:10 APO.viktorm.jaeger 7060 jaeger i fvc_perc_90=3D23.0 09:42:35 apogee.expose=3DapRaw-41280051, 43, A, Object 09:42:42 boss exposure 900.0, file=3D340619 09:50:17 apogee.expose=3DapRaw-41280052, 43, B, Object =3D=3D=3D=3D=3D=3D=3D=3D 09:59:26 Config Loaded: 101806-46829-4983 10:02:15 APO.viktorm.jaeger 7090 jaeger d text=3D"Saving processed image /d= ata/fcam/59690/proc-fimg-fvc1n-0042.fits" 10:02:13 APO.viktorm.jaeger 7090 jaeger i fvc_perc_90=3D24.9 10:06:07 apogee.expose=3DapRaw-41280053, 43, B, Object 10:06:14 boss exposure 900.0, file=3D340620 10:13:53 apogee.expose=3DapRaw-41280054, 43, A, Object =3D=3D=3D=3D=3D=3D=3D=3D 10:21:24 Config Loaded: 101806-46830-4984 10:24:56 APO.viktorm.jaeger 7115 jaeger d text=3D"Saving processed image /d= ata/fcam/59690/proc-fimg-fvc1n-0044.fits" 10:24:55 APO.viktorm.jaeger 7115 jaeger i fvc_perc_90=3D22.7 10:28:55 apogee.expose=3DapRaw-41280055, 43, A, Object 10:29:01 boss exposure 900.0, file=3D340621 10:37:57 apogee.expose=3DapRaw-41280056, 43, B, Object Last APOGEE did not start automatically because of the error: 10:32:45 APO.viktorm.apogee 7133 apogee f errorMessage=3D"cannot execute co= mmand: Controller is busy..." 10:32:45 CmdDone: APO.viktorm.apogee 7133 apogee expose time=3D459.0 object= =3Dobject 10:32:45 APO.viktorm 60 hal e error=3D"Command 'apogee expose time=3D459.0 = object=3Dobject' failed." We started it manually. =3D=3D=3D=3D=3D=3D=3D=3D 10:45:28 Config Loaded: 103229-51302-4985 We've got a robot collision: 10:46:46 .jaeger 0 jaeger f text=3D"A collision was detected in positioner = 902. Sending SEND_TRAJECTORY_ABORT and locking the FPS." Tried to unwind, but it did not work well 10:50:40 OBSERVER.viktorm 204 jaeger f error=3D"Command 'unwind' failed bec= ause of an uncaught error 'RuntimeError: Grid is Kaiju-collided, cannot gen= erate paths'. See traceback in the log for more information." 10:50:40 CmdDone: OBSERVER.viktorm 204 jaeger unwind We were able to explode more than one time and then unwind, but repeated at= tempts to load this design brought up the same problem with the robot 902. We decided to skip the design. Some 18 minutes lost. =3D=3D=3D=3D=3D=3D=3D=3D 11:09:15 Config Loaded: 100569-40386-4986 11:11:19 APO.viktorm.jaeger 7181 jaeger d text=3D"Saving processed image /d= ata/fcam/59690/proc-fimg-fvc1n-0046.fits" 11:11:17 APO.viktorm.jaeger 7181 jaeger i fvc_perc_90=3D24.1 11:17:34 apogee.expose=3DapRaw-41280057, 43, B, Object 11:17:40 boss exposure 900.0, file=3D340624 11:25:20 apogee.expose=3DapRaw-41280058, 43, A, Object 11:42:29 encl25m : closed; Calibrations started. ---- Mirror Numbers ---- PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A: 0.0060 0.0040 Axial B: 0.0040 0.0030 Axial C: -0.1180 -0.1150 Trans D: -0.0337 -0.0338 Lateral E: 0.3315 0.3315 Lateral F: 0.5345 0.5350 GALILS Commanded: 12500 -1600 5300 47050 -8800 -8400 Actual: 12500 -1600 5300 47050 -8800 -8400 SETMIR VALUES PrimDesOrient: 0.0 -59.5 -9.15 -512.0 1906.3 PrimOrient: 1.07 -59.51 -9.21 -512.13 1906.31 SECONDARY: ---------- Focus: 0.0000 Air temp: 16.22 Altitude: 30.0 MIGS TONIGHT NOMINAL Axial A: -9.6540 -9.6570 Axial B: -9.6650 -9.6710 Axial C: -9.6980 -9.6970 Trans D: 0.1160 0.1140 GALILS Commanded: 4883550 4877900 4903850 -50 -3450 Actual: 4859079 4859117 4859142 -50 -3450 SETMIR VALUES SecDesOrient: 3857.0 0.0 0.0 0.0 42.22 SecOrient: 3856.92 -0.01 0.01 0.01 42.87 ---- Telescope Status ---- 00:50Z Started ventilation 01:46Z encl25m : open 11:42Z encl25m : closed No interlocks bypassed Status at: 11:47Z Telescope stowed at: 121.0, 30.0, 0.0 mount Instrument mounted: FPS Counterweights at: 685/703 LN2 autofill systems: Connected and turned on 180L LN2 dewar scales: SP1 186 lbs, 23 psi FPS Status =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D CP10 Loop =3D 7.01 L/min GFA Loop =3D 1.81 L/min Supply pressure =3D 41.0 psi Return pressure =3D 26.9 psi Chiller Status =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Set temperarature =3D 9.5 C Flow =3D 3.6 Gpm ---- Software Used ---- Get Versions, 04/20/22, 23:57:35 -- STUI software: STUI : 1.7.17b1 alerts : 2.2.0 boss : 6.0.1 -boss.specMec : 3.0.4 apo : 3.0.1 fliswarm : 0.2.1-alpha.0 jaeger : 0.14.1-alpha.0 kaiju : 1.2.2-alpha.1 coordio : 1.2.2-alpha.1 fps_calibrati : APO.2021.12.25+b1cf2fd cherno : 0.1.1-alpha.0 hal : 0.1.1-alpha.0 hartmann : 2.0.2 hub : 5.0.1a0 mcp : 2.1 -mcp.plc : 34 -mcp.azFiduci : v1_148 -mcp.altFiduc : v1_148 -mcp.rotFiduc : v1_148 tcc : 1.4.0 apogee : not availble apogeecal : not availble apogeeql : v1_7_0dev STUIAdditions : 3.1.13 --------------Other Versions-------------- sdss-obstools : 3.7.14 TPM : 1.2-31-gd7eb MCP : 2.1 PLC : V34 Fiducials : v1_148 idlspec2d : v6_0_7 Kronos : 1.2.1a roboscheduler : 1.2.2a /dev/sdb1 2.0T 170G 1.9T 9% /home 10.25.1.230:/data 21T 18T 2.1T 90% /data ---- APOGEE Data Summary ---- APOGEE data list, mjd=3D59690, all raw_data: /data/apogee/utr_cdr/59690/apRaw-*.fits archive: /data/apogee/archive/59690/apR-[a,b,c]-*.apz quickred: /data/apogee/quickred/59690/apq-*.fits ---------------------------------------------------------------------------= --- mjd i UT File/Exp Imtype Nread A/B Dth Design-Cfg Arch Offset %Fl= ux Comm --------------------------------------------------------------------------= ---- 59690 1 23:58 41280001 Dark 3 A 13.0 ---------- a-b-c - - 59690 2 00:10 41280002 Dark 60 A 13.0 ---------- a-b-c - - 59690 3 00:12 41280003 QuarFlat 10 A 13.0 ---------- a-b-c - 10= 0 59690 4 00:14 41280004 Arc-Thar 12 B 13.5 ---------- a-b-c 1.92 - 59690 5 00:21 41280005 Arc-Une 40 B 13.5 ---------- a-b-c - - 59690 6 00:24 41280006 Arc-Thar 12 A 13.0 ---------- a-b-c 1.53 - 59690 7 00:31 41280007 Arc-Une 40 A 13.0 ---------- a-b-c - - 59690 8 00:42 41280008 Dark 60 A 13.0 ---------- a-b-c - - 59690 9 00:47 41280009 Arc---- 30 A 13.0 ---------- a-b-c - -= FPI 59690 10 01:53 41280010 DomeFlat 5 A 13.0 -999-4960 a-b-c - 332 59690 11 01:55 41280011 DomeFlat 5 A 13.0 -999-4960 a-b-c - 141 59690 12 01:57 41280012 DomeFlat 5 A 13.0 -999-4960 a-b-c - 137 59690 13 02:27 41280013 Object 43 A 13.0 50361-4961 a-b-c 1.42 - 59690 14 02:35 41280014 Object 43 B 13.5 50361-4961 a-b-c 1.94 - 59690 15 02:56 41280015 Object 43 B 13.5 44085-4962 a-b-c 1.94 - 59690 16 03:04 41280016 Object 43 A 13.0 44085-4962 a-b-c 1.42 - 59690 17 03:13 41280017 Object 43 A 13.0 44085-4962 a-b-c 1.42 - 59690 18 03:21 41280018 Object 43 B 13.5 44085-4962 a-b-c 1.94 - 59690 19 03:39 41280019 Object 43 B 13.5 44086-4963 a-b-c 1.94 - 59690 20 03:46 41280020 Object 43 A 13.0 44086-4963 a-b-c 1.42 - 59690 21 04:05 41280021 Object 46 A 13.0 44087-4964 a-b-c 1.42 - 59690 22 04:13 41280022 Object 46 B 13.5 44087-4964 a-b-c 1.93 - 59690 23 04:21 41280023 Object 43 B 13.5 44087-4964 a-b-c 1.94 - 59690 24 04:29 41280024 Object 43 A 13.0 44087-4964 a-b-c 1.42 - 59690 25 04:48 41280025 Object 46 A 13.0 44088-4965 a-b-c 1.42 - 59690 26 04:57 41280026 Object 46 B 13.5 44088-4965 a-b-c 1.94 - 59690 27 05:04 41280027 Object 43 B 13.5 44088-4965 a-b-c 1.94 - 59690 28 05:12 41280028 Object 43 A 13.0 44088-4965 a-b-c 1.42 - 59690 29 05:40 41280029 Object 30 A 13.0 35987-4966 a-b-c 1.43 - 59690 30 06:00 41280030 Object 30 A 13.0 35987-4969 a-b-c 1.43 - 59690 31 06:21 41280031 Object 30 A 13.0 35987-4972 a-b-c 1.43 - 59690 32 06:59 41280032 Object 30 A 13.0 35987-4977 a-b-c 1.43 - 59690 33 07:10 41280033 Object 30 A 13.0 35987-4977 a-b-c 1.43 - 59690 34 07:18 41280034 Object 30 A 13.0 35987-4977 a-b-c 1.43 - 59690 35 07:24 41280035 Object 30 A 13.0 35987-4977 a-b-c 1.44 - 59690 36 07:32 41280036 Object 30 A 13.0 35987-4977 a-b-c 1.43 - 59690 37 07:40 41280037 Object 30 A 13.0 35987-4977 a-b-c 1.43 - 59690 38 07:48 41280038 Object 30 A 13.0 35987-4977 a-b-c 1.42 - 59690 39 07:55 41280039 Object 30 A 13.0 35987-4977 a-b-c 1.42 - 59690 40 08:02 41280040 Object 30 A 13.0 35987-4977 a-b-c 1.43 - 59690 41 08:09 41280041 Object 30 A 13.0 35987-4977 a-b-c 1.44 - 59690 42 08:16 41280042 Object 30 A 13.0 35987-4977 a-b-c 1.43 - 59690 43 08:23 41280043 Object 30 A 13.0 35987-4977 a-b-c 1.43 - 59690 44 08:29 41280044 Object 30 A 13.0 35987-4977 a-b-c 1.43 - 59690 45 08:35 41280045 Object 30 A 13.0 35987-4977 a-b-c 1.43 - 59690 46 08:41 41280046 Object 30 A 13.0 35987-4977 a-b-c 1.43 - 59690 47 09:01 41280047 Object 43 A 13.0 48449-4980 a-b-c 1.43 - 59690 48 09:09 41280048 Object 43 B 13.5 48449-4980 a-b-c 1.95 - 59690 49 09:27 41280049 Object 43 B 13.5 46827-4981 a-b-c 1.97 - 59690 50 09:35 41280050 Object 43 A 13.0 46827-4981 a-b-c 1.45 - 59690 51 09:49 41280051 Object 43 A 13.0 46828-4982 a-b-c 1.44 - 59690 52 09:57 41280052 Object 43 B 13.5 46828-4982 a-b-c 1.96 - 59690 53 10:13 41280053 Object 43 B 13.5 46829-4983 a-b-c 1.96 - 59690 54 10:21 41280054 Object 43 A 13.0 46829-4983 a-b-c 1.43 - 59690 55 10:36 41280055 Object 43 A 13.0 46830-4984 a-b-c 1.69 - 59690 56 10:45 41280056 Object 43 B 13.5 46830-4984 a-b-c 1.96 - 59690 57 11:24 41280057 Object 43 B 13.5 40386-4986 a-b-c 2.11 - 59690 58 11:32 41280058 Object 43 A 13.0 40386-4986 a-b-c 1.61 - 59690 59 11:34 41280059 DomeFlat 5 A 13.0 40386-4986 a-b-c - 14= 3 59690 60 11:35 41280060 DomeFlat 5 A 13.0 40386-4986 a-b-c - 13= 8 59690 61 11:37 41280061 DomeFlat 5 A 13.0 40386-4986 a-b-c - 13= 8 cannot read file : /data/apogee/utr_cdr/59690/apRaw-41280062.fits ---------------------------------------------------------------------------= --- ---- eBOSS Data Summary ---- Reading FITS from /data/spectro//59690 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -= - n Time Filename design-field cfg flav shu Exp colA colB = colC Hrt - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -= - 1 00:00 b1-00340585 ----------- ---- bia Clo 0.0 1579 1728 333 2 00:00 r1-00340585 ----------- ---- bia Clo 0.0 1579 1728 333 3 00:01 b1-00340586 ----------- ---- dar Clo 20.0 1579 1728 333 4 00:01 r1-00340586 ----------- ---- dar Clo 20.0 1579 1728 333 5 00:04 b1-00340587 55179-104197 4959 arc Lef 4.1 1579 1728 333= HART 6 00:04 r1-00340587 55179-104197 4959 arc Lef 4.1 1579 1728 333= HART 7 00:05 b1-00340588 55179-104197 4959 arc Rig 4.1 1579 1728 333= HART 8 00:05 r1-00340588 55179-104197 4959 arc Rig 4.1 1579 1728 333= HART 9 00:07 b1-00340589 55179-104197 4959 arc Out 4.1 626 776 -621 10 00:07 r1-00340589 55179-104197 4959 arc Out 4.1 626 776 -621 11 00:09 b1-00340590 55179-104197 4959 fla Out 25.1 626 776 -621 12 00:09 r1-00340590 55179-104197 4959 fla Out 25.1 626 776 -621 13 02:08 r1-00340591 50361-102959 4961 arc Lef 4.1 626 776 -621= HART 14 02:08 b1-00340591 50361-102959 4961 arc Lef 4.1 626 776 -621= HART 15 02:09 b1-00340592 50361-102959 4961 arc Rig 4.1 626 776 -621= HART 16 02:09 r1-00340592 50361-102959 4961 arc Rig 4.1 626 776 -621= HART 17 02:10 b1-00340593 50361-102959 4961 arc Out 4.1 2108 2257 860 18 02:10 r1-00340593 50361-102959 4961 arc Out 4.1 2108 2257 860 19 02:12 b1-00340594 50361-102959 4961 fla Out 25.1 2108 2257 860 20 02:12 r1-00340594 50361-102959 4961 fla Out 25.1 2108 2257 860 21 02:20 r1-00340595 50361-102959 4961 sci Out 900.1 2108 2257 860 22 02:20 b1-00340595 50361-102959 4961 sci Out 900.1 2108 2257 860 23 02:40 r1-00340596 44085-101240 4962 arc Lef 4.1 2108 2257 860= HART 24 02:40 b1-00340596 44085-101240 4962 arc Lef 4.1 2108 2257 860= HART 25 02:41 b1-00340597 44085-101240 4962 arc Rig 4.1 2108 2257 860= HART 26 02:41 r1-00340597 44085-101240 4962 arc Rig 4.1 2108 2257 860= HART 27 02:42 b1-00340598 44085-101240 4962 arc Out 4.1 2465 2614 1218 28 02:42 r1-00340598 44085-101240 4962 arc Out 4.1 2465 2614 1218 29 02:43 b1-00340599 44085-101240 4962 fla Out 25.1 2465 2614 1218 30 02:43 r1-00340599 44085-101240 4962 fla Out 25.1 2465 2614 1218 31 02:49 r1-00340600 44085-101240 4962 sci Out 900.1 2465 2614 1218 32 02:49 b1-00340600 44085-101240 4962 sci Out 900.1 2465 2614 1218 33 03:06 r1-00340601 44085-101240 4962 sci Out 900.1 2465 2614 1218 34 03:06 b1-00340601 44085-101240 4962 sci Out 900.1 2465 2614 1218 35 03:32 b1-00340602 44086-101240 4963 sci Out 900.1 2465 2614 1218 36 03:32 r1-00340602 44086-101240 4963 sci Out 900.1 2465 2614 1218 37 03:57 r1-00340603 44087-101240 4964 sci Out 900.1 2465 2614 1218 38 03:57 b1-00340603 44087-101240 4964 sci Out 900.1 2465 2614 1218 39 04:14 r1-00340604 44087-101240 4964 sci Out 900.1 2465 2614 1218 40 04:14 b1-00340604 44087-101240 4964 sci Out 900.1 2465 2614 1218 41 04:41 r1-00340605 44088-101240 4965 sci Out 900.1 2465 2614 1218 42 04:41 b1-00340605 44088-101240 4965 sci Out 900.1 2465 2614 1218 43 04:57 r1-00340606 44088-101240 4965 sci Out 900.1 2465 2614 1218 44 04:57 b1-00340606 44088-101240 4965 sci Out 900.1 2465 2614 1218 45 05:35 b1-00340607 35987-20956 4966 sci Out 300.1 2465 2614 1218 46 05:35 r1-00340607 35987-20956 4966 sci Out 300.1 2465 2614 1218 47 05:55 r1-00340608 35987-20956 4969 sci Out 300.1 2465 2614 1218 48 05:55 b1-00340608 35987-20956 4969 sci Out 300.1 2465 2614 1218 49 06:16 b1-00340609 35987-20956 4972 sci Out 300.1 2465 2614 1218 50 06:16 r1-00340609 35987-20956 4972 sci Out 300.1 2465 2614 1218 51 06:54 b1-00340610 35987-20956 4977 sci Out 300.1 2465 2614 1218 52 06:54 r1-00340610 35987-20956 4977 sci Out 300.1 2465 2614 1218 53 08:47 b1-00340611 48449-102300 4980 arc Lef 4.1 2465 2614 1218= HART 54 08:47 r1-00340611 48449-102300 4980 arc Lef 4.1 2465 2614 1218= HART 55 08:48 b1-00340612 48449-102300 4980 arc Rig 4.1 2465 2614 1218= HART 56 08:48 r1-00340612 48449-102300 4980 arc Rig 4.1 2465 2614 1218= HART 57 08:49 b1-00340613 48449-102300 4980 arc Out 4.1 1035 1184 -211 58 08:49 r1-00340613 48449-102300 4980 arc Out 4.1 1035 1184 -211 59 08:50 b1-00340614 48449-102300 4980 fla Out 25.1 1035 1184 -211 60 08:50 r1-00340614 48449-102300 4980 fla Out 25.1 1035 1184 -211 61 08:54 r1-00340615 48449-102300 4980 sci Out 900.1 1035 1184 -211 62 08:54 b1-00340615 48449-102300 4980 sci Out 900.1 1035 1184 -211 63 09:14 r1-00340616 46827-101806 4981 arc Out 4.1 1035 1184 -211 64 09:14 b1-00340616 46827-101806 4981 arc Out 4.1 1035 1184 -211 65 09:16 b1-00340617 46827-101806 4981 fla Out 25.1 1035 1184 -211 66 09:16 r1-00340617 46827-101806 4981 fla Out 25.1 1035 1184 -211 67 09:20 b1-00340618 46827-101806 4981 sci Out 900.1 1035 1184 -211 68 09:20 r1-00340618 46827-101806 4981 sci Out 900.1 1035 1184 -211 69 09:42 b1-00340619 46828-101806 4982 sci Out 900.1 1035 1184 -211 70 09:42 r1-00340619 46828-101806 4982 sci Out 900.1 1035 1184 -211 71 10:06 r1-00340620 46829-101806 4983 sci Out 900.1 1035 1184 -211 72 10:06 b1-00340620 46829-101806 4983 sci Out 900.1 1035 1184 -211 73 10:29 r1-00340621 46830-101806 4984 sci Out 900.1 1035 1184 -211 74 10:29 b1-00340621 46830-101806 4984 sci Out 900.1 1035 1184 -211 75 11:12 r1-00340622 40386-100569 4986 arc Out 4.1 1035 1184 -211 76 11:12 b1-00340622 40386-100569 4986 arc Out 4.1 1035 1184 -211 77 11:14 b1-00340623 40386-100569 4986 fla Out 25.1 1035 1184 -211 78 11:14 r1-00340623 40386-100569 4986 fla Out 25.1 1035 1184 -211 79 11:17 r1-00340624 40386-100569 4986 sci Out 900.1 1035 1184 -211 80 11:17 b1-00340624 40386-100569 4986 sci Out 900.1 1035 1184 -211 ---- BOSS Spectrographs Collimation ---- Time TSP1 R1 B1 Move1 B1Resid =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 00:05:30 7.8 -1649 0.8 -953 -2.2 02:10:00 7.6 329 -8.3 1482 -3.6 02:41:50 7.4 329 -1.2 357 -0.1 08:48:50 6.3 -1319 4.8 -1430 0.3 ---- Weather Log ---- Wind 1 um DIMM IRSC Time Temp C Dewp C MPH Dir Dust fwhm mean sigma 18:00 13.2 -11.6 26.1 275.5 1535 18:15 13.1 -11.6 26.7 272.7 1863 18:30 12.6 -11.7 31.0 286.8 2036 18:45 13.7 -11.5 30.7 278.1 2107 19:00 14.3 -11.7 23.0 262.3 2910 19:15 15.8 -11.4 23.6 274.0 2366 19:30 16.2 -11.3 19.2 263.3 1424 19:45 15.2 -11.7 27.4 273.8 1695 20:00 14.5 -12.6 31.3 270.5 2391 20:15 15.4 -12.6 23.8 277.8 2426 20:30 15.0 -11.8 21.3 279.1 4430 20:45 15.8 -12.4 21.5 245.2 4953 21:00 15.7 -12.9 29.2 257.2 10209 21:15 16.5 -12.7 28.4 271.7 7783 21:30 15.4 -13.3 24.0 275.0 6716 21:45 17.4 -11.9 32.2 259.3 5971 22:00 16.1 -13.3 23.5 261.0 6090 22:15 16.9 -11.9 23.0 260.7 3487 22:30 17.2 -12.6 21.2 254.5 4368 22:45 17.2 -12.6 19.1 281.4 3001 23:00 17.2 -12.8 25.6 266.0 2965 23:15 17.3 -13.0 35.6 243.2 3031 23:30 16.7 -12.9 26.2 244.0 2543 23:45 16.8 -12.7 22.9 249.1 3182 00:00 16.4 -13.2 24.7 247.7 2985 00:15 15.9 -13.5 21.8 255.3 3463 00:30 15.7 -13.4 23.0 277.7 3059 00:45 15.3 -13.6 22.0 251.2 2625 01:00 15.2 -13.3 28.2 250.9 2196 01:15 14.6 -13.3 23.7 265.0 2282 01:30 14.0 -13.6 20.5 259.8 2197 01:45 13.8 -13.3 25.7 268.5 2166 02:00 13.4 -13.8 22.5 259.3 2038 02:15 13.2 -13.8 21.4 266.0 2220 02:30 13.2 -13.6 15.8 263.5 2137 88 9.94 02:45 13.3 -13.6 14.7 258.1 2224 92 9.40 03:00 13.1 -13.8 16.0 261.6 2401 107 3.92 03:15 13.2 -13.8 23.0 266.7 2144 102 3.75 03:30 12.8 -13.7 19.3 272.1 2094 141 1.40 03:45 12.9 -13.6 22.0 266.3 1934 273 20.32 04:00 12.2 -13.6 19.4 275.0 1944 88 9.09 04:15 12.6 -13.8 14.3 284.6 1968 249 19.53 04:30 12.7 -14.0 17.6 279.1 1955 87 12.03 04:45 12.4 -13.7 20.7 275.9 2005 244 21.01 05:00 12.5 -13.7 23.0 281.6 1857 88 9.94 05:15 12.4 -13.6 28.0 292.7 1723 215 11.59 05:30 12.4 -13.4 16.3 272.2 1956 156 3.40 05:45 12.1 -13.3 15.4 269.5 1901 141 2.47 06:00 12.0 -13.4 20.1 291.0 1819 146 4.71 06:15 12.0 -13.3 20.0 275.5 1905 156 3.84 06:30 11.9 -13.3 15.8 275.4 1740 87 10.51 06:45 11.6 -13.3 15.5 266.6 1715 239 17.74 07:00 11.6 -13.2 12.1 264.6 1811 88 7.55 07:15 11.5 -12.9 14.2 275.0 1814 107 2.07 07:30 11.5 -13.0 17.3 290.1 1718 200 6.03 07:45 11.2 -12.9 17.8 279.1 1712 112 5.09 08:00 11.1 -12.9 17.4 291.0 1667 102 9.65 08:15 11.1 -12.9 13.9 316.6 1715 166 3.87 08:30 10.9 -12.8 13.0 282.0 1707 161 4.86 08:45 10.8 -12.7 10.7 276.6 1705 151 1.68 09:00 10.8 -12.8 14.0 287.3 1672 88 13.60 09:15 10.5 -12.7 13.8 266.1 1672 88 10.81 09:30 10.0 -12.9 11.7 265.5 161 1.26 09:45 9.9 -13.1 10.9 277.7 332 34.37 10:00 9.9 -12.8 11.5 276.1 88 13.59 10:15 9.7 -12.9 14.6 280.9 151 1.48 10:30 9.8 -12.6 13.0 279.7 88 12.25 10:45 9.5 -12.2 12.8 311.3 166 2.57 11:00 9.3 -12.6 15.9 315.6 92 9.02 11:15 9.3 -11.9 14.8 305.3 269 22.07 11:30 9.3 -11.8 16.2 291.8 88 11.21 11:45 9.0 -11.9 10.2 293.5 149 5.79 ---- Problems in Detail ---- -- During Afternoon: fliswarm reconnect -- run 3 times : first and second was: 00:15:49 OBSERVER.viktorm 75 fliswarm f error=3D"Command 'reconnect' failed= because of an uncaught error 'ConnectionRefusedError: [Errno 111] Connect = call failed ('10.25.1.120', 19995)'. See traceback in the log for more info= rmation." 00:16:33 OBSERVER.viktorm 76 fliswarm f error=3D"Command 'reconnect' failed= because of an uncaught error 'ConnectionRefusedError: [Errno 111] Connect = call failed ('10.25.1.125', 19995)'. See traceback in the log for more info= rmation." 00:17:20 CmdDone: OBSERVER.viktorm 77 fliswarm reconnect ----------------- ____________________________________________________________ Logs of Nightly APO Activities https://urldefense.com/v3/__https://trac.sdss.org/tracmailman/browser/priva= te/sloan-25mlog/index.html__;!!K6Z8K8YTIA!UTJ-3w1pSsyQGs4vue3KOY7dBd4rRWt3D= rik8iVlanIO0Lv7VBQa8sa8QjWe5cc$ --_000_MW4PR02MB745922A772444ADA2CB6BB37D8F49MW4PR02MB7459namp_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable P {margin-top:0;margin-bo= ttom:0;} Hi John, & Jon, This seems like the right call -- Jon, please do consult MWM-exec & let= us know if we should change course! Thanks, -kevin From: Donor, John Sent: Thursday, April 21, 2022 12:18 PM To: Kevin Covey ; dmbiz@apo.nmsu.edu ; obs-report-25m@apo.nmsu.edu Cc: Jon Bird ; Drew Chojnowski <= ;drewski@nmsu.edu> Subject: Re: [sloan-25mlog 6024] Re: 2.5m obslog 20220420 (SJD 59690= ) Hey y'all, Since this is time sensitive (the entire epoch will be automatically mar= ked not started soon because of the short max_length for bright time) I wen= t ahead and marked that design manually complete. If that is a problem let = me know and we'll mark the whole epoch not started so it can be rescheduled. -John From: sloan-25mlog on behalf of Donor, John Sent: Thursday, April 21, 2022 11:14:53 AM To: Kevin Covey; dmbiz@apo.nmsu.edu; obs-report-25m@apo.nmsu.edu Cc: Jon Bird; Drew Chojnowski Subject: [sloan-25mlog 6024] Re: 2.5m obslog 20220420 (SJD 59690) I realized shortly after sending that email that coming back would mean = repeating all 4 designs, because bright time requires all observations= occur within a few hours. So I think we should definitely just hit "c= omplete" and deal with the one design's worth of no dither? But someone should sign off on that. -John From: Kevin Covey Sent: Thursday, April 21, 2022 11:09:43 AM To: Donor, John; dmbiz@apo.nmsu.edu; obs-report-25m@apo.nmsu.edu Cc: Jon Bird; Drew Chojnowski; Nidever,David Subject: Re: [sloan-25mlog 6021] 2.5m obslog 20220420 (SJD 59690) [EXTERNAL EMAIL WARNING= ] DO NOT CLICK LINKS or open attachments unless you recognize= the sender and know the content is safe. Hi John, I am AOK with not coming back, but I'm also cc'ing Drew & David so they= can assess if that will cause issues for the APOGEE pipeline... -kevin From: Donor, John Sent: Thursday, April 21, 2022 8:12 AM To: dmbiz@apo.nmsu.edu ; obs-report-25m@ap= o.nmsu.edu Cc: Kevin Covey ; Jon Bird Subject: Re: [sloan-25mlog 6021] 2.5m obslog 20220420 (SJD 59690) Hi Dmitry, 46830, the last design on field 101806 that you commented on, has two ex= posures taken at the same dither position (looks like "B", 13.499). Is there a reason you didn't dither on the last exposure? The automatic check won't allow a design to finish without a quali= ty A and B exposure. But I can manually mark this design finished if we don= 't want to come back for a single dither position (CC Kevin and Jon B.) -John From: sloan-25mlog &l= t;sloan-25mlog-bounces@sdss.org> on behalf of dmbiz@apo.nmsu.edu Sent: Thursday, April 21, 2022 6:53:43 AM To: obs-report-25m@apo.nmsu.edu Subject: [sloan-25mlog 6021] 2.5m obslog 20220420 (SJD 59690) [EXTERNAL EMAIL WARNING] DO NOT CLICK LINKS or= open attachments unless you recognize the sender and know the content is s= afe. &nb= sp; APACHE POINT OBSE= RVATORY SDSS 2.5M &nb= sp; = OBSERVING LOG &nb= sp; = Wednesday April 20, 2022 &nb= sp; = (SJD 59690) ---- Observing Team ---- Viktor Malanushenko Dmitry Bizyaev ---- Observing Plan ---- Monday-Friday, April 16-20 If clear, queue, then observe one dither field each night. Select from desi= gns 35987 (APOGEE), 35988 (BOSS), 35989 (APOGEE), 35990 (BOSS), until you g= et one successful complete dither on each. If there are clouds, then just observe from the queue. Observability windows should be good for second half of the night: put 3598= 7, 35988, 35989, 35990 into Kronos VizPreview for observability windows ---- Observing Summary ---- Clear, productive night with decent seeing. Observed: Science fields from the queue 02:06:22 Config Loaded: 102959-50361-4961 1x900s 1xAB DON= E 02:37:20 Config Loaded: 101240-44085-4962 2x900s 2xAB don= e design 03:23:01 Config Loaded: 101240-44086-4963 1x900s 1xAB don= e design 03:49:49 Config Loaded: 101240-44087-4964 2x900s 2xAB don= e design 04:29:30 Config Loaded: 101240-44088-4965 2x900s 2xAB DON= E FIELD 08:45:36 Config Loaded: 102300-48449-4980 1x900s 1xAB DON= E 09:09:29 Config Loaded: 101806-46827-4981 1x900s 1xAB DON= E 09:35:25 Config Loaded: 101806-46828-4982 1x900s 1xAB DON= E 09:59:26 Config Loaded: 101806-46829-4983 1x900s 1xAB DON= E 10:21:24 Config Loaded: 101806-46830-4984 1x900s 1xAB not done, altho= ugh the APOGEE signal is more than sufficient. 11:09:15 Config Loaded: 100569-40386-4986 1x900s 1xAB DON= E 10:45:28 Config Loaded: 103229-51302-4985 Jaeger collision: 902 - many times, see log. We were una= ble to load and execute the design because of the positioner 902 got locked every time. Ski= pped this field. Telescope dither test with design 35987 (APOGEE). 12pt : see log from 05:20Z to 08:41Z on the clear sky and 1.6"->1.3&= quot; seeing. Notes ----- -- dust accumulated: 13.9K -- in the afternoon it took us 3 attempts to make "fliswarm reco= nnect" successful. See "Problems in Detail" for errors= . -- We've got a line in the Kronos display right after we reloaded 'Obs Queu= e ': ERROR: ['could not schedule field at 59690.489695835284'] But in 10 minutes the error disappeared. -- We didn't have problems with the robot 1340 tonight. -- We finished all 4 designs for Field 101806, which is bright_8x4_v1= . Last design 46830 has got a very good signal, but was not turn= ed green in Kronos. -- Not unexpected, but the back/open motion of the enclosure is still running on slow speed ---- Engineering Issues ---- None ---- Observing Log ---- 00:33ZAfternoon: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --Test Actors, 23:56:44 -- APOGEE Short Dark - 23:57:34 -- 23:59:37 -- APOGEE: Evening Cals -- 00:48:08-- done -- boss bias, dark: files=3D340585,340586 00:05:42 Hartmann collimate output sp1: offset: r=3D -1649 (Out of focus); b=3D 0.8 (In Foc= us) sp1: pred. move: spAverageMove=3D -953 sp1: pred. resid: r=3D -696, b=3D -2.2, txt=3D OK, spTemp =3D = 8.2 BOSS collimation was checked. No adjustment was made. ---------- motorPosition sp1: 626,  = ; 776, -621; ---------- Median spectrographs temperatures: Tsp1=3D 8.1 Outside air temperature=3D 16.3; ---------- boss arc, flat: files=3D340589,340590 jaeger power on --gfas fliswarm reconnect -- run 3 times : first and second was errors (in section= "Problems in Detail") 00:17:20 CmdDone: OBSERVER.viktorm 77 fliswarm reconnect 00:19:07 CmdStarted: OBSERVER.viktorm 78 fliswarm talk -c gfa temperature -= 20 (00:26:13 CmdDone: OBSERVER.viktorm 78 fliswarm talk -c gfa temperature -20= ) 00:20:05 CmdDone: OBSERVER.viktorm 79 jaeger unwind 00:22:16 CmdDone: OBSERVER.viktorm 84 jaeger configuration load --from-posi= tions Checked and confirmed all robots are folded. Robot position checked with "jaeger fvc loop --no-apply". 00:23:32 OBSERVER.viktorm 86 jaeger i fvc_rms=3D43.42 00:23:32 OBSERVER.viktorm 86 jaeger i fvc_perc_90=3D68.9 00:23:32 OBSERVER.viktorm 86 jaeger i fvc_percent_reached=3D1.8 00:50Z start ventilation 01:00Z on the telescope all checked and ok. Night: =3D=3D=3D=3D=3D=3D 01:46:29 encl25m : open; =3D=3D=3D=3D=3D=3D=3D=3D 02:06:22 Config Loaded: 102959-50361-4961 02:10:09 Hartmann collimate output sp1: offset: r=3D 329 (In Focus); b=3D -8.3 (Out of focu= s) sp1: pred. move: spAverageMove=3D 1482 sp1: pred. resid: r=3D -1153, b=3D -3.6, txt=3D OK, spTemp =3D= 7.8 02:11:19 APO.viktorm.jaeger 6756 jaeger d text=3D"Saving processed ima= ge /data/fcam/59690/proc-fimg-fvc1n-0003.fits" 02:11:17 APO.viktorm.jaeger 6756 jaeger i text=3D"[FVC]: RMS full fit = 15.290 um." 02:11:17 APO.viktorm.jaeger 6756 jaeger i fvc_perc_90=3D23.5 02:20:44 apogee.expose=3DapRaw-41280013, 43, A, Object 02:20:50 boss exposure 900.0, file=3D340595 02:28:24 apogee.expose=3DapRaw-41280014, 43, B, Object =3D=3D=3D=3D=3D=3D=3D 02:37:20 Config Loaded: 101240-44085-4962 02:41:53 Hartmann collimate output sp1: offset: r=3D 329 (In Focus); b=3D -1.2 (In Focus) sp1: pred. move: spAverageMove=3D 357 sp1: pred. resid: r=3D -28, b=3D -0.1, txt=3D OK, spTemp =3D 7= .7 02:43:03 APO.viktorm.jaeger 6797 jaeger d text=3D"Saving processed ima= ge /data/fcam/59690/proc-fimg-fvc1n-0005.fits" 02:43:02 APO.viktorm.jaeger 6797 jaeger i text=3D"[FVC]: RMS full fit = 14.590 um." 02:43:02 APO.viktorm.jaeger 6797 jaeger i fvc_perc_90=3D24.7 02:49:29 apogee.expose=3DapRaw-41280015, 43, B, Object 02:49:35 boss exposure 900.0, file=3D340600 02:57:14 apogee.expose=3DapRaw-41280016, 43, A, Object 03:06:30 apogee.expose=3DapRaw-41280017, 43, A, Object 03:06:36 boss exposure 900.0, file=3D340601 03:14:11 apogee.expose=3DapRaw-41280018, 43, B, Object =3D=3D=3D=3D=3D=3D=3D=3D 03:23:01 Config Loaded: 101240-44086-4963 03:26:40 APO.viktorm.jaeger 6831 jaeger d text=3D"Saving processed ima= ge /data/fcam/59690/proc-fimg-fvc1n-0007.fits" 03:26:39 APO.viktorm.jaeger 6831 jaeger i text=3D"[FVC]: RMS full fit = 45.860 um." 03:26:39 APO.viktorm.jaeger 6831 jaeger i fvc_perc_90=3D24.6 03:32:00 apogee.expose=3DapRaw-41280019, 43, B, Object 03:32:06 boss exposure 900.0, file=3D340602 03:39:45 apogee.expose=3DapRaw-41280020, 43, A, Object =3D=3D=3D=3D=3D=3D=3D=3D 03:49:49 Config Loaded: 101240-44087-4964 03:53:11 APO.viktorm.jaeger 6861 jaeger d text=3D"Saving processed ima= ge /data/fcam/59690/proc-fimg-fvc1n-0009.fits" 03:53:10 APO.viktorm.jaeger 6861 jaeger i text=3D"[FVC]: RMS full fit = 31.620 um." 03:53:10 APO.viktorm.jaeger 6861 jaeger i fvc_perc_90=3D27.4 03:57:47 apogee.expose=3DapRaw-41280021, 46, A, Object 03:57:54 boss exposure 900.0, file=3D340603 04:06:01 apogee.expose=3DapRaw-41280022, 46, B, Object 04:14:10 apogee.expose=3DapRaw-41280023, 43, B, Object 04:14:17 boss exposure 900.0, file=3D340604 04:21:55 apogee.expose=3DapRaw-41280024, 43, A, Object =3D=3D=3D=3D=3D=3D=3D=3D 04:29:30 Config Loaded: 101240-44088-4965 04:33:03 APO.viktorm.jaeger 6885 jaeger d text=3D"Saving processed ima= ge /data/fcam/59690/proc-fimg-fvc1n-0011.fits" 04:33:02 APO.viktorm.jaeger 6885 jaeger i text=3D"[FVC]: RMS full fit = 386.140 um." We decided to re-do the FVC loop again: 04:37:27 APO.viktorm.jaeger 6891 jaeger i text=3D"[FVC]: RMS full fit = 385.660 um." 04:37:28 APO.viktorm.jaeger 6891 jaeger d text=3D"Saving processed ima= ge /data/fcam/59690/proc-fimg-fvc1n-0013.fits" 04:37:27 APO.viktorm.jaeger 6891 jaeger i fvc_perc_90=3D25.8 04:41:09 apogee.expose=3DapRaw-41280025, 46, A, Object 04:41:15 boss exposure 900.0, file=3D340605 04:49:21 apogee.expose=3DapRaw-41280026, 46, B, Object 04:57:31 apogee.expose=3DapRaw-41280027, 43, B, Object 04:57:38 boss exposure 900.0, file=3D340606 05:05:17 apogee.expose=3DapRaw-41280028, 43, A, Object (observers39) [observer@sdss5-hub: ~]$ field 101240 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Design/Field S/N^2=3D b1 &nb= sp; r1 MJDs =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Design 44085 2.90&nb= sp; 8.99 59690 Design 44086 1.59&nb= sp; 4.89 59690 Design 44087 2.91&nb= sp; 8.57 59690 Design 44088 2.78&nb= sp; 7.20 59690 This Field 101240 10.17 29.65 =3D=3D=3D=3D=3D=3D=3D=3D Telescope dither test with design 35987 (APOGEE) = =3D=3D=3D=3D=3D=3D=3D=3D 05:20Z started. The sky is clear. Seeing ~ 1.6" at the beginning= , 1.3 at the end. 05:22:43 CmdStarted: OBSERVER.viktorm 119 jaeger configuration load --no-cl= one 35987 05:26:14 CmdStarted: OBSERVER.viktorm 120 jaeger fvc loop --max-iterations = 4 --target-90-percentile 10 05:26:52 OBSERVER.viktorm 120 jaeger d text=3D"Saving processed image = /data/fcam/59690/proc-fimg-fvc1n-0014.fits" 05:27:30 OBSERVER.viktorm 120 jaeger d text=3D"Saving processed image = /data/fcam/59690/proc-fimg-fvc1n-0015.fits" 05:28:08 OBSERVER.viktorm 120 jaeger d text=3D"Saving processed image = /data/fcam/59690/proc-fimg-fvc1n-0016.fits" 05:28:36 OBSERVER.viktorm 120 jaeger d text=3D"Saving processed image = /data/fcam/59690/proc-fimg-fvc1n-0017.fits" 05:26:39 OBSERVER.viktorm 120 jaeger i fvc_perc_90=3D67.9 05:27:19 OBSERVER.viktorm 120 jaeger i fvc_perc_90=3D25.8 05:27:57 OBSERVER.viktorm 120 jaeger i fvc_perc_90=3D25.2 05:28:35 OBSERVER.viktorm 120 jaeger i fvc_perc_90=3D25.4 05:26:39 OBSERVER.viktorm 120 jaeger i text=3D"[FVC]: RMS full fit 54.= 450 um." 05:27:19 OBSERVER.viktorm 120 jaeger i text=3D"[FVC]: RMS full fit 19.= 020 um." 05:27:57 OBSERVER.viktorm 120 jaeger i text=3D"[FVC]: RMS full fit 19.= 070 um." 05:28:35 OBSERVER.viktorm 120 jaeger i text=3D"[FVC]: RMS full fit 18.= 880 um." 05:29:45 CmdStarted: OBSERVER.viktorm 122 jaeger configuration slew 05:23:46 Config Loaded: 20956-35987-4966 05:35:18 apogee.expose=3DapRaw-41280029, 30, A, Object 05:35:24 boss exposure 300.0, file=3D340607 05:44:29 CmdStarted: OBSERVER.viktorm 131 jaeger configuration load --no-cl= one 35987 05:48:01 OBSERVER.viktorm 132 jaeger d text=3D"Saving processed image = /data/fcam/59690/proc-fimg-fvc1n-0018.fits" 05:48:39 OBSERVER.viktorm 132 jaeger d text=3D"Saving processed image = /data/fcam/59690/proc-fimg-fvc1n-0019.fits" 05:49:17 OBSERVER.viktorm 132 jaeger d text=3D"Saving processed image = /data/fcam/59690/proc-fimg-fvc1n-0020.fits" 05:49:45 OBSERVER.viktorm 132 jaeger d text=3D"Saving processed image = /data/fcam/59690/proc-fimg-fvc1n-0021.fits" 05:47:48 OBSERVER.viktorm 132 jaeger i text=3D"[FVC]: RMS full fit 54.= 600 um." 05:48:28 OBSERVER.viktorm 132 jaeger i text=3D"[FVC]: RMS full fit 40.= 050 um." 05:49:06 OBSERVER.viktorm 132 jaeger i text=3D"[FVC]: RMS full fit 18.= 590 um." 05:49:44 OBSERVER.viktorm 132 jaeger i text=3D"[FVC]: RMS full fit 17.= 980 um." 05:47:48 OBSERVER.viktorm 132 jaeger i fvc_perc_90=3D69.7 05:48:28 OBSERVER.viktorm 132 jaeger i fvc_perc_90=3D24.7 05:49:06 OBSERVER.viktorm 132 jaeger i fvc_perc_90=3D23.3 05:49:44 OBSERVER.viktorm 132 jaeger i fvc_perc_90=3D15.3 05:52:34 CmdDone: OBSERVER.viktorm 133 jaeger configuration slew 05:45:28 Config Loaded: 20956-35987-4969 05:55:31 apogee.expose=3DapRaw-41280030, 30, A, Object 05:55:38 boss exposure 300.0, file=3D340608 06:08:29 OBSERVER.viktorm 140 jaeger i text=3D"[FVC]: RMS full fit 54.= 300 um." 06:09:09 OBSERVER.viktorm 140 jaeger i text=3D"[FVC]: RMS full fit 33.= 100 um." 06:09:47 OBSERVER.viktorm 140 jaeger i text=3D"[FVC]: RMS full fit 19.= 130 um." 06:10:25 OBSERVER.viktorm 140 jaeger i text=3D"[FVC]: RMS full fit 19.= 430 um." 06:08:42 OBSERVER.viktorm 140 jaeger d text=3D"Saving processed image = /data/fcam/59690/proc-fimg-fvc1n-0022.fits" 06:09:20 OBSERVER.viktorm 140 jaeger d text=3D"Saving processed image = /data/fcam/59690/proc-fimg-fvc1n-0023.fits" 06:09:58 OBSERVER.viktorm 140 jaeger d text=3D"Saving processed image = /data/fcam/59690/proc-fimg-fvc1n-0024.fits" 06:10:26 OBSERVER.viktorm 140 jaeger d text=3D"Saving processed image = /data/fcam/59690/proc-fimg-fvc1n-0025.fits" 06:08:29 OBSERVER.viktorm 140 jaeger i fvc_perc_90=3D70.1 06:09:09 OBSERVER.viktorm 140 jaeger i fvc_perc_90=3D26.4 06:09:47 OBSERVER.viktorm 140 jaeger i fvc_perc_90=3D25.1 06:10:25 OBSERVER.viktorm 140 jaeger i fvc_perc_90=3D27.6 06:13:08 CmdDone: OBSERVER.viktorm 141 jaeger configuration slew 06:05:36 Config Loaded: 20956-35987-4972 06:16:06 apogee.expose=3DapRaw-41280031, 30, A, Object 06:16:13 boss exposure 300.0, file=3D340609 06:30:23 CmdDone: OBSERVER.viktorm 152 jaeger configuration reverse 06:30:50 CmdStarted: OBSERVER.viktorm 153 jaeger configuration load --no-cl= one --epoch-delay 4800 35987 06:32:17 .jaeger 0 jaeger w text=3D"Positioner 437 replied to SEND_TRA= JECTORY_DATA UID=3D4379 with 'VALUE_OUT_OF_RANGE'. (JaegerUserWarning)"= ; 06:32:17 OBSERVER.viktorm 154 jaeger f error=3D"Trajectory failed with= error: Something went wrong sending the trajectory: At least one SEND_TRAJ= ECTORY_COMMAND failed" 06:32:17 CmdDone: OBSERVER.viktorm 154 jaeger configuration execute Trying again with different epochdelay: 06:35:36 CmdStarted: OBSERVER.viktorm 157 jaeger configuration load --no-cl= one --epoch-delay 3600 35987 06:38:56 OBSERVER.viktorm 159 jaeger d text=3D"Saving processed image = /data/fcam/59690/proc-fimg-fvc1n-0026.fits" 06:39:34 OBSERVER.viktorm 159 jaeger d text=3D"Saving processed image = /data/fcam/59690/proc-fimg-fvc1n-0027.fits" 06:40:12 OBSERVER.viktorm 159 jaeger d text=3D"Saving processed image = /data/fcam/59690/proc-fimg-fvc1n-0028.fits" 06:40:40 OBSERVER.viktorm 159 jaeger d text=3D"Saving processed image = /data/fcam/59690/proc-fimg-fvc1n-0029.fits" 06:38:45 OBSERVER.viktorm 159 jaeger i fvc_perc_90=3D71.2 06:39:23 OBSERVER.viktorm 159 jaeger i fvc_perc_90=3D25.4 06:40:01 OBSERVER.viktorm 159 jaeger i fvc_perc_90=3D24.6 06:40:39 OBSERVER.viktorm 159 jaeger i fvc_perc_90=3D22.8 06:38:45 OBSERVER.viktorm 159 jaeger i text=3D"[FVC]: RMS full fit 54.= 690 um." 06:39:23 OBSERVER.viktorm 159 jaeger i text=3D"[FVC]: RMS full fit 32.= 500 um." 06:40:01 OBSERVER.viktorm 159 jaeger i text=3D"[FVC]: RMS full fit 17.= 500 um." 06:40:39 OBSERVER.viktorm 159 jaeger i text=3D"[FVC]: RMS full fit 16.= 420 um." ** Execute the following command 5 times: jaeger fvc loop --no-apply --no-w= rite-summary. Record these 5 exposure numbers in the log 06:42:05 OBSERVER.viktorm 160 jaeger d text=3D"Saving processed image = /data/fcam/59690/proc-fimg-fvc1n-0030.fits" 06:42:55 OBSERVER.viktorm 161 jaeger d text=3D"Saving processed image = /data/fcam/59690/proc-fimg-fvc1n-0031.fits" 06:43:44 OBSERVER.viktorm 162 jaeger d text=3D"Saving processed image = /data/fcam/59690/proc-fimg-fvc1n-0032.fits" 06:44:30 OBSERVER.viktorm 163 jaeger d text=3D"Saving processed image = /data/fcam/59690/proc-fimg-fvc1n-0033.fits" 06:45:12 OBSERVER.viktorm 164 jaeger d text=3D"Saving processed image = /data/fcam/59690/proc-fimg-fvc1n-0034.fits" 06:42:03 OBSERVER.viktorm 160 jaeger i text=3D"[FVC]: RMS full fit 17.= 230 um." 06:42:54 OBSERVER.viktorm 161 jaeger i text=3D"[FVC]: RMS full fit 18.= 850 um." 06:43:42 OBSERVER.viktorm 162 jaeger i text=3D"[FVC]: RMS full fit 16.= 820 um." 06:44:28 OBSERVER.viktorm 163 jaeger i text=3D"[FVC]: RMS full fit 14.= 790 um." 06:45:11 OBSERVER.viktorm 164 jaeger i text=3D"[FVC]: RMS full fit 18.= 200 um." 06:42:03 OBSERVER.viktorm 160 jaeger i fvc_perc_90=3D23.1 06:42:54 OBSERVER.viktorm 161 jaeger i fvc_perc_90=3D26.0 06:43:42 OBSERVER.viktorm 162 jaeger i fvc_perc_90=3D23.5 06:44:28 OBSERVER.viktorm 163 jaeger i fvc_perc_90=3D19.2 06:45:11 OBSERVER.viktorm 164 jaeger i fvc_perc_90=3D26.2 06:45:46 CmdStarted: OBSERVER.viktorm 165 jaeger configuration slew ** base offset 06:54:27 apogee.expose=3DapRaw-41280032, 30, A, Object 06:54:34 boss exposure 300.0, file=3D340610 07:02:13 CmdStarted: OBSERVER.viktorm 168 cherno offset -1.20 0.58 -432 07:05:22 apogee.expose=3DapRaw-41280033, 30, A, Object 07:10:45 CmdStarted: OBSERVER.viktorm 169 cherno offset -0.43 0.34 -432 07:13:16 apogee.expose=3DapRaw-41280034, 30, A, Object 07:18:27 CmdStarted: OBSERVER.viktorm 170 cherno offset -0.35 -0.21 -432 07:19:52 apogee.expose=3DapRaw-41280035, 30, A, Object 07:25:23 CmdStarted: OBSERVER.viktorm 171 cherno offset 0.70 0.33 -432 07:27:46 apogee.expose=3DapRaw-41280036, 30, A, Object 07:33:00 CmdStarted: OBSERVER.dmbiz 36 cherno offset -0.41 -1.05 -432 07:35:10 apogee.expose=3DapRaw-41280037, 30, A, Object 07:40:37 CmdDone: OBSERVER.viktorm 174 cherno offset 0.31 -0.08 -432 07:43:25 apogee.expose=3DapRaw-41280038, 30, A, Object 07:48:56 CmdStarted: OBSERVER.viktorm 175 cherno offset 0 0 -432 07:50:24 apogee.expose=3DapRaw-41280039, 30, A, Object 07:55:47 CmdStarted: OBSERVER.viktorm 176 cherno offset 1.97 0.35 -432 07:57:26 apogee.expose=3DapRaw-41280040, 30, A, Object 08:02:41 CmdStarted: OBSERVER.viktorm 177 cherno offset -2.07 -0.02 -432 08:04:49 apogee.expose=3DapRaw-41280041, 30, A, Object 08:10:19 CmdStarted: OBSERVER.viktorm 180 cherno offset -0.12 0.92 -432 08:11:47 apogee.expose=3DapRaw-41280042, 30, A, Object 08:17:03 CmdStarted: OBSERVER.viktorm 181 cherno offset 0.71 0.36 -432 08:18:10 apogee.expose=3DapRaw-41280043, 30, A, Object 08:23:21 CmdStarted: OBSERVER.viktorm 182 cherno offset -0.54 -0.40 -432 08:24:37 apogee.expose=3DapRaw-41280044, 30, A, Object 08:29:47 CmdStarted: OBSERVER.viktorm 183 cherno offset 0.77 0.11 -432 08:30:44 apogee.expose=3DapRaw-41280045, 30, A, Object 08:35:54 CmdStarted: OBSERVER.viktorm 184 cherno offset 0 0 -432 08:37:00 apogee.expose=3DapRaw-41280046, 30, A, Object =3D=3D=3D=3D=3D=3D=3D=3D Returning back to the queue observations =3D=3D=3D= =3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D 08:45:36 Config Loaded: 102300-48449-4980 08:49:03 sp1.motor.move=3D -1430, -1430, -1429 08:49:03 Hartmann collimate output sp1: offset: r=3D -1319 (Out of focus); b=3D 4.8 (Out of= focus) sp1: pred. move: spAverageMove=3D -1430 sp1: pred. resid: r=3D 111, b=3D 0.3, txt=3D OK, spTemp =3D 6.= 4 08:50:13 APO.viktorm.jaeger 6999 jaeger d text=3D"Saving processed ima= ge /data/fcam/59690/proc-fimg-fvc1n-0036.fits" 08:50:12 APO.viktorm.jaeger 6999 jaeger i text=3D"[FVC]: RMS full fit = 363.660 um." 08:50:12 APO.viktorm.jaeger 6999 jaeger i fvc_perc_90=3D22.9 08:54:16 apogee.expose=3DapRaw-41280047, 43, A, Object 08:54:22 boss exposure 900.0, file=3D340615 09:01:57 apogee.expose=3DapRaw-41280048, 43, B, Object =3D=3D=3D=3D=3D=3D=3D=3D 09:09:29 Config Loaded: 101806-46827-4981 09:13:30 APO.viktorm.jaeger 7031 jaeger d text=3D"Saving processed ima= ge /data/fcam/59690/proc-fimg-fvc1n-0038.fits" 09:13:29 APO.viktorm.jaeger 7031 jaeger i fvc_perc_90=3D24.9 09:13:29 APO.viktorm.jaeger 7031 jaeger i text=3D"[FVC]: RMS full fit = 41.860 um." 09:13:30 APO.viktorm.jaeger 7031 jaeger d text=3D"Saving processed ima= ge /data/fcam/59690/proc-fimg-fvc1n-0038.fits" 09:13:29 APO.viktorm.jaeger 7031 jaeger i text=3D"[FVC]: RMS full fit = 41.860 um." 09:13:29 APO.viktorm.jaeger 7031 jaeger i fvc_perc_90=3D24.9 09:20:10 apogee.expose=3DapRaw-41280049, 43, B, Object 09:20:16 boss exposure 900.0, file=3D340618 09:27:55 apogee.expose=3DapRaw-41280050, 43, A, Object =3D=3D=3D=3D=3D=3D=3D=3D 09:35:25 Config Loaded: 101806-46828-4982 09:38:15 APO.viktorm.jaeger 7060 jaeger d text=3D"Saving processed ima= ge /data/fcam/59690/proc-fimg-fvc1n-0040.fits" 09:38:10 APO.viktorm.jaeger 7060 jaeger i fvc_perc_90=3D23.0 09:42:35 apogee.expose=3DapRaw-41280051, 43, A, Object 09:42:42 boss exposure 900.0, file=3D340619 09:50:17 apogee.expose=3DapRaw-41280052, 43, B, Object =3D=3D=3D=3D=3D=3D=3D=3D 09:59:26 Config Loaded: 101806-46829-4983 10:02:15 APO.viktorm.jaeger 7090 jaeger d text=3D"Saving processed ima= ge /data/fcam/59690/proc-fimg-fvc1n-0042.fits" 10:02:13 APO.viktorm.jaeger 7090 jaeger i fvc_perc_90=3D24.9 10:06:07 apogee.expose=3DapRaw-41280053, 43, B, Object 10:06:14 boss exposure 900.0, file=3D340620 10:13:53 apogee.expose=3DapRaw-41280054, 43, A, Object =3D=3D=3D=3D=3D=3D=3D=3D 10:21:24 Config Loaded: 101806-46830-4984 10:24:56 APO.viktorm.jaeger 7115 jaeger d text=3D"Saving processed ima= ge /data/fcam/59690/proc-fimg-fvc1n-0044.fits" 10:24:55 APO.viktorm.jaeger 7115 jaeger i fvc_perc_90=3D22.7 10:28:55 apogee.expose=3DapRaw-41280055, 43, A, Object 10:29:01 boss exposure 900.0, file=3D340621 10:37:57 apogee.expose=3DapRaw-41280056, 43, B, Object Last APOGEE did not start automatically because of the error: 10:32:45 APO.viktorm.apogee 7133 apogee f errorMessage=3D"cannot execu= te command: Controller is busy..." 10:32:45 CmdDone: APO.viktorm.apogee 7133 apogee expose time=3D459.0 object= =3Dobject 10:32:45 APO.viktorm 60 hal e error=3D"Command 'apogee expose time=3D4= 59.0 object=3Dobject' failed." We started it manually. =3D=3D=3D=3D=3D=3D=3D=3D 10:45:28 Config Loaded: 103229-51302-4985 We've got a robot collision: 10:46:46 .jaeger 0 jaeger f text=3D"A collision was detected in positi= oner 902. Sending SEND_TRAJECTORY_ABORT and locking the FPS." Tried to unwind, but it did not work well 10:50:40 OBSERVER.viktorm 204 jaeger f error=3D"Command 'unwind' faile= d because of an uncaught error 'RuntimeError: Grid is Kaiju-collided, canno= t generate paths'. See traceback in the log for more information." 10:50:40 CmdDone: OBSERVER.viktorm 204 jaeger unwind We were able to explode more than one time and then unwind, but repeated at= tempts to load this design brought up the same problem with the robot 902. We decided to skip the design. Some 18 minutes lost. =3D=3D=3D=3D=3D=3D=3D=3D 11:09:15 Config Loaded: 100569-40386-4986 11:11:19 APO.viktorm.jaeger 7181 jaeger d text=3D"Saving processed ima= ge /data/fcam/59690/proc-fimg-fvc1n-0046.fits" 11:11:17 APO.viktorm.jaeger 7181 jaeger i fvc_perc_90=3D24.1 11:17:34 apogee.expose=3DapRaw-41280057, 43, B, Object 11:17:40 boss exposure 900.0, file=3D340624 11:25:20 apogee.expose=3DapRaw-41280058, 43, A, Object 11:42:29 encl25m : closed; Calibrations started. ---- Mirror Numbers ---- PRIMARY: -------- Scale: 1.000000 MIGS TONI= GHT NOMINAL Axial A: 0.0060 0.004= 0 Axial B: 0.0040 0.003= 0 Axial C: -0.1180 -0.1150 Trans D: -0.0337 -0.0338 Lateral E: 0.3315 0.3315 Lateral F: 0.5345 0.5350 GALILS Commanded: 12500 -1600 5300 47050 -8800 -8400 Actual: 12500 -1600 5300 47050 -8800 -8400 SETMIR VALUES PrimDesOrient: 0.0 -59.5 -9.15 -512.0 1906.3 PrimOrient: 1.07 -59.51 -9.21 -512.13 1906.31 SECONDARY: ---------- Focus: 0.0000 Air temp: 16.22 Altitude: 30.0 MIGS TONI= GHT NOMINAL Axial A: -9.6540 -9.6570 Axial B: -9.6650 -9.6710 Axial C: -9.6980 -9.6970 Trans D: 0.1160 0.114= 0 GALILS Commanded: 4883550 4877900 4903850 -50 -3450 Actual: 4859079 4859117 4859142 -50 -3450 SETMIR VALUES SecDesOrient: 3857.0 0.0 0.0 0.0 42.22 SecOrient: 3856.92 -0.01 0.01 0.01 42.87 ---- Telescope Status ---- 00:50Z Started ventilation 01:46Z encl25m : open 11:42Z encl25m : closed No interlocks bypassed Status at: 11:47Z Telescope stowed at: 121.0, 30.0, 0.0 m= ount Instrument mounted: FPS Counterweights at: 68= 5/703 LN2 autofill systems: Connected and turned on 180L LN2 dewar scales: SP1 186 lb= s, 23 psi FPS Status =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D CP10 Loop =3D 7.01 L/min GFA Loop =3D 1.81 L/min Supply pressure =3D 41.0 psi Return pressure =3D 26.9 psi Chiller Status =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Set temperarature =3D 9.5 C Flow =3D 3.6 Gpm ---- Software Used ---- Get Versions, 04/20/22, 23:57:35 -- STUI software: STUI : 1.7.17b1 alerts : 2.2.0 boss : 6.0.1 -boss.specMec : 3.0.4 apo : 3.0.1 fliswarm : 0.2.1-alpha.0 jaeger : 0.14.1-alpha.0 kaiju : 1.2.2-alpha.1 coordio : 1.2.2-alpha.1 fps_calibrati : APO.2021.12.25+b1cf2fd cherno : 0.1.1-alpha.0 hal : 0.1.1-alp= ha.0 hartmann : 2.0.2 hub : 5.0.1a0 mcp : 2.1 -mcp.plc : 34 -mcp.azFiduci : v1_148 -mcp.altFiduc : v1_148 -mcp.rotFiduc : v1_148 tcc : 1.4.0 apogee : not availble apogeecal : not availble apogeeql : v1_7_0dev STUIAdditions : 3.1.13 --------------Other Versions-------------- sdss-obstools : 3.7.14 TPM = : 1.2-31-gd7eb MCP = : 2.1 PLC = : V34 Fiducials : v1_= 148 idlspec2d : v6_= 0_7 Kronos &nb= sp; : 1.2.1a roboscheduler : 1.2.2a /dev/sdb1 = 2.0T 170G 1.9T 9% /home 10.25.1.230:/data 21T = 18T 2.1T 90% /data ---- APOGEE Data Summary ---- APOGEE data list, mjd=3D59690, all raw_data: /data/apogee/utr_cdr/59690/apRaw-*.fits archive: /data/apogee/archive/59690/apR-[a,b,c]-*.apz quickred: /data/apogee/quickred/59690/apq-*.fits ---------------------------------------------------------------------------= --- mjd i UT File/Exp Imtype Nr= ead A/B Dth Design-Cfg Arch Offset %Flux Comm ---------------------------------------------------------------------= --------- 59690 1 23:58 41280001 Dark = 3 A 13.0 ---------- a-b-c = - - 59690 2 00:10 41280002 Dark = 60 A 13.0 ---------- a-b-c -&nbs= p; - 59690 3 00:12 41280003 QuarFlat 10 A 13= .0 ---------- a-b-c - 100 59690 4 00:14 41280004 Arc-Thar 12 B 13= .5 ---------- a-b-c 1.92 - 59690 5 00:21 41280005 Arc-Une 40 B&nbs= p; 13.5 ---------- a-b-c - - 59690 6 00:24 41280006 Arc-Thar 12 A 13= .0 ---------- a-b-c 1.53 - 59690 7 00:31 41280007 Arc-Une 40 A&nbs= p; 13.0 ---------- a-b-c - - 59690 8 00:42 41280008 Dark = 60 A 13.0 ---------- a-b-c -&nbs= p; - 59690 9 00:47 41280009 Arc---- 30 A&nbs= p; 13.0 ---------- a-b-c - - = ; FPI 59690 10 01:53 41280010 DomeFlat 5 A 13= .0 -999-4960 a-b-c - 332 59690 11 01:55 41280011 DomeFlat 5 A 13= .0 -999-4960 a-b-c - 141 59690 12 01:57 41280012 DomeFlat 5 A 13= .0 -999-4960 a-b-c - 137 59690 13 02:27 41280013 Object 43 A&nbs= p; 13.0 50361-4961 a-b-c 1.42 - 59690 14 02:35 41280014 Object 43 B&nbs= p; 13.5 50361-4961 a-b-c 1.94 - 59690 15 02:56 41280015 Object 43 B&nbs= p; 13.5 44085-4962 a-b-c 1.94 - 59690 16 03:04 41280016 Object 43 A&nbs= p; 13.0 44085-4962 a-b-c 1.42 - 59690 17 03:13 41280017 Object 43 A&nbs= p; 13.0 44085-4962 a-b-c 1.42 - 59690 18 03:21 41280018 Object 43 B&nbs= p; 13.5 44085-4962 a-b-c 1.94 - 59690 19 03:39 41280019 Object 43 B&nbs= p; 13.5 44086-4963 a-b-c 1.94 - 59690 20 03:46 41280020 Object 43 A&nbs= p; 13.0 44086-4963 a-b-c 1.42 - 59690 21 04:05 41280021 Object 46 A&nbs= p; 13.0 44087-4964 a-b-c 1.42 - 59690 22 04:13 41280022 Object 46 B&nbs= p; 13.5 44087-4964 a-b-c 1.93 - 59690 23 04:21 41280023 Object 43 B&nbs= p; 13.5 44087-4964 a-b-c 1.94 - 59690 24 04:29 41280024 Object 43 A&nbs= p; 13.0 44087-4964 a-b-c 1.42 - 59690 25 04:48 41280025 Object 46 A&nbs= p; 13.0 44088-4965 a-b-c 1.42 - 59690 26 04:57 41280026 Object 46 B&nbs= p; 13.5 44088-4965 a-b-c 1.94 - 59690 27 05:04 41280027 Object 43 B&nbs= p; 13.5 44088-4965 a-b-c 1.94 - 59690 28 05:12 41280028 Object 43 A&nbs= p; 13.0 44088-4965 a-b-c 1.42 - 59690 29 05:40 41280029 Object 30 A&nbs= p; 13.0 35987-4966 a-b-c 1.43 - 59690 30 06:00 41280030 Object 30 A&nbs= p; 13.0 35987-4969 a-b-c 1.43 - 59690 31 06:21 41280031 Object 30 A&nbs= p; 13.0 35987-4972 a-b-c 1.43 - 59690 32 06:59 41280032 Object 30 A&nbs= p; 13.0 35987-4977 a-b-c 1.43 - 59690 33 07:10 41280033 Object 30 A&nbs= p; 13.0 35987-4977 a-b-c 1.43 - 59690 34 07:18 41280034 Object 30 A&nbs= p; 13.0 35987-4977 a-b-c 1.43 - 59690 35 07:24 41280035 Object 30 A&nbs= p; 13.0 35987-4977 a-b-c 1.44 - 59690 36 07:32 41280036 Object 30 A&nbs= p; 13.0 35987-4977 a-b-c 1.43 - 59690 37 07:40 41280037 Object 30 A&nbs= p; 13.0 35987-4977 a-b-c 1.43 - 59690 38 07:48 41280038 Object 30 A&nbs= p; 13.0 35987-4977 a-b-c 1.42 - 59690 39 07:55 41280039 Object 30 A&nbs= p; 13.0 35987-4977 a-b-c 1.42 - 59690 40 08:02 41280040 Object 30 A&nbs= p; 13.0 35987-4977 a-b-c 1.43 - 59690 41 08:09 41280041 Object 30 A&nbs= p; 13.0 35987-4977 a-b-c 1.44 - 59690 42 08:16 41280042 Object 30 A&nbs= p; 13.0 35987-4977 a-b-c 1.43 - 59690 43 08:23 41280043 Object 30 A&nbs= p; 13.0 35987-4977 a-b-c 1.43 - 59690 44 08:29 41280044 Object 30 A&nbs= p; 13.0 35987-4977 a-b-c 1.43 - 59690 45 08:35 41280045 Object 30 A&nbs= p; 13.0 35987-4977 a-b-c 1.43 - 59690 46 08:41 41280046 Object 30 A&nbs= p; 13.0 35987-4977 a-b-c 1.43 - 59690 47 09:01 41280047 Object 43 A&nbs= p; 13.0 48449-4980 a-b-c 1.43 - 59690 48 09:09 41280048 Object 43 B&nbs= p; 13.5 48449-4980 a-b-c 1.95 - 59690 49 09:27 41280049 Object 43 B&nbs= p; 13.5 46827-4981 a-b-c 1.97 - 59690 50 09:35 41280050 Object 43 A&nbs= p; 13.0 46827-4981 a-b-c 1.45 - 59690 51 09:49 41280051 Object 43 A&nbs= p; 13.0 46828-4982 a-b-c 1.44 - 59690 52 09:57 41280052 Object 43 B&nbs= p; 13.5 46828-4982 a-b-c 1.96 - 59690 53 10:13 41280053 Object 43 B&nbs= p; 13.5 46829-4983 a-b-c 1.96 - 59690 54 10:21 41280054 Object 43 A&nbs= p; 13.0 46829-4983 a-b-c 1.43 - 59690 55 10:36 41280055 Object 43 A&nbs= p; 13.0 46830-4984 a-b-c 1.69 - 59690 56 10:45 41280056 Object 43 B&nbs= p; 13.5 46830-4984 a-b-c 1.96 - 59690 57 11:24 41280057 Object 43 B&nbs= p; 13.5 40386-4986 a-b-c 2.11 - 59690 58 11:32 41280058 Object 43 A&nbs= p; 13.0 40386-4986 a-b-c 1.61 - 59690 59 11:34 41280059 DomeFlat 5 A 13= .0 40386-4986 a-b-c - 143 59690 60 11:35 41280060 DomeFlat 5 A 13= .0 40386-4986 a-b-c - 138 59690 61 11:37 41280061 DomeFlat 5 A 13= .0 40386-4986 a-b-c - 138 cannot read file : /data/apogee/utr_cdr/59690/apRaw-4128= 0062.fits ---------------------------------------------------------------------------= --- ---- eBOSS Data Summary ---- Reading FITS from /data/spectro//59690 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -= - n Time Filename design-field cfg = flav shu Exp colA co= lB colC Hrt - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -= - 1 00:00 b1-00340585 ----------- ---- bia Clo 0.0&n= bsp; 1579 1728 333 2 00:00 r1-00340585 ----------- ---- bia Clo 0.0&n= bsp; 1579 1728 333 3 00:01 b1-00340586 ----------- ---- dar Clo 20.0 &= nbsp; 1579 1728 333 4 00:01 r1-00340586 ----------- ---- dar Clo 20.0 &= nbsp; 1579 1728 333 5 00:04 b1-00340587 55179-104197 4959 arc Lef 4.1&= nbsp; 1579 1728 333 HART 6 00:04 r1-00340587 55179-104197 4959 arc Lef 4.1&= nbsp; 1579 1728 333 HART 7 00:05 b1-00340588 55179-104197 4959 arc Rig 4.1&= nbsp; 1579 1728 333 HART 8 00:05 r1-00340588 55179-104197 4959 arc Rig 4.1&= nbsp; 1579 1728 333 HART 9 00:07 b1-00340589 55179-104197 4959 arc Out 4.1&= nbsp; 626 776 -621 10 00:07 r1-00340589 55179-104197 4959 arc Out 4.1&= nbsp; 626 776 -621 11 00:09 b1-00340590 55179-104197 4959 fla Out 25.1 = 626 776 -621 12 00:09 r1-00340590 55179-104197 4959 fla Out 25.1 = 626 776 -621 13 02:08 r1-00340591 50361-102959 4961 arc Lef 4.1&= nbsp; 626 776 -621 HART 14 02:08 b1-00340591 50361-102959 4961 arc Lef 4.1&= nbsp; 626 776 -621 HART 15 02:09 b1-00340592 50361-102959 4961 arc Rig 4.1&= nbsp; 626 776 -621 HART 16 02:09 r1-00340592 50361-102959 4961 arc Rig 4.1&= nbsp; 626 776 -621 HART 17 02:10 b1-00340593 50361-102959 4961 arc Out 4.1&= nbsp; 2108 2257 860 18 02:10 r1-00340593 50361-102959 4961 arc Out 4.1&= nbsp; 2108 2257 860 19 02:12 b1-00340594 50361-102959 4961 fla Out 25.1 = 2108 2257 860 20 02:12 r1-00340594 50361-102959 4961 fla Out 25.1 = 2108 2257 860 21 02:20 r1-00340595 50361-102959 4961 sci Out 900.1  = ; 2108 2257 860 22 02:20 b1-00340595 50361-102959 4961 sci Out 900.1  = ; 2108 2257 860 23 02:40 r1-00340596 44085-101240 4962 arc Lef 4.1&= nbsp; 2108 2257 860 HART 24 02:40 b1-00340596 44085-101240 4962 arc Lef 4.1&= nbsp; 2108 2257 860 HART 25 02:41 b1-00340597 44085-101240 4962 arc Rig 4.1&= nbsp; 2108 2257 860 HART 26 02:41 r1-00340597 44085-101240 4962 arc Rig 4.1&= nbsp; 2108 2257 860 HART 27 02:42 b1-00340598 44085-101240 4962 arc Out 4.1&= nbsp; 2465 2614 1218 28 02:42 r1-00340598 44085-101240 4962 arc Out 4.1&= nbsp; 2465 2614 1218 29 02:43 b1-00340599 44085-101240 4962 fla Out 25.1 = 2465 2614 1218 30 02:43 r1-00340599 44085-101240 4962 fla Out 25.1 = 2465 2614 1218 31 02:49 r1-00340600 44085-101240 4962 sci Out 900.1  = ; 2465 2614 1218 32 02:49 b1-00340600 44085-101240 4962 sci Out 900.1  = ; 2465 2614 1218 33 03:06 r1-00340601 44085-101240 4962 sci Out 900.1  = ; 2465 2614 1218 34 03:06 b1-00340601 44085-101240 4962 sci Out 900.1  = ; 2465 2614 1218 35 03:32 b1-00340602 44086-101240 4963 sci Out 900.1  = ; 2465 2614 1218 36 03:32 r1-00340602 44086-101240 4963 sci Out 900.1  = ; 2465 2614 1218 37 03:57 r1-00340603 44087-101240 4964 sci Out 900.1  = ; 2465 2614 1218 38 03:57 b1-00340603 44087-101240 4964 sci Out 900.1  = ; 2465 2614 1218 39 04:14 r1-00340604 44087-101240 4964 sci Out 900.1  = ; 2465 2614 1218 40 04:14 b1-00340604 44087-101240 4964 sci Out 900.1  = ; 2465 2614 1218 41 04:41 r1-00340605 44088-101240 4965 sci Out 900.1  = ; 2465 2614 1218 42 04:41 b1-00340605 44088-101240 4965 sci Out 900.1  = ; 2465 2614 1218 43 04:57 r1-00340606 44088-101240 4965 sci Out 900.1  = ; 2465 2614 1218 44 04:57 b1-00340606 44088-101240 4965 sci Out 900.1  = ; 2465 2614 1218 45 05:35 b1-00340607 35987-20956 4966 sci Out 300.1 = 2465 2614 1218 46 05:35 r1-00340607 35987-20956 4966 sci Out 300.1 = 2465 2614 1218 47 05:55 r1-00340608 35987-20956 4969 sci Out 300.1 = 2465 2614 1218 48 05:55 b1-00340608 35987-20956 4969 sci Out 300.1 = 2465 2614 1218 49 06:16 b1-00340609 35987-20956 4972 sci Out 300.1 = 2465 2614 1218 50 06:16 r1-00340609 35987-20956 4972 sci Out 300.1 = 2465 2614 1218 51 06:54 b1-00340610 35987-20956 4977 sci Out 300.1 = 2465 2614 1218 52 06:54 r1-00340610 35987-20956 4977 sci Out 300.1 = 2465 2614 1218 53 08:47 b1-00340611 48449-102300 4980 arc Lef 4.1&= nbsp; 2465 2614 1218 HART 54 08:47 r1-00340611 48449-102300 4980 arc Lef 4.1&= nbsp; 2465 2614 1218 HART 55 08:48 b1-00340612 48449-102300 4980 arc Rig 4.1&= nbsp; 2465 2614 1218 HART 56 08:48 r1-00340612 48449-102300 4980 arc Rig 4.1&= nbsp; 2465 2614 1218 HART 57 08:49 b1-00340613 48449-102300 4980 arc Out 4.1&= nbsp; 1035 1184 -211 58 08:49 r1-00340613 48449-102300 4980 arc Out 4.1&= nbsp; 1035 1184 -211 59 08:50 b1-00340614 48449-102300 4980 fla Out 25.1 = 1035 1184 -211 60 08:50 r1-00340614 48449-102300 4980 fla Out 25.1 = 1035 1184 -211 61 08:54 r1-00340615 48449-102300 4980 sci Out 900.1  = ; 1035 1184 -211 62 08:54 b1-00340615 48449-102300 4980 sci Out 900.1  = ; 1035 1184 -211 63 09:14 r1-00340616 46827-101806 4981 arc Out 4.1&= nbsp; 1035 1184 -211 64 09:14 b1-00340616 46827-101806 4981 arc Out 4.1&= nbsp; 1035 1184 -211 65 09:16 b1-00340617 46827-101806 4981 fla Out 25.1 = 1035 1184 -211 66 09:16 r1-00340617 46827-101806 4981 fla Out 25.1 = 1035 1184 -211 67 09:20 b1-00340618 46827-101806 4981 sci Out 900.1  = ; 1035 1184 -211 68 09:20 r1-00340618 46827-101806 4981 sci Out 900.1  = ; 1035 1184 -211 69 09:42 b1-00340619 46828-101806 4982 sci Out 900.1  = ; 1035 1184 -211 70 09:42 r1-00340619 46828-101806 4982 sci Out 900.1  = ; 1035 1184 -211 71 10:06 r1-00340620 46829-101806 4983 sci Out 900.1  = ; 1035 1184 -211 72 10:06 b1-00340620 46829-101806 4983 sci Out 900.1  = ; 1035 1184 -211 73 10:29 r1-00340621 46830-101806 4984 sci Out 900.1  = ; 1035 1184 -211 74 10:29 b1-00340621 46830-101806 4984 sci Out 900.1  = ; 1035 1184 -211 75 11:12 r1-00340622 40386-100569 4986 arc Out 4.1&= nbsp; 1035 1184 -211 76 11:12 b1-00340622 40386-100569 4986 arc Out 4.1&= nbsp; 1035 1184 -211 77 11:14 b1-00340623 40386-100569 4986 fla Out 25.1 = 1035 1184 -211 78 11:14 r1-00340623 40386-100569 4986 fla Out 25.1 = 1035 1184 -211 79 11:17 r1-00340624 40386-100569 4986 sci Out 900.1  = ; 1035 1184 -211 80 11:17 b1-00340624 40386-100569 4986 sci Out 900.1  = ; 1035 1184 -211 ---- BOSS Spectrographs Collimation ---- Time TSP1 R1 &nb= sp; B1 Move1 B1Resid =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 00:05:30 7.8 -1649 0.8 -953= -2.2 02:10:00 7.6 329 -8.3 = 1482 -3.6 02:41:50 7.4 329 -1.2 = 357 -0.1 08:48:50 6.3 -1319 4.8 -1430 = ; 0.3 ---- Weather Log ---- &nb= sp; = Wind 1 um  = ; DIMM &nb= sp; IRSC Time Temp C Dewp C MPH  = ; Dir Dust fwhm = mean sigma 18:00 13.2 -11.6 26.1 = 275.5 1535 18:15 13.1 -11.6 26.7 = 272.7 1863 18:30 12.6 -11.7 31.0 = 286.8 2036 18:45 13.7 -11.5 30.7 = 278.1 2107 19:00 14.3 -11.7 23.0 = 262.3 2910 19:15 15.8 -11.4 23.6 = 274.0 2366 19:30 16.2 -11.3 19.2 = 263.3 1424 19:45 15.2 -11.7 27.4 = 273.8 1695 20:00 14.5 -12.6 31.3 = 270.5 2391 20:15 15.4 -12.6 23.8 = 277.8 2426 20:30 15.0 -11.8 21.3 = 279.1 4430 20:45 15.8 -12.4 21.5 = 245.2 4953 21:00 15.7 -12.9 29.2 = 257.2 10209 21:15 16.5 -12.7 28.4 = 271.7 7783 21:30 15.4 -13.3 24.0 = 275.0 6716 21:45 17.4 -11.9 32.2 = 259.3 5971 22:00 16.1 -13.3 23.5 = 261.0 6090 22:15 16.9 -11.9 23.0 = 260.7 3487 22:30 17.2 -12.6 21.2 = 254.5 4368 22:45 17.2 -12.6 19.1 = 281.4 3001 23:00 17.2 -12.8 25.6 = 266.0 2965 23:15 17.3 -13.0 35.6 = 243.2 3031 23:30 16.7 -12.9 26.2 = 244.0 2543 23:45 16.8 -12.7 22.9 = 249.1 3182 00:00 16.4 -13.2 24.7 = 247.7 2985 00:15 15.9 -13.5 21.8 = 255.3 3463 00:30 15.7 -13.4 23.0 = 277.7 3059 00:45 15.3 -13.6 22.0 = 251.2 2625 01:00 15.2 -13.3 28.2 = 250.9 2196 01:15 14.6 -13.3 23.7 = 265.0 2282 01:30 14.0 -13.6 20.5 = 259.8 2197 01:45 13.8 -13.3 25.7 = 268.5 2166 02:00 13.4 -13.8 22.5 = 259.3 2038 02:15 13.2 -13.8 21.4 = 266.0 2220 02:30 13.2 -13.6 15.8 = 263.5 2137 &nbs= p; 88 9.94 02:45 13.3 -13.6 14.7 = 258.1 2224 &nbs= p; 92 9.40 03:00 13.1 -13.8 16.0 = 261.6 2401 &nbs= p; 107 3.92 03:15 13.2 -13.8 23.0 = 266.7 2144 &nbs= p; 102 3.75 03:30 12.8 -13.7 19.3 = 272.1 2094 &nbs= p; 141 1.40 03:45 12.9 -13.6 22.0 = 266.3 1934 &nbs= p; 273 20.32 04:00 12.2 -13.6 19.4 = 275.0 1944 &nbs= p; 88 9.09 04:15 12.6 -13.8 14.3 = 284.6 1968 &nbs= p; 249 19.53 04:30 12.7 -14.0 17.6 = 279.1 1955 &nbs= p; 87 12.03 04:45 12.4 -13.7 20.7 = 275.9 2005 &nbs= p; 244 21.01 05:00 12.5 -13.7 23.0 = 281.6 1857 &nbs= p; 88 9.94 05:15 12.4 -13.6 28.0 = 292.7 1723 &nbs= p; 215 11.59 05:30 12.4 -13.4 16.3 = 272.2 1956 &nbs= p; 156 3.40 05:45 12.1 -13.3 15.4 = 269.5 1901 &nbs= p; 141 2.47 06:00 12.0 -13.4 20.1 = 291.0 1819 &nbs= p; 146 4.71 06:15 12.0 -13.3 20.0 = 275.5 1905 &nbs= p; 156 3.84 06:30 11.9 -13.3 15.8 = 275.4 1740 &nbs= p; 87 10.51 06:45 11.6 -13.3 15.5 = 266.6 1715 &nbs= p; 239 17.74 07:00 11.6 -13.2 12.1 = 264.6 1811 &nbs= p; 88 7.55 07:15 11.5 -12.9 14.2 = 275.0 1814 &nbs= p; 107 2.07 07:30 11.5 -13.0 17.3 = 290.1 1718 &nbs= p; 200 6.03 07:45 11.2 -12.9 17.8 = 279.1 1712 &nbs= p; 112 5.09 08:00 11.1 -12.9 17.4 = 291.0 1667 &nbs= p; 102 9.65 08:15 11.1 -12.9 13.9 = 316.6 1715 &nbs= p; 166 3.87 08:30 10.9 -12.8 13.0 = 282.0 1707 &nbs= p; 161 4.86 08:45 10.8 -12.7 10.7 = 276.6 1705 &nbs= p; 151 1.68 09:00 10.8 -12.8 14.0 = 287.3 1672 &nbs= p; 88 13.60 09:15 10.5 -12.7 13.8 = 266.1 1672 &nbs= p; 88 10.81 09:30 10.0 -12.9 11.7 = 265.5 &nb= sp; 161 1= .26 09:45 9.9 -13.1 10.9 &= nbsp; 277.7 &nbs= p; 332 &nb= sp; 34.37 10:00 9.9 -12.8 11.5 &= nbsp; 276.1 &nbs= p; 88 &nbs= p; 13.59 10:15 9.7 -12.9 14.6 &= nbsp; 280.9 &nbs= p; 151 &nb= sp; 1.48 10:30 9.8 -12.6 13.0 &= nbsp; 279.7 &nbs= p; 88 &nbs= p; 12.25 10:45 9.5 -12.2 12.8 &= nbsp; 311.3 &nbs= p; 166 &nb= sp; 2.57 11:00 9.3 -12.6 15.9 &= nbsp; 315.6 &nbs= p; 92 &nbs= p; 9.02 11:15 9.3 -11.9 14.8 &= nbsp; 305.3 &nbs= p; 269 &nb= sp; 22.07 11:30 9.3 -11.8 16.2 &= nbsp; 291.8 &nbs= p; 88 &nbs= p; 11.21 11:45 9.0 -11.9 10.2 &= nbsp; 293.5 &nbs= p; 149 &nb= sp; 5.79 ---- Problems in Detail ---- -- During Afternoon: fliswarm reconnect -- run 3 times : first and second was: 00:15:49 OBSERVER.viktorm 75 fliswarm f error=3D"Command 'reconnect' f= ailed because of an uncaught error 'ConnectionRefusedError: [Errno 111] Con= nect call failed ('10.25.1.120', 19995)'. See traceback in the log for more= information." 00:16:33 OBSERVER.viktorm 76 fliswarm f error=3D"Command 'reconnect' f= ailed because of an uncaught error 'ConnectionRefusedError: [Errno 111] Con= nect call failed ('10.25.1.125', 19995)'. See traceback in the log for more= information." 00:17:20 CmdDone: OBSERVER.viktorm 77 fliswarm reconnect ----------------- ____________________________________________________________ Logs of Nightly APO Activities https://urldefense.com/v3/__htt= ps://trac.sdss.org/tracmailman/browser/private/sloan-25mlog/index.html__;!!= K6Z8K8YTIA!UTJ-3w1pSsyQGs4vue3KOY7dBd4rRWt3Drik8iVlanIO0Lv7VBQa8sa8QjWe5cc$= --_000_MW4PR02MB745922A772444ADA2CB6BB37D8F49MW4PR02MB7459namp_--