Subject: 2.5m obslog 20220309 (SJD 59648) From: kgrabowski@apo.nmsu.edu Date: Thu, 10 Mar 2022 12:38:18 +0000 (UTC) APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Wednesday March 09, 2022 (SJD 59648) ---- Observing Team ---- Viktor Malanushenko Katie Grabowski ---- Observing Plan ---- Wednesday March 9 Please start with a couple of commissioning items, before returning to run from the queue: 1. observe design 12798 : please do only one configuration and take 6 sets of APOGEE+BOSS exposures, i.e. six BOSSx900 and 6 APOGEE AB pairs (so 1.5 hours total). You can load the design from the command line or, if you prefer, try to put the design at the top of the queue and observe from there. If you load this design into the queue and run it from the queue, do not use the Kronos exposure completion criteria; just get the 1.5 hours no matter what. 2. observe designs 106811 and 106812. These can be APOGEE only, with 1 AB pair (43 reads) each. 3. if conditions are good, do a joint BOSS-APOGEE dither test on design 106921. Following instructions on the observing instructions page, take both BOSS (300s) and APOGEE (single 30 read) at each dither position. If conditions are not stable/good, skip this, or do later. 4. refresh the Kronos queue, and observe from the queue ---- Observing Summary ---- Design 12798, Configuration 3921, Field 19092 - 6B+6xAB Design 106811, Configuration 3922, Field 21200 - 1xAB Design 106812, Configuration 3923, Field 21200 - 1xAB Design 48169, Configuration 3930, Field 102102- 1B+1xAB Design 106934, Configuration 3931, Field 21324 (dither test, seeing too high to attempt) 8 Cloned designs for dark_RM field - 8xB (no APOGEE targets) Design 106970-77, Configuration 3932-39, Field 112359 May have collected 30k dust counts. Closed for 1.5 hours due to high dust counts. Altitude out-of-closed-loop moving twice between designs. Boson froze between the first and second designs and needed to be force quit. First time we attempted the dither test with the script, we got a new error: 05:17:25 OBSERVER.kgrabow 25 jaeger f error="Failed processing image: 'Metrology'" We were unable to move past jaeger fvc loop --no-apply. In returning to the parent configuration, the RMS was >1.4 and wouldn't decrease. Cherno was only applying the rotator correction, but the measured rotator offset was -600. We were unable to solve this, so we refreshed Kronos and moved on to observing from the queue. We saw the same problem on the first field from the queue. It was noticed that the warm observer's RA,DEC, Rot, and focus were not checked, so the guider couldn't automatically adjust. Once this was discovered, the RMS values decreased enough to observe. Kronos picked a design (48169) with a green window at about this time, but the altitude is 33 degrees and the airmass is 1.8. The blue visibility window was later in the night. Closed for 1.5 hours due to high dust counts. ---- Observing Log ---- Afternoon: ========== --Test Actors, 23:26:53 -- APOGEE Short Dark - 23:28:02 -- 23:42:48 -- APOGEE: Evening Cals 23:32:40 boss exposure 0.0, file=338634 23:35:34 boss exposure 20.0, file=338635 23:42:30 Hartmann collimate output sp1: offset: r= 1319 (Out of focus); b= -8.3 (Out of focus) sp1: pred. move: spAverageMove= 1977 sp1: pred. resid: r= -658, b= -2.1, txt= OK, spTemp = 3.1 ---------- motorPosition sp1: -5690, -5547, -6930; ---------- Median spectrographs temperatures: Tsp1= 3.2 Outside air temperature= 4.9; ---------- 23:44:05 boss exposure 4.0, file=338638 23:46:00 boss exposure 25.0, file=338639 00:15Z Started ventilation Night ===== 01:11:09 encl25m : open; Design 12798, Configuration 3921, Field 19092 01:45:04 boss exposure 4.0, file=338640 01:45:46 boss exposure 4.0, file=338641 01:46:33 sp1.motor.move= 2793, 2795, 2795 01:46:33 Hartmann collimate output sp1: offset: r= 2308 (Out of focus); b= -10.3 (Out of focus) sp1: pred. move: spAverageMove= 2793 sp1: pred. resid: r= -485, b= -1.5, txt= OK, spTemp = 1.5 01:46:53 boss exposure 4.0, file=338642 01:48:42 boss exposure 25.0, file=338643 01:46:36 APO.viktorm.jaeger 3347 jaeger i text="FVC iteration 1" 01:47:16 APO.viktorm.jaeger 3347 jaeger d fvc_filename=/data/fcam/59648/fimg-fvc1n-0001.fits 01:47:21 APO.viktorm.jaeger 3347 jaeger i text="[FVC]: RMS full fit 48.510 um." 01:47:21 APO.viktorm.jaeger 3347 jaeger d text="[FVC]: Finished processing /data/fcam/59648/fimg-fvc1n-0001.fits" 01:47:21 APO.viktorm.jaeger 3347 jaeger i fvc_rms=48.51 01:47:21 APO.viktorm.jaeger 3347 jaeger i fvc_perc_90=70.8 01:47:21 APO.viktorm.jaeger 3347 jaeger i fvc_percent_reached=1.3 01:47:37 APO.viktorm.jaeger 3347 jaeger i text="FVC iteration 2" 01:47:57 APO.viktorm.jaeger 3347 jaeger d fvc_filename=/data/fcam/59648/fimg-fvc1n-0002.fits 01:48:01 APO.viktorm.jaeger 3347 jaeger d text="[FVC]: Finished processing /data/fcam/59648/fimg-fvc1n-0002.fits" 01:48:01 APO.viktorm.jaeger 3347 jaeger i fvc_rms=15.59 01:48:01 APO.viktorm.jaeger 3347 jaeger i fvc_deltarms=32.92 01:48:01 APO.viktorm.jaeger 3347 jaeger i fvc_perc_90=24.2 01:48:01 APO.viktorm.jaeger 3347 jaeger i fvc_percent_reached=18.6 01:57:30 apogee.expose=apRaw-40860010, 5, A, DomeFlat 01:58:52 apogee.expose=apRaw-40860011, 5, A, DomeFlat 02:00:21 apogee.expose=apRaw-40860012, 5, A, DomeFlat 02:05:51 apogee.expose=apRaw-40860013, 43, A, Object 02:06:01 boss exposure 900.0, file=338644 02:13:32 apogee.expose=apRaw-40860014, 43, B, Object 02:22:39 boss exposure 900.0, file=338645 02:22:41 apogee.expose=apRaw-40860015, 43, A, Object 02:30:21 apogee.expose=apRaw-40860016, 43, B, Object 2:39:14 boss exposure 900.0, file=338646 02:39:16 apogee.expose=apRaw-40860017, 43, A, Object 02:46:56 apogee.expose=apRaw-40860018, 43, B, Object 02:55:51 boss exposure 900.0, file=338647 02:55:53 apogee.expose=apRaw-40860019, 43, A, Object 03:03:33 apogee.expose=apRaw-40860020, 43, B, Object 03:13:16 boss exposure 900.0, file=338648 03:13:18 apogee.expose=apRaw-40860021, 43, A, Object 03:20:59 apogee.expose=apRaw-40860022, 43, B, Object 03:30:51 boss exposure 900.0, file=338649 03:30:53 apogee.expose=apRaw-40860023, 43, A, Object 03:38:33 apogee.expose=apRaw-40860024, 43, B, Object Design 106811, Configuration 3922, Field 21200 03:54:19 APO.viktorm.jaeger 3446 jaeger i text="FVC iteration 1" 03:54:43 .jaeger 0 jaeger w text="[FVC]: Found 6 positioners with dubious centroid matches." 03:54:44 APO.viktorm.jaeger 3446 jaeger i text="[FVC]: RMS full fit 52.550 um." 03:54:44 APO.viktorm.jaeger 3446 jaeger i fvc_rms=52.55 03:54:44 APO.viktorm.jaeger 3446 jaeger i fvc_perc_90=81.8 03:54:44 APO.viktorm.jaeger 3446 jaeger i fvc_percent_reached=3.1 03:55:23 APO.viktorm.jaeger 3446 jaeger w text="[FVC]: Found 5 positioners with dubious centroid matches." 03:55:24 APO.viktorm.jaeger 3446 jaeger i text="[FVC]: RMS full fit 18.520 um." 03:55:24 APO.viktorm.jaeger 3446 jaeger i fvc_rms=18.52 03:55:24 APO.viktorm.jaeger 3446 jaeger i fvc_deltarms=34.03 03:55:24 APO.viktorm.jaeger 3446 jaeger i fvc_perc_90=24.6 03:55:24 APO.viktorm.jaeger 3446 jaeger i fvc_percent_reached=22.8 04:07:17 apogee.expose=apRaw-40860025, 46, A, Object- cancelled exposure 04:08:24 apogee.expose=apRaw-40860026, 43, A, Object 04:16:06 apogee.expose=apRaw-40860027, 43, B, Object Design 106812, Configuration 3923, Field 21201 04:28:07 APO.viktorm.jaeger 3476 jaeger i text="FVC iteration 1" 04:28:07 APO.viktorm.jaeger 3476 jaeger d text="Taking exposure with fliswarm." 04:28:24 .jaeger 0 jaeger d alive_at=1646886504.374083 04:28:32 .jaeger 0 jaeger w text="[FVC]: Found 5 positioners with dubious centroid matches." 04:28:32 APO.viktorm.jaeger 3476 jaeger i text="[FVC]: RMS full fit 46.530 um." 04:28:32 APO.viktorm.jaeger 3476 jaeger i fvc_rms=46.53 04:28:32 APO.viktorm.jaeger 3476 jaeger i fvc_perc_90=68.2 04:28:32 APO.viktorm.jaeger 3476 jaeger i fvc_percent_reached=1.4 04:28:48 APO.viktorm.jaeger 3476 jaeger i text="FVC iteration 2" 04:28:48 APO.viktorm.jaeger 3476 jaeger d text="Taking exposure with fliswarm." 04:29:08 APO.viktorm.jaeger 3476 jaeger d fvc_filename=/data/fcam/59648/fimg-fvc1n-0006.fits 04:29:12 APO.viktorm.jaeger 3476 jaeger w text="[FVC]: Found 5 positioners with dubious centroid matches." 04:29:12 APO.viktorm.jaeger 3476 jaeger i text="[FVC]: RMS full fit 14.890 um." 04:29:12 APO.viktorm.jaeger 3476 jaeger i fvc_rms=14.89 04:29:12 APO.viktorm.jaeger 3476 jaeger i fvc_deltarms=31.64 04:29:12 APO.viktorm.jaeger 3476 jaeger i fvc_perc_90=24.0 04:29:12 APO.viktorm.jaeger 3476 jaeger i fvc_percent_reached=15.8 04:32:07 apogee.expose=apRaw-40860028, 43, A, Object 04:39:48 apogee.expose=apRaw-40860029, 43, B, Object Design 106921, Configuration 3294, Field 21310 Parent configuration (APOGEE pair, not a single APOGEE read) 04:58:49 boss exposure 300.0, file=338650 (had to delete it) 04:58:51 apogee.expose=apRaw-40860030, 15, A, Object 05:01:35 apogee.expose=apRaw-40860031, 15, B, Object 05:05:15 OBSERVER.kg.boss 3508 boss e text="sp1daq readout failed: timed out (>)." 05:05:18 APO.viktorm.cherno 3502 cherno w offset=0.0,0.0,-432.0 05:05:43 OBSERVER.kg 22 hal e error="Command 'boss exposure science itime=300.0' timed out." 05:05:43 OBSERVER.kg 22 hal w text="Running cleanup tasks." 05:05:45 .tcc 0 tcc W Text="number keys does not match number of values" 05:05:48 OBSERVER.kg 22 hal w text="BOSS exposure is running. Not cancelling it." 05:05:48 OBSERVER.kg 22 hal f 05:05:48 CmdDone: OBSERVER.kg 22 hal expose --pairs -b 300 --count 1 Stuck reading out at zero. Cleared exposure, and started a bias. It also stopped reading at zero boss ping returns 3-OKs for sp1daq, sp1cam, and sp1mech Reloaded controllers. Bias reads out completely Started dither test: 05:07:20 OBSERVER.kgrabow 8 jaeger i text="FVC iteration 1" 05:07:20 OBSERVER.kgrabow 8 jaeger d text="Taking exposure with fliswarm." 05:07:27 .jaeger 0 jaeger d alive_at=1646888847.1605537 05:07:40 OBSERVER.kgrabow 8 jaeger d fvc_filename=/data/fcam/59648/fimg-fvc1n-0009.fits 05:07:40 OBSERVER.kgrabow 8 jaeger i text="[FVC]: Processing raw image /data/fcam/59648/fimg-fvc1n-0009.fits" 05:07:40 OBSERVER.kgrabow 8 jaeger i text="[FVC]: Max counts in image: 48682" 05:07:44 .jaeger 0 jaeger w text="[FVC]: Found 7 positioners with dubious centroid matches." 05:07:44 OBSERVER.kgrabow 8 jaeger w text="[FVC]: Found 7 positioners with dubious centroid matches." 05:07:44 OBSERVER.kgrabow 8 jaeger f error="Failed processing image: 'Metrology'" 05:07:44 CmdDone: OBSERVER.kgrabow 8 jaeger fvc loop --no-apply 05:07:44 OBSERVER.kgrabow 8 jaeger f error="Failed processing image: 'Metrology'" 05:07:44 CmdDone: OBSERVER.kgrabow 8 jaeger fvc loop --no-apply 05:07:44 .jaeger 54 jaeger w text="Cannot write processed image." 05:17:24 OBSERVER.kgrabow 25 jaeger w text="[FVC]: Found 7 positioners with dubious centroid matches." 05:17:25 OBSERVER.kgrabow 25 jaeger f error="Failed processing image: 'Metrology'" 05:17:25 CmdDone: OBSERVER.kgrabow 25 jaeger fvc loop --no-apply Restarted. Unwound robots. 05:25:30 .jaeger 0 jaeger w text="Positioner 1370 replied to SEND_TRAJECTORY_DATA UID=4020 with 'VALUE_OUT_OF_RANGE'. (JaegerUserWarning)" 5:27:46 OBSERVER.kg.jaeger 3538 jaeger i text="FVC iteration 1" 05:28:06 OBSERVER.kg.jaeger 3538 jaeger d fvc_filename=/data/fcam/59648/fimg-fvc1n-0012.fits 05:28:10 OBSERVER.kg.jaeger 3538 jaeger w text="[FVC]: Found 7 positioners with dubious centroid matches." 05:28:10 OBSERVER.kg.jaeger 3538 jaeger i text="[FVC]: RMS full fit 51.240 um." 05:28:10 OBSERVER.kg.jaeger 3538 jaeger i fvc_rms=51.24 05:28:10 OBSERVER.kg.jaeger 3538 jaeger i fvc_perc_90=68.8 05:28:10 OBSERVER.kg.jaeger 3538 jaeger i fvc_percent_reached=2.3 5:28:26 OBSERVER.kg.jaeger 3538 jaeger i text="FVC iteration 2" 05:28:47 OBSERVER.kg.jaeger 3538 jaeger d fvc_filename=/data/fcam/59648/fimg-fvc1n-0013.fits 05:28:51 OBSERVER.kg.jaeger 3538 jaeger w text="[FVC]: Found 7 positioners with dubious centroid matches." 05:28:51 OBSERVER.kg.jaeger 3538 jaeger i text="[FVC]: RMS full fit 24.350 um." 05:28:51 OBSERVER.kg.jaeger 3538 jaeger i fvc_rms=24.35 05:28:51 OBSERVER.kg.jaeger 3538 jaeger i fvc_deltarms=26.89 05:28:51 OBSERVER.kg.jaeger 3538 jaeger i fvc_perc_90=24.6 05:28:51 OBSERVER.kg.jaeger 3538 jaeger i fvc_percent_reached=14.4 5:57:34 OBSERVER.kg.jaeger 3591 jaeger i text="FVC iteration 1" 05:57:55 OBSERVER.kg.jaeger 3591 jaeger d fvc_filename=/data/fcam/59648/fimg-fvc1n-0014.fits 05:57:59 .jaeger 0 jaeger w text="[FVC]: Found 7 positioners with dubious centroid matches." 05:57:59 OBSERVER.kg.jaeger 3591 jaeger i text="[FVC]: RMS full fit 48.310 um." 05:57:59 OBSERVER.kg.jaeger 3591 jaeger i fvc_rms=48.31 05:57:59 OBSERVER.kg.jaeger 3591 jaeger i fvc_perc_90=67.6 05:57:59 OBSERVER.kg.jaeger 3591 jaeger i fvc_percent_reached=1.3 05:58:15 OBSERVER.kg.jaeger 3591 jaeger i text="FVC iteration 2" 05:58:35 OBSERVER.kg.jaeger 3591 jaeger d fvc_filename=/data/fcam/59648/fimg-fvc1n-0015.fits 05:58:39 OBSERVER.kg.jaeger 3591 jaeger w text="[FVC]: Found 7 positioners with dubious centroid matches." 05:58:39 OBSERVER.kg.jaeger 3591 jaeger i text="[FVC]: RMS full fit 15.530 um." 05:58:39 OBSERVER.kg.jaeger 3591 jaeger i fvc_rms=15.53 05:58:39 OBSERVER.kg.jaeger 3591 jaeger i fvc_deltarms=32.78 05:58:39 OBSERVER.kg.jaeger 3591 jaeger i fvc_perc_90=24.2 05:58:39 OBSERVER.kg.jaeger 3591 jaeger i fvc_percent_reached=13.6 05:58:39 OBSERVER.kg.jaeger 3591 jaeger i text="Target 90% percentile reached." We tried to manually correct the rotator offset, which improved the RMS down to 0.8, still a factor of 10 too large to continue with the dither test. We aren't sure what is causing this, so we will refresh Kronos and observe from the queue. Before the end of the night, we'll try a dither test on the other field suggested. Design 48169, Configuration 3930, Field 102102 06:13:46 APO.viktorm.jaeger 3610 jaeger i text="FVC iteration 1" 06:14:07 APO.viktorm.jaeger 3610 jaeger d fvc_filename=/data/fcam/59648/fimg-fvc1n-0016.fits 06:14:11 APO.viktorm.jaeger 3610 jaeger w text="[FVC]: Found 4 positioners with dubious centroid matches." 06:14:11 APO.viktorm.jaeger 3610 jaeger i text="[FVC]: RMS full fit 47.580 um." 06:14:11 APO.viktorm.jaeger 3610 jaeger i fvc_rms=47.58 06:14:11 APO.viktorm.jaeger 3610 jaeger i fvc_perc_90=68.0 06:14:11 APO.viktorm.jaeger 3610 jaeger i fvc_percent_reached=1.3 06:14:27 APO.viktorm.jaeger 3610 jaeger i text="FVC iteration 2" 06:14:27 APO.viktorm.jaeger 3610 jaeger d text="Taking exposure with fliswarm." 06:14:28 .jaeger 0 jaeger d alive_at=1646892868.0682912 06:14:51 APO.viktorm.jaeger 3610 jaeger w text="[FVC]: Found 4 positioners with dubious centroid matches." 06:14:51 APO.viktorm.jaeger 3610 jaeger i text="[FVC]: RMS full fit 18.320 um." 06:14:51 APO.viktorm.jaeger 3610 jaeger d text="[FVC]: Finished processing /data/fcam/59648/fimg-fvc1n-0017.fits" 06:14:51 APO.viktorm.jaeger 3610 jaeger i fvc_rms=18.32 06:14:51 APO.viktorm.jaeger 3610 jaeger i fvc_deltarms=29.26 06:14:51 APO.viktorm.jaeger 3610 jaeger i fvc_perc_90=24.3 06:14:51 APO.viktorm.jaeger 3610 jaeger i fvc_percent_reached=18.9 06:16:06 boss exposure 4.0, file=338654 06:17:40 boss exposure 25.0, file=338655 Rotator off by -4k counts The fiducials have been fine. We're not at the edge of the rotator wrap limit. The rotator moves the way we expect it to when we physically look at it. Slewed back to the field. Rotator still has a measured offset of -4k counts Manually moving it back, the RMS improves to 0.2, but no better. THe airmass may be high for this field, but we can't get down to 0.07 manually. It was noticed that the warm observer's RA,DEC, Rot, and focus were not checked, so the guider couldn't automatically adjust. Working from the cold observer's computer, we tried to set up the dither test. cherno set axes tcc axis stop jaeger configuration dither 0.1 jaeger fvc loop --no-apply (and then it failed with the metrology problem) so we never re-enabled guiding. We may try cherno stop in the future. 6:39:43 APO.viktorm.jaeger 3633 jaeger i text="FVC iteration 1" 06:39:43 CmdStarted: client.v1772.fliswarm 82 fliswarm talk -c fvc expose 5.0 06:40:03 client.v1772.fliswarm 82 fliswarm i filename=fvc,fvc1n,/data/fcam/59648/fimg-fvc1n-0018.fits 06:40:08 APO.viktorm.jaeger 3633 jaeger w text="[FVC]: Found 3 positioners with dubious centroid matches." 06:40:08 APO.viktorm.jaeger 3633 jaeger i text="[FVC]: RMS full fit 46.990 um." 06:40:08 APO.viktorm.jaeger 3633 jaeger i fvc_rms=46.99 06:40:08 APO.viktorm.jaeger 3633 jaeger i fvc_perc_90=68.3 06:40:08 APO.viktorm.jaeger 3633 jaeger i fvc_percent_reached=0.2 06:40:24 APO.viktorm.jaeger 3633 jaeger i text="FVC iteration 2" 06:40:24 CmdStarted: client.v1772.fliswarm 83 fliswarm talk -c fvc expose 5.0 6:40:48 APO.viktorm.jaeger 3633 jaeger i text="[FVC]: RMS full fit 16.520 um." 06:40:48 APO.viktorm.jaeger 3633 jaeger i fvc_rms=16.52 06:40:48 APO.viktorm.jaeger 3633 jaeger i fvc_deltarms=30.47 06:40:48 APO.viktorm.jaeger 3633 jaeger i fvc_perc_90=24.1 06:40:48 APO.viktorm.jaeger 3633 jaeger i fvc_percent_reached=23.5 06:59:02 boss exposure 900.0, file=338656 06:59:02 apogee.expose=apRaw-40860032, 43, A, Object 07:06:43 apogee.expose=apRaw-40860033, 43, B, Object Collected about 20k dust counts. The range has been 6-8k, so we will close. 07:22:10 encl25m : closed; Accepted fiducial error for rotator when we slewed to stow. 09:03:10 encl25m : open; Design 106934, Configuration 33931, Field 21324 09:11:38 boss exposure 4.0, file=338657 09:12:19 boss exposure 4.0, file=338658 09:13:05 sp1.motor.move= -2898, -2899, -2903 09:13:05 Hartmann collimate output sp1: offset: r= -2968 (Out of focus); b= 8.9 (Out of focus) sp1: pred. move: spAverageMove= -2898 sp1: pred. resid: r= -70, b= -0.2, txt= OK, spTemp = 1.2 09:13:25 boss exposure 4.0, file=338659 09:14:57 boss exposure 25.0, file=338660 9:13:08 APO.viktorm.jaeger 3658 jaeger i text="FVC iteration 1" 09:13:28 APO.viktorm.jaeger 3658 jaeger d fvc_filename=/data/fcam/59648/fimg-fvc1n-0020.fits 09:13:32 APO.viktorm.jaeger 3658 jaeger w text="[FVC]: Found 3 positioners with dubious centroid matches." 09:13:33 APO.viktorm.jaeger 3658 jaeger i text="[FVC]: RMS full fit 45.950 um." 09:13:33 APO.viktorm.jaeger 3658 jaeger i fvc_rms=45.95 09:13:33 APO.viktorm.jaeger 3658 jaeger i fvc_perc_90=68.6 09:13:33 APO.viktorm.jaeger 3658 jaeger i fvc_percent_reached=1.0 09:13:49 APO.viktorm.jaeger 3658 jaeger i text="FVC iteration 2" 09:14:09 APO.viktorm.jaeger 3658 jaeger d fvc_filename=/data/fcam/59648/fimg-fvc1n-0021.fits 09:14:13 APO.viktorm.jaeger 3658 jaeger w text="[FVC]: Found 3 positioners with dubious centroid matches." 09:14:13 APO.viktorm.jaeger 3658 jaeger i text="[FVC]: RMS full fit 15.880 um." 09:14:13 APO.viktorm.jaeger 3658 jaeger i fvc_rms=15.88 09:14:13 APO.viktorm.jaeger 3658 jaeger i fvc_deltarms=30.07 09:14:13 APO.viktorm.jaeger 3658 jaeger i fvc_perc_90=23.7 09:14:13 APO.viktorm.jaeger 3658 jaeger i fvc_percent_reached=16.5 Current seeing ~1.8", altitude of 65, airmass ~1.01 Didn't improve, so we will run from the queue rather than try the dither test. Design 106970, Configuration 3932, Field 112359 09:36:23 boss exposure 4.0, file=338661 09:37:56 boss exposure 25.0, file=338662 9:34:03 APO.viktorm.jaeger 3684 jaeger i text="FVC iteration 1" 09:34:23 APO.viktorm.jaeger 3684 jaeger d fvc_filename=/data/fcam/59648/fimg-fvc1n-0022.fits 09:34:27 APO.viktorm.jaeger 3684 jaeger w text="[FVC]: Found 5 positioners with dubious centroid matches." 09:34:28 APO.viktorm.jaeger 3684 jaeger i text="[FVC]: RMS full fit 51.440 um." 09:34:28 APO.viktorm.jaeger 3684 jaeger i fvc_rms=51.44 09:34:28 APO.viktorm.jaeger 3684 jaeger i fvc_perc_90=72.0 09:34:28 APO.viktorm.jaeger 3684 jaeger i fvc_percent_reached=0.8 09:34:42 APO.viktorm.jaeger 3684 jaeger i text="FVC iteration 2" 09:35:02 APO.viktorm.jaeger 3684 jaeger d fvc_filename=/data/fcam/59648/fimg-fvc1n-0023.fits 09:35:07 APO.viktorm.jaeger 3684 jaeger w text="[FVC]: Found 6 positioners with dubious centroid matches." 09:35:07 APO.viktorm.jaeger 3684 jaeger i text="[FVC]: RMS full fit 14.430 um." 09:35:07 APO.viktorm.jaeger 3684 jaeger i fvc_rms=14.43 09:35:07 APO.viktorm.jaeger 3684 jaeger i fvc_deltarms=37.01 09:35:07 APO.viktorm.jaeger 3684 jaeger i fvc_perc_90=23.2 09:35:07 APO.viktorm.jaeger 3684 jaeger i fvc_percent_reached=24.4 09:42:16 boss exposure 900.0, file=338663 09:42:18 apogee.expose=apRaw-40860034, 43, A, Object 09:49:59 apogee.expose=apRaw-40860035, 43, B, Object Design 106971, Configuration 3933, Field 112359 - Cloned dark_RM field 10:00:50 boss exposure 900.0, file=338664 10:00:51 apogee.expose=apRaw-40860036, 43, A, Object 10:08:32 apogee.expose=apRaw-40860037, 43, B, Object Design 106973, Configuration 3934, Field 112359 - Cloned dark_RM field 10:18:56 boss exposure 900.0, file=338665 10:18:58 apogee.expose=apRaw-40860038, 43, A, Object 10:26:39 apogee.expose=apRaw-40860039, 43, B, Object Design 106973, Configuration 3935, Field 112359 - Cloned dark_RM field 10:36:39 boss exposure 900.0, file=338666 10:36:40 apogee.expose=apRaw-40860040, 43, A, Object 10:44:21 apogee.expose=apRaw-40860041, 43, B, Object Design 106974, Configuration 3936, Field 112359 - Cloned dark_RM field 10:54:28 boss exposure 900.0, file=338667 10:54:29 apogee.expose=apRaw-40860042, 43, A, Object 11:02:10 apogee.expose=apRaw-40860043, 43, B, Object Design 106975, Configuration 3937, Field 112359 - Cloned dark_RM field 11:12:33 boss exposure 900.0, file=338668 11:12:34 apogee.expose=apRaw-40860044, 43, A, Object 11:20:16 apogee.expose=apRaw-40860045, 43, B, Object Design 106976, Configuration 3938, Field 112359 - Cloned dark_RM field 11:30:24 boss exposure 900.0, file=338669 11:30:25 apogee.expose=apRaw-40860046, 43, A, Object 11:38:06 apogee.expose=apRaw-40860047, 43, B, Object Design 106977, Configuration 3939, Field 112359 11:52:16 APO.viktorm.jaeger 3760 jaeger i text="FVC iteration 1" 11:52:36 APO.viktorm.jaeger 3760 jaeger d fvc_filename=/data/fcam/59648/fimg-fvc1n-0024.fits 11:52:41 APO.viktorm.jaeger 3760 jaeger w text="[FVC]: Found 5 positioners with dubious centroid matches." 11:52:41 APO.viktorm.jaeger 3760 jaeger i text="[FVC]: RMS full fit 47.570 um." 11:52:41 APO.viktorm.jaeger 3760 jaeger i fvc_rms=47.57 11:52:41 APO.viktorm.jaeger 3760 jaeger i fvc_perc_90=70.7 11:52:41 APO.viktorm.jaeger 3760 jaeger i fvc_percent_reached=1.9 11:52:55 APO.viktorm.jaeger 3760 jaeger i text="FVC iteration 2" 11:53:15 APO.viktorm.jaeger 3760 jaeger d fvc_filename=/data/fcam/59648/fimg-fvc1n-0025.fits 11:53:19 APO.viktorm.jaeger 3760 jaeger w text="[FVC]: Found 5 positioners with dubious centroid matches." 11:53:19 APO.viktorm.jaeger 3760 jaeger i text="[FVC]: RMS full fit 17.490 um." 11:53:19 APO.viktorm.jaeger 3760 jaeger i fvc_rms=17.49 11:53:19 APO.viktorm.jaeger 3760 jaeger i fvc_deltarms=30.08 11:53:19 APO.viktorm.jaeger 3760 jaeger i fvc_perc_90=25.4 11:53:20 APO.viktorm.jaeger 3760 jaeger i fvc_percent_reached=19.8 12:00:18 boss exposure 900.0, file=338672 12:00:19 apogee.expose=apRaw-40860048, 43, A, Object 12:08:00 apogee.expose=apRaw-40860049, 43, B, Object 12:19:08 apogee.expose=apRaw-40860050, 5, B, DomeFlat 12:20:35 apogee.expose=apRaw-40860051, 5, B, DomeFlat 12:22:00 apogee.expose=apRaw-40860052, 5, B, DomeFlat 12:30:10 encl25m : closed; 12:33Z- Started APOGEE cals ---- Mirror Numbers ---- PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A: 0.0020 0.0040 Axial B: -0.0010 0.0030 Axial C: -0.1100 -0.1150 Trans D: -0.0342 -0.0338 Lateral E: 0.3315 0.3315 Lateral F: 0.5350 0.5350 GALILS Commanded: 12500 -1600 5300 47050 -8800 -8400 Actual: 12500 -1600 5300 47050 -8800 -8400 SETMIR VALUES PrimDesOrient: 0.0 -59.5 -9.15 -512.0 1906.3 PrimOrient: 1.07 -59.51 -9.21 -512.13 1906.31 SECONDARY: ---------- Focus: 0.0000 Air temp: 4.44 Altitude: 30.0 MIGS TONIGHT NOMINAL Axial A: -9.6550 -9.6570 Axial B: -9.6690 -9.6710 Axial C: -9.6950 -9.6970 Trans D: 0.1230 0.1140 GALILS Commanded: 4894550 4894900 4914050 -50 -3450 Actual: 4859073 4859306 4859293 -50 -3450 SETMIR VALUES SecDesOrient: 3857.0 0.0 0.0 0.0 42.22 SecOrient: 3857.01 -0.02 0.07 0.05 42.88 ---- Telescope Status ---- 00:15Z Started ventilation 01:11Z, 09:03Z encl25m : open 07:22Z, 12:30Z encl25m : closed No interlocks bypassed Status at:12:36Z Telescope stowed at: 121.0, 30.0, 0.0 mount Instrument mounted: Instrument = FPS Counterweights at: 686 687 705 704 LN2 autofill systems: Connected and turned on 180L LN2 dewar scales: SP1 175 lbs, 23.9 psi ---- Software Used ---- Get Versions, 03/09/22, 23:58:12 -- STUI software: STUI : 1.7.17b1 alerts : 2.2.0 boss : 6.0.1 -boss.specMec : 3.0.4 guider : FAILED ecamera : FAILED apo : 3.0.1 fliswarm : 0.2.1-alpha.0 jaeger : 0.14.1-alpha.0 kaiju : 1.2.2-alpha.1 coordio : 1.2.2-alpha.1 fps_calibrati : APO.2021.12.25+960cb05 cherno : 0.1.1-alpha.0 hal : 0.1.1-alpha.0 hartmann : 2.0.2 hub : 5.0.1a0 mcp : 2.1 -mcp.plc : 34 -mcp.azFiduci : v1_148 -mcp.altFiduc : v1_148 -mcp.rotFiduc : v1_148 tcc : 1.4.0 apogee : not availble apogeecal : not availble apogeeql : v1_7_0dev STUIAdditions : 3.1.12 --------------Other Versions-------------- sdss-obstools : 3.7.10 TPM : 1.2-31-gd7eb MCP : 2.1 PLC : V34 Fiducials : v1_148 idlspec2d : v6_0_5 Kronos : 1.1.2a roboscheduler : 1.1.0 /dev/sdb1 2.0T 166G 1.9T 9% /home 10.25.1.230:/data 21T 19T 1.1T 95% /data ---- APOGEE Data Summary ---- APOGEE data list, mjd=59648, all raw_data: /data/apogee/utr_cdr/59648/apRaw-*.fits archive: /data/apogee/archive/59648/apR-[a,b,c]-*.apz quickred: /data/apogee/quickred/59648/apq-*.fits ------------------------------------------------------------------------------ mjd i UT File/Exp Imtype Nread A/B Dth Design-Cfg Arch Offset %Flux Comm ------------------------------------------------------------------------------ 59648 1 23:28 40860001 Dark 3 A 13.0 ---------- a-b-c - - 59648 2 23:53 40860002 Dark 60 A 13.0 ---------- a-b-c - - 59648 3 23:55 40860003 QuarFlat 10 A 13.0 ---------- a-b-c - 97 59648 4 23:57 40860004 Arc-Thar 12 B 13.5 ---------- a-b-c 1.91 - 59648 5 00:04 40860005 Arc-Une 40 B 13.5 ---------- a-b-c - - 59648 6 00:07 40860006 Arc-Thar 12 A 13.0 ---------- a-b-c 1.51 - 59648 7 00:14 40860007 Arc-Une 40 A 13.0 ---------- a-b-c - - 59648 8 00:25 40860008 Dark 60 A 13.0 ---------- a-b-c - - 59648 9 00:30 40860009 Arc---- 30 A 13.0 ---------- a-b-c - - FPI 59648 10 01:58 40860010 DomeFlat 5 A 13.0 12798-3921 a-b-c - 143 59648 11 01:59 40860011 DomeFlat 5 A 13.0 12798-3921 a-b-c - 140 59648 12 02:00 40860012 DomeFlat 5 A 13.0 12798-3921 a-b-c - 139 59648 13 02:13 40860013 Object 43 A 13.0 12798-3921 a-b-c 1.42 - 59648 14 02:20 40860014 Object 43 B 13.5 12798-3921 a-b-c 1.95 - 59648 15 02:29 40860015 Object 43 A 13.0 12798-3921 a-b-c 1.42 - 59648 16 02:37 40860016 Object 43 B 13.5 12798-3921 a-b-c 1.95 - 59648 17 02:46 40860017 Object 43 A 13.0 12798-3921 a-b-c 1.43 - 59648 18 02:54 40860018 Object 43 B 13.5 12798-3921 a-b-c 1.95 - 59648 19 03:03 40860019 Object 43 A 13.0 12798-3921 a-b-c 1.43 - 59648 20 03:10 40860020 Object 43 B 13.5 12798-3921 a-b-c 1.95 - 59648 21 03:20 40860021 Object 43 A 13.0 12798-3921 a-b-c 1.43 - 59648 22 03:28 40860022 Object 43 B 13.5 12798-3921 a-b-c 1.95 - 59648 23 03:38 40860023 Object 43 A 13.0 12798-3921 a-b-c 1.43 - 59648 24 03:45 40860024 Object 43 B 13.5 12798-3921 a-b-c 1.95 - 59648 25 04:07 40860025 Object 4 A 13.0 106811-3922 a-b-c 1.45 - 59648 26 04:15 40860026 Object 43 A 13.0 106811-3922 a-b-c 1.43 - 59648 27 04:23 40860027 Object 43 B 13.5 106811-3922 a-b-c 1.95 - 59648 28 04:39 40860028 Object 43 A 13.0 106812-3923 a-b-c 1.43 - 59648 29 04:47 40860029 Object 43 B 13.5 106812-3923 a-b-c 1.95 - 59648 30 05:01 40860030 Object 15 A 13.0 106921-3924 a-b-c 1.44 - 59648 31 05:03 40860031 Object 15 B 13.5 106921-3924 a-b-c 1.94 - 59648 32 07:06 40860032 Object 43 A 13.0 48169-3930 a-b-c 1.44 - 59648 33 07:13 40860033 Object 43 B 13.5 48169-3930 a-b-c 1.96 - 59648 34 09:49 40860034 Object 43 A 13.0 106970-3932 a-b-c 1.44 - 59648 35 09:57 40860035 Object 43 B 13.5 106970-3932 a-b-c 1.95 - 59648 36 10:08 40860036 Object 43 A 13.0 106971-3933 a-b-c 1.43 - 59648 37 10:15 40860037 Object 43 B 13.5 106971-3933 a-b-c 1.96 - 59648 38 10:26 40860038 Object 43 A 13.0 106972-3934 a-b-c 1.44 - 59648 39 10:33 40860039 Object 43 B 13.5 106972-3934 a-b-c 1.95 - 59648 40 10:43 40860040 Object 43 A 13.0 106973-3935 a-b-c 1.43 - 59648 41 10:51 40860041 Object 43 B 13.5 106973-3935 a-b-c 1.95 - 59648 42 11:01 40860042 Object 43 A 13.0 106974-3936 a-b-c 1.44 - 59648 43 11:09 40860043 Object 43 B 13.5 106974-3936 a-b-c 1.95 - 59648 44 11:19 40860044 Object 43 A 13.0 106975-3937 a-b-c 1.44 - 59648 45 11:27 40860045 Object 43 B 13.5 106975-3937 a-b-c 1.96 - 59648 46 11:37 40860046 Object 43 A 13.0 106976-3938 a-b-c 1.43 - 59648 47 11:45 40860047 Object 43 B 13.5 106976-3938 a-b-c 1.96 - 59648 48 12:07 40860048 Object 43 A 13.0 106977-3939 a-b-c 1.44 - 59648 49 12:15 40860049 Object 43 B 13.5 106977-3939 a-b-c 1.95 - 59648 50 12:19 40860050 DomeFlat 5 B 13.5 106977-3939 a-b-c - 145 59648 51 12:21 40860051 DomeFlat 5 B 13.5 106977-3939 a-b-c - 147 59648 52 12:22 40860052 DomeFlat 5 B 13.5 106977-3939 a-b-c - 145 cannot read file : /data/apogee/utr_cdr/59648/apRaw-40860053.fits ------------------------------------------------------------------------------ ---- eBOSS Data Summary ---- Reading FITS from /data/spectro//59648 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - n Time Filename design-field cfg flav shu Exp colA colB colC Hrt - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 1 23:32 b1-00338634 ----------- ---- bia Clo 0.0 -7668 -7525 -8907 2 23:32 r1-00338634 ----------- ---- bia Clo 0.0 -7668 -7525 -8907 3 23:35 b1-00338635 ----------- ---- dar Clo 20.0 -7668 -7525 -8907 4 23:35 r1-00338635 ----------- ---- dar Clo 20.0 -7668 -7525 -8907 5 23:41 r1-00338636 36908-100159 3920 arc Lef 4.1 -7668 -7525 -8907 HART 6 23:41 b1-00338636 36908-100159 3920 arc Lef 4.1 -7668 -7525 -8907 HART 7 23:41 b1-00338637 36908-100159 3920 arc Rig 4.1 -7668 -7525 -8907 HART 8 23:41 r1-00338637 36908-100159 3920 arc Rig 4.1 -7668 -7525 -8907 HART 9 23:44 b1-00338638 36908-100159 3920 arc Out 4.1 -5690 -5547 -6930 10 23:44 r1-00338638 36908-100159 3920 arc Out 4.1 -5690 -5547 -6930 11 23:46 b1-00338639 36908-100159 3920 fla Out 25.1 -5690 -5547 -6930 12 23:46 r1-00338639 36908-100159 3920 fla Out 25.1 -5690 -5547 -6930 13 01:45 r1-00338640 12798-19092 3921 arc Lef 4.1 -5690 -5547 -6930 HART 14 01:45 b1-00338640 12798-19092 3921 arc Lef 4.1 -5690 -5547 -6930 HART 15 01:45 b1-00338641 12798-19092 3921 arc Rig 4.1 -5690 -5547 -6930 HART 16 01:45 r1-00338641 12798-19092 3921 arc Rig 4.1 -5690 -5547 -6930 HART 17 01:46 b1-00338642 12798-19092 3921 arc Out 4.1 -2897 -2752 -4135 18 01:46 r1-00338642 12798-19092 3921 arc Out 4.1 -2897 -2752 -4135 19 01:48 r1-00338643 12798-19092 3921 fla Out 25.1 -2897 -2752 -4135 20 01:48 b1-00338643 12798-19092 3921 fla Out 25.1 -2897 -2752 -4135 21 02:06 b1-00338644 12798-19092 3921 sci Out 900.1 -2897 -2752 -4135 22 02:06 r1-00338644 12798-19092 3921 sci Out 900.1 -2897 -2752 -4135 23 02:22 r1-00338645 12798-19092 3921 sci Out 900.1 -2897 -2752 -4135 24 02:22 b1-00338645 12798-19092 3921 sci Out 900.1 -2897 -2752 -4135 25 02:39 r1-00338646 12798-19092 3921 sci Out 900.1 -2897 -2752 -4135 26 02:39 b1-00338646 12798-19092 3921 sci Out 900.1 -2897 -2752 -4135 27 02:55 r1-00338647 12798-19092 3921 sci Out 900.1 -2897 -2752 -4135 28 02:55 b1-00338647 12798-19092 3921 sci Out 900.1 -2897 -2752 -4135 29 03:13 r1-00338648 12798-19092 3921 sci Out 900.1 -2897 -2752 -4135 30 03:13 b1-00338648 12798-19092 3921 sci Out 900.1 -2897 -2752 -4135 31 03:30 b1-00338649 12798-19092 3921 sci Out 900.1 -2897 -2752 -4135 32 03:30 r1-00338649 12798-19092 3921 sci Out 900.1 -2897 -2752 -4135 33 05:15 r1-00338653 ----------- ---- bia Clo 0.0 -2897 -2752 -4135 34 05:15 b1-00338653 ----------- ---- bia Clo 0.0 -2897 -2752 -4135 35 06:16 r1-00338654 48169-102102 3930 arc Out 4.1 -2897 -2752 -4135 36 06:16 b1-00338654 48169-102102 3930 arc Out 4.1 -2897 -2752 -4135 37 06:17 r1-00338655 48169-102102 3930 fla Out 25.1 -2897 -2752 -4135 38 06:17 b1-00338655 48169-102102 3930 fla Out 25.1 -2897 -2752 -4135 39 06:59 r1-00338656 48169-102102 3930 sci Out 900.1 -2897 -2752 -4135 40 06:59 b1-00338656 48169-102102 3930 sci Out 900.1 -2897 -2752 -4135 41 09:11 r1-00338657 106935-21324 3931 arc Lef 4.1 -2897 -2752 -4135 HART 42 09:11 b1-00338657 106935-21324 3931 arc Lef 4.1 -2897 -2752 -4135 HART 43 09:12 r1-00338658 106935-21324 3931 arc Rig 4.1 -2897 -2752 -4135 HART 44 09:12 b1-00338658 106935-21324 3931 arc Rig 4.1 -2897 -2752 -4135 HART 45 09:13 r1-00338659 106935-21324 3931 arc Out 4.1 -5795 -5651 -7038 46 09:13 b1-00338659 106935-21324 3931 arc Out 4.1 -5795 -5651 -7038 47 09:14 b1-00338660 106935-21324 3931 fla Out 25.1 -5795 -5651 -7038 48 09:14 r1-00338660 106935-21324 3931 fla Out 25.1 -5795 -5651 -7038 49 09:36 r1-00338661 106970-112359 3932 arc Out 4.1 -5795 -5651 -7038 50 09:36 b1-00338661 106970-112359 3932 arc Out 4.1 -5795 -5651 -7038 51 09:37 r1-00338662 106970-112359 3932 fla Out 25.1 -5795 -5651 -7038 52 09:37 b1-00338662 106970-112359 3932 fla Out 25.1 -5795 -5651 -7038 53 09:42 b1-00338663 106970-112359 3932 sci Out 900.1 -5795 -5651 -7038 54 09:42 r1-00338663 106970-112359 3932 sci Out 900.1 -5795 -5651 -7038 55 10:00 r1-00338664 106971-112359 3933 sci Out 900.0 -5795 -5651 -7038 56 10:00 b1-00338664 106971-112359 3933 sci Out 900.0 -5795 -5651 -7038 57 10:18 r1-00338665 106972-112359 3934 sci Out 900.1 -5795 -5651 -7038 58 10:18 b1-00338665 106972-112359 3934 sci Out 900.1 -5795 -5651 -7038 59 10:36 r1-00338666 106973-112359 3935 sci Out 900.1 -5795 -5651 -7038 60 10:36 b1-00338666 106973-112359 3935 sci Out 900.1 -5795 -5651 -7038 61 10:54 r1-00338667 106974-112359 3936 sci Out 900.1 -5795 -5651 -7038 62 10:54 b1-00338667 106974-112359 3936 sci Out 900.1 -5795 -5651 -7038 63 11:12 r1-00338668 106975-112359 3937 sci Out 900.1 -5795 -5651 -7038 64 11:12 b1-00338668 106975-112359 3937 sci Out 900.1 -5795 -5651 -7038 65 11:30 r1-00338669 106976-112359 3938 sci Out 900.0 -5795 -5651 -7038 66 11:30 b1-00338669 106976-112359 3938 sci Out 900.0 -5795 -5651 -7038 67 11:54 r1-00338670 106977-112359 3939 arc Out 4.1 -5795 -5651 -7038 68 11:54 b1-00338670 106977-112359 3939 arc Out 4.1 -5795 -5651 -7038 69 11:56 b1-00338671 106977-112359 3939 fla Out 25.1 -5795 -5651 -7038 70 11:56 r1-00338671 106977-112359 3939 fla Out 25.1 -5795 -5651 -7038 71 12:00 b1-00338672 106977-112359 3939 sci Out 900.1 -5795 -5651 -7038 72 12:00 r1-00338672 106977-112359 3939 sci Out 900.1 -5795 -5651 -7038 ---- BOSS Spectrographs Collimation ---- Time TSP1 R1 B1 Move1 B1Resid ============================================= 23:42:20 2.8 1319 -8.3 1977 -2.1 01:46:20 1.2 2308 -10.3 2793 -1.5 09:12:50 0.6 -2968 8.9 -2898 -0.2