Subject: 2.5m obslog 20220122 (SJD 59602) From: asimmons@apo.nmsu.edu Date: Sun, 23 Jan 2022 12:33:01 +0000 (UTC) APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Saturday January 22, 2022 (SJD 59602) ---- Observing Team ---- Dmitry Bizyaev Audrey Oravetz Jose Sanchez-Gallego (support) ZOOM: Dylan, Katie, Viktor, Kaike Natalie Ulloa, Arlin Cortes, Consuelo Gonzalez Avila, Pablo Vera ---- Observing Plan ---- FPS commissioning - Robot dithers with various APOGEE and BOSS fields, and another run at a science field with FPI fibers plugged in. ---- Observing Summary ---- Jose logged on with us remotely for a large portion of the night and was a tremendous help. Many changes were made within hal goto-field and expose, with also improvements with the snapshot window, logging, and jaeger. We are running version 0.0.4-alpha.20 of BOSON now. - Adjusted the tcc track command to work within the acquire phase of goto-field instead of the FVC loop, in case the RMS doesn't reach a certain threshold. - Added an axis stop during prepare & acquire to help with a slew cannot compute error when clearing the axes. - Increased jaeger robot timeout for non-movement to greater than 2 seconds (which was the issue with occasional failed dithers of 0.2). - A bottom scroll bar was added to the snapshot window within BOSON for easier navigation. - We now have the ability to click on the design ID in the HAL window in BOSON and it will become in a text box to load a new design (not ready to be tested yet though). - Practiced filtering through BOSON's new log windows. - Fixed a hal expose FPI calbox shutter issue. DESIGNS OBSERVED TONIGHT: ========================= - APOGEE robot 12-pt dither sequence for 35897 - BOSS robot 12-pt dither sequence for 35940 - Science design for both APOGEE and BOSS 12798 - 5 exposures with FPI fibers (though the first APOGEE exposure was 38 reads instead of 43) - APOGEE robot 12-pt dither sequence for 35847 NOTES: ====== !!!!!*****NO ROBOT COLLISIONS OR TRAJECTORY ERRORS FOR ALL DESIGNS TONIGHT*****!!!!! Encountered serveral error messages after executing an fvc loop --no-apply the last design of the night: 10:07:41 jaeger f error="Failed processing image: The exposure succeeded but did not output the filename." The FPI fibers were tricky again tonight. It is not easy pushing up and locking them gently in cold temperatures and at odd orientations (trying not to slew the telescope with the fibers plugged in). It can take ~10 minutes or so to get them plugged, unless you have the magic touch. Any changes to the parameters (min_npix, sigma, etc) of the guider will remain for every subsequent field unless they are manually altered or if cherno is restarted. They will not return to their defaulted values automatically. BOSON froze for both of us at 08:15Z, gave the 'rainbow wheel of death', 'BOSON not responding' message in command/option/esc menu. CPU maxed out at 100% with 'top'. Reloading it with Command-R did not help. PID COMMAND %CPU TIME #TH #WQ #PORT MEM PURG CMPRS PGRP PPID STATE BOOSTS %CPU_ME %CPU_OTHRS UID FAULTS COW MSGSENT MSGRECV SYSBSD SYSMACH 1601 boson 100.3 16:57.45 29/1 1 435 2090M+ 0B 531M 1601 1 running *0[58] 0.00000 0.00000 507 651729+ 616 425352+ 180293+ 64731332+ 3548629+ 1020 STUI 32.9 02:31:54 7 1 389+ 12G+ 10M- 41G- 1020 1 sleeping *1[9] 0.47622 0.00000 507 59979850+ 89641+ 33106379+ 3846337+ 357843250+ 244040746+ The cal box shutter remained open after the hal expose routine was finished. The east door weather stripping ripped and was found dangling next to the entrance. ---- Observing Log ---- Afternoon --------- All systems found responsive 3:15:38 Hartmann collimate output sp1: offset: r= -989 (In Focus); b= 0.8 (In Focus) sp1: pred. move: spAverageMove= -623 sp1: pred. resid: r= -366, b= -1.1, txt= OK, spTemp = 3.1 The blue ring is kept unchanged. 23:20Z ventilation started. APOGEE instrument window reported the collimator with all 1 1 1 1 1 1 at the limit. An apogee coll init was issued. The outer weather strip on the northern side of the eastern door was found hanged and detached from the door. It is left on desk inside the enclosure now. Night ----- 00:30Z - Opened the enclosure and filled. ***APOGEE DITHER ROBOT SEQUENCE WITH DESIGN 35897 01:21:19 Config Loaded: 35897-2620 Used HAL goto-field. The sequence failed at the 'acquire' phase with these error messages: 01:28:32 CmdDone: OBSERVER.dmbiz_2.jaeger 116 jaeger fvc loop 1:28:32 OBSERVER.dmbiz_2 25 hal i text="Re-slewing to field." 01:28:32 CmdStarted: OBSERVER.dmbiz_2.tcc 117 tcc axis status 01:28:32 OBSERVER.dmbiz_2.tcc 117 tcc i axisConnState=Connected,Connected,Connected 01:28:32 OBSERVER.dmbiz_2.tcc 117 tcc i RotStat=205.943956,0.000000,5149618112.54287,0x00000000; RotDTime=0.02 01:28:32 OBSERVER.dmbiz_2.tcc 117 tcc i AzStat=27.160600,-0.014739,5149618112.60830,0x00000000; AzDTime=0.04 01:28:32 CmdDone: OBSERVER.dmbiz_2.tcc 117 tcc axis status 01:28:32 CmdStarted: OBSERVER.dmbiz_2.tcc 118 tcc axis init 01:28:32 .tcc 0 tcc w AxisCmdState=Halting,Halting,Halted; AxisErrCode=HaltRequested,HaltRequested,HaltRequested; AxisNoTrack=TTT 01:28:32 .mcp 0 mcp i trace="Resetting amp for axis rotator: 4" 01:28:33 .mcp 0 mcp i ab_status=0,1,1,0 01:28:33 .mcp 0 mcp i command="init" 01:28:33 .mcp 0 mcp i trace="Resetting amp for axis altitude: 2" 01:28:33 .mcp 0 mcp i trace="Resetting amp for axis altitude: 3" 01:28:33 .mcp 0 mcp i ab_status=0,1,8,0 01:28:33 .mcp 0 mcp w RawLine="\"" 01:28:33 .tcc 0 tcc i AxisCmdState=Halting,Halted,Halted 01:28:34 .mcp 0 mcp i trace="Resetting amp for axis azimuth: 0" 01:28:34 .mcp 0 mcp i trace="Resetting amp for axis azimuth: 1" 01:28:34 .mcp 0 mcp i ab_status=0,8,0,0 01:28:34 .mcp 0 mcp w RawLine="\"" 01:28:34 .tcc 0 tcc i AxisCmdState=Halted,Halted,Halted 01:28:35 OBSERVER.dmbiz_2.tcc 119 tcc i AxisCmdState=Halting,Halting,Halting; AxisErrCode=CannotCompute,CannotCompute,CannotCompute 01:28:39 OBSERVER.dmbiz_2.tcc 119 tcc i AxisCmdState=Halted,Halted,Halted 01:28:39 OBSERVER.dmbiz_2.tcc 119 tcc : SlewEnd 01:28:40 OBSERVER.dmbiz_2 25 hal e error="Axes must be tracking for acquisition." 01:28:15 FVC Loop - RMS:11.97 Delta RMS: -0.13 Image #: 0004 01:26:12 OBSERVER.dmbiz_2.jaeger 116 jaeger i text="[FVC]: RMS full fit 54.580 um." 01:26:53 OBSERVER.dmbiz_2.jaeger 116 jaeger i text="[FVC]: RMS full fit 13.640 um." 01:27:34 OBSERVER.dmbiz_2.jaeger 116 jaeger i text="[FVC]: RMS full fit 11.840 um." 01:28:15 OBSERVER.dmbiz_2.jaeger 116 jaeger i text="[FVC]: RMS full fit 11.970 um." 01:31:46 FVC Loop - RMS:11.53 Delta RMS: 0.48 Image #: 0007 01:30:24 OBSERVER.dmbiz 35 jaeger i text="[FVC]: RMS full fit 18.440 um." 01:31:05 OBSERVER.dmbiz 35 jaeger i text="[FVC]: RMS full fit 12.010 um." 01:31:46 OBSERVER.dmbiz 35 jaeger i text="[FVC]: RMS full fit 11.530 um." 01:49:59 apogee.expose=apRaw-40400010, 28, A, Object (using HAL, but failed to use 'prepare') 01:55:55 apogee.expose=apRaw-40400011, 30, A, Object Best focus determined to be 400microns. Seeing 1.0", clear. Beginning this sequence of dither commands, looped 12 times: jaeger configuration dither 0.1 tcc axis stop cherno set axes jaeger fvc loop --no-apply tcc axis init tcc track tui wait 3 cherno set axes radec rot tui wait 20 apogee dither namedpos=A apogee expose nreads=30 object=Object tui wait 2 jaeger configuration reverse tui wait 3 02:01:21 Config Loaded: 35897-2621 02:03:10 apogee.expose=apRaw-40400012, 30, A, Object 02:08:28 Config Loaded: 35897-2620 02:08:39 Config Loaded: 35897-2622 02:10:27 apogee.expose=apRaw-40400013, 30, A, Object 02:15:48 Config Loaded: 35897-2620 02:15:59 Config Loaded: 35897-2623 02:17:46 apogee.expose=apRaw-40400014, 30, A, Object 02:23:04 Config Loaded: 35897-2620 02:23:16 Config Loaded: 35897-2624 02:25:05 apogee.expose=apRaw-40400015, 30, A, Object 02:30:26 Config Loaded: 35897-2620 02:30:37 Config Loaded: 35897-2625 02:32:25 apogee.expose=apRaw-40400016, 30, A, Object 02:37:45 Config Loaded: 35897-2620 02:37:57 Config Loaded: 35897-2626 02:39:46 apogee.expose=apRaw-40400017, 30, A, Object 02:45:06 Config Loaded: 35897-2620 02:45:18 Config Loaded: 35897-2627 02:47:04 apogee.expose=apRaw-40400018, 30, A, Object 02:52:24 Config Loaded: 35897-2620 02:52:36 Config Loaded: 35897-2628 02:54:21 apogee.expose=apRaw-40400019, 30, A, Object 02:59:40 Config Loaded: 35897-2620 02:59:51 Config Loaded: 35897-2629 03:01:40 apogee.expose=apRaw-40400020, 30, A, Object 03:07:00 Config Loaded: 35897-2620 03:07:11 Config Loaded: 35897-2630 03:08:59 apogee.expose=apRaw-40400021, 30, A, Object 03:14:18 Config Loaded: 35897-2620 03:14:30 Config Loaded: 35897-2631 03:16:18 apogee.expose=apRaw-40400022, 30, A, Object 03:21:37 Config Loaded: 35897-2620 03:21:49 Config Loaded: 35897-2632 03:23:35 apogee.expose=apRaw-40400023, 30, A, Object 03:28:54 Config Loaded: 35897-2620 03:35:26 apogee.expose=apRaw-40400024, 5, A, DomeFlat FVC images #8 - #19 02:01:59 OBSERVER.dmbiz 71 jaeger i text="[FVC]: RMS full fit 54.230 um." 02:09:18 OBSERVER.dmbiz 84 jaeger i text="[FVC]: RMS full fit 54.640 um." 02:16:36 OBSERVER.dmbiz 98 jaeger i text="[FVC]: RMS full fit 54.960 um." 02:23:56 OBSERVER.dmbiz 111 jaeger i text="[FVC]: RMS full fit 53.650 um." 02:31:16 OBSERVER.dmbiz 125 jaeger i text="[FVC]: RMS full fit 55.220 um." 02:38:37 OBSERVER.dmbiz 137 jaeger i text="[FVC]: RMS full fit 54.000 um." 02:45:56 OBSERVER.dmbiz 149 jaeger i text="[FVC]: RMS full fit 54.970 um." 02:53:13 OBSERVER.dmbiz 162 jaeger i text="[FVC]: RMS full fit 55.250 um." 03:00:30 OBSERVER.dmbiz 172 jaeger i text="[FVC]: RMS full fit 56.780 um." 03:07:50 OBSERVER.dmbiz 185 jaeger i text="[FVC]: RMS full fit 57.620 um." 03:15:09 OBSERVER.dmbiz 196 jaeger i text="[FVC]: RMS full fit 56.400 um." 03:22:27 OBSERVER.dmbiz 209 jaeger i text="[FVC]: RMS full fit 56.240 um." jaeger configuration reverse (now robots are back home) ***BOSS DITHER ROBOT SEQUENCE WITH DESIGN 35940 03:38:27 Config Loaded: 35940-2633 03:40:28 boss exposure 4.0, file=336463 03:41:09 boss exposure 4.0, file=336464 03:41:53 sp1.motor.move= 1668, 1669, 1668 03:41:54 Hartmann collimate output sp1: offset: r= 2308 (Out of focus); b= -3.2 (In Focus) sp1: pred. move: spAverageMove= 1668 sp1: pred. resid: r= 640, b= 2.0, txt= OK, spTemp = 0.5 03:47:26 boss exposure 4.0, file=336465 03:49:00 boss exposure 25.0, file=336466 03:54:04 boss exposure 300.0, file=336467 03:42:21 OBSERVER.dmbiz_2.jaeger 149 jaeger i text="[FVC]: RMS full fit 63.250 um." 03:43:02 OBSERVER.dmbiz_2.jaeger 149 jaeger i text="[FVC]: RMS full fit 23.130 um." 03:43:42 OBSERVER.dmbiz_2.jaeger 149 jaeger i text="[FVC]: RMS full fit 12.100 um." 03:44:23 OBSERVER.dmbiz_2.jaeger 149 jaeger i text="[FVC]: RMS full fit 8.770 um." 03:45:03 OBSERVER.dmbiz_2.jaeger 149 jaeger i text="[FVC]: RMS full fit 6.280 um." 03:42:18 OBSERVER.dmbiz_2.jaeger 149 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0020.fits 03:42:59 OBSERVER.dmbiz_2.jaeger 149 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0021.fits 03:43:40 OBSERVER.dmbiz_2.jaeger 149 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0022.fits 03:44:20 OBSERVER.dmbiz_2.jaeger 149 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0023.fits 03:45:00 OBSERVER.dmbiz_2.jaeger 149 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0024.fits jaeger configuration dither 0.2 boss exposure science itime=300 tui wait 2 jaeger configuration reverse tui wait 3 04:01:08 Config Loaded: 35940-2634 04:04:02 boss exposure 300.0, file=336468 04:10:19 Config Loaded: 35940-2633 04:10:49 .jaeger 0 jaeger i snapshot=/data/logs/jaeger/snapshots/59602/fps_snapshot_59602_0039.pdf 04:10:49 OBSERVER.dmbiz.jaeger 114 jaeger f error="Trajectory failed with error: Something went wrong during the trajectory: Move failed to start. Trajectory failed 5.79 seconds after start" 04:10:49 .hub 0 cmds d CmdDone=135920,"f" 04:10:49 CmdDone: OBSERVER.dmbiz.jaeger 114 jaeger configuration execute 04:10:49 OBSERVER.dmbiz 242 jaeger f error="Failed executing configuration." 04:10:49 .hub 0 cmds d CmdDone=135915,"f" 04:10:49 CmdDone: OBSERVER.dmbiz 242 jaeger configuration dither 0.2 04:10:49 .jaeger 144 jaeger i text="Command 'configuration dither 0.2' failed because of an uncaught error 'RuntimeError: cannot modify a done command.'. See traceback in the log for more information." 04:11:02 OBSERVER.dmbiz 243 msg : msg="2022-01-23T04:11:02","4:10:49 OBSERVER.dmbiz.jaeger 114 jaeger f error=\"Trajectory failed with error: Something went wrong during the trajectory: Move failed to start. Trajectory failed 5.79 seconds after start\"" 04:12:10 CmdStarted: OBSERVER.dmbiz 246 jaeger explode 5 04:12:35 CmdStarted: OBSERVER.dmbiz 247 jaeger configuration reverse 04:12:45 OBSERVER.dmbiz 247 jaeger f error="Trajectory failed with error: Something went wrong during the trajectory: Move failed to start. Trajectory failed 6.03 seconds after start" 04:12:45 .hub 0 cmds d CmdDone=135953,"f" 04:12:45 CmdDone: OBSERVER.dmbiz 247 jaeger configuration reverse 04:14:23 APO.Jose: I think I know what's going on, but to fix it I'd need to restart jaeger and we'd lose the configuration, so go on with a smaller dither size. 04:17:13 APO.Jose: Ok, if you go back to folded I can fix it. 04:20:20 APO.Jose: Ok. Restart jaeger from systemctl and I think it will work. Done. 04:21:30 APO.Jose: The problem is that the trajectories were taking longer than 2 seconds. 2 second was the threshold at which jaeger does a check to see if the robots are moving. The robots take a bit to report that they are moving so if it's a very short trajectory and you check too early they report they are not moving when they are. I thought 2 seconds was safe but it seems not ... I've disabled that check for now. Starting over: 04:23:50 CmdDone: OBSERVER.dmbiz 257 jaeger configuration load 35940 04:23:47 Config Loaded: 35940-2636 04:27:54 FVC Loop - RMS: 5.16 Delta RMS: 0.46 Image #: 0028 04:25:51 OBSERVER.dmbiz 261 jaeger i text="[FVC]: RMS full fit 50.680 um." 04:26:32 OBSERVER.dmbiz 261 jaeger i text="[FVC]: RMS full fit 8.020 um." 04:27:13 OBSERVER.dmbiz 261 jaeger i text="[FVC]: RMS full fit 5.620 um." 04:27:54 OBSERVER.dmbiz 261 jaeger i text="[FVC]: RMS full fit 5.160 um." 04:25:48 OBSERVER.dmbiz 261 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0025.fits 04:26:29 OBSERVER.dmbiz 261 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0026.fits 04:27:10 OBSERVER.dmbiz 261 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0027.fits 04:27:51 OBSERVER.dmbiz 261 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0028.fits HAL's expose within BOSON worked to only start a 900 second BOSS exposure. 04:33:32 boss exposure 300.0, file=336469 (BASE) jaeger configuration dither 0.2 boss exposure science itime=300 tui wait 2 jaeger configuration reverse tui wait 3 04:40:28 Config Loaded: 35940-2637 04:40:58 boss exposure 300.0, file=336470 04:47:14 Config Loaded: 35940-2636 04:47:26 Config Loaded: 35940-2638 04:47:55 boss exposure 300.0, file=336471 04:54:12 Config Loaded: 35940-2636 04:54:24 Config Loaded: 35940-2639 04:54:56 boss exposure 300.0, file=336472 05:01:13 Config Loaded: 35940-2636 05:01:24 Config Loaded: 35940-2640 05:01:56 boss exposure 300.0, file=336473 05:08:12 Config Loaded: 35940-2636 05:08:24 Config Loaded: 35940-2641 05:08:55 boss exposure 300.0, file=336474 05:15:12 Config Loaded: 35940-2636 05:15:24 Config Loaded: 35940-2642 05:15:55 boss exposure 300.0, file=336475 05:22:13 Config Loaded: 35940-2636 05:22:24 Config Loaded: 35940-2643 05:22:56 boss exposure 300.0, file=336476 05:29:13 Config Loaded: 35940-2636 05:29:24 Config Loaded: 35940-2644 05:29:56 boss exposure 300.0, file=336477 05:36:12 Config Loaded: 35940-2636 05:36:24 Config Loaded: 35940-2645 05:36:56 boss exposure 300.0, file=336478 05:43:13 Config Loaded: 35940-2636 05:43:24 Config Loaded: 35940-2646 05:43:56 boss exposure 300.0, file=336479 05:50:12 Config Loaded: 35940-2636 05:50:24 Config Loaded: 35940-2647 05:50:56 boss exposure 300.0, file=336480 05:57:12 Config Loaded: 35940-2636 05:57:24 Config Loaded: 35940-2648 05:57:56 boss exposure 300.0, file=336481 06:04:13 Config Loaded: 35940-2636 jaeger configuration reverse - to send back home ***APOGEE AND BOSS SCIENCE TEST WITH FPI, WITH DESIGN 12798 Jose worked on some modifications to HAL to help with the axes halting from before. It was restarted. 06:06:58 CmdStarted: OBSERVER.dmbiz 328 jaeger configuration load 12798 06:10:56 boss exposure 4.0, file=336482 06:11:36 boss exposure 4.0, file=336483 06:12:20 sp1.motor.move= -1917, -1915, -1921 06:12:20 Hartmann collimate output sp1: offset: r= -1649 (Out of focus); b= 6.8 (Out of focus) sp1: pred. move: spAverageMove= -1917 sp1: pred. resid: r= 268, b= 0.8, txt= OK, spTemp = 0.5 06:14:09 FVC Loop - RMS:38.78 Delta RMS: -2.06 Image #: 0031 06:16:55 FVC Loop - RMS:37.95 Delta RMS: 0.46 Image #: 0033 06:20:43 boss exposure 4.0, file=336484 06:22:17 boss exposure 25.0, file=336485 06:12:47 OBSERVER.dmbiz_2.jaeger 119 jaeger i text="[FVC]: RMS full fit 60.110 um." 06:13:29 OBSERVER.dmbiz_2.jaeger 119 jaeger i text="[FVC]: RMS full fit 36.720 um." 06:14:09 OBSERVER.dmbiz_2.jaeger 119 jaeger i text="[FVC]: RMS full fit 38.780 um." 06:16:14 OBSERVER.dmbiz_2.jaeger 127 jaeger i text="[FVC]: RMS full fit 38.410 um." 06:16:55 OBSERVER.dmbiz_2.jaeger 127 jaeger i text="[FVC]: RMS full fit 37.950 um." 06:12:45 OBSERVER.dmbiz_2.jaeger 119 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0029.fits 06:13:26 OBSERVER.dmbiz_2.jaeger 119 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0030.fits 06:14:06 OBSERVER.dmbiz_2.jaeger 119 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0031.fits 06:16:11 OBSERVER.dmbiz_2.jaeger 127 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0032.fits 06:16:52 OBSERVER.dmbiz_2.jaeger 127 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0033.fits Restarting hal to verify changes to the track command within the goto-field sequence (moving to acquire instead of fvc stage), as well as cal box shutter errors. It worked. 06:46Z - Plugged in the FPI fibers. 07:12:00 apogee.expose=apRaw-40400025, 39, A, Object 07:12:06 boss exposure 900.0, file=336486 07:18:58 apogee.expose=apRaw-40400026, 39, B, Object We tested Boson's new hal expose GUI, and such the exposure time for APOGEE is not what it should be during this sequence. We fixed it for the following exposures. We noticed that we are still running the guider with the same parameters as we used the previous night. Our guiding has been excellent, so we do not want to change anything. But it is good to be aware of the guider status at all times while observing. 07:44:51 APO.aoravetz 29 cherno i text="status: EXPOSING | FAILED | NON_IDLE" 07:44:51 APO.aoravetz 29 cherno i text="offset: (0.0, 0.0, -432.0)" 07:44:51 APO.aoravetz 29 cherno i text="exposure_time: 20.0" 07:44:51 APO.aoravetz 29 cherno i text="guide_loop: {'radec': {'max_correction': 600," 07:44:51 APO.aoravetz 29 cherno i text=" 'min_correction': 0.05," 07:44:51 APO.aoravetz 29 cherno i text=" 'pid': {'k': 0.7}}," 07:44:51 APO.aoravetz 29 cherno i text=" 'rot': {'max_correction': 7200," 07:44:51 APO.aoravetz 29 cherno i text=" 'min_correction': 3," 07:44:51 APO.aoravetz 29 cherno i text=" 'min_isolated_correction': 30," 07:44:51 APO.aoravetz 29 cherno i text=" 'pid': {'k': 0.7}}}" 07:44:51 APO.aoravetz 29 cherno i text="acquisition: {'cpulimit': 15.0, 'min_npix': 20.0, 'sigma': 4.0}" 07:44:51 APO.aoravetz 29 cherno i text="enabled_cameras: ['gfa2', 'gfa3', 'gfa4', 'gfa5', 'gfa6']" 07:44:51 APO.aoravetz 29 cherno i text="enabled_axes: ['radec', 'rot']" Seeing 2.0". Clear. 07:12:00 apogee.expose=apRaw-40400025, 39, A, Object 07:12:06 boss exposure 900.0, file=336486 07:18:58 apogee.expose=apRaw-40400026, 39, B, Object 07:29:37 apogee.expose=apRaw-40400027, 43, B, Object 07:29:43 boss exposure 900.0, file=336487 07:37:23 apogee.expose=apRaw-40400028, 43, A, Object 07:52:04 apogee.expose=apRaw-40400029, 43, A, Object 07:52:10 boss exposure 900.0, file=336488 07:59:45 apogee.expose=apRaw-40400030, 43, B, Object 08:14:29 apogee.expose=apRaw-40400031, 43, B, Object 08:14:35 boss exposure 900.0, file=336489 08:22:14 apogee.expose=apRaw-40400032, 43, A, Object 08:31:00 apogee.expose=apRaw-40400033, 43, A, Object 08:31:07 boss exposure 900.0, file=336490 08:38:41 apogee.expose=apRaw-40400034, 43, B, Object 08:48:42 apogee.expose=apRaw-40400035, 5, B, DomeFlat jaeger unwind *** APOGEE DITHER ROBOT SEQUENCE WITH DESIGN 35947 9:01:37 CmdStarted: OBSERVER.dmbiz_2 27 hal goto-field -s prepare,slew,fvc,acquire,guide --guider-time 10 Failed with error messages, and brakes engaged with all axes: 09:01:37 OBSERVER.dmbiz_2 27 hal > 09:01:38 CmdStarted: OBSERVER.dmbiz_2.tcc 154 tcc axis stop 09:01:38 OBSERVER.dmbiz_2 27 hal i stages=goto_field,prepare,slew,fvc,acquire,guide,cleanup 09:01:38 OBSERVER.dmbiz_2 27 hal i all_stages=goto_field,prepare,slew,reconfigure,boss_hartmann,fvc,boss_arcs,boss_flat,acquire,guide,cleanup 09:01:38 .tcc 0 tcc w AxisCmdState=Halting,Halting,Halting; AxisErrCode=HaltRequested,HaltRequested,HaltRequested; AxisNoTrack=TTT 09:01:38 .tcc 0 tcc i AxisCmdState=Halting,Halting,Halted 09:01:38 .tcc 0 tcc i AxisCmdState=Halting,Halted,Halted 09:01:38 .tcc 0 tcc i AxisCmdState=Halted,Halted,Halted 09:01:38 CmdDone: OBSERVER.dmbiz_2.tcc 154 tcc axis stop 09:01:38 CmdStarted: OBSERVER.dmbiz_2.tcc 155 tcc track 135.0, 40.0 icrs /rottype=object/rotang=0/rotwrap=mid 09:01:38 OBSERVER.dmbiz_2 27 hal i text="Slewing to (ra, dec, rot) == (135.0000, 40.0000, 0)" 09:01:39 OBSERVER.dmbiz_2.tcc 155 tcc i AxisCmdState=Slewing,Slewing,Slewing 09:01:39 OBSERVER.dmbiz_2.tcc 155 tcc > SlewBeg=5149645324.73; SlewDuration=25.94; SlewNumIter=2; MoveItems=NYNNYYYYY 09:01:47 .tcc 0 tcc w BadRotStatus; RotStat=77.720944,0.000000,5149645307.19934,0x00000800; RotDTime=0.04 09:01:48 .mcp 0 mcp i trace="MGT: bad az state ABORT_EVENT: ID_ERROR_LIMIT" 09:01:49 .tcc 0 tcc w BadAzStatus; AzStat=227.424319,0.000000,5149645309.27996,0x00000800; AzDTime=0.01 09:01:50 .mcp 0 mcp i command="brake_on" 09:01:51 .tcc 0 tcc w BadAltStatus; AltStat=64.620111,0.000000,5149645311.42721,0x00000800; AltDTime=0.04 09:01:58 .tcc 0 tcc w BadAzStatus; AzStat=227.424081,0.000000,5149645317.65500,0x00001800; AzDTime=0.01 09:01:59 .tcc 0 tcc i AxisCmdState=Slewing,Slewing,Slewing; AxisErrCode=OK,OK,OK; AxisNoTrack=FFF 09:02:02 .tcc 0 tcc w BadAltStatus; AltStat=64.650576,0.000000,5149645321.49502,0x00001800; AltDTime=0.04 09:02:05 OBSERVER.dmbiz_2 27 hal e error="TCC slew command ended with some bad bits set: 0x1800,0x1800,0x800" 09:02:05 OBSERVER.dmbiz_2 27 hal e error="Failed to complete slew: see TCC messages for details." Everything worked just fine the second time around: 09:02:38 .tcc 0 tcc i AxisCmdState=Halted,Halted,Halted; AxisErrCode=HaltRequested,HaltRequested,HaltRequested; AxisNoTrack=TTT 09:02:38 .alerts 0 alerts i activeAlerts 09:02:38 .alerts 0 alerts i disabledAlertRules 09:02:38 CmdStarted: OBSERVER.dmbiz_2 28 hal goto-field -s prepare,slew,fvc,acquire,guide --guider-time 10 08:59:48 Config Loaded: 35947-2651 9:03:33 OBSERVER.dmbiz_2.jaeger 162 jaeger i text="[FVC]: RMS full fit 65.560 um." 09:04:14 OBSERVER.dmbiz_2.jaeger 162 jaeger i text="[FVC]: RMS full fit 21.840 um." 09:04:56 OBSERVER.dmbiz_2.jaeger 162 jaeger i text="[FVC]: RMS full fit 12.450 um." 09:05:37 OBSERVER.dmbiz_2.jaeger 162 jaeger i text="[FVC]: RMS full fit 9.580 um." 09:06:18 OBSERVER.dmbiz_2.jaeger 162 jaeger i text="[FVC]: RMS full fit 7.650 um." 09:03:30 OBSERVER.dmbiz_2.jaeger 162 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0034.fits 09:04:12 OBSERVER.dmbiz_2.jaeger 162 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0035.fits 09:04:53 OBSERVER.dmbiz_2.jaeger 162 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0036.fits 09:05:34 OBSERVER.dmbiz_2.jaeger 162 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0037.fits 09:06:16 OBSERVER.dmbiz_2.jaeger 162 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0038.fits 09:21:15 apogee.expose=apRaw-40400036, 30, B, Object 09:27:50 Config Loaded: 35947-2652 09:31:12 Config Loaded: 35947-2651 09:28:23 OBSERVER.dmbiz.fliswarm 52 fliswarm w error=fvc,fvc1n,"Failed writing image to disk: 6132 requested and 0 written" 09:28:23 OBSERVER.dmbiz.fliswarm 52 fliswarm w error=fvc,"One or more cameras failed to expose." 09:28:23 OBSERVER.dmbiz.fliswarm 52 fliswarm : 09:28:23 CmdDone: OBSERVER.dmbiz.fliswarm 52 fliswarm talk -c fvc expose 5.0 09:28:23 OBSERVER.dmbiz 441 jaeger f error="Failed processing image: The exposure succeeded but did not output the filename." 09:28:23 CmdDone: OBSERVER.dmbiz 441 jaeger fvc loop --no-apply 09:28:23 .jaeger 80 jaeger i text="Saving confSummaryF file." 09:28:23 .jaeger 80 jaeger i text="Command 'fvc loop --no-apply' failed because of an uncaught error 'FVCError: No fibre data.'. See traceback in the log for more information." Not sure what happened to the fvc loop --no-apply, but it does not matter because we noticed that the data we have taken thus far is in the B dither position from the last design. We are gong to start the loop over with A dithering and then go back to take another base image with dither A at the end. Clear. Refocused. Seeing 1.0". jaeger configuration dither 0.1 tcc axis stop cherno set axes jaeger fvc loop --no-apply tcc axis init tcc track tui wait 3 cherno set axes radec rot tui wait 20 apogee dither namedpos=A apogee expose nreads=30 object=Object tui wait 2 jaeger configuration reverse tui wait 3 09:32:07 apogee.expose=apRaw-40400037, 30, A, Object - NOT GOOD. 9:38:19 Config Loaded: 35947-2653 09:40:15 apogee.expose=apRaw-40400038, 30, A, Object 09:45:35 Config Loaded: 35947-2651 09:51:08 Config Loaded: 35947-2654 09:52:54 apogee.expose=apRaw-40400039, 30, A, expose 09:53:25 Config Loaded: 35947-2651 We are unsure what happened and where the robots are. We are going to start completely over as a precaution. jaeger unwind 09:55:28 CmdDone: OBSERVER.dmbiz 468 jaeger configuration load 35947 09:55:32 CmdStarted: OBSERVER.dmbiz 469 jaeger configuration execute 09:57:00 CmdStarted: OBSERVER.dmbiz 473 jaeger fvc loop 09:55:26 Config Loaded: 35947-2655 09:58:50 FVC Loop - RMS: 4.90 Delta RMS: 3.00 Image #: 0044 10:00:30 apogee.expose=apRaw-40400040, 30, A, Object 09:51:46 OBSERVER.dmbiz 458 jaeger i text="[FVC]: RMS full fit 63.650 um." 09:57:27 OBSERVER.dmbiz 473 jaeger i text="[FVC]: RMS full fit 55.720 um." 09:58:09 OBSERVER.dmbiz 473 jaeger i text="[FVC]: RMS full fit 7.900 um." 09:58:50 OBSERVER.dmbiz 473 jaeger i text="[FVC]: RMS full fit 4.900 um." 09:38:55 OBSERVER.dmbiz 448 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0040.fits 09:51:43 OBSERVER.dmbiz 458 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0041.fits 09:57:24 OBSERVER.dmbiz 473 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0042.fits 09:58:06 OBSERVER.dmbiz 473 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0043.fits 09:58:47 OBSERVER.dmbiz 473 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0044.fits jaeger configuration dither 0.1 tcc axis stop cherno set axes jaeger fvc loop --no-apply tcc axis init tcc track tui wait 3 cherno set axes radec rot tui wait 20 apogee dither namedpos=A apogee expose nreads=30 object=Object tui wait 2 jaeger configuration reverse tui wait 3 Error messages seen after executing an fvc loop --no-apply: 10:07:40 "Failed writing image to disk: 6132 requested and 0 written" 10:07:41 fliswarm w error=fvc,"One or more cameras failed to expose." 10:07:41 jaeger f error="Failed processing image: The exposure succeeded but did not output the filename." Accidentally commanded *TWO* configuration dither 0.1 commands. Then reversed the robots twice, which sent them home. Trying again: 10:12:55 OBSERVER.dmbiz 491 jaeger : text="Configuration 2658 loaded." 10:12:55 CmdDone: OBSERVER.dmbiz 491 jaeger configuration load 35947 10:13:49 CmdDone: OBSERVER.dmbiz 492 jaeger configuration execute 10:13:56 CmdStarted: OBSERVER.dmbiz 493 jaeger fvc loop 10:08:54 OBSERVER.dmbiz 484 jaeger i text="[FVC]: RMS full fit 56.040 um." 10:14:23 OBSERVER.dmbiz 493 jaeger i text="[FVC]: RMS full fit 67.480 um." 10:15:05 OBSERVER.dmbiz 493 jaeger i text="[FVC]: RMS full fit 27.200 um." 10:15:45 OBSERVER.dmbiz 493 jaeger i text="[FVC]: RMS full fit 26.010 um." 10:16:26 OBSERVER.dmbiz 493 jaeger i text="[FVC]: RMS full fit 25.240 um." 10:17:18 OBSERVER.dmbiz 494 jaeger i text="[FVC]: RMS full fit 24.710 um." 10:17:59 OBSERVER.dmbiz 494 jaeger i text="[FVC]: RMS full fit 24.160 um." 10:18:47 OBSERVER.dmbiz 495 jaeger i text="[FVC]: RMS full fit 23.840 um." 10:19:28 OBSERVER.dmbiz 495 jaeger i text="[FVC]: RMS full fit 23.410 um." It never got any better. Trying again. Unwind. *** Start from here **** 0:22:33 Config Loaded: 35947-2659 10:26:11 FVC Loop - RMS: 5.65 Delta RMS: 0.97 Image #: 0058 10:24:05 OBSERVER.dmbiz 500 jaeger i text="[FVC]: RMS full fit 56.330 um." 10:24:48 OBSERVER.dmbiz 500 jaeger i text="[FVC]: RMS full fit 8.850 um." 10:25:30 OBSERVER.dmbiz 500 jaeger i text="[FVC]: RMS full fit 6.620 um." 10:26:11 OBSERVER.dmbiz 500 jaeger i text="[FVC]: RMS full fit 5.650 um." 10:24:02 OBSERVER.dmbiz 500 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0055.fits 10:24:45 OBSERVER.dmbiz 500 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0056.fits 10:25:28 OBSERVER.dmbiz 500 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0057.fits 10:26:08 OBSERVER.dmbiz 500 jaeger d fvc_filename=/data/fcam/59602/fimg-fvc1n-0058.fits 10:30:52 apogee.expose=apRaw-40400041, 30, A, Object 10:40:20 Config Loaded: 35947-2660 10:42:01 apogee.expose=apRaw-40400042, 30, A, Object 10:47:19 Config Loaded: 35947-2659 10:47:31 Config Loaded: 35947-2661 10:49:11 apogee.expose=apRaw-40400043, 30, A, Object 10:54:32 Config Loaded: 35947-2659 10:54:43 Config Loaded: 35947-2662 10:56:26 apogee.expose=apRaw-40400044, 30, A, Object 11:01:46 Config Loaded: 35947-2659 11:01:58 Config Loaded: 35947-2663 11:03:40 apogee.expose=apRaw-40400045, 30, A, Object 11:09:01 Config Loaded: 35947-2659 11:09:13 Config Loaded: 35947-2664 11:10:55 apogee.expose=apRaw-40400046, 30, A, Object 11:16:16 Config Loaded: 35947-2659 11:16:27 Config Loaded: 35947-2665 11:18:09 apogee.expose=apRaw-40400047, 30, A, Object 11:23:30 Config Loaded: 35947-2659 11:23:42 Config Loaded: 35947-2666 11:24:16 OBSERVER.dmbiz.fliswarm 166 fliswarm w error=fvc,fvc1n,"Failed writing image to disk: 6132 requested and 0 written" 11:24:16 OBSERVER.dmbiz.fliswarm 166 fliswarm w error=fvc,"One or more cameras failed to expose." 11:24:16 OBSERVER.dmbiz.fliswarm 166 fliswarm : 11:24:16 CmdDone: OBSERVER.dmbiz.fliswarm 166 fliswarm talk -c fvc expose 5.0 11:24:16 OBSERVER.dmbiz 561 jaeger f error="Failed processing image: The exposure succeeded but did not output the filename." Tried again and failed an additional two times before it worked. 11:33:54 apogee.expose=apRaw-40400048, 30, A, Object 11:39:14 Config Loaded: 35947-2659 11:40:43 Config Loaded: 35947-2667 11:42:25 apogee.expose=apRaw-40400049, 30, A, Object 11:47:46 Config Loaded: 35947-2659 11:47:58 Config Loaded: 35947-2668 11:49:39 apogee.expose=apRaw-40400050, 30, A, Object 11:55:00 Config Loaded: 35947-2659 11:55:12 Config Loaded: 35947-2669 11:56:54 apogee.expose=apRaw-40400051, 30, A, Object 12:02:14 Config Loaded: 35947-2659 12:02:26 Config Loaded: 35947-2670 12:04:07 apogee.expose=apRaw-40400052, 30, A, Object 12:09:27 Config Loaded: 35947-2659 12:09:39 Config Loaded: 35947-2671 12:11:20 apogee.expose=apRaw-40400053, 30, A, Object 12:16:41 Config Loaded: 35947-2659 12:18:24 apogee.expose=apRaw-40400054, 5, A, DomeFlat FVC Images from 59-73 10:40:57 OBSERVER.dmbiz 507 jaeger i text="[FVC]: RMS full fit 54.980 um." 10:48:09 OBSERVER.dmbiz 516 jaeger i text="[FVC]: RMS full fit 54.970 um." 10:55:23 OBSERVER.dmbiz 525 jaeger i text="[FVC]: RMS full fit 55.950 um." 11:02:37 OBSERVER.dmbiz 534 jaeger i text="[FVC]: RMS full fit 55.380 um." 11:09:53 OBSERVER.dmbiz 543 jaeger i text="[FVC]: RMS full fit 55.080 um." 11:17:07 OBSERVER.dmbiz 552 jaeger i text="[FVC]: RMS full fit 54.690 um." 11:31:35 OBSERVER.dmbiz 564 jaeger i text="[FVC]: RMS full fit 54.770 um." 11:41:23 OBSERVER.dmbiz 573 jaeger i text="[FVC]: RMS full fit 55.170 um." 11:48:36 OBSERVER.dmbiz 582 jaeger i text="[FVC]: RMS full fit 55.780 um." 11:55:52 OBSERVER.dmbiz 591 jaeger i text="[FVC]: RMS full fit 57.100 um." 12:03:05 OBSERVER.dmbiz 600 jaeger i text="[FVC]: RMS full fit 56.640 um." 12:10:19 OBSERVER.dmbiz 609 jaeger i text="[FVC]: RMS full fit 57.310 um." jaeger unwind. Power off. 12:21Z - Closed enclosure and started end of night cals. ---- Mirror Numbers ---- PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A: 0.0030 0.0040 Axial B: 0.0130 0.0030 Axial C: -0.1120 -0.1150 Trans D: -0.0342 -0.0338 Lateral E: 0.3315 0.3315 Lateral F: 0.5350 0.5350 GALILS Commanded: 12500 -1600 5300 47050 -8800 -8400 Actual: 12500 -1600 5300 47050 -8800 -8400 SETMIR VALUES PrimDesOrient: 0.0 -59.5 -9.15 -512.0 1906.3 PrimOrient: 1.07 -59.51 -9.21 -512.13 1906.31 SECONDARY: ---------- Focus: 0.0000 Air temp: 1.00 Altitude: 30.0 MIGS TONIGHT NOMINAL Axial A: -9.6580 -9.6570 Axial B: -9.6700 -9.6710 Axial C: -9.6950 -9.6970 Trans D: 0.1100 0.1140 GALILS Commanded: 4899350 4898850 4916750 -100 -3450 Actual: 4861410 4861466 4861397 -100 -3450 SETMIR VALUES SecDesOrient: 3858.73 0.0 0.0 0.0 43.54 SecOrient: 3858.75 0.02 0.01 -0.6 43.47 ---- Telescope Status ---- 23:20Z Started ventilation 00:30Z encl25m : open 12:32Z encl25m : closed No interlocks bypassed Status at: 12:32Z Telescope stowed at: 121.0, 30.0, 0.0 mount Instrument mounted: cartridge = FPS Counterweights at: 680/710 LN2 autofill systems: Connected and turned on 180L LN2 dewar scales: SP1 211 lbs, 20 psi ---- Software Used ---- STUI : 1.7.16 alerts : 2.2.0 apo : 3.0.1 boss : 6.0.1 -boss.specMec : 3.0.4 guider : FAILED ecamera : FAILED fliswarm : 0.2.1-alpha.0 jaeger : 0.13.2-alpha.0 kaiju : 1.2.2-alpha.1 coordio : 1.2.1-alpha.2 fps_calibrati : APO.2021.12.25 cherno : 0.1.1-alpha.0 hal : 0.1.1-alpha.0 hartmann : 2.0.2 hub : 5.0.1a0 mcp : 2.1 -mcp.plc : 34 -mcp.azFiduci : v1_148 -mcp.altFiduc : v1_148 -mcp.rotFiduc : v1_148 tcc : 1.4.0 apogee : not availble apogeecal : not availble apogeeql : v1_7_0dev STUIAdditions : 3.1.11 ---- APOGEE Data Summary ---- Last morning: 59601 48 11:30 40390048 Dark 60 A 13.0 ---------- a-b-c - - 59601 49 11:41 40390049 Dark 60 A 13.0 ---------- a-b-c - - 59601 50 11:51 40390050 Dark 60 A 13.0 ---------- a-b-c - - 59601 51 11:54 40390051 QuarFlat 10 A 13.0 ---------- a-b-c - 98 59601 52 11:55 40390052 QuarFlat 10 A 13.0 ---------- a-b-c - 98 59601 53 11:57 40390053 QuarFlat 10 A 13.0 ---------- a-b-c - 99 59601 54 11:59 40390054 Arc-Thar 12 B 13.5 ---------- a-b-c 0.51 - 59601 55 12:07 40390055 Arc-Une 40 B 13.5 ---------- a-b-c - - 59601 56 12:09 40390056 Arc-Thar 12 A 13.0 ---------- a-b-c -0.08 - 59601 57 12:16 40390057 Arc-Une 40 A 13.0 ---------- a-b-c - - 59601 58 12:22 40390058 Dark 30 A 13.0 ---------- a-b-c - - 59601 59 12:27 40390059 InteFlat 30 A 13.0 ---------- a-b-c - 279 59601 60 12:32 40390060 InteFlat 30 A 13.0 ---------- a-b-c - 279 59601 61 12:38 40390061 InteFlat 30 A 13.0 ---------- a-b-c - 279 59601 62 12:43 40390062 Dark 30 A 13.0 ---------- a-b-c - - 59601 63 12:49 40390063 Arc---- 30 A 13.0 ---------- a-b-c - - FPI FPI image has signal. APOGEE data list, mjd=59602, all raw_data: /data/apogee/utr_cdr/59602/apRaw-*.fits archive: /data/apogee/archive/59602/apR-[a,b,c]-*.apz quickred: /data/apogee/quickred/59602/apq-*.fits ------------------------------------------------------------------------------ mjd i UT File/Exp Imtype Nread A/B Dth Design-Cfg Arch Offset %Flux Comm ------------------------------------------------------------------------------ 59602 1 23:08 40400001 Arc---- 3 A 13.0 ---------- a-b-c - - test 59602 2 23:21 40400002 Dark 60 A 13.0 ---------- a-b-c - - 59602 3 23:23 40400003 QuarFlat 10 A 13.0 ---------- a-b-c - 99 59602 4 23:25 40400004 Arc-Thar 12 B 13.5 ---------- a-b-c 0.49 - 59602 5 23:32 40400005 Arc-Une 40 B 13.5 ---------- a-b-c - - 59602 6 23:35 40400006 Arc-Thar 12 A 13.0 ---------- a-b-c -0.12 - 59602 7 23:42 40400007 Arc-Une 40 A 13.0 ---------- a-b-c - - 59602 8 23:53 40400008 Dark 60 A 13.0 ---------- a-b-c - - 59602 9 23:58 40400009 Arc---- 30 A 13.0 ---------- a-b-c - - FPI 59602 10 01:54 40400010 Object 28 A 13.0 35897-2620 a-b-c -0.11 - 59602 11 02:00 40400011 Object 30 A 13.0 35897-2620 a-b-c -0.11 - 35897 base 59602 12 02:08 40400012 Object 30 A 13.0 35897-2621 a-b-c -0.11 - 59602 13 02:15 40400013 Object 30 A 13.0 35897-2622 a-b-c -0.11 - 59602 14 02:22 40400014 Object 30 A 13.0 35897-2623 a-b-c -0.11 - 59602 15 02:30 40400015 Object 30 A 13.0 35897-2624 a-b-c -0.11 - 59602 16 02:37 40400016 Object 30 A 13.0 35897-2625 a-b-c -0.12 - 59602 17 02:44 40400017 Object 30 A 13.0 35897-2626 a-b-c -0.11 - 59602 18 02:52 40400018 Object 30 A 13.0 35897-2627 a-b-c -0.11 - 59602 19 02:59 40400019 Object 30 A 13.0 35897-2628 a-b-c -0.11 - 59602 20 03:06 40400020 Object 30 A 13.0 35897-2629 a-b-c -0.11 - 59602 21 03:13 40400021 Object 30 A 13.0 35897-2630 a-b-c -0.11 - 59602 22 03:21 40400022 Object 30 A 13.0 35897-2631 a-b-c -0.11 - 59602 23 03:28 40400023 Object 30 A 13.0 35897-2632 a-b-c -0.11 - 59602 24 03:35 40400024 DomeFlat 5 A 13.0 35897-2620 a-b-c - 126 59602 25 07:18 40400025 Object 39 A 13.0 12798-2649 a-b-c -0.11 - 59602 26 07:25 40400026 Object 39 B 13.5 12798-2649 a-b-c 0.40 - 59602 27 07:36 40400027 Object 43 B 13.5 12798-2649 a-b-c 0.40 - 59602 28 07:44 40400028 Object 43 A 13.0 12798-2649 a-b-c -0.10 - 59602 29 07:59 40400029 Object 43 A 13.0 12798-2649 a-b-c -0.10 - 59602 30 08:06 40400030 Object 43 B 13.5 12798-2649 a-b-c 0.40 - 59602 31 08:21 40400031 Object 43 B 13.5 12798-2649 a-b-c 0.40 - 59602 32 08:29 40400032 Object 43 A 13.0 12798-2649 a-b-c -0.09 - 59602 33 08:38 40400033 Object 43 A 13.0 12798-2649 a-b-c -0.10 - 59602 34 08:45 40400034 Object 43 B 13.5 12798-2649 a-b-c 0.41 - 59602 35 08:49 40400035 DomeFlat 5 B 13.5 12798-2649 a-b-c - 127 59602 36 09:26 40400036 Object 30 B 13.5 35947-2651 a-b-c 0.41 - 35947 base 59602 37 09:37 40400037 Object 30 A 13.0 35947-2651 a-b-c -0.10 - 35947 base 59602 38 09:45 40400038 Object 30 A 13.0 35947-2653 a-b-c -0.10 - 59602 39 09:52 40400039 Object 30 A 13.0 ---------- a-b-c - - 59602 40 10:05 40400040 Object 30 A 13.0 35947-2655 a-b-c -0.09 - 35947 base 59602 41 10:35 40400041 Object 30 A 13.0 35947-2659 a-b-c -0.10 - 35947 base 59602 42 10:46 40400042 Object 30 A 13.0 35947-2660 a-b-c -0.09 - 59602 43 10:54 40400043 Object 30 A 13.0 35947-2661 a-b-c -0.10 - 59602 44 11:01 40400044 Object 30 A 13.0 35947-2662 a-b-c -0.10 - 59602 45 11:08 40400045 Object 30 A 13.0 35947-2663 a-b-c -0.11 - 59602 46 11:15 40400046 Object 30 A 13.0 35947-2664 a-b-c -0.11 - 59602 47 11:23 40400047 Object 30 A 13.0 35947-2665 a-b-c -0.10 - 59602 48 11:38 40400048 Object 30 A 13.0 35947-2666 a-b-c -0.10 - 59602 49 11:47 40400049 Object 30 A 13.0 35947-2667 a-b-c -0.10 - 59602 50 11:54 40400050 Object 30 A 13.0 35947-2668 a-b-c -0.10 - 59602 51 12:01 40400051 Object 30 A 13.0 35947-2669 a-b-c -0.10 - 59602 52 12:09 40400052 Object 30 A 13.0 35947-2670 a-b-c -0.10 - 59602 53 12:16 40400053 Object 30 A 13.0 35947-2671 a-b-c -0.10 - 59602 54 12:18 40400054 DomeFlat 5 A 13.0 35947-2659 a-b-c - 128 59602 55 12:31 40400055 Dark 60 A 13.0 ---------- x-x-x - - ------------------------------------------------------------------------------ ---- eBOSS Data Summary ---- Reading FITS from /data/spectro//59602 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - n Time Filename design-field cfg flav shu Exp colA colB colC Hrt - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 1 23:08 r1-00336459 ----------- ---- bia Clo 0.0 -6305 -6263 -6326 2 23:08 b1-00336459 ----------- ---- bia Clo 0.0 -6305 -6263 -6326 3 23:11 r1-00336460 ----------- ---- dar Clo 20.0 -6305 -6263 -6326 4 23:11 b1-00336460 ----------- ---- dar Clo 20.0 -6305 -6263 -6326 5 23:14 b1-00336461 35889-16188 2611 arc Lef 4.1 -6305 -6263 -6326 HART 6 23:14 r1-00336461 35889-16188 2611 arc Lef 4.1 -6305 -6263 -6326 HART 7 23:14 b1-00336462 35889-16188 2611 arc Rig 4.1 -6305 -6263 -6326 HART 8 23:14 r1-00336462 35889-16188 2611 arc Rig 4.1 -6305 -6263 -6326 HART 9 03:40 b1-00336463 35940-20909 2633 arc Lef 4.1 -6928 -6885 -6951 HART 10 03:40 r1-00336463 35940-20909 2633 arc Lef 4.1 -6928 -6885 -6951 HART 11 03:41 r1-00336464 35940-20909 2633 arc Rig 4.0 -6928 -6885 -6951 HART 12 03:41 b1-00336464 35940-20909 2633 arc Rig 4.0 -6928 -6885 -6951 HART 13 03:47 r1-00336465 35940-20909 2633 arc Out 4.1 -5260 -5216 -5283 14 03:47 b1-00336465 35940-20909 2633 arc Out 4.1 -5260 -5216 -5283 15 03:48 b1-00336466 35940-20909 2633 fla Out 25.1 -5260 -5216 -5283 16 03:48 r1-00336466 35940-20909 2633 fla Out 25.1 -5260 -5216 -5283 17 03:54 r1-00336467 35940-20909 2633 sci Out 300.1 -5260 -5216 -5283 18 03:54 b1-00336467 35940-20909 2633 sci Out 300.1 -5260 -5216 -5283 19 04:04 r1-00336468 35940-20909 2634 sci Out 300.1 -5260 -5216 -5283 20 04:04 b1-00336468 35940-20909 2634 sci Out 300.1 -5260 -5216 -5283 21 04:33 b1-00336469 35940-20909 2636 sci Out 300.1 -5260 -5216 -5283 22 04:33 r1-00336469 35940-20909 2636 sci Out 300.1 -5260 -5216 -5283 23 04:40 r1-00336470 35940-20909 2637 sci Out 300.1 -5260 -5216 -5283 24 04:40 b1-00336470 35940-20909 2637 sci Out 300.1 -5260 -5216 -5283 25 04:47 b1-00336471 35940-20909 2638 sci Out 300.1 -5260 -5216 -5283 26 04:47 r1-00336471 35940-20909 2638 sci Out 300.1 -5260 -5216 -5283 27 04:54 b1-00336472 35940-20909 2639 sci Out 300.1 -5260 -5216 -5283 28 04:54 r1-00336472 35940-20909 2639 sci Out 300.1 -5260 -5216 -5283 29 05:01 b1-00336473 35940-20909 2640 sci Out 300.1 -5260 -5216 -5283 30 05:01 r1-00336473 35940-20909 2640 sci Out 300.1 -5260 -5216 -5283 31 05:08 r1-00336474 35940-20909 2641 sci Out 300.1 -5260 -5216 -5283 32 05:08 b1-00336474 35940-20909 2641 sci Out 300.1 -5260 -5216 -5283 33 05:15 r1-00336475 35940-20909 2642 sci Out 300.1 -5260 -5216 -5283 34 05:15 b1-00336475 35940-20909 2642 sci Out 300.1 -5260 -5216 -5283 35 05:22 r1-00336476 35940-20909 2643 sci Out 300.1 -5260 -5216 -5283 36 05:22 b1-00336476 35940-20909 2643 sci Out 300.1 -5260 -5216 -5283 37 05:29 r1-00336477 35940-20909 2644 sci Out 300.1 -5260 -5216 -5283 38 05:29 b1-00336477 35940-20909 2644 sci Out 300.1 -5260 -5216 -5283 39 05:36 r1-00336478 35940-20909 2645 sci Out 300.1 -5260 -5216 -5283 40 05:36 b1-00336478 35940-20909 2645 sci Out 300.1 -5260 -5216 -5283 41 05:43 r1-00336479 35940-20909 2646 sci Out 300.1 -5260 -5216 -5283 42 05:43 b1-00336479 35940-20909 2646 sci Out 300.1 -5260 -5216 -5283 43 05:50 r1-00336480 35940-20909 2647 sci Out 300.1 -5260 -5216 -5283 44 05:50 b1-00336480 35940-20909 2647 sci Out 300.1 -5260 -5216 -5283 45 05:57 b1-00336481 35940-20909 2648 sci Out 300.1 -5260 -5216 -5283 46 05:57 r1-00336481 35940-20909 2648 sci Out 300.1 -5260 -5216 -5283 47 06:10 r1-00336482 12798-19092 2649 arc Lef 4.1 -5260 -5216 -5283 HART 48 06:10 b1-00336482 12798-19092 2649 arc Lef 4.1 -5260 -5216 -5283 HART 49 06:11 r1-00336483 12798-19092 2649 arc Rig 4.1 -5260 -5216 -5283 HART 50 06:11 b1-00336483 12798-19092 2649 arc Rig 4.1 -5260 -5216 -5283 HART 51 06:20 r1-00336484 12798-19092 2649 arc Out 4.1 -7177 -7131 -7204 52 06:20 b1-00336484 12798-19092 2649 arc Out 4.1 -7177 -7131 -7204 53 06:22 r1-00336485 12798-19092 2649 fla Out 25.1 -7177 -7131 -7204 54 06:22 b1-00336485 12798-19092 2649 fla Out 25.1 -7177 -7131 -7204 55 07:12 r1-00336486 12798-19092 2649 sci Out 900.1 -7177 -7131 -7204 56 07:12 b1-00336486 12798-19092 2649 sci Out 900.1 -7177 -7131 -7204 57 07:29 r1-00336487 12798-19092 2649 sci Out 900.1 -7177 -7131 -7204 58 07:29 b1-00336487 12798-19092 2649 sci Out 900.1 -7177 -7131 -7204 59 07:52 r1-00336488 12798-19092 2649 sci Out 900.1 -7177 -7131 -7204 60 07:52 b1-00336488 12798-19092 2649 sci Out 900.1 -7177 -7131 -7204 61 08:14 b1-00336489 12798-19092 2649 sci Out 900.1 -7177 -7131 -7204 62 08:14 r1-00336489 12798-19092 2649 sci Out 900.1 -7177 -7131 -7204 63 08:31 b1-00336490 12798-19092 2649 sci Out 900.1 -7177 -7131 -7204 64 08:31 r1-00336490 12798-19092 2649 sci Out 900.1 -7177 -7131 -7204 ---- Weather Log ---- Wind 1 um DIMM IRSC Time Temp C Dewp C MPH Dir Dust fwhm mean sigma 19:00 -0.1 -7.9 13.6 111.7 862 19:15 -0.4 -8.2 19.6 130.4 694 19:30 -0.1 -8.2 25.7 125.4 783 19:45 0.1 -7.6 9.0 117.2 725 20:00 -0.6 -7.7 11.5 83.3 757 20:15 0.2 -7.8 9.9 110.8 761 20:30 0.3 -8.1 18.4 130.2 690 20:45 1.2 -7.7 16.1 128.6 738 21:00 0.4 -7.7 7.1 105.9 831 21:15 0.4 -8.4 12.2 126.4 722 21:30 0.5 -8.4 2.5 177.5 777 21:45 1.2 -7.6 2.3 153.7 828 22:00 1.4 -8.4 2.5 200.8 846 22:15 1.2 -7.8 1.3 203.9 908 22:30 0.8 -8.1 3.8 109.7 945 22:45 0.7 -7.7 1.1 181.3 973 23:00 0.9 -8.3 0.7 205.0 971 23:15 -0.1 -8.1 4.6 101.7 1068 23:30 0.5 -8.6 1.5 221.4 1045 23:45 -0.2 -8.1 1.0 171.3 1106 00:00 -0.4 -7.8 1.5 138.9 1127 00:15 -1.0 -7.9 6.3 139.8 1114 00:30 -1.7 -8.1 8.7 128.5 1142 00:45 -1.7 -8.3 11.8 136.6 1231 01:00 -1.0 -8.4 15.8 145.0 1168 239 20.22 01:15 -1.3 -8.1 15.1 154.0 1221 169 4.48 01:30 -1.4 -7.6 12.9 156.3 1349 89 10.53 01:45 -1.7 -7.3 13.0 162.3 1425 202 13.10 02:00 -1.5 -7.3 13.7 158.2 1416 84 10.04 02:15 -1.3 -7.1 13.3 141.5 1443 84 11.71 02:30 -1.3 -6.9 15.6 146.0 1458 98 5.37 02:45 -0.9 -7.3 18.4 138.3 1510 174 7.79 03:00 -0.6 -7.5 19.6 139.7 1419 235 20.73 03:15 -0.7 -8.1 16.5 132.7 1184 141 1.53 03:30 -0.4 -8.5 16.5 135.6 962 136 1.56 03:45 -0.4 -8.6 14.6 145.5 925 127 4.58 04:00 -0.7 -8.6 10.9 165.8 912 89 6.81 04:15 -0.8 -8.6 12.3 146.5 879 136 2.86 04:30 -0.9 -8.6 11.4 169.8 770 277 23.62 04:45 -0.9 -8.6 11.2 175.8 782 84 10.20 05:00 -0.9 -8.4 12.4 172.3 704 183 8.55 05:15 -0.9 -8.4 12.1 174.8 754 108 5.87 05:30 -1.2 -8.4 14.3 154.5 773 226 14.91 05:45 -1.4 -8.2 12.8 162.6 786 84 9.25 06:00 -1.1 -8.3 10.5 164.2 764 103 11.90 06:15 -0.9 -8.4 10.7 146.9 756 89 9.62 06:30 -0.9 -8.2 18.3 143.5 791 179 7.63 06:45 -0.8 -7.9 16.6 138.6 842 179 6.54 07:00 -0.9 -7.9 15.8 135.1 882 84 11.65 07:15 -0.8 -7.9 15.1 126.1 783 131 3.55 07:30 0.2 -9.5 12.2 169.3 745 155 4.61 07:45 0.4 -10.7 6.4 184.6 601 94 7.35 08:00 0.0 -9.8 13.3 152.4 526 136 4.35 08:15 -0.3 -9.8 10.3 160.9 560 301 30.63 08:30 0.3 -9.9 10.8 157.4 535 169 6.26 08:45 0.5 -10.2 11.1 176.0 509 146 1.67 09:00 0.6 -10.2 10.6 169.1 469 178 9.22 09:15 0.9 -10.3 13.2 150.3 521 164 3.18 09:30 0.7 -10.2 8.4 150.3 575 245 20.89 09:45 0.4 -10.1 13.3 152.7 528 108 5.74 10:00 0.6 -9.9 10.2 162.8 480 197 13.28 10:15 0.7 -10.1 9.4 165.4 560 169 3.54 10:30 0.6 -9.8 8.5 162.7 589 193 10.94 10:45 0.5 -9.7 12.5 144.6 587 136 6.96 11:00 0.6 -9.5 11.7 141.8 571 136 1.67 11:15 0.6 -9.5 11.4 137.7 623 84 13.40 11:30 -0.2 -9.3 11.6 129.1 589 84 8.43 11:45 0.5 -8.6 13.7 140.8 679 254 21.40 12:00 0.3 -8.4 13.7 141.1 803 164 4.11 12:15 -0.2 -8.1 16.9 126.7 803 164 6.56 12:30 -0.6 -8.2 15.8 110.6 754 240 17.20