Subject: 2.5m obslog 20200320 (SJD 58929) From: kinemuchi@apo.nmsu.edu Date: Sat, 21 Mar 2020 12:56:00 +0000 (UTC) APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Friday March 20, 2020 (SJD 58929) ---- Observing Team ---- Christian Carter Karen Kinemuchi ---- Observing Plan ---- Fri 20 MAR 2020 58929.0 0.12 02:25 11:58 07:18 16:52 ---- SPIDERS until 6:04 (3.8 h), then MaNGA ( 5.8h) ---- Observing Summary ---- Partly cloudy skies at the beginning of the night. SPIDERS OBSERVING: ================== Cart 12 (Plate 12525) - Offset DEC=+5"; 2x900s --> not done Cart 16 (Plate 12545) - GSOGTF; 5x900s --> not done MANGA OBSERVING: =============== Cart 1 (Plate 12491) - GSOGTF; 2xNSE + APOGEE 6xAB --> not done Missing fibers: [211, 273] Faint fibers: ['1 - 30', 32, '34 - 35', '37 - 38', '40 - 42', '45 - 47', 49, '51 - 55', '57 - 90', '121 - 180', 227, 292] Cart 2 (Plate 12483) - GSOGTF; 2xNSE + APOGEE 6xAB --> DONE! Missing fibers: [38, 57, 59, 198, 211, 259, 273] Faint fibers: ['61 - 90', 137, 162, '169 - 173', 175, 177, '212 - 240'] Cart 6 (Plate 11970) - GSOGTF; 1xNSE+1xN + APOGEE 4xAB --> not done Missing fibers: [15, 181, 211, 273] Faint fibers: [1, '3 - 14', '16 - 24', '26 - 37', '39 - 55', '58 - 90', 105, 121, 123, 133, 148, '151 - 164', '166 - 173', '175 - 180', '212 - 240'] Notes to day staff and developers: =================================== * On Cart 1, DoS reported: sdR-r1-00310925.fit.gz: WARNING: Scattered light max = 65 electrons (Warm CCD or twi?) * On Cart 6, DoS reported: sdR-b2-00310954.fit.gz: WARNING: pixel bias rms for camera b2 is 6.75722x above nominal limit- is there an electronics problem?? sdR-r2-00310954.fit.gz: WARNING: pixel bias rms for camera r2 is 2.08674x above nominal limit- is there an electronics problem?? * We had trouble with the AZ axis out of closed loop events at the end of the night. See problems in detail of the description of the problem. * Dust ~5500 ---- Observing Log ---- ============================================================================ AFTERNOON CHECKOUT ============================================================================ Tested actors. Running MCP Version 1.12. SoS is alive. motorPosition sp1: 17696, 16877, 15268; sp2: 14475, 14266, 14078; Median spectrographs temperatures: Tsp1= 3.7, Tsp2= 4.2; Outside air temperature= 6.2; Filesystem Size Used Avail Use% Mounted on /dev/vda2 7.7G 6.1G 1.3G 83% / tmpfs 3.4G 0 3.4G 0% /dev/shm /dev/vda1 194M 128M 57M 70% /boot /dev/vdb 19G 8.8G 9.1G 49% /exports/hub-local /dev/vdd 688G 543G 145G 79% /home /dev/vdc1 11T 7.4T 3.5T 68% /data BOSS cals: 22:42:56 boss exposure 0.0, file=310893 22:44:16 boss exposure 20.0, file=310894 22:58:19 boss exposure 25.0, file=310895 23:01:08 boss exposure 4.0, file=310896 23:06:55 sp1.motor.move= -954, -956, -957 23:06:59 sp2.motor.move= 949, 947, 946 23:06:59 Hartmann collimate output on cart #17 sp1: offset: r= -1649 (Out of focus); b= 0.8 (In Focus) sp2: offset: r= 1578 (Out of focus); b= -1.0 (In Focus) sp1: pred. move: spAverageMove= -953 sp2: pred. move: spAverageMove= 946 sp1: pred. resid: r= -696, b= -2.2, txt= OK, spTemp = 3.8 sp2: pred. resid: r= 631, b= 2.0, txt= OK, spTemp = 4.5 APOGEE dither from morning cals: [observer@sdss4-hub 58929]$ list_ap -m 58928 -morning APOGEE data list, mjd=58928, morning raw_data: /data/apogee/utr_cdr/58928/apRaw-*.fits archive: /data/apogee/archive/58928/apR-[a,b,c]-*.apz quickred: /data/apogee/quickred/58928/ap2D-a-*.fits.fz -------------------------------------------------------------------------------- mjd i UT File/Exp Imtype Nread A/B Dth Ct-Plate Arch Offset %Flux Comm -------------------------------------------------------------------------------- 58928 1 12:29 33660059 Dark 60 A 13.0 6-11970 a-b-c - - 58928 2 12:40 33660060 Dark 60 A 13.0 17-12544 a-b-c - - 58928 3 12:51 33660061 Dark 60 A 13.0 17-12544 a-b-c - - 58928 4 13:01 33660062 QuarFlat 10 A 13.0 17-12544 a-b-c - 43 58928 5 13:03 33660063 QuarFlat 10 A 13.0 17-12544 a-b-c - 43 58928 6 13:05 33660064 QuarFlat 10 A 13.0 17-12544 a-b-c - 42 58928 7 13:07 33660065 Arc-Thar 12 A 13.0 17-12544 a-b-c -5.11 - 58928 8 13:09 33660066 Arc-Une 40 A 13.0 17-12544 a-b-c - - 58928 9 13:16 33660067 Arc-Thar 12 B 13.5 17-12544 a-b-c -4.61 - 58928 10 13:19 33660068 Arc-Une 40 B 13.5 17-12544 a-b-c - - 58928 11 13:26 33660069 Dark 30 A 13.0 17-12544 a-b-c - - 58928 12 13:31 33660070 InteFlat 30 A 13.0 17-12544 a-b-c - 0 58928 13 13:37 33660071 InteFlat 30 A 13.0 17-12544 a-b-c - 0 58928 14 13:42 33660072 InteFlat 30 A 13.0 17-12544 a-b-c - 0 58928 15 13:47 33660073 Dark 30 A 13.0 17-12544 a-b-c - - -------------------------------------------------------------------------------- Did not adjust focus rings. Visually inspected all APOGEE evening cals. STUI memory at podium = 8.3GB; restarted STUI. 23:21Z High altitude clouds on site with more on the way. 00:00Z Ventilation begins ============================================================================ NIGHT ============================================================================ 01:19:08 encl25m : open; Swapped cartridges as requested by the SPIDERS team: 01:42:11 loadCart: Cart 12, Plate 12525; 01:42:11 survey: eBOSS; lead: None HARTMANN: 01:45:37 boss exposure 4.0, file=310899 01:46:15 boss exposure 4.0, file=310900 01:47:38 sp1.motor.move= 3287, 3287, 3288 01:47:46 sp2.motor.move= 2521, 2522, 2521 01:47:46 Hartmann collimate output on cart #12 sp1: offset: r= 3298 (Out of focus); b= -10.3 (Out of focus) sp2: offset: r= 2524 (Out of focus); b= -7.9 (Out of focus) sp1: pred. move: spAverageMove= 3288 sp2: pred. move: spAverageMove= 2522 sp1: pred. resid: r= 10, b= 0.0, txt= OK, spTemp = 2.9 sp2: pred. resid: r= 1, b= 0.0, txt= OK, spTemp = 3.5 CALIBS: 01:48:05 boss exposure 4.0, file=310901 01:49:32 boss exposure 25.0, file=310902 No guide stars; applied offset DEC=+5"; mostly cloudy; seeing 1.4" 02:23:31 boss exposure 900.0, file=310903 02:39:51 boss exposure 900.0, file=310904 02:56:12 boss exposure 900.0, file=310905 -------------------------------------------------------------------------------- 02:22 12-12525A 1.000181 -1790 -689 14 35.6 50.9 3.5 18 198 1.5 boss 02:39 12-12525A 1.000175 -1729 -645 14 29.7 52.8 3.7 13 201 1.7 boss 02:55 12-12525A 1.000159 -1573 -602 -67 23.3 54.3 3.7 16 193 1.4 boss 04:39 16-12545A 1.000041 -404 312 -113 -5.3 56.5 4.3 11 189 1.5 boss 04:55 16-12545A 1.000050 -490 241 -113 -12.6 56.0 4.3 8 188 1.2 boss 05:12 16-12545A 1.000054 -528 222 -104 -19.6 55.0 3.8 9 188 1.2 boss 05:28 16-12545A 1.000052 -512 235 -107 -26.2 53.7 3.3 8 183 1.5 boss 05:44 16-12545A 1.000060 -592 202 -81 -32.4 52.0 4.3 13 182 1.3 boss 06:20 1-12491A 1.000074 -729 27 -122 -4.3 59.3 4.4 12 193 1.1 boss 06:36 1-12491A 1.000074 -729 34 -116 -12.3 58.8 4.1 7 195 1.1 boss 06:53 1-12491A 1.000084 -828 -39 -110 -19.9 57.8 3.2 5 186 1.6 boss 07:09 1-12491A 1.000084 -828 -24 -99 -27.0 56.4 3.1 7 182 1.4 boss 07:25 1-12491A 1.000104 -1028 -163 -79 -33.6 54.7 2.7 8 210 1.0 boss 07:42 1-12491A 1.000104 -1028 -157 -79 -39.6 52.6 2.7 9 189 1.5 boss 08:22 2-12483A 1.000064 -630 47 -148 262.7 79.5 4.6 11 197 1.0 boss 08:38 2-12483A 1.000069 -684 -12 -167 262.5 76.1 4.2 12 189 1.0 boss 08:54 2-12483A 1.000086 -847 -112 -138 262.0 72.7 4.3 11 185 1.0 boss 09:11 2-12483A 1.000098 -965 -178 -111 261.2 69.2 1.4 9 173 1.4 boss 09:27 2-12483A 1.000106 -1041 -220 -95 260.3 65.8 2.1 6 201 1.2 boss 09:43 2-12483A 1.000124 -1223 -338 -71 259.3 62.4 1.9 8 154 1.8 boss 10:23 6-11970A 1.000205 -2022 -963 -23 -9.4 67.3 2.5 14 182 1.6 boss 10:39 6-11970A 1.000144 -1423 -594 -152 -19.5 66.5 3.2 9 181 1.2 boss 10:55 6-11970A 1.000132 -1302 -545 -207 -28.7 65.1 1.0 6 167 1.7 boss 11:12 6-11970A 1.000142 -1401 -625 -210 -36.9 63.2 1.5 10 177 1.1 boss 11:32 6-11970A 1.000149 -1468 -680 -217 -45.5 60.4 3.9 14 184 1.3 boss 12:43 6-11970A 1.000151 -1487 -580 -207 121.0 30.0 2.7 12 183 1.3 boss ---- Telescope Status ---- 00:00Z Started ventilation 01:19Z encl25m : open 12:14Z encl25m : closed No interlocks bypassed Status at: 12:38:07 Telescope stowed at: 121.0, 30.0, -0.0 mount Instrument mounted: cartridge = 6 Counterweights at: 302, 352, 336, 305 LN2 autofill systems: Connected and turned on 180L LN2 dewar scales: SP1 153.8 lbs, 24.5 psi SP2 145.8 lbs, 21.3 psi ---- Software Used ---- -- STUI software: STUI : 1.7.8 alerts : 2.0.0 apo : trunk boss : v4_1_2 -boss.specMec : 3.0.4 gcamera : v2_2 guider : 3.8.1 hartmann : 1.7.1 hub : v4.2.4 mcp : 1.12 -mcp.plc : 34 -mcp.azFiduci : v1_141 -mcp.altFiduc : v1_141 -mcp.rotFiduc : v1_141 platedb : 2.8.0 sop : 3.12.7 tcc : 1.4.0 apogee : not availble apogeecal : not availble apogeeql : v1_6_1 APO-local : trunk -- Other versions ---- idlspec2d : v5_7_12 plate-mapper3 : NOCVS:v6_0_12 petunia : 1.5.0 autoscheduler : 1.5.1 sdss_python_module: trunk mangadrp : trunk exported ---- APOGEE Data Summary ---- APOGEE data list, mjd=58929, all raw_data: /data/apogee/utr_cdr/58929/apRaw-*.fits archive: /data/apogee/archive/58929/apR-[a,b,c]-*.apz quickred: /data/apogee/quickred/58929/ap2D-a-*.fits.fz -------------------------------------------------------------------------------- mjd i UT File/Exp Imtype Nread A/B Dth Ct-Plate Arch Offset %Flux Comm -------------------------------------------------------------------------------- 58929 1 22:34 33670001 Dark 3 A 13.0 17-12544 a-b-c - - 58929 2 22:39 33670002 Dark 60 A 13.0 17-12544 a-b-c - - 58929 3 22:50 33670003 QuarFlat 10 A 13.0 17-12544 a-b-c - 43 58929 4 22:51 33670004 Arc-Thar 12 A 13.0 17-12544 a-b-c -5.11 - 58929 5 22:54 33670005 Arc-Une 40 A 13.0 17-12544 a-b-c - - 58929 6 23:01 33670006 Arc-Thar 12 B 13.5 17-12544 a-b-c -4.63 - 58929 7 23:03 33670007 Arc-Une 40 B 13.5 17-12544 a-b-c - - 58929 8 23:11 33670008 Dark 60 A 13.0 17-12544 a-b-c - - 58929 9 01:24 33670009 Dark 10 A 13.0 17-12544 a-b-c - - 58929 10 01:25 33670010 Dark 10 A 13.0 17-12544 a-b-c - - 58929 11 06:07 33670011 Dark 10 A 13.0 16-12545 a-b-c - - 58929 12 06:08 33670012 Dark 10 A 13.0 16-12545 a-b-c - - 58929 13 06:20 33670013 Object 42 A 13.0 1-12491 a-b-c - - 58929 14 06:28 33670014 Object 42 B 13.5 1-12491 a-b-c - - 58929 15 06:36 33670015 Object 42 B 13.5 1-12491 a-b-c - - 58929 16 06:44 33670016 Object 42 A 13.0 1-12491 a-b-c - - 58929 17 06:53 33670017 Object 42 A 13.0 1-12491 a-b-c - - 58929 18 07:00 33670018 Object 42 B 13.5 1-12491 a-b-c - - 58929 19 07:09 33670019 Object 42 B 13.5 1-12491 a-b-c - - 58929 20 07:17 33670020 Object 42 A 13.0 1-12491 a-b-c - - 58929 21 07:26 33670021 Object 42 A 13.0 1-12491 a-b-c - - 58929 22 07:33 33670022 Object 42 B 13.5 1-12491 a-b-c - - 58929 23 07:42 33670023 Object 42 B 13.5 1-12491 a-b-c - - 58929 24 07:50 33670024 Object 42 A 13.0 1-12491 a-b-c - - 58929 25 08:04 33670025 DomeFlat 5 A 13.0 1-12491 a-b-c - 24 58929 26 08:06 33670026 Dark 10 A 13.0 1-12491 a-b-c - - 58929 27 08:07 33670027 Dark 10 A 13.0 1-12491 a-b-c - - 58929 28 08:22 33670028 Object 42 A 13.0 2-12483 a-b-c - - 58929 29 08:29 33670029 Object 42 B 13.5 2-12483 a-b-c - - 58929 30 08:38 33670030 Object 42 B 13.5 2-12483 a-b-c - - 58929 31 08:46 33670031 Object 42 A 13.0 2-12483 a-b-c - - 58929 32 08:55 33670032 Object 42 A 13.0 2-12483 a-b-c - - 58929 33 09:02 33670033 Object 42 B 13.5 2-12483 a-b-c - - 58929 34 09:11 33670034 Object 42 B 13.5 2-12483 a-b-c - - 58929 35 09:19 33670035 Object 42 A 13.0 2-12483 a-b-c - - 58929 36 09:27 33670036 Object 42 A 13.0 2-12483 a-b-c - - 58929 37 09:35 33670037 Object 42 B 13.5 2-12483 a-b-c - - 58929 38 09:44 33670038 Object 42 B 13.5 2-12483 a-b-c - - 58929 39 09:51 33670039 Object 42 A 13.0 2-12483 a-b-c - - 58929 40 10:06 33670040 DomeFlat 5 A 13.0 2-12483 a-b-c - 60 58929 41 10:07 33670041 Dark 10 A 13.0 2-12483 a-b-c - - 58929 42 10:09 33670042 Dark 10 A 13.0 2-12483 a-b-c - - 58929 43 10:23 33670043 Object 42 A 13.0 6-11970 a-b-c - - 58929 44 10:30 33670044 Object 42 B 13.5 6-11970 a-b-c - - 58929 45 10:39 33670045 Object 42 B 13.5 6-11970 a-b-c - - 58929 46 10:47 33670046 Object 42 A 13.0 6-11970 a-b-c - - 58929 47 10:56 33670047 Object 42 A 13.0 6-11970 a-b-c - - 58929 48 11:03 33670048 Object 42 B 13.5 6-11970 a-b-c - - 58929 49 11:12 33670049 Object 42 B 13.5 6-11970 a-b-c - - 58929 50 11:20 33670050 Object 42 A 13.0 6-11970 a-b-c - - 58929 51 11:32 33670051 Object 42 A 13.0 6-11970 a-b-c - - 58929 52 12:02 33670052 DomeFlat 5 A 13.0 6-11970 a-b-c - 33 58929 53 12:17 33670053 Dark 60 A 13.0 6-11970 a-b-c - - 58929 54 12:38 33670054 Dark 60 A 13.0 ------- a-x-x - - 58929 55 12:42 33670055 Dark 60 A 13.0 ------- x-x-x - - -------------------------------------------------------------------------------- ---- eBOSS Data Summary ---- Reading FITS from /data/spectro/58929/ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - n time filename cart plt flav hrt exp colA colB colC Hrt - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 1 22:42 b1-00310893 17 12544 bia Clo 0.0 17696 16877 15268 22:42 b2-00310893 17 12544 bia Clo 0.0 14475 14266 14078 22:42 r1-00310893 17 12544 bia Clo 0.0 17696 16877 15268 22:42 r2-00310893 17 12544 bia Clo 0.0 14475 14266 14078 2 22:44 b1-00310894 17 12544 dar Clo 20.0 17696 16877 15268 22:44 b2-00310894 17 12544 dar Clo 20.0 14475 14266 14078 22:44 r1-00310894 17 12544 dar Clo 20.0 17696 16877 15268 22:44 r2-00310894 17 12544 dar Clo 20.0 14475 14266 14078 3 22:58 b1-00310895 17 12544 fla Out 25.1 17696 16877 15268 22:58 b2-00310895 17 12544 fla Out 25.2 14475 14266 14078 22:58 r1-00310895 17 12544 fla Out 25.1 17696 16877 15268 22:58 r2-00310895 17 12544 fla Out 25.2 14475 14266 14078 4 23:01 b1-00310896 17 12544 arc Out 4.1 17696 16877 15268 23:01 b2-00310896 17 12544 arc Out 4.2 14475 14266 14078 23:01 r1-00310896 17 12544 arc Out 4.1 17696 16877 15268 23:01 r2-00310896 17 12544 arc Out 4.2 14475 14266 14078 5 23:03 b1-00310897 17 12544 arc Lef 4.1 17696 16877 15268 HART 23:03 b2-00310897 17 12544 arc Lef 4.2 14475 14266 14078 HART 23:03 r1-00310897 17 12544 arc Lef 4.1 17696 16877 15268 HART 23:03 r2-00310897 17 12544 arc Lef 4.2 14475 14266 14078 HART 6 23:04 b1-00310898 17 12544 arc Rig 4.1 17696 16877 15268 HART 23:04 b2-00310898 17 12544 arc Rig 4.2 14475 14266 14078 HART 23:04 r1-00310898 17 12544 arc Rig 4.1 17696 16877 15268 HART 23:04 r2-00310898 17 12544 arc Rig 4.2 14475 14266 14078 HART 7 01:45 b1-00310899 12 12525 arc Lef 4.1 16742 15921 14311 HART 01:45 b2-00310899 12 12525 arc Lef 4.2 15424 15213 15024 HART 01:45 r1-00310899 12 12525 arc Lef 4.1 16742 15921 14311 HART 01:45 r2-00310899 12 12525 arc Lef 4.2 15424 15213 15024 HART 8 01:46 b1-00310900 12 12525 arc Rig 4.1 16742 15921 14311 HART 01:46 b2-00310900 12 12525 arc Rig 4.1 15424 15213 15024 HART 01:46 r1-00310900 12 12525 arc Rig 4.1 16742 15921 14311 HART 01:46 r2-00310900 12 12525 arc Rig 4.1 15424 15213 15024 HART 9 01:48 b1-00310901 12 12525 arc Out 4.1 20029 19208 17599 01:48 b2-00310901 12 12525 arc Out 4.2 17945 17735 17545 01:48 r1-00310901 12 12525 arc Out 4.1 20029 19208 17599 01:48 r2-00310901 12 12525 arc Out 4.2 17945 17735 17545 10 01:49 b1-00310902 12 12525 fla Out 25.1 20029 19208 17599 01:49 b2-00310902 12 12525 fla Out 25.2 17945 17735 17545 01:49 r1-00310902 12 12525 fla Out 25.1 20029 19208 17599 01:49 r2-00310902 12 12525 fla Out 25.2 17945 17735 17545 11 02:23 b1-00310903 12 12525 sci Out 900.1 20029 19208 17599 02:23 b2-00310903 12 12525 sci Out 900.2 17945 17735 17545 02:23 r1-00310903 12 12525 sci Out 900.1 20029 19208 17599 02:23 r2-00310903 12 12525 sci Out 900.2 17945 17735 17545 12 02:39 b1-00310904 12 12525 sci Out 900.1 20029 19208 17599 02:39 b2-00310904 12 12525 sci Out 900.2 17945 17735 17545 02:39 r1-00310904 12 12525 sci Out 900.1 20029 19208 17599 02:39 r2-00310904 12 12525 sci Out 900.2 17945 17735 17545 13 02:56 b1-00310905 12 12525 sci Out 459.6 20029 19208 17599 02:56 b2-00310905 12 12525 sci Out 459.8 17945 17735 17545 02:56 r1-00310905 12 12525 sci Out 459.6 20029 19208 17599 02:56 r2-00310905 12 12525 sci Out 459.8 17945 17735 17545 14 04:31 b1-00310906 16 12545 arc Lef 4.1 20029 19208 17599 HART 04:31 b2-00310906 16 12545 arc Lef 4.2 17945 17735 17545 HART 04:31 r1-00310906 16 12545 arc Lef 4.1 20029 19208 17599 HART 04:31 r2-00310906 16 12545 arc Lef 4.2 17945 17735 17545 HART 15 04:31 b1-00310907 16 12545 arc Rig 4.1 20029 19208 17599 HART 04:31 b2-00310907 16 12545 arc Rig 4.2 17945 17735 17545 HART 04:31 r1-00310907 16 12545 arc Rig 4.1 20029 19208 17599 HART 04:31 r2-00310907 16 12545 arc Rig 4.2 17945 17735 17545 HART 16 04:33 b1-00310908 16 12545 arc Out 4.1 18451 17629 16019 04:33 b2-00310908 16 12545 arc Out 4.2 19048 18838 18648 04:33 r1-00310908 16 12545 arc Out 4.1 18451 17629 16019 04:33 r2-00310908 16 12545 arc Out 4.2 19048 18838 18648 17 04:34 b1-00310909 16 12545 fla Out 25.1 18451 17629 16019 04:34 b2-00310909 16 12545 fla Out 25.2 19048 18838 18648 04:34 r1-00310909 16 12545 fla Out 25.1 18451 17629 16019 04:34 r2-00310909 16 12545 fla Out 25.2 19048 18838 18648 18 04:39 b1-00310910 16 12545 sci Out 900.1 18451 17629 16019 04:39 b2-00310910 16 12545 sci Out 900.2 19048 18838 18648 04:39 r1-00310910 16 12545 sci Out 900.1 18451 17629 16019 04:39 r2-00310910 16 12545 sci Out 900.2 19048 18838 18648 19 04:56 b1-00310911 16 12545 sci Out 900.1 18451 17629 16019 04:56 b2-00310911 16 12545 sci Out 900.2 19048 18838 18648 04:56 r1-00310911 16 12545 sci Out 900.1 18451 17629 16019 04:56 r2-00310911 16 12545 sci Out 900.2 19048 18838 18648 20 05:12 b1-00310912 16 12545 sci Out 900.1 18451 17629 16019 05:12 b2-00310912 16 12545 sci Out 900.2 19048 18838 18648 05:12 r1-00310912 16 12545 sci Out 900.1 18451 17629 16019 05:12 r2-00310912 16 12545 sci Out 900.2 19048 18838 18648 21 05:28 b1-00310913 16 12545 sci Out 900.1 18451 17629 16019 05:28 b2-00310913 16 12545 sci Out 900.2 19048 18838 18648 05:28 r1-00310913 16 12545 sci Out 900.1 18451 17629 16019 05:28 r2-00310913 16 12545 sci Out 900.2 19048 18838 18648 22 05:45 b1-00310914 16 12545 sci Out 900.1 18451 17629 16019 05:45 b2-00310914 16 12545 sci Out 900.2 19048 18838 18648 05:45 r1-00310914 16 12545 sci Out 900.1 18451 17629 16019 05:45 r2-00310914 16 12545 sci Out 900.2 19048 18838 18648 23 06:02 b1-00310915 16 12545 fla Out 25.1 18451 17629 16019 06:02 b2-00310915 16 12545 fla Out 25.2 19048 18838 18648 06:02 r1-00310915 16 12545 fla Out 25.1 18451 17629 16019 06:02 r2-00310915 16 12545 fla Out 25.2 19048 18838 18648 24 06:05 b1-00310916 16 12545 arc Out 4.1 18451 17629 16019 06:05 b2-00310916 16 12545 arc Out 4.2 19048 18838 18648 06:05 r1-00310916 16 12545 arc Out 4.1 18451 17629 16019 06:05 r2-00310916 16 12545 arc Out 4.2 19048 18838 18648 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - --- OK --- ---- MaNGA Data Summary ---- Reading FITS from /data/spectro/58929/ 1 06:14 b1-00310917 1 12491 arc Lef C 4.1 18451 17629 16019 HART 06:14 b2-00310917 1 12491 arc Lef C 4.2 19048 18836 18648 HART 06:14 r1-00310917 1 12491 arc Lef C 4.1 18451 17629 16019 HART 06:14 r2-00310917 1 12491 arc Lef C 4.2 19048 18836 18648 HART 2 06:14 b1-00310918 1 12491 arc Rig C 4.1 18451 17629 16019 HART 06:14 b2-00310918 1 12491 arc Rig C 4.2 19048 18836 18648 HART 06:14 r1-00310918 1 12491 arc Rig C 4.1 18451 17629 16019 HART 06:14 r2-00310918 1 12491 arc Rig C 4.2 19048 18836 18648 HART 3 06:16 b1-00310919 1 12491 arc Out C 4.1 18002 17180 15565 06:16 b2-00310919 1 12491 arc Out C 4.2 14476 14264 14077 06:16 r1-00310919 1 12491 arc Out C 4.1 18002 17180 15565 06:16 r2-00310919 1 12491 arc Out C 4.2 14476 14264 14077 4 06:18 b1-00310920 1 12491 fla Out C 25.1 18002 17180 15565 06:18 b2-00310920 1 12491 fla Out C 25.2 14476 14264 14077 06:18 r1-00310920 1 12491 fla Out C 25.1 18002 17180 15565 06:18 r2-00310920 1 12491 fla Out C 25.2 14476 14264 14077 5 06:20 b1-00310921 1 12491 sci Out N 900.1 18002 17180 15565 06:20 b2-00310921 1 12491 sci Out N 900.2 14476 14264 14077 06:20 r1-00310921 1 12491 sci Out N 900.1 18002 17180 15565 06:20 r2-00310921 1 12491 sci Out N 900.2 14476 14264 14077 6 06:37 b1-00310922 1 12491 sci Out S 900.1 18002 17180 15565 06:37 b2-00310922 1 12491 sci Out S 900.2 14476 14264 14077 06:37 r1-00310922 1 12491 sci Out S 900.1 18002 17180 15565 06:37 r2-00310922 1 12491 sci Out S 900.2 14476 14264 14077 7 06:53 b1-00310923 1 12491 sci Out E 900.1 18002 17180 15565 06:53 b2-00310923 1 12491 sci Out E 900.2 14476 14264 14077 06:53 r1-00310923 1 12491 sci Out E 900.1 18002 17180 15565 06:53 r2-00310923 1 12491 sci Out E 900.2 14476 14264 14077 8 07:10 b1-00310924 1 12491 sci Out N 900.1 18002 17180 15565 07:10 b2-00310924 1 12491 sci Out N 900.2 14476 14264 14077 07:10 r1-00310924 1 12491 sci Out N 900.1 18002 17180 15565 07:10 r2-00310924 1 12491 sci Out N 900.2 14476 14264 14077 9 07:26 b1-00310925 1 12491 sci Out S 900.1 18002 17180 15565 07:26 b2-00310925 1 12491 sci Out S 900.2 14476 14264 14077 07:26 r1-00310925 1 12491 sci Out S 900.1 18002 17180 15565 07:26 r2-00310925 1 12491 sci Out S 900.2 14476 14264 14077 10 07:42 b1-00310926 1 12491 sci Out E 900.1 18002 17180 15565 07:42 b2-00310926 1 12491 sci Out E 900.2 14476 14264 14077 07:42 r1-00310926 1 12491 sci Out E 900.1 18002 17180 15565 07:42 r2-00310926 1 12491 sci Out E 900.2 14476 14264 14077 11 07:59 b1-00310927 1 12491 fla Out C 25.1 18002 17180 15565 07:59 b2-00310927 1 12491 fla Out C 25.1 14476 14264 14077 07:59 r1-00310927 1 12491 fla Out C 25.1 18002 17180 15565 07:59 r2-00310927 1 12491 fla Out C 25.1 14476 14264 14077 12 08:02 b1-00310928 1 12491 arc Out C 4.1 18002 17180 15565 08:02 b2-00310928 1 12491 arc Out C 4.2 14476 14264 14077 08:02 r1-00310928 1 12491 arc Out C 4.1 18002 17180 15565 08:02 r2-00310928 1 12491 arc Out C 4.2 14476 14264 14077 13 08:13 b1-00310929 2 12483 arc Lef C 4.1 18002 17180 15565 HART 08:13 b2-00310929 2 12483 arc Lef C 4.2 14476 14264 14077 HART 08:13 r1-00310929 2 12483 arc Lef C 4.1 18002 17180 15565 HART 08:13 r2-00310929 2 12483 arc Lef C 4.2 14476 14264 14077 HART 14 08:14 b1-00310930 2 12483 arc Rig C 4.1 18002 17180 15565 HART 08:14 b2-00310930 2 12483 arc Rig C 4.2 14476 14264 14077 HART 08:14 r1-00310930 2 12483 arc Rig C 4.1 18002 17180 15565 HART 08:14 r2-00310930 2 12483 arc Rig C 4.2 14476 14264 14077 HART 15 08:16 b1-00310931 2 12483 arc Out C 4.1 18524 17704 16088 08:16 b2-00310931 2 12483 arc Out C 4.2 17157 16945 16758 08:16 r1-00310931 2 12483 arc Out C 4.1 18524 17704 16088 08:16 r2-00310931 2 12483 arc Out C 4.2 17157 16945 16758 16 08:17 b1-00310932 2 12483 fla Out C 25.1 18524 17704 16088 08:17 b2-00310932 2 12483 fla Out C 25.2 17157 16945 16758 08:17 r1-00310932 2 12483 fla Out C 25.1 18524 17704 16088 08:17 r2-00310932 2 12483 fla Out C 25.2 17157 16945 16758 17 08:22 b1-00310933 2 12483 sci Out N 900.1 18524 17704 16088 08:22 b2-00310933 2 12483 sci Out N 900.2 17157 16945 16758 08:22 r1-00310933 2 12483 sci Out N 900.1 18524 17704 16088 08:22 r2-00310933 2 12483 sci Out N 900.2 17157 16945 16758 18 08:39 b1-00310934 2 12483 sci Out S 900.1 18524 17704 16088 08:39 b2-00310934 2 12483 sci Out S 900.2 17157 16945 16758 08:39 r1-00310934 2 12483 sci Out S 900.1 18524 17704 16088 08:39 r2-00310934 2 12483 sci Out S 900.2 17157 16945 16758 19 08:55 b1-00310935 2 12483 sci Out E 900.1 18524 17704 16088 08:55 b2-00310935 2 12483 sci Out E 900.2 17157 16945 16758 08:55 r1-00310935 2 12483 sci Out E 900.1 18524 17704 16088 08:55 r2-00310935 2 12483 sci Out E 900.2 17157 16945 16758 20 09:11 b1-00310936 2 12483 sci Out N 900.1 18524 17704 16088 09:11 b2-00310936 2 12483 sci Out N 900.2 17157 16945 16758 09:11 r1-00310936 2 12483 sci Out N 900.1 18524 17704 16088 09:11 r2-00310936 2 12483 sci Out N 900.2 17157 16945 16758 21 09:28 b1-00310937 2 12483 sci Out S 900.1 18524 17704 16088 09:28 b2-00310937 2 12483 sci Out S 900.2 17157 16945 16758 09:28 r1-00310937 2 12483 sci Out S 900.1 18524 17704 16088 09:28 r2-00310937 2 12483 sci Out S 900.2 17157 16945 16758 22 09:44 b1-00310938 2 12483 sci Out E 900.1 18524 17704 16088 09:44 b2-00310938 2 12483 sci Out E 900.2 17157 16945 16758 09:44 r1-00310938 2 12483 sci Out E 900.1 18524 17704 16088 09:44 r2-00310938 2 12483 sci Out E 900.2 17157 16945 16758 23 10:01 b1-00310939 2 12483 fla Out C 25.1 18524 17704 16088 10:01 b2-00310939 2 12483 fla Out C 25.2 17157 16945 16758 10:01 r1-00310939 2 12483 fla Out C 25.1 18524 17704 16088 10:01 r2-00310939 2 12483 fla Out C 25.2 17157 16945 16758 24 10:04 b1-00310940 2 12483 arc Out C 4.1 18524 17704 16088 10:04 b2-00310940 2 12483 arc Out C 4.2 17157 16945 16758 10:04 r1-00310940 2 12483 arc Out C 4.1 18524 17704 16088 10:04 r2-00310940 2 12483 arc Out C 4.2 17157 16945 16758 25 10:15 b1-00310941 6 11970 arc Lef C 4.1 18524 17704 16088 HART 10:15 b2-00310941 6 11970 arc Lef C 4.2 17157 16945 16758 HART 10:15 r1-00310941 6 11970 arc Lef C 4.1 18524 17704 16088 HART 10:15 r2-00310941 6 11970 arc Lef C 4.2 17157 16945 16758 HART 26 10:16 b1-00310942 6 11970 arc Rig C 4.1 18524 17704 16088 HART 10:16 b2-00310942 6 11970 arc Rig C 4.2 17157 16945 16758 HART 10:16 r1-00310942 6 11970 arc Rig C 4.1 18524 17704 16088 HART 10:16 r2-00310942 6 11970 arc Rig C 4.2 17157 16945 16758 HART 27 10:17 b1-00310943 6 11970 arc Out C 4.1 20348 19529 17914 10:17 b2-00310943 6 11970 arc Out C 4.2 17315 17102 16916 10:17 r1-00310943 6 11970 arc Out C 4.1 20348 19529 17914 10:17 r2-00310943 6 11970 arc Out C 4.2 17315 17102 16916 28 10:19 b1-00310944 6 11970 fla Out C 25.1 20348 19529 17914 10:19 b2-00310944 6 11970 fla Out C 25.2 17315 17102 16916 10:19 r1-00310944 6 11970 fla Out C 25.1 20348 19529 17914 10:19 r2-00310944 6 11970 fla Out C 25.2 17315 17102 16916 29 10:23 b1-00310945 6 11970 sci Out N 900.1 20348 19529 17914 10:23 b2-00310945 6 11970 sci Out N 900.2 17315 17102 16916 10:23 r1-00310945 6 11970 sci Out N 900.1 20348 19529 17914 10:23 r2-00310945 6 11970 sci Out N 900.2 17315 17102 16916 30 10:40 b1-00310946 6 11970 sci Out S 900.1 20348 19529 17914 10:40 b2-00310946 6 11970 sci Out S 900.2 17315 17102 16916 10:40 r1-00310946 6 11970 sci Out S 900.1 20348 19529 17914 10:40 r2-00310946 6 11970 sci Out S 900.2 17315 17102 16916 31 10:56 b1-00310947 6 11970 sci Out E 900.1 20348 19529 17914 10:56 b2-00310947 6 11970 sci Out E 900.2 17315 17102 16916 10:56 r1-00310947 6 11970 sci Out E 900.1 20348 19529 17914 10:56 r2-00310947 6 11970 sci Out E 900.2 17315 17102 16916 32 11:12 b1-00310948 6 11970 sci Out N 900.1 20348 19529 17914 11:12 b2-00310948 6 11970 sci Out N 900.2 17315 17102 16916 11:12 r1-00310948 6 11970 sci Out N 900.1 20348 19529 17914 11:12 r2-00310948 6 11970 sci Out N 900.2 17315 17102 16916 33 11:32 b1-00310949 6 11970 sci Out S 68.8 20348 19529 17914 11:32 b2-00310949 6 11970 sci Out S 68.9 17315 17102 16916 11:32 r1-00310949 6 11970 sci Out S 68.8 20348 19529 17914 11:32 r2-00310949 6 11970 sci Out S 68.9 17315 17102 16916 34 11:51 b1-00310950 6 11970 fla Out S 25.1 20348 19529 17914 11:51 b2-00310950 6 11970 fla Out S 25.2 17315 17102 16916 11:51 r1-00310950 6 11970 fla Out S 25.1 20348 19529 17914 11:51 r2-00310950 6 11970 fla Out S 25.2 17315 17102 16916 35 11:54 b1-00310951 6 11970 arc Out S 4.1 20348 19529 17914 11:54 b2-00310951 6 11970 arc Out S 4.2 17315 17102 16916 11:54 r1-00310951 6 11970 arc Out S 4.1 20348 19529 17914 11:54 r2-00310951 6 11970 arc Out S 4.2 17315 17102 16916 36 12:17 b1-00310952 6 11970 bia Clo S 0.0 20348 19529 17914 12:17 b2-00310952 6 11970 bia Clo S 0.0 17315 17100 16916 12:17 r1-00310952 6 11970 bia Clo S 0.0 20348 19529 17914 12:17 r2-00310952 6 11970 bia Clo S 0.0 17315 17100 16916 37 12:19 b1-00310953 6 11970 bia Clo S 0.0 20348 19529 17914 12:19 b2-00310953 6 11970 bia Clo S 0.0 17315 17100 16916 12:19 r1-00310953 6 11970 bia Clo S 0.0 20348 19529 17914 12:19 r2-00310953 6 11970 bia Clo S 0.0 17315 17100 16916 38 12:20 b1-00310954 6 11970 bia Clo S 0.0 20348 19529 17914 12:20 b2-00310954 6 11970 bia Clo S 0.0 17315 17100 16916 12:20 r1-00310954 6 11970 bia Clo S 0.0 20348 19529 17914 12:20 r2-00310954 6 11970 bia Clo S 0.0 17315 17100 16916 39 12:21 b1-00310955 6 11970 bia Clo S 0.0 20348 19529 17914 12:21 b2-00310955 6 11970 bia Clo S 0.0 17315 17100 16916 12:21 r1-00310955 6 11970 bia Clo S 0.0 20348 19529 17914 12:21 r2-00310955 6 11970 bia Clo S 0.0 17315 17100 16916 40 12:23 b1-00310956 6 11970 bia Clo S 0.0 20348 19529 17914 12:23 b2-00310956 6 11970 bia Clo S 0.0 17315 17100 16916 12:23 r1-00310956 6 11970 bia Clo S 0.0 20348 19529 17914 12:23 r2-00310956 6 11970 bia Clo S 0.0 17315 17100 16916 41 12:24 b1-00310957 6 11970 bia Clo S 0.0 20348 19529 17914 12:24 b2-00310957 6 11970 bia Clo S 0.0 17315 17100 16916 12:24 r1-00310957 6 11970 bia Clo S 0.0 20348 19529 17914 12:24 r2-00310957 6 11970 bia Clo S 0.0 17315 17100 16916 42 12:25 b1-00310958 6 11970 bia Clo S 0.0 20348 19529 17914 12:25 b2-00310958 6 11970 bia Clo S 0.0 17315 17100 16916 12:25 r1-00310958 6 11970 bia Clo S 0.0 20348 19529 17914 12:25 r2-00310958 6 11970 bia Clo S 0.0 17315 17100 16916 43 12:27 b1-00310959 6 11970 bia Clo S 0.0 20348 19529 17914 12:27 b2-00310959 6 11970 bia Clo S 0.0 17315 17100 16916 12:27 r1-00310959 6 11970 bia Clo S 0.0 20348 19529 17914 12:27 r2-00310959 6 11970 bia Clo S 0.0 17315 17100 16916 44 12:28 b1-00310960 6 11970 bia Clo S 0.0 20348 19529 17914 12:28 b2-00310960 6 11970 bia Clo S 0.0 17315 17100 16916 12:28 r1-00310960 6 11970 bia Clo S 0.0 20348 19529 17914 12:28 r2-00310960 6 11970 bia Clo S 0.0 17315 17100 16916 45 12:29 b1-00310961 6 11970 bia Clo S 0.0 20348 19529 17914 12:29 b2-00310961 6 11970 bia Clo S 0.0 17315 17100 16916 12:29 r1-00310961 6 11970 bia Clo S 0.0 20348 19529 17914 12:29 r2-00310961 6 11970 bia Clo S 0.0 17315 17100 16916 46 12:31 b1-00310962 6 11970 bia Clo S 0.0 20348 19529 17914 12:31 b2-00310962 6 11970 bia Clo S 0.0 17315 17100 16916 12:31 r1-00310962 6 11970 bia Clo S 0.0 20348 19529 17914 12:31 r2-00310962 6 11970 bia Clo S 0.0 17315 17100 16916 47 12:32 b1-00310963 6 11970 bia Clo S 0.0 20348 19529 17914 12:32 b2-00310963 6 11970 bia Clo S 0.0 17315 17100 16916 12:32 r1-00310963 6 11970 bia Clo S 0.0 20348 19529 17914 12:32 r2-00310963 6 11970 bia Clo S 0.0 17315 17100 16916 48 12:33 b1-00310964 6 11970 bia Clo S 0.0 20348 19529 17914 12:33 b2-00310964 6 11970 bia Clo S 0.0 17315 17100 16916 12:33 r1-00310964 6 11970 bia Clo S 0.0 20348 19529 17914 12:33 r2-00310964 6 11970 bia Clo S 0.0 17315 17100 16916 49 12:35 b1-00310965 6 11970 bia Clo S 0.0 20348 19529 17914 12:35 b2-00310965 6 11970 bia Clo S 0.0 17315 17100 16916 12:35 r1-00310965 6 11970 bia Clo S 0.0 20348 19529 17914 12:35 r2-00310965 6 11970 bia Clo S 0.0 17315 17100 16916 50 12:36 b1-00310966 6 11970 bia Clo S 0.0 20348 19529 17914 12:36 b2-00310966 6 11970 bia Clo S 0.0 17315 17100 16916 12:36 r1-00310966 6 11970 bia Clo S 0.0 20348 19529 17914 12:36 r2-00310966 6 11970 bia Clo S 0.0 17315 17100 16916 51 12:37 b1-00310967 6 11970 bia Clo S 0.0 20348 19529 17914 12:37 b2-00310967 6 11970 bia Clo S 0.0 17315 17100 16916 12:37 r1-00310967 6 11970 bia Clo S 0.0 20348 19529 17914 12:37 r2-00310967 6 11970 bia Clo S 0.0 17315 17100 16916 52 12:39 b1-00310968 6 11970 bia Clo S 0.0 20348 19529 17914 12:39 b2-00310968 6 11970 bia Clo S 0.0 17315 17100 16916 12:39 r1-00310968 6 11970 bia Clo S 0.0 20348 19529 17914 12:39 r2-00310968 6 11970 bia Clo S 0.0 17315 17100 16916 53 12:40 b1-00310969 6 11970 bia Clo S 0.0 20348 19529 17914 12:40 b2-00310969 6 11970 bia Clo S 0.0 17315 17100 16916 12:40 r1-00310969 6 11970 bia Clo S 0.0 20348 19529 17914 12:40 r2-00310969 6 11970 bia Clo S 0.0 17315 17100 16916 54 12:41 b1-00310970 6 11970 bia Clo S 0.0 20348 19529 17914 12:41 b2-00310970 6 11970 bia Clo S 0.0 17315 17100 16916 12:41 r1-00310970 6 11970 bia Clo S 0.0 20348 19529 17914 12:41 r2-00310970 6 11970 bia Clo S 0.0 17315 17100 16916 ---- Weather Log ---- --- Weather --- Time Inst Temp DP Dif Humid Wind Dir Dust,1um IRSC IRSCm -------------------------------------------------------------------------------- 02:22 12-12525A 3.5 -3.3 6.8 60 18 198 676 37.1 333 1.5 boss 02:39 12-12525A 3.7 -3.6 7.3 58 13 201 680 26.7 335 1.7 boss 02:55 12-12525A 3.7 -4.2 7.9 56 16 193 608 38.7 337 1.4 boss 04:39 16-12545A 4.3 -4.7 9.0 51 11 189 452 39.9 338 1.5 boss 04:55 16-12545A 4.3 -4.9 9.2 51 8 188 437 19.5 338 1.2 boss 05:12 16-12545A 3.8 -4.2 8.0 55 9 188 432 30.6 341 1.2 boss 05:28 16-12545A 3.3 -3.8 7.1 59 8 183 440 50.4 338 1.5 boss 05:44 16-12545A 4.3 -4.4 8.7 53 13 182 496 39.4 334 1.3 boss 06:20 1-12491A 4.4 -5.1 9.5 50 12 193 409 52.0 336 1.1 boss 06:36 1-12491A 4.1 -4.7 8.8 52 7 195 480 68.2 337 1.1 boss 06:53 1-12491A 3.2 -3.7 6.9 60 5 186 557 50.7 345 1.6 boss 07:09 1-12491A 3.1 -4.2 7.3 58 7 182 475 78.6 351 1.4 boss 07:25 1-12491A 2.7 -3.8 6.5 62 8 210 511 84.8 371 1.0 boss 07:42 1-12491A 2.7 -3.8 6.5 62 9 189 441 47.6 337 1.5 boss 08:22 2-12483A 4.6 -5.4 10.0 48 11 197 302 24.6 334 1.0 boss 08:38 2-12483A 4.2 -4.0 8.2 55 12 189 556 58.7 324 1.0 boss 08:54 2-12483A 4.3 -4.3 8.6 53 11 185 536 34.7 334 1.0 boss 09:11 2-12483A 1.4 -3.8 5.2 68 9 173 513 27.3 334 1.4 boss 09:27 2-12483A 2.1 -3.6 5.7 66 6 201 646 71.0 351 1.2 boss 09:43 2-12483A 1.9 -3.3 5.2 68 8 154 628 33.1 337 1.8 boss 10:23 6-11970A 2.5 -4.2 6.7 61 14 182 648 30.1 324 1.6 boss 10:39 6-11970A 3.2 -4.3 7.5 57 9 181 491 36.1 339 1.2 boss 10:55 6-11970A 1.0 -3.3 4.3 72 6 167 615 32.2 338 1.7 boss 11:12 6-11970A 1.5 -3.3 4.8 70 10 177 581 58.8 336 1.1 boss 11:32 6-11970A 3.9 -5.0 8.9 52 14 184 441 42.8 341 1.3 boss 12:43 6-11970A 2.7 -3.0 5.7 65 12 183 806 5.6 553 1.3 boss ---- Problems in Detail ---- Az axis went out of closed loop during the North dither. We stopped the MaNGA sequence. We issued a "tcc axis init" and then "tcc track" to track again on field. Resumed the MaNGA exposures, but the Az went out of closed loop again. We stopped the exposure sequence again and decided to go to the telescope to see if there was anything really wrong. The telescope was pointed due west and Az was at -44 degrees. We checked the lower enclosure and walked around the Az drive to make sure nothing was obstructing drive. We did not see anything out of the ordinary. At the telescope, we did test slews to gang change position and stow (took the post cals and dome flat as well), and could not get the Az to go out of closed loop. After we finished the dome flat, we sent the telescope to stow again, but Az went out of closed loop again at 12:03Z an at Az=-52 deg. We checked the interlocks, and az_mtr_cw_perm, az_mtr_ccw_perm, wind_mtr_cw_perm, wind_mtr_ccw_perm were red. We tried tcc axis init and tcc track 121,30 mount commands to return to stow. The Az went out of closed loop again when it was at -6.9 deg. Did another axis init and tcc track 121,45 mount command, and then the rot went out of closed loop (it was already at rot=0 from an earlier test slew). Telescope arrived at 121,45,0 postion. At this point, it was past 15 degree morning twilight, so we decided to close. 12:48Z We checked the tron logs for mcp, and there does appear to be a bunch of Az reposition frame break errors. We documented the first instance of this problem at 11:28Z. From /data/logs/tron/mcp/current.log file: 2020-03-21 11:25:14,911Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:25:19,910Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:25:24,910Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:25:29,909Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:25:34,909Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:25:39,909Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:25:44,909Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:25:49,909Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:25:54,934Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:25:59,954Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:26:04,909Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:26:09,988Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:26:14,977Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:26:20,034Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:26:25,009Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:26:30,024Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:26:34,908Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:26:40,002Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:26:45,005Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:26:49,920Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:26:55,007Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:26:59,908Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:27:04,907Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:27:09,907Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:27:14,907Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:27:19,907Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:27:24,913Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:27:29,913Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:27:34,907Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:27:39,994Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:27:44,913Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:27:49,913Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:27:54,913Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:27:59,913Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:28:04,912Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:28:09,912Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:28:14,921Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:28:19,995Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:28:24,912Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:28:29,912Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:28:34,912Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:28:39,912Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:28:44,912Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:28:49,987Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:28:54,911Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:28:59,911Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:29:04,945Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:29:09,911Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:29:14,911Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:29:19,917Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:29:24,911Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:29:29,992Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:29:34,967Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:29:39,948Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:29:44,911Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:29:49,982Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:29:54,910Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:29:59,910Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:33:01,675Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:33:06,644Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:33:11,644Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:33:16,644Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:33:21,644Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:33:26,644Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:33:31,671Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:33:36,643Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:33:41,706Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:33:46,650Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:33:51,649Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:33:56,693Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:34:01,674Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:34:06,642Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:34:11,642Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:34:16,649Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:34:21,649Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:34:26,648Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:34:31,648Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:34:36,648Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:34:41,686Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:34:46,648Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:34:51,648Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:34:56,648Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:35:01,648Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:35:06,648Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:35:11,647Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:35:16,648Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:35:21,704Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:35:26,647Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:35:31,647Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:35:36,739Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:35:41,713Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:37:31,807Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 11:37:36,801Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 12:04:58,517Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 12:05:03,517Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 12:07:46,826Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 12:07:51,831Z < 0 0 i trace="azimuth reposition caused frame break" 2020-03-21 12:07:56,826Z < 0 0 i trace="azimuth reposition caused frame break"