Subject: 2.5m obslog 20191009 (SJD 58766) From: kpan@apo.nmsu.edu Date: Thu, 10 Oct 2019 13:16:50 +0000 (UTC) APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Wednesday October 09, 2019 (SJD 58766) ---- Observing Team ---- Katie Grabowski Kaike Pan Jeff Hagen (remote support) JohnD (phone support) ---- Observing Plan ---- Wed 9 OCT 2019 58766.0 0.85 01:31 11:57 19:42 06:08 ---- APOGEE-2 until 10:08 ( 8.6 h), then MaNGA ( 1.8 h) ---- Observing Summary ---- It was windy (30-35 mph) in the first 4 hours, then settled down to ~25 mph. clear sky with good to moderate seeing. We observed 6 APOGEE plates, completed 4 of them. APOGEE and MaNGA lost 1.4 and 1.8 hours due to the MCP problem. APOGEE-led: ========== cart 5, pl 12162, field: CVZTILE_098+30_btx. A-led. Completed. 24X83s for MaNGA. 4XAB/BA for APOGEE. Missing fibers: [211, 273] Faint fibers: ['121 - 123', '126 - 129', 133, 145, '147 - 148', 154] cart 9, pl 11421, field: N6819-RV_btx. only Apogee fibers. not done. 4XAB/BA. Missing fibers: [36, 211, 245, 273] Faint fibers: [123, '126 - 127', 133, 253] cart 8, pl 11480, field:K2_C12_077-54_btx. A-led, only Apogee fibers.Completed. 2XDAB/BA. Missing fibers: [35, 184, 211, 227, 273] Faint fibers: [123, '127 - 128', 148] cart 7, pl 11466, field:NGC7789_btx. Only Apogee fibers. Completed. 5XAB/BA. Missing fibers: [211, 273, 283] Faint fibers: [123, '126 - 127', 133, 136, 202] cart 1, pl 12132, field: K2_C4_168-28_btx. A-led. Completed. 27X 28 s for MaNGA. 3XAB/BA for Apogee. Missing fibers: [211, 273] Faint fibers: [121, 123, '126 - 127', 129, 133, 136, 148, 227] Notes from developers and engineers: ==================================== 1. MCP dead twice at 08:01 (versin 3.1) and 09:48Z (version 1.2). We left MCP 1.1 running at the end of the night. (see problem details). 2. We used "goto GangChange" on SOP GUI for the first 5 cartridges under MCP 3.1, i.e. our normal procedure, "goto GangChange" takes a dome flat, then slews to the gang change position. We didn't encounter any problem. 3. We notice two guiding glitches, at 1:47Z and 7:31 UT. The later corresponds to guiding frame 1041 and 1042. The dRA was ~0.1" on frame 1040, and dramatically increased to 3" on 1041, and returned back to normal at 1045. 4. MCP version 1.2 output mcp info in STUI much more frequently then the version 1.1. Version 1.2 outputs these messages in 3 seconds or so. ---- Observing Log ---- Afternoon: =========== All systems alive and responsive. 22:26Z- Started APOGEE and BOSS calibrations 22:30:52 boss exposure 4.0, file=296492, C 22:32:03 boss exposure 4.0, file=296493, C 22:33:56 sp1.motor.move= 513, 509, 511 22:33:59 sp2.motor.move= 632, 629, 630 22:34:00 Hartmann collimate output on cart #4 sp1: offset: r= -329 (In Focus); b= -4.2 (Out of focus) sp2: offset: r= 946 (In Focus); b= -1.0 (In Focus) sp1: pred. move: spAverageMove= 510 sp2: pred. move: spAverageMove= 630 sp1: pred. resid: r= -839, b= -2.6, txt= OK, spTemp = 8.6 sp2: pred. resid: r= 315, b= 1.0, txt= OK, spTemp = 12.1 Did not adjust blue rings. 22:34:40 boss exposure 25.0, file=296494, C 22:37:29 boss exposure 4.0, file=296495, C Restarted podium laptop. Mounted Cart.5. Delayed ventilation for 30mph wind gusts. Checked dust with handheld monitor because other instruments were unavailable. At 00:05Z, the 0.1 micron dust counts integrated over a minute and 0.1 cubic feet was about 700. Ventilation started 00:10Z Night: ======= 00:42Z enclosure open. 01:00Z: cart 5, pl 12162, field: CVZTILE_098+30_btx. A-led. Completed. 01:05:24 Hartmann collimate output on cart #5 sp1: offset: r= 3958 (Out of focus); b= -17.3 (Out of focus) sp2: offset: r= 2840 (Out of focus); b= -3.9 (Out of focus) sp1: pred. move: spAverageMove= 4742 sp2: pred. move: spAverageMove= 2050 sp1: pred. resid: r= -784, b= -2.5, txt= OK, spTemp = 6.9 sp2: pred. resid: r= 790, b= 2.5, txt= OK, spTemp = 9.5 01:14Z: started 4XAB/BA for APOGEE. 24X83 for MaNGA. Windy (30-35 mph), clear, guider seeing at 1.5". MaNGA: 1st exp: 01:14:16 boss exposure 83.0, file=296500, C last exp: 02:18:16 boss exposure 83.0, file=296523, C APOGEE: 1st exp: 01:14:10 apogee.expose=apRaw-32040011, 47, A, Object last exp: 02:12:53 apogee.expose=apRaw-32040018, 47, A, Object 02:21:44 apogee.expose=apRaw-32040019, 5, A, DomeFlat 02:24:27 apogee.expose=apRaw-32040020, 10, A, Dark 02:26:15 apogee.expose=apRaw-32040021, 10, A, Dark Missing fibers: [211, 273] Faint fibers: ['121 - 123', '126 - 129', 133, 145, '147 - 148', 154] 02:26Z: cart 9, pl 11421, field: N6819-RV_btx. only Apogee fibers. 02:34Z: started 4XAB/BA. windy, clear, seeing at 1.2". 02:34:00 apogee.expose=apRaw-32040022, 47, A, Object 02:42:23 apogee.expose=apRaw-32040023, 47, B, Object 02:50:43 apogee.expose=apRaw-32040024, 47, B, Object 02:59:11 apogee.expose=apRaw-32040025, 47, A, Object 03:07:31 apogee.expose=apRaw-32040026, 47, A, Object 03:15:54 apogee.expose=apRaw-32040027, 47, B, Object 03:24:14 apogee.expose=apRaw-32040028, 47, B, Object 03:32:42 apogee.expose=apRaw-32040029, 47, A, Object 03:41:13 apogee.expose=apRaw-32040030, 5, A, DomeFlat 03:44:04 apogee.expose=apRaw-32040031, 10, A, Dark 03:45:52 apogee.expose=apRaw-32040032, 10, A, Dark Missing fibers: [36, 211, 245, 273] Faint fibers: [123, '126 - 127', 133, 253] 03:46Z: cart 8, pl 11480, field: K2_C12_077-54_btx. Completed. 03:53Z: started 2XDAB/BA. Clear, seeing at 1.4". 03:53:43 apogee.expose=apRaw-32040033, 94, A, Object 04:10:24 apogee.expose=apRaw-32040034, 94, B, Object 04:27:03 apogee.expose=apRaw-32040035, 94, B, Object 04:43:49 apogee.expose=apRaw-32040036, 94, A, Object 05:01:00 apogee.expose=apRaw-32040037, 5, A, DomeFlat 05:03:50 apogee.expose=apRaw-32040038, 10, A, Dark 05:05:37 apogee.expose=apRaw-32040039, 10, A, Dark Missing fibers: [35, 184, 211, 227, 273] Faint fibers: [123, '127 - 128', 148] 05:06Z: cart 7, pl 11466, field:NGC7789_btx. Only Apogee fibers. Completed. 5XAB/BA. clear, seeing at 1.3". Because the next plate is 1 hour away, We added one pair on this plate. 1st exp: 05:11:54 apogee.expose=apRaw-32040040, 47, A, Object last exp: 06:27:20 apogee.expose=apRaw-32040049, 47, B, Object 06:35:53 apogee.expose=apRaw-32040050, 5, B, DomeFlat 06:38:29 apogee.expose=apRaw-32040051, 10, B, Dark 06:40:17 apogee.expose=apRaw-32040052, 10, B, Dark Missing fibers: [211, 273, 283] Faint fibers: [123, '126 - 127', 133, 136, 202] 06:40Z: cart 1, pl 12132, field: K2_C4_168-28_btx. A-led. Completed. 06:47:03 Hartmann collimate output on cart #1 sp1: offset: r= -5937 (Out of focus); b= 22.0 (Out of focus) sp2: offset: r= -6312 (Out of focus); b= 17.8 (Out of focus) sp1: pred. move: spAverageMove= -6471 sp2: pred. move: spAverageMove= -5993 sp1: pred. resid: r= 534, b= 1.7, txt= OK, spTemp = 6.3 sp2: pred. resid: r= -319, b= -1.0, txt= OK, spTemp = 9.2 06:45Z: started 3XAB/BA. Clear, seeing at 1.7". MaNGA: 27 X 28 s 1st exp: 06:54:42 boss exposure 28.0, file=296528, C last exp: 07:42:46 boss exposure 28.0, file=296554, C APOEE: 3XAB/BA 1st exp: 06:54:36 apogee.expose=apRaw-32040053, 47, B, Object last exp:07:36:36 apogee.expose=apRaw-32040058, 47, A, Object 07:45:18 apogee.expose=apRaw-32040059, 5, A, DomeFlat 07:47:31 apogee.expose=apRaw-32040060, 10, A, Dark 07:49:19 apogee.expose=apRaw-32040061, 10, A, Dark Missing fibers: [211, 273] Faint fibers: [121, 123, '126 - 127', 129, 133, 136, 148, 227] 07:51Z: cart 6, pl 12131. Field: K2_C4_168-21_btx. A-led. not done. 07:58:17 Hartmann collimate output on cart #6 sp1: offset: r= 1979 (Out of focus); b= -3.2 (In Focus) sp2: offset: r= 2524 (Out of focus); b= -10.9 (Out of focus) sp1: pred. move: spAverageMove= 1503 sp2: pred. move: spAverageMove= 2995 sp1: pred. resid: r= 475, b= 1.5, txt= OK, spTemp = 6.2 sp2: pred. resid: r= -471, b= -1.5, txt= OK, spTemp = 8.9 08:00:33 boss exposure 4.0, file=296557, C gotoField failed after the arc exposure with an error message: 08:01:42 OBSERVER.kgrabow 155 sop f text="Failed to prepare for flats" 08:01:42 CmdDone: OBSERVER.kgrabow 155 sop gotoField arcTime=4.0 flatTime=25.0 guiderFlatTime=0.5 guiderTime=5.0 Then we notice MCP dead. "MCP is unavailable" on MCP menu. We tried to restart MCP using Jeff's instruction but it failed (see problem section), then we called Conor, no luck. We then called Jeff. He restarted MCP with the previous version 1.2. 09:01Z: we re-gotoField with MCP-1.2. I accepted fiducials on both AZ and Alt, but no guiding stars. Slow away from the field, slow back several times, no luck. Then we accepted fiduical error of 1500 ticket (~20 arcseond, re-slow to the field and found stars this time. We don't think ~20" offset on rotator should make this different. 09:01:55 Hartmann collimate output on cart #6 sp1: offset: r= 989 (In Focus); b= -0.2 (In Focus) sp2: offset: r= 0 (In Focus); b= -1.0 (In Focus) sp1: pred. move: spAverageMove= 526 sp2: pred. move: spAverageMove= 157 sp1: pred. resid: r= 462, b= 1.4, txt= OK, spTemp = 6.0 sp2: pred. resid: r= -157, b= -0.5, txt= OK, spTemp = 8.5 09:24Z: started 2XAB/BA but MCP dead again on the third exposure, but get one good pair.Clear, seeing at 1.5". 14 X 28 s for MaNGA. APOGEE: 1XAB + B. 09:24:49 apogee.expose=apRaw-32040062, 47, A, Object 09:33:13 apogee.expose=apRaw-32040063, 47, B, Object 09:41:33 apogee.expose=apRaw-32040064, 47, B, Object 10:55:12 apogee.expose=apRaw-32040065, 5, B, DomeFlat 11:28:52 apogee.expose=apRaw-32040066, 10, B, Dark 11:30:40 apogee.expose=apRaw-32040067, 10, B, Dark Missing fibers: [181, 211, 273] Faint fibers: [15, 34, 105, 121, 123, 127] 09:48Z: MCP dead again. (see problem section for more details). 11:40Z: close. 12:50: Started morning calib. ---- BOSS Spectrographs Collimation Summary ---- --- Hartmann --- Time Inst r1 b1 move1 b1pred Tsp1 r2 b2 move2 b2pred Tsp2 -------------- ---------------------------- ---------------------------- 22:33 4-10224A -329 -4.2 510 -2.6 8.6 946 -1.0 630 1.0 12.1 Time Inst r1 b1 move1 b1pred Tsp1 r2 b2 move2 b2pred Tsp2 -------------- ---------------------------- ---------------------------- 01:04 5-12162A 3958 -17.3 4742 -2.5 6.9 2840 -3.9 2050 2.5 9.5 06:46 1-12132A -5937 22.0 -6471 1.7 6.3 -6312 17.8 -5993 -1.0 9.2 07:57 6-12131A 1979 -3.2 1503 1.5 6.2 2524 -10.9 2995 -1.5 8.9 09:01 6-12131A 989 -0.2 526 1.4 6.0 0 -1.0 157 -0.5 8.5 ---- Mirror Numbers ---- PRIMARY: -------- Scale: 1.000000; MIGS TONIGHT NOMINAL Axial A: -5.331 -5.329 Axial B: -5.053 -5.05 Axial C: -5.271 -5.268 Trans D: -0.578 -0.577 Lateral E: 9.3472 9.3472 Lateral F: 14.4399 14.4526 GALILS Commanded: 11200 -1850 6850 29650 21250 21050 Actual: 11200 -1850 6850 29650 21250 21050 SETMIR VALUES PrimDesOrient: 0.0 -11.37 6.87 375.47 621.12 PrimOrient: 0.0 -11.4 6.58 375.38 620.77 SECONDARY: ---------- Focus: 0.0000 Air temp: 12.3 Altitude: 30.0 MIGS TONIGHT NOMINAL Axial A: -6.665 -6.664 Axial B: -0.031 -0.028 Axial C: -6.668 -6.669 Trans D: -0.074 -0.076 GALILS Commanded: 1094500 1134950 1118750 -7200 -6550 Actual: 1058737 1101420 1079701 -7200 -6550 SETMIR VALUES SecDesOrient: 857.0 2.7 12.75 -38.35 156.51 SecOrient: 857.07 2.66 12.77 -37.84 156.17 ---- Telescope Offsets and Scale ---- Time Inst Az Alt Rot objArcOff guideRot calibOff guideRMS -------------------------------------------------------------------------------- 01:13 5-12162A 192.9 52.9 29.5 ( -1, 1) (-23) ( 0, 0, 0) 0.482 apog 01:21 5-12162A 194.0 52.5 32.3 ( -1, 0) (-24) ( 0, 0, 0) 0.417 apog 01:30 5-12162A 195.1 52.1 35.1 ( -1, 0) (-11) ( 0, 0, 0) 0.440 apog 01:38 5-12162A 196.1 51.6 37.9 ( -1, 1) ( -7) ( 0, 0, 0) 0.393 apog 01:47 5-12162A 197.1 51.1 40.6 ( -1, 1) (-15) ( 0, 0, 0) 0.390 apog 01:55 5-12162A 198.0 50.5 43.2 ( 1, 2) (-11) ( 0, 0, 0) 0.387 apog 02:03 5-12162A 198.9 50.0 45.9 ( 0, 3) (-16) ( 0, 0, 0) 0.370 apog 02:12 5-12162A 199.7 49.4 48.5 ( 0, 3) (-15) ( 0, 0, 0) 0.388 apog 02:33 9-11421A 238.3 75.6 67.5 ( -5, 4) (-48) ( 0, 0, 0) 0.366 apog 02:41 9-11421A 240.5 74.1 71.0 ( -5, 5) (-46) ( 0, 0, 0) 0.360 apog 02:50 9-11421A 242.2 72.5 73.9 ( -5, 5) (-42) ( 0, 0, 0) 0.399 apog 02:58 9-11421A 243.5 71.0 76.5 ( -5, 5) (-41) ( 0, 0, 0) 0.387 apog 03:06 9-11421A 244.5 69.4 78.7 ( -5, 5) (-41) ( 0, 0, 0) 0.356 apog 03:15 9-11421A 245.3 67.8 80.8 ( -4, 5) (-42) ( 0, 0, 0) 0.356 apog 03:23 9-11421A 245.8 66.2 82.6 ( -5, 5) (-41) ( 0, 0, 0) 0.418 apog 03:32 9-11421A 246.3 64.5 84.4 ( -4, 5) (-41) ( 0, 0, 0) 0.365 apog 03:53 8-11480A 30.2 53.0 204.9 ( -1, -2) (-34) ( 0, 0, 0) 0.671 apog 04:09 8-11480A 23.6 54.6 199.6 ( -0, -0) (-38) ( 0, 0, 0) 0.502 apog 04:26 8-11480A 16.7 55.8 193.8 ( 0, 0) (-28) ( 0, 0, 0) 0.556 apog 04:43 8-11480A 9.2 56.6 187.6 ( 0, 1) (-22) ( 0, 0, 0) 0.523 apog 05:11 7-11466A 168.1 65.2 -18.6 ( 2, 2) ( 18) ( 0, 0, 0) 0.395 apog 05:19 7-11466A 170.7 65.5 -14.4 ( 2, 2) ( 17) ( 0, 0, 0) 0.375 apog 05:28 7-11466A 173.4 65.7 -10.2 ( 2, 2) ( 17) ( 0, 0, 0) 0.377 apog 05:36 7-11466A 176.2 65.9 -5.8 ( 2, 3) ( 18) ( 0, 0, 0) 0.333 apog 05:44 7-11466A 179.0 66.0 -1.5 ( 2, 3) ( 25) ( 0, 0, 0) 0.378 apog 05:53 7-11466A 181.9 66.0 2.8 ( 2, 3) ( 23) ( 0, 0, 0) 0.365 apog 06:01 7-11466A 184.6 65.9 7.1 ( 2, 4) ( 20) ( 0, 0, 0) 0.386 apog 06:09 7-11466A 187.4 65.7 11.4 ( 2, 4) ( 24) ( 0, 0, 0) 0.389 apog 06:18 7-11466A 190.1 65.4 15.6 ( 2, 4) ( 21) ( 0, 0, 0) 0.683 apog 06:26 7-11466A 192.7 65.1 19.7 ( 2, 4) ( 22) ( 0, 0, 0) 0.414 apog 06:53 1-12132A 79.9 53.1 241.8 ( -2, -1) (-38) ( 0, 0, 0) 0.479 apog 07:02 1-12132A 78.2 54.9 241.2 ( -2, 0) (-35) ( 0, 0, 0) 0.491 apog 07:10 1-12132A 76.5 56.6 240.6 ( -2, 0) (-35) ( 0, 0, 0) 0.439 apog 07:19 1-12132A 74.7 58.3 239.7 ( -1, 1) (-41) ( 0, 0, 0) 0.381 apog 07:27 1-12132A 72.8 60.0 238.8 ( -1, 1) (-40) ( 0, 0, 0) 0.491 apog 07:35 1-12132A 70.6 61.7 237.6 ( 4, -1) (-40) ( 0, 0, 0) 0.387 apog 09:24 6-12131A 32.7 80.9 210.1 ( 3, 0) ( 3) ( 0, 0, 0) 0.287 apog 09:32 6-12131A 21.3 81.7 199.7 ( 3, 0) ( 6) ( 0, 0, 0) 0.246 apog 09:40 6-12131A 8.2 82.1 187.6 ( 3, 0) ( 12) ( 0, 0, 0) 0.294 apog 11:56 6-12131A 121.0 30.0 0.0 ( 0, 0) ( 0) ( 0, 0, 0) 2.152 boss ---- Focus Log ---- Time Inst Scale M1 M2 Focus Az Alt Temp Wind Dir -------------------------------------------------------------------------------- 01:13 5-12162A 1.000150 -1481 -445 8 192.9 52.9 12.2 27 254 1.5 apog 01:21 5-12162A 1.000180 -1777 -668 29 194.0 52.5 12.2 17 250 1.6 apog 01:30 5-12162A 1.000180 -1777 -659 36 195.1 52.1 12.2 17 249 1.5 apog 01:38 5-12162A 1.000180 -1777 -657 37 196.1 51.6 12.2 17 236 1.4 apog 01:47 5-12162A 1.000180 -1777 -656 37 197.1 51.1 12.2 28 251 1.7 apog 01:55 5-12162A 1.000180 -1777 -673 18 198.0 50.5 12.2 22 255 1.5 apog 02:03 5-12162A 1.000180 -1777 -692 -2 198.9 50.0 12.2 26 259 1.3 apog 02:12 5-12162A 1.000180 -1777 -728 -39 199.7 49.4 12.2 27 254 1.2 apog 02:33 9-11421A 1.000010 -98 602 -38 238.3 75.6 12.2 19 260 1.2 apog 02:41 9-11421A 0.999990 98 767 -38 240.5 74.1 12.2 16 253 1.3 apog 02:50 9-11421A 0.999990 98 762 -46 242.2 72.5 12.2 20 243 1.3 apog 02:58 9-11421A 0.999990 98 747 -64 243.5 71.0 12.2 19 242 1.1 apog 03:06 9-11421A 0.999990 98 739 -74 244.5 69.4 12.2 16 262 1.3 apog 03:15 9-11421A 0.999990 98 733 -82 245.3 67.8 12.2 19 254 1.1 apog 03:23 9-11421A 0.999990 98 727 -92 245.8 66.2 12.2 15 264 1.3 apog 03:32 9-11421A 0.999990 98 730 -92 246.3 64.5 12.2 21 265 1.1 apog 03:53 8-11480A 1.000020 -197 480 -127 30.2 53.0 12.2 15 266 1.3 apog 04:09 8-11480A 1.000020 -197 435 -168 23.6 54.6 12.2 20 267 1.4 apog 04:26 8-11480A 1.000000 0 591 -172 16.7 55.8 12.2 21 259 1.6 apog 04:43 8-11480A 1.000000 0 589 -172 9.2 56.6 12.2 20 264 1.3 apog 05:11 7-11466A 1.000030 -296 300 -195 168.1 65.2 12.2 20 263 1.3 apog 05:19 7-11466A 1.000030 -296 284 -210 170.7 65.5 12.2 21 254 1.3 apog 05:28 7-11466A 1.000050 -493 121 -209 173.4 65.7 12.2 19 248 1.3 apog 05:36 7-11466A 1.000050 -493 119 -210 176.2 65.9 12.2 22 266 1.4 apog 05:44 7-11466A 1.000050 -493 118 -211 179.0 66.0 12.2 17 264 1.4 apog 05:53 7-11466A 1.000050 -493 120 -209 181.9 66.0 12.2 18 261 1.4 apog 06:01 7-11466A 1.000050 -493 120 -209 184.6 65.9 12.2 21 258 1.4 apog 06:09 7-11466A 1.000050 -493 121 -209 187.4 65.7 12.2 20 260 1.5 apog 06:18 7-11466A 1.000070 -691 -41 -209 190.1 65.4 12.2 19 262 1.3 apog 06:26 7-11466A 1.000070 -691 -40 -209 192.7 65.1 12.2 22 260 1.4 apog 06:53 1-12132A 1.000220 -2172 -1219 -192 79.9 53.1 12.2 17 260 1.6 apog 07:02 1-12132A 1.000220 -2172 -1224 -192 78.2 54.9 12.2 15 262 1.7 apog 07:10 1-12132A 1.000220 -2172 -1229 -192 76.5 56.6 12.2 17 253 1.5 apog 07:19 1-12132A 1.000220 -2172 -1233 -192 74.7 58.3 12.2 14 254 1.4 apog 07:27 1-12132A 1.000200 -1975 -1074 -192 72.8 60.0 12.2 20 240 1.4 apog 07:35 1-12132A 1.000200 -1975 -1078 -192 70.6 61.7 12.2 20 252 1.2 apog 09:24 6-12131A 1.000200 -1975 -1041 -124 32.7 80.9 12.2 22 270 1.1 apog 09:32 6-12131A 1.000220 -2172 -1205 -124 21.3 81.7 12.2 17 268 1.2 apog 09:40 6-12131A 1.000220 -2172 -1206 -124 8.2 82.1 12.2 17 267 1.2 apog 11:56 6-12131A 1.000000 0 771 -85 121.0 30.0 12.2 17 264 1.1 boss ---- Telescope Status ---- 00:10Z Started ventilation 00:42Z encl25m : open 11:50Z encl25m : closed No interlocks bypassed Status at: 13:15 Telescope stowed at: 121.0, 30.0, 0.0 mount Instrument mounted: cartridge = 6 Counterweights at: 330 LN2 autofill systems: Connected and turned on 180L LN2 dewar scales: SP1 132 lbs, 22 psi SP2 134 lbs, 20 psi ---- Software Used ---- Filesystem Size Used Avail Use% Mounted on /dev/vdd 688G 643G 46G 94% /home /dev/vdc1 11T 6.6T 4.2T 62% /data -- STUI software: STUI : 1.7.7 alerts : trunk apo : trunk boss : v4_1_1 -boss.specMec : 3.0.4 gcamera : v2_2 guider : 3.8.1 hartmann : 1.6.0 hub : Tag: v4.2 mcp : 1.3|2019-10-09|11:46 -mcp.plc : 34 -mcp.azFiduci : v1_140 -mcp.altFiduc : v1_140 -mcp.rotFiduc : v1_140 platedb : 2.8.0 sop : 3.12.3 tcc : 1.3.1 apogee : not availble apogeecal : not availble apogeeql : v1_6_1 APO-local : trunk -- Other versions ---- idlspec2d : v5_7_12 plate-mapper3 : NOCVS:v6_0_12 petunia : git autoscheduler : 1.3.0 sdss_python_module: trunk mangadrp : trunk exported ---- APOGEE Data Summary ---- APOGEE data list, mjd=58766, all raw_data: /data/apogee/utr_cdr/58766/apRaw-*.fits archive: /data/apogee/archive/58766/apR-[a,b,c]-*.apz quickred: /data/apogee/quickred/58766/ap2D-a-*.fits.fz -------------------------------------------------------------------------------- mjd i UT File/Exp Imtype Nread A/B Dth Ct-Plate Arch Offset %Flux Comm -------------------------------------------------------------------------------- 58766 1 22:21 32040001 Dark 3 A 13.0 4-10224 a-b-c - - 58766 2 22:26 32040002 Dark 60 A 13.0 4-10224 a-b-c - - 58766 3 22:37 32040003 QuarFlat 10 A 13.0 4-10224 a-b-c - 46 58766 4 22:39 32040004 Arc-Thar 12 A 13.0 4-10224 a-b-c -2.83 - 58766 5 22:41 32040005 Arc-Une 40 A 13.0 4-10224 a-b-c - - 58766 6 22:48 32040006 Arc-Thar 12 B 13.5 4-10224 a-b-c -2.30 - 58766 7 22:50 32040007 Arc-Une 40 B 13.5 4-10224 a-b-c - - 58766 8 22:58 32040008 Dark 60 A 13.0 5-12162 a-b-c - - 58766 9 00:46 32040009 Dark 10 A 13.0 5-12162 a-b-c - - 58766 10 00:47 32040010 Dark 10 A 13.0 5-12162 a-b-c - - 58766 11 01:13 32040011 Object 47 A 13.0 5-12162 a-b-c - - 58766 12 01:22 32040012 Object 47 B 13.5 5-12162 a-b-c - - 58766 13 01:30 32040013 Object 47 B 13.5 5-12162 a-b-c - - 58766 14 01:39 32040014 Object 47 A 13.0 5-12162 a-b-c - - 58766 15 01:47 32040015 Object 47 A 13.0 5-12162 a-b-c - - 58766 16 01:55 32040016 Object 47 B 13.5 5-12162 a-b-c - - 58766 17 02:04 32040017 Object 47 B 13.5 5-12162 a-b-c - - 58766 18 02:12 32040018 Object 47 A 13.0 5-12162 a-b-c - - 58766 19 02:21 32040019 DomeFlat 5 A 13.0 5-12162 a-b-c - 81 58766 20 02:24 32040020 Dark 10 A 13.0 5-12162 a-b-c - - 58766 21 02:26 32040021 Dark 10 A 13.0 5-12162 a-b-c - - 58766 22 02:33 32040022 Object 47 A 13.0 9-11421 a-b-c - - 58766 23 02:42 32040023 Object 47 B 13.5 9-11421 a-b-c - - 58766 24 02:50 32040024 Object 47 B 13.5 9-11421 a-b-c - - 58766 25 02:58 32040025 Object 47 A 13.0 9-11421 a-b-c - - 58766 26 03:07 32040026 Object 47 A 13.0 9-11421 a-b-c - - 58766 27 03:15 32040027 Object 47 B 13.5 9-11421 a-b-c - - 58766 28 03:24 32040028 Object 47 B 13.5 9-11421 a-b-c - - 58766 29 03:32 32040029 Object 47 A 13.0 9-11421 a-b-c - - 58766 30 03:41 32040030 DomeFlat 5 A 13.0 9-11421 a-b-c - 88 58766 31 03:43 32040031 Dark 10 A 13.0 9-11421 a-b-c - - 58766 32 03:45 32040032 Dark 10 A 13.0 9-11421 a-b-c - - 58766 33 03:53 32040033 Object 94 A 13.0 8-11480 a-b-c - - 58766 34 04:10 32040034 Object 94 B 13.5 8-11480 a-b-c - - 58766 35 04:26 32040035 Object 94 B 13.5 8-11480 a-b-c - - 58766 36 04:43 32040036 Object 94 A 13.0 8-11480 a-b-c - - 58766 37 05:00 32040037 DomeFlat 5 A 13.0 8-11480 a-b-c - 83 58766 38 05:03 32040038 Dark 10 A 13.0 8-11480 a-b-c - - 58766 39 05:05 32040039 Dark 10 A 13.0 8-11480 a-b-c - - 58766 40 05:11 32040040 Object 47 A 13.0 7-11466 a-b-c - - 58766 41 05:20 32040041 Object 47 B 13.5 7-11466 a-b-c - - 58766 42 05:28 32040042 Object 47 B 13.5 7-11466 a-b-c - - 58766 43 05:36 32040043 Object 47 A 13.0 7-11466 a-b-c - - 58766 44 05:45 32040044 Object 47 A 13.0 7-11466 a-b-c - - 58766 45 05:53 32040045 Object 47 B 13.5 7-11466 a-b-c - - 58766 46 06:01 32040046 Object 47 B 13.5 7-11466 a-b-c - - 58766 47 06:10 32040047 Object 47 A 13.0 7-11466 a-b-c - - 58766 48 06:18 32040048 Object 47 A 13.0 7-11466 a-b-c - - 58766 49 06:27 32040049 Object 47 B 13.5 7-11466 a-b-c - - 58766 50 06:35 32040050 DomeFlat 5 B 13.5 7-11466 a-b-c - 87 58766 51 06:38 32040051 Dark 10 B 13.5 7-11466 a-b-c - - 58766 52 06:40 32040052 Dark 10 B 13.5 7-11466 a-b-c - - 58766 53 06:54 32040053 Object 47 B 13.5 1-12132 a-b-c - - 58766 54 07:02 32040054 Object 47 A 13.0 1-12132 a-b-c - - 58766 55 07:11 32040055 Object 47 A 13.0 1-12132 a-b-c - - 58766 56 07:19 32040056 Object 47 B 13.5 1-12132 a-b-c - - 58766 57 07:27 32040057 Object 47 B 13.5 1-12132 a-b-c - - 58766 58 07:36 32040058 Object 47 A 13.0 1-12132 a-b-c - - 58766 59 07:45 32040059 DomeFlat 5 A 13.0 1-12132 a-b-c - 80 58766 60 07:47 32040060 Dark 10 A 13.0 1-12132 a-b-c - - 58766 61 07:49 32040061 Dark 10 A 13.0 1-12132 a-b-c - - 58766 62 09:24 32040062 Object 47 A 13.0 6-12131 a-b-c - - 58766 63 09:33 32040063 Object 47 B 13.5 6-12131 a-b-c - - 58766 64 09:41 32040064 Object 47 B 13.5 6-12131 a-b-c - - 58766 65 10:55 32040065 DomeFlat 5 B 13.5 6-12131 a-b-c - 73 58766 66 11:28 32040066 Dark 10 B 13.5 6-12131 a-b-c - - 58766 67 11:30 32040067 Dark 10 B 13.5 6-12131 a-b-c - - 58766 68 11:32 32040068 Dark 60 B 13.5 6-12131 a-b-c - - 58766 69 11:43 32040069 Dark 60 B 13.5 6-12131 a-b-c - - 58766 70 11:53 32040070 Dark 60 B 13.5 6-12131 a-b-c - - 58766 71 12:04 32040071 QuarFlat 10 B 13.5 6-12131 a-b-c - 52 58766 72 12:06 32040072 QuarFlat 10 B 13.5 6-12131 a-b-c - 52 58766 73 12:08 32040073 QuarFlat 10 B 13.5 6-12131 a-b-c - 52 58766 74 12:10 32040074 Arc-Thar 12 A 13.0 6-12131 a-b-c -2.79 - 58766 75 12:12 32040075 Arc-Une 40 A 13.0 6-12131 a-b-c - - 58766 76 12:19 32040076 Arc-Thar 12 B 13.5 6-12131 a-b-c -2.27 - 58766 77 12:22 32040077 Arc-Une 40 B 13.5 6-12131 a-b-c - - 58766 78 12:29 32040078 Dark 30 A 13.0 6-12131 a-b-c - - 58766 79 12:34 32040079 InteFlat 30 A 13.0 6-12131 a-b-c - 0 58766 80 12:40 32040080 InteFlat 30 A 13.0 6-12131 a-b-c - 0 58766 81 12:45 32040081 InteFlat 30 A 13.0 6-12131 a-b-c - 0 58766 82 12:50 32040082 Dark 30 A 13.0 6-12131 a-b-c - - -------------------------------------------------------------------------------- ---- MaNGA Data Summary ---- Reading FITS from /data/spectro/58766/ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - n time filename crt plt flav Shu Dth exp colA colB colC Hrt b1 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 1 22:26 b1-00296490 4 10224 bia Clo C 0.0 22447 21596 20337 22:26 b2-00296490 4 10224 bia Clo C 0.0 23102 22899 22736 22:26 r1-00296490 4 10224 bia Clo C 0.0 22447 21596 20337 22:26 r2-00296490 4 10224 bia Clo C 0.0 23102 22899 22736 2 22:28 b1-00296491 4 10224 dar Clo C 20.0 22447 21596 20337 22:28 b2-00296491 4 10224 dar Clo C 20.0 23102 22899 22736 22:28 r1-00296491 4 10224 dar Clo C 20.0 22447 21596 20337 22:28 r2-00296491 4 10224 dar Clo C 20.0 23102 22899 22736 3 22:30 b1-00296492 4 10224 arc Lef C 4.1 22447 21596 20337 HART 22:30 b2-00296492 4 10224 arc Lef C 4.2 23102 22899 22736 HART 22:30 r1-00296492 4 10224 arc Lef C 4.1 22447 21596 20337 HART 22:30 r2-00296492 4 10224 arc Lef C 4.2 23102 22899 22736 HART 4 22:32 b1-00296493 4 10224 arc Rig C 4.1 22447 21596 20337 HART 22:32 b2-00296493 4 10224 arc Rig C 4.2 23102 22899 22736 HART 22:32 r1-00296493 4 10224 arc Rig C 4.1 22447 21596 20337 HART 22:32 r2-00296493 4 10224 arc Rig C 4.2 23102 22899 22736 HART 5 22:34 b1-00296494 4 10224 fla Out C 25.0 22960 22105 20848 22:34 b2-00296494 4 10224 fla Out C 25.2 23734 23528 23366 22:34 r1-00296494 4 10224 fla Out C 25.0 22960 22105 20848 22:34 r2-00296494 4 10224 fla Out C 25.2 23734 23528 23366 6 22:37 b1-00296495 4 10224 arc Out C 4.1 22960 22105 20848 22:37 b2-00296495 4 10224 arc Out C 4.2 23734 23528 23366 22:37 r1-00296495 4 10224 arc Out C 4.1 22960 22105 20848 22:37 r2-00296495 4 10224 arc Out C 4.2 23734 23528 23366 7 01:03 b1-00296496 5 12162 arc Lef C 4.1 22960 22105 20848 HART 01:03 b2-00296496 5 12162 arc Lef C 4.2 23734 23528 23366 HART 01:03 r1-00296496 5 12162 arc Lef C 4.1 22960 22105 20848 HART 01:03 r2-00296496 5 12162 arc Lef C 4.2 23734 23528 23366 HART 8 01:03 b1-00296497 5 12162 arc Rig C 4.1 22960 22105 20848 HART 01:03 b2-00296497 5 12162 arc Rig C 4.2 23734 23528 23366 HART 01:03 r1-00296497 5 12162 arc Rig C 4.1 22960 22105 20848 HART 01:03 r2-00296497 5 12162 arc Rig C 4.2 23734 23528 23366 HART 9 01:07 b1-00296498 5 12162 arc Out C 4.1 27701 26847 25589 01:07 b2-00296498 5 12162 arc Out C 4.2 25783 25573 25417 01:07 r1-00296498 5 12162 arc Out C 4.1 27701 26847 25589 01:07 r2-00296498 5 12162 arc Out C 4.2 25783 25573 25417 10 01:09 b1-00296499 5 12162 fla Out C 25.1 27701 26847 25589 01:09 b2-00296499 5 12162 fla Out C 25.2 25783 25573 25417 01:09 r1-00296499 5 12162 fla Out C 25.1 27701 26847 25589 01:09 r2-00296499 5 12162 fla Out C 25.2 25783 25573 25417 11 01:14 b1-00296500 5 12162 sci Out C 83.1 27701 26847 25589 01:14 b2-00296500 5 12162 sci Out C 83.2 25783 25573 25417 01:14 r1-00296500 5 12162 sci Out C 83.1 27701 26847 25589 01:14 r2-00296500 5 12162 sci Out C 83.2 25783 25573 25417 12 01:17 b1-00296501 5 12162 sci Out C 83.1 27701 26847 25589 01:17 b2-00296501 5 12162 sci Out C 83.2 25783 25573 25417 01:17 r1-00296501 5 12162 sci Out C 83.1 27701 26847 25589 01:17 r2-00296501 5 12162 sci Out C 83.2 25783 25573 25417 13 01:19 b1-00296502 5 12162 sci Out C 83.1 27701 26847 25589 01:19 b2-00296502 5 12162 sci Out C 83.2 25783 25573 25417 01:19 r1-00296502 5 12162 sci Out C 83.1 27701 26847 25589 01:19 r2-00296502 5 12162 sci Out C 83.2 25783 25573 25417 14 01:22 b1-00296503 5 12162 sci Out C 83.1 27701 26847 25589 01:22 b2-00296503 5 12162 sci Out C 83.2 25783 25573 25417 01:22 r1-00296503 5 12162 sci Out C 83.1 27701 26847 25589 01:22 r2-00296503 5 12162 sci Out C 83.2 25783 25573 25417 15 01:25 b1-00296504 5 12162 sci Out C 83.1 27701 26847 25589 01:25 b2-00296504 5 12162 sci Out C 83.2 25783 25573 25417 01:25 r1-00296504 5 12162 sci Out C 83.1 27701 26847 25589 01:25 r2-00296504 5 12162 sci Out C 83.2 25783 25573 25417 16 01:28 b1-00296505 5 12162 sci Out C 83.1 27701 26847 25589 01:28 b2-00296505 5 12162 sci Out C 83.2 25783 25573 25417 01:28 r1-00296505 5 12162 sci Out C 83.1 27701 26847 25589 01:28 r2-00296505 5 12162 sci Out C 83.2 25783 25573 25417 17 01:31 b1-00296506 5 12162 sci Out C 83.1 27701 26847 25589 01:31 b2-00296506 5 12162 sci Out C 83.2 25783 25573 25417 01:31 r1-00296506 5 12162 sci Out C 83.1 27701 26847 25589 01:31 r2-00296506 5 12162 sci Out C 83.2 25783 25573 25417 18 01:33 b1-00296507 5 12162 sci Out C 83.1 27701 26847 25589 01:33 b2-00296507 5 12162 sci Out C 83.2 25783 25573 25417 01:33 r1-00296507 5 12162 sci Out C 83.1 27701 26847 25589 01:33 r2-00296507 5 12162 sci Out C 83.2 25783 25573 25417 19 01:36 b1-00296508 5 12162 sci Out C 83.1 27701 26847 25589 01:36 b2-00296508 5 12162 sci Out C 83.2 25783 25573 25417 01:36 r1-00296508 5 12162 sci Out C 83.1 27701 26847 25589 01:36 r2-00296508 5 12162 sci Out C 83.2 25783 25573 25417 20 01:39 b1-00296509 5 12162 sci Out C 83.1 27701 26847 25589 01:39 b2-00296509 5 12162 sci Out C 83.2 25783 25573 25417 01:39 r1-00296509 5 12162 sci Out C 83.1 27701 26847 25589 01:39 r2-00296509 5 12162 sci Out C 83.2 25783 25573 25417 21 01:42 b1-00296510 5 12162 sci Out C 83.1 27701 26847 25589 01:42 b2-00296510 5 12162 sci Out C 83.2 25783 25573 25417 01:42 r1-00296510 5 12162 sci Out C 83.1 27701 26847 25589 01:42 r2-00296510 5 12162 sci Out C 83.2 25783 25573 25417 22 01:44 b1-00296511 5 12162 sci Out C 83.1 27701 26847 25589 01:44 b2-00296511 5 12162 sci Out C 83.2 25783 25573 25417 01:44 r1-00296511 5 12162 sci Out C 83.1 27701 26847 25589 01:44 r2-00296511 5 12162 sci Out C 83.2 25783 25573 25417 23 01:47 b1-00296512 5 12162 sci Out C 83.1 27701 26847 25589 01:47 b2-00296512 5 12162 sci Out C 83.2 25783 25573 25417 01:47 r1-00296512 5 12162 sci Out C 83.1 27701 26847 25589 01:47 r2-00296512 5 12162 sci Out C 83.2 25783 25573 25417 24 01:50 b1-00296513 5 12162 sci Out C 83.1 27701 26847 25589 01:50 b2-00296513 5 12162 sci Out C 83.2 25783 25573 25417 01:50 r1-00296513 5 12162 sci Out C 83.1 27701 26847 25589 01:50 r2-00296513 5 12162 sci Out C 83.2 25783 25573 25417 25 01:53 b1-00296514 5 12162 sci Out C 83.1 27701 26847 25589 01:53 b2-00296514 5 12162 sci Out C 83.2 25783 25573 25417 01:53 r1-00296514 5 12162 sci Out C 83.1 27701 26847 25589 01:53 r2-00296514 5 12162 sci Out C 83.2 25783 25573 25417 26 01:56 b1-00296515 5 12162 sci Out C 83.1 27701 26847 25589 01:56 b2-00296515 5 12162 sci Out C 83.2 25783 25573 25417 01:56 r1-00296515 5 12162 sci Out C 83.1 27701 26847 25589 01:56 r2-00296515 5 12162 sci Out C 83.2 25783 25573 25417 27 01:58 b1-00296516 5 12162 sci Out C 83.1 27701 26847 25589 01:58 b2-00296516 5 12162 sci Out C 83.2 25783 25573 25417 01:58 r1-00296516 5 12162 sci Out C 83.1 27701 26847 25589 01:58 r2-00296516 5 12162 sci Out C 83.2 25783 25573 25417 28 02:01 b1-00296517 5 12162 sci Out C 83.1 27701 26847 25589 02:01 b2-00296517 5 12162 sci Out C 83.2 25783 25573 25417 02:01 r1-00296517 5 12162 sci Out C 83.1 27701 26847 25589 02:01 r2-00296517 5 12162 sci Out C 83.2 25783 25573 25417 29 02:04 b1-00296518 5 12162 sci Out C 83.1 27701 26847 25589 02:04 b2-00296518 5 12162 sci Out C 83.2 25783 25573 25417 02:04 r1-00296518 5 12162 sci Out C 83.1 27701 26847 25589 02:04 r2-00296518 5 12162 sci Out C 83.2 25783 25573 25417 30 02:07 b1-00296519 5 12162 sci Out C 83.1 27701 26847 25589 02:07 b2-00296519 5 12162 sci Out C 83.2 25783 25573 25417 02:07 r1-00296519 5 12162 sci Out C 83.1 27701 26847 25589 02:07 r2-00296519 5 12162 sci Out C 83.2 25783 25573 25417 31 02:10 b1-00296520 5 12162 sci Out C 83.1 27701 26847 25589 02:10 b2-00296520 5 12162 sci Out C 83.2 25783 25573 25417 02:10 r1-00296520 5 12162 sci Out C 83.1 27701 26847 25589 02:10 r2-00296520 5 12162 sci Out C 83.2 25783 25573 25417 32 02:12 b1-00296521 5 12162 sci Out C 83.1 27701 26847 25589 02:12 b2-00296521 5 12162 sci Out C 83.2 25783 25573 25417 02:12 r1-00296521 5 12162 sci Out C 83.1 27701 26847 25589 02:12 r2-00296521 5 12162 sci Out C 83.2 25783 25573 25417 33 02:15 b1-00296522 5 12162 sci Out C 83.1 27701 26847 25589 02:15 b2-00296522 5 12162 sci Out C 83.2 25783 25573 25417 02:15 r1-00296522 5 12162 sci Out C 83.1 27701 26847 25589 02:15 r2-00296522 5 12162 sci Out C 83.2 25783 25573 25417 34 02:18 b1-00296523 5 12162 sci Out C 83.1 27701 26847 25589 02:18 b2-00296523 5 12162 sci Out C 83.2 25783 25573 25417 02:18 r1-00296523 5 12162 sci Out C 83.1 27701 26847 25589 02:18 r2-00296523 5 12162 sci Out C 83.2 25783 25573 25417 35 06:44 b1-00296524 1 12132 arc Lef C 4.1 27701 26847 25589 HART 06:44 b2-00296524 1 12132 arc Lef C 4.2 25783 25571 25416 HART 06:44 r1-00296524 1 12132 arc Lef C 4.1 27701 26847 25589 HART 06:44 r2-00296524 1 12132 arc Lef C 4.2 25783 25571 25416 HART 36 06:45 b1-00296525 1 12132 arc Rig C 4.1 27701 26847 25589 HART 06:45 b2-00296525 1 12132 arc Rig C 4.2 25783 25571 25416 HART 06:45 r1-00296525 1 12132 arc Rig C 4.1 27701 26847 25589 HART 06:45 r2-00296525 1 12132 arc Rig C 4.2 25783 25571 25416 HART 37 06:49 b1-00296526 1 12132 arc Out C 4.1 21230 20376 19118 06:49 b2-00296526 1 12132 arc Out C 4.2 19789 19577 19423 06:49 r1-00296526 1 12132 arc Out C 4.1 21230 20376 19118 06:49 r2-00296526 1 12132 arc Out C 4.2 19789 19577 19423 38 06:50 b1-00296527 1 12132 fla Out C 25.1 21230 20376 19118 06:50 b2-00296527 1 12132 fla Out C 25.2 19789 19577 19423 06:50 r1-00296527 1 12132 fla Out C 25.1 21230 20376 19118 06:50 r2-00296527 1 12132 fla Out C 25.2 19789 19577 19423 39 06:54 b1-00296528 1 12132 sci Out C 28.1 21230 20376 19118 06:54 b2-00296528 1 12132 sci Out C 28.1 19789 19577 19423 06:54 r1-00296528 1 12132 sci Out C 28.1 21230 20376 19118 06:54 r2-00296528 1 12132 sci Out C 28.1 19789 19577 19423 40 06:56 b1-00296529 1 12132 sci Out C 28.1 21230 20376 19118 06:56 b2-00296529 1 12132 sci Out C 28.1 19789 19577 19423 06:56 r1-00296529 1 12132 sci Out C 28.1 21230 20376 19118 06:56 r2-00296529 1 12132 sci Out C 28.1 19789 19577 19423 41 06:58 b1-00296530 1 12132 sci Out C 28.1 21230 20376 19118 06:58 b2-00296530 1 12132 sci Out C 28.2 19789 19577 19423 06:58 r1-00296530 1 12132 sci Out C 28.1 21230 20376 19118 06:58 r2-00296530 1 12132 sci Out C 28.2 19789 19577 19423 42 07:00 b1-00296531 1 12132 sci Out C 28.1 21230 20376 19118 07:00 b2-00296531 1 12132 sci Out C 28.2 19789 19577 19423 07:00 r1-00296531 1 12132 sci Out C 28.1 21230 20376 19118 07:00 r2-00296531 1 12132 sci Out C 28.2 19789 19577 19423 43 07:01 b1-00296532 1 12132 sci Out C 28.1 21230 20376 19118 07:01 b2-00296532 1 12132 sci Out C 28.2 19789 19577 19423 07:01 r1-00296532 1 12132 sci Out C 28.1 21230 20376 19118 07:01 r2-00296532 1 12132 sci Out C 28.2 19789 19577 19423 44 07:03 b1-00296533 1 12132 sci Out C 28.1 21230 20376 19118 07:03 b2-00296533 1 12132 sci Out C 28.1 19789 19577 19423 07:03 r1-00296533 1 12132 sci Out C 28.1 21230 20376 19118 07:03 r2-00296533 1 12132 sci Out C 28.1 19789 19577 19423 45 07:05 b1-00296534 1 12132 sci Out C 28.1 21230 20376 19118 07:05 b2-00296534 1 12132 sci Out C 28.2 19789 19577 19423 07:05 r1-00296534 1 12132 sci Out C 28.1 21230 20376 19118 07:05 r2-00296534 1 12132 sci Out C 28.2 19789 19577 19423 46 07:07 b1-00296535 1 12132 sci Out C 28.1 21230 20376 19118 07:07 b2-00296535 1 12132 sci Out C 28.2 19789 19577 19423 07:07 r1-00296535 1 12132 sci Out C 28.1 21230 20376 19118 07:07 r2-00296535 1 12132 sci Out C 28.2 19789 19577 19423 47 07:09 b1-00296536 1 12132 sci Out C 28.1 21230 20376 19118 07:09 b2-00296536 1 12132 sci Out C 28.2 19789 19577 19423 07:09 r1-00296536 1 12132 sci Out C 28.1 21230 20376 19118 07:09 r2-00296536 1 12132 sci Out C 28.2 19789 19577 19423 48 07:11 b1-00296537 1 12132 sci Out C 28.1 21230 20376 19118 07:11 b2-00296537 1 12132 sci Out C 28.2 19789 19577 19423 07:11 r1-00296537 1 12132 sci Out C 28.1 21230 20376 19118 07:11 r2-00296537 1 12132 sci Out C 28.2 19789 19577 19423 49 07:13 b1-00296538 1 12132 sci Out C 28.1 21230 20376 19118 07:13 b2-00296538 1 12132 sci Out C 28.2 19789 19577 19423 07:13 r1-00296538 1 12132 sci Out C 28.1 21230 20376 19118 07:13 r2-00296538 1 12132 sci Out C 28.2 19789 19577 19423 50 07:15 b1-00296539 1 12132 sci Out C 28.1 21230 20376 19118 07:15 b2-00296539 1 12132 sci Out C 28.2 19789 19577 19423 07:15 r1-00296539 1 12132 sci Out C 28.1 21230 20376 19118 07:15 r2-00296539 1 12132 sci Out C 28.2 19789 19577 19423 51 07:16 b1-00296540 1 12132 sci Out C 28.1 21230 20376 19118 07:16 b2-00296540 1 12132 sci Out C 28.2 19789 19577 19423 07:16 r1-00296540 1 12132 sci Out C 28.1 21230 20376 19118 07:16 r2-00296540 1 12132 sci Out C 28.2 19789 19577 19423 52 07:18 b1-00296541 1 12132 sci Out C 28.1 21230 20376 19118 07:18 b2-00296541 1 12132 sci Out C 28.2 19789 19577 19423 07:18 r1-00296541 1 12132 sci Out C 28.1 21230 20376 19118 07:18 r2-00296541 1 12132 sci Out C 28.2 19789 19577 19423 53 07:20 b1-00296542 1 12132 sci Out C 28.1 21230 20376 19118 07:20 b2-00296542 1 12132 sci Out C 28.2 19789 19577 19423 07:20 r1-00296542 1 12132 sci Out C 28.1 21230 20376 19118 07:20 r2-00296542 1 12132 sci Out C 28.2 19789 19577 19423 54 07:22 b1-00296543 1 12132 sci Out C 28.1 21230 20376 19118 07:22 b2-00296543 1 12132 sci Out C 28.2 19789 19577 19423 07:22 r1-00296543 1 12132 sci Out C 28.1 21230 20376 19118 07:22 r2-00296543 1 12132 sci Out C 28.2 19789 19577 19423 55 07:24 b1-00296544 1 12132 sci Out C 28.1 21230 20376 19118 07:24 b2-00296544 1 12132 sci Out C 28.2 19789 19577 19423 07:24 r1-00296544 1 12132 sci Out C 28.1 21230 20376 19118 07:24 r2-00296544 1 12132 sci Out C 28.2 19789 19577 19423 56 07:26 b1-00296545 1 12132 sci Out C 28.1 21230 20376 19118 07:26 b2-00296545 1 12132 sci Out C 28.2 19789 19577 19423 07:26 r1-00296545 1 12132 sci Out C 28.1 21230 20376 19118 07:26 r2-00296545 1 12132 sci Out C 28.2 19789 19577 19423 57 07:28 b1-00296546 1 12132 sci Out C 28.1 21230 20376 19118 07:28 b2-00296546 1 12132 sci Out C 28.2 19789 19577 19423 07:28 r1-00296546 1 12132 sci Out C 28.1 21230 20376 19118 07:28 r2-00296546 1 12132 sci Out C 28.2 19789 19577 19423 58 07:30 b1-00296547 1 12132 sci Out C 28.1 21230 20376 19118 07:30 b2-00296547 1 12132 sci Out C 28.2 19789 19577 19423 07:30 r1-00296547 1 12132 sci Out C 28.1 21230 20376 19118 07:30 r2-00296547 1 12132 sci Out C 28.2 19789 19577 19423 59 07:31 b1-00296548 1 12132 sci Out C 28.1 21230 20376 19118 07:31 b2-00296548 1 12132 sci Out C 28.2 19789 19577 19423 07:31 r1-00296548 1 12132 sci Out C 28.1 21230 20376 19118 07:31 r2-00296548 1 12132 sci Out C 28.2 19789 19577 19423 60 07:33 b1-00296549 1 12132 sci Out C 28.1 21230 20376 19118 07:33 b2-00296549 1 12132 sci Out C 28.1 19789 19577 19423 07:33 r1-00296549 1 12132 sci Out C 28.1 21230 20376 19118 07:33 r2-00296549 1 12132 sci Out C 28.1 19789 19577 19423 61 07:35 b1-00296550 1 12132 sci Out C 28.1 21230 20376 19118 07:35 b2-00296550 1 12132 sci Out C 28.2 19789 19577 19423 07:35 r1-00296550 1 12132 sci Out C 28.1 21230 20376 19118 07:35 r2-00296550 1 12132 sci Out C 28.2 19789 19577 19423 62 07:37 b1-00296551 1 12132 sci Out C 28.1 21230 20376 19118 07:37 b2-00296551 1 12132 sci Out C 28.2 19789 19577 19423 07:37 r1-00296551 1 12132 sci Out C 28.1 21230 20376 19118 07:37 r2-00296551 1 12132 sci Out C 28.2 19789 19577 19423 63 07:39 b1-00296552 1 12132 sci Out C 28.1 21230 20376 19118 07:39 b2-00296552 1 12132 sci Out C 28.2 19789 19577 19423 07:39 r1-00296552 1 12132 sci Out C 28.1 21230 20376 19118 07:39 r2-00296552 1 12132 sci Out C 28.2 19789 19577 19423 64 07:40 b1-00296553 1 12132 sci Out C 28.1 21230 20376 19118 07:40 b2-00296553 1 12132 sci Out C 28.2 19789 19577 19423 07:40 r1-00296553 1 12132 sci Out C 28.1 21230 20376 19118 07:40 r2-00296553 1 12132 sci Out C 28.2 19789 19577 19423 65 07:42 b1-00296554 1 12132 sci Out C 28.1 21230 20376 19118 07:42 b2-00296554 1 12132 sci Out C 28.2 19789 19577 19423 07:42 r1-00296554 1 12132 sci Out C 28.1 21230 20376 19118 07:42 r2-00296554 1 12132 sci Out C 28.2 19789 19577 19423 66 07:56 b1-00296555 6 12131 arc Lef C 4.1 21230 20376 19118 HART 07:56 b2-00296555 6 12131 arc Lef C 4.2 19789 19577 19423 HART 07:56 r1-00296555 6 12131 arc Lef C 4.1 21230 20376 19118 HART 07:56 r2-00296555 6 12131 arc Lef C 4.2 19789 19577 19423 HART 67 07:56 b1-00296556 6 12131 arc Rig C 4.1 21230 20376 19118 HART 07:56 b2-00296556 6 12131 arc Rig C 4.2 19789 19577 19423 HART 07:56 r1-00296556 6 12131 arc Rig C 4.1 21230 20376 19118 HART 07:56 r2-00296556 6 12131 arc Rig C 4.2 19789 19577 19423 HART 68 08:00 b1-00296557 6 12131 arc Out C 4.1 22733 21879 20619 08:00 b2-00296557 6 12131 arc Out C 4.2 22785 22572 22418 08:00 r1-00296557 6 12131 arc Out C 4.1 22733 21879 20619 08:00 r2-00296557 6 12131 arc Out C 4.2 22785 22572 22418 69 08:59 b1-00296558 6 12131 arc Lef C 4.1 22733 21879 20619 HART 08:59 b2-00296558 6 12131 arc Lef C 4.2 22785 22572 22418 HART 08:59 r1-00296558 6 12131 arc Lef C 4.1 22733 21879 20619 HART 08:59 r2-00296558 6 12131 arc Lef C 4.2 22785 22572 22418 HART 70 09:00 b1-00296559 6 12131 arc Rig C 4.1 22733 21879 20619 HART 09:00 b2-00296559 6 12131 arc Rig C 4.2 22785 22572 22418 HART 09:00 r1-00296559 6 12131 arc Rig C 4.1 22733 21879 20619 HART 09:00 r2-00296559 6 12131 arc Rig C 4.2 22785 22572 22418 HART 71 09:04 b1-00296560 6 12131 arc Out C 4.1 23258 22405 21144 09:04 b2-00296560 6 12131 arc Out C 4.2 22940 22729 22575 09:04 r1-00296560 6 12131 arc Out C 4.1 23258 22405 21144 09:04 r2-00296560 6 12131 arc Out C 4.2 22940 22729 22575 72 09:05 b1-00296561 6 12131 fla Out C 25.1 23258 22405 21144 09:05 b2-00296561 6 12131 fla Out C 25.2 22940 22729 22575 09:05 r1-00296561 6 12131 fla Out C 25.1 23258 22405 21144 09:05 r2-00296561 6 12131 fla Out C 25.2 22940 22729 22575 73 09:24 b1-00296562 6 12131 sci Out C 28.1 23258 22405 21144 09:24 b2-00296562 6 12131 sci Out C 28.2 22940 22729 22575 09:24 r1-00296562 6 12131 sci Out C 28.1 23258 22405 21144 09:24 r2-00296562 6 12131 sci Out C 28.2 22940 22729 22575 74 09:26 b1-00296563 6 12131 sci Out C 28.1 23258 22405 21144 09:26 b2-00296563 6 12131 sci Out C 28.2 22940 22729 22575 09:26 r1-00296563 6 12131 sci Out C 28.1 23258 22405 21144 09:26 r2-00296563 6 12131 sci Out C 28.2 22940 22729 22575 75 09:28 b1-00296564 6 12131 sci Out C 28.1 23258 22405 21144 09:28 b2-00296564 6 12131 sci Out C 28.2 22940 22729 22575 09:28 r1-00296564 6 12131 sci Out C 28.1 23258 22405 21144 09:28 r2-00296564 6 12131 sci Out C 28.2 22940 22729 22575 76 09:30 b1-00296565 6 12131 sci Out C 28.1 23258 22405 21144 09:30 b2-00296565 6 12131 sci Out C 28.1 22940 22729 22575 09:30 r1-00296565 6 12131 sci Out C 28.1 23258 22405 21144 09:30 r2-00296565 6 12131 sci Out C 28.1 22940 22729 22575 77 09:32 b1-00296566 6 12131 sci Out C 28.1 23258 22405 21144 09:32 b2-00296566 6 12131 sci Out C 28.1 22940 22729 22575 09:32 r1-00296566 6 12131 sci Out C 28.1 23258 22405 21144 09:32 r2-00296566 6 12131 sci Out C 28.1 22940 22729 22575 78 09:34 b1-00296567 6 12131 sci Out C 28.1 23258 22405 21144 09:34 b2-00296567 6 12131 sci Out C 28.2 22940 22729 22575 09:34 r1-00296567 6 12131 sci Out C 28.1 23258 22405 21144 09:34 r2-00296567 6 12131 sci Out C 28.2 22940 22729 22575 79 09:35 b1-00296568 6 12131 sci Out C 28.1 23258 22405 21144 09:35 b2-00296568 6 12131 sci Out C 28.1 22940 22729 22575 09:35 r1-00296568 6 12131 sci Out C 28.1 23258 22405 21144 09:35 r2-00296568 6 12131 sci Out C 28.1 22940 22729 22575 80 09:37 b1-00296569 6 12131 sci Out C 28.1 23258 22405 21144 09:37 b2-00296569 6 12131 sci Out C 28.1 22940 22729 22575 09:37 r1-00296569 6 12131 sci Out C 28.1 23258 22405 21144 09:37 r2-00296569 6 12131 sci Out C 28.1 22940 22729 22575 81 09:39 b1-00296570 6 12131 sci Out C 28.1 23258 22405 21144 09:39 b2-00296570 6 12131 sci Out C 28.2 22940 22729 22575 09:39 r1-00296570 6 12131 sci Out C 28.1 23258 22405 21144 09:39 r2-00296570 6 12131 sci Out C 28.2 22940 22729 22575 82 09:41 b1-00296571 6 12131 sci Out C 28.1 23258 22405 21144 09:41 b2-00296571 6 12131 sci Out C 28.2 22940 22729 22575 09:41 r1-00296571 6 12131 sci Out C 28.1 23258 22405 21144 09:41 r2-00296571 6 12131 sci Out C 28.2 22940 22729 22575 83 09:43 b1-00296572 6 12131 sci Out C 28.1 23258 22405 21144 09:43 b2-00296572 6 12131 sci Out C 28.2 22940 22729 22575 09:43 r1-00296572 6 12131 sci Out C 28.1 23258 22405 21144 09:43 r2-00296572 6 12131 sci Out C 28.2 22940 22729 22575 84 09:45 b1-00296573 6 12131 sci Out C 28.1 23258 22405 21144 09:45 b2-00296573 6 12131 sci Out C 28.1 22940 22729 22575 09:45 r1-00296573 6 12131 sci Out C 28.1 23258 22405 21144 09:45 r2-00296573 6 12131 sci Out C 28.1 22940 22729 22575 85 09:47 b1-00296574 6 12131 sci Out C 28.1 23258 22405 21144 09:47 b2-00296574 6 12131 sci Out C 28.2 22940 22729 22575 09:47 r1-00296574 6 12131 sci Out C 28.1 23258 22405 21144 09:47 r2-00296574 6 12131 sci Out C 28.2 22940 22729 22575 86 09:48 b1-00296575 6 12131 sci Out C 12.7 23258 22405 21144 09:48 b2-00296575 6 12131 sci Out C 12.6 22940 22729 22575 09:48 r1-00296575 6 12131 sci Out C 12.7 23258 22405 21144 09:48 r2-00296575 6 12131 sci Out C 12.6 22940 22729 22575 87 11:30 b1-00296576 6 12131 bia Clo C 0.0 23258 22405 21144 11:30 b2-00296576 6 12131 bia Clo C 0.0 22940 22728 22575 11:30 r1-00296576 6 12131 bia Clo C 0.0 23258 22405 21144 11:30 r2-00296576 6 12131 bia Clo C 0.0 22940 22728 22575 88 11:32 b1-00296577 6 12131 bia Clo C 0.0 23258 22405 21144 11:32 b2-00296577 6 12131 bia Clo C 0.0 22940 22728 22575 11:32 r1-00296577 6 12131 bia Clo C 0.0 23258 22405 21144 11:32 r2-00296577 6 12131 bia Clo C 0.0 22940 22728 22575 89 11:33 b1-00296578 6 12131 bia Clo C 0.0 23258 22405 21144 11:33 b2-00296578 6 12131 bia Clo C 0.0 22940 22728 22575 11:33 r1-00296578 6 12131 bia Clo C 0.0 23258 22405 21144 11:33 r2-00296578 6 12131 bia Clo C 0.0 22940 22728 22575 90 11:34 b1-00296579 6 12131 bia Clo C 0.0 23258 22405 21144 11:34 b2-00296579 6 12131 bia Clo C 0.0 22940 22728 22575 11:34 r1-00296579 6 12131 bia Clo C 0.0 23258 22405 21144 11:34 r2-00296579 6 12131 bia Clo C 0.0 22940 22728 22575 91 11:36 b1-00296580 6 12131 bia Clo C 0.0 23258 22405 21144 11:36 b2-00296580 6 12131 bia Clo C 0.0 22940 22728 22575 11:36 r1-00296580 6 12131 bia Clo C 0.0 23258 22405 21144 11:36 r2-00296580 6 12131 bia Clo C 0.0 22940 22728 22575 92 11:37 b1-00296581 6 12131 bia Clo C 0.0 23258 22405 21144 11:37 b2-00296581 6 12131 bia Clo C 0.0 22940 22728 22575 11:37 r1-00296581 6 12131 bia Clo C 0.0 23258 22405 21144 11:37 r2-00296581 6 12131 bia Clo C 0.0 22940 22728 22575 93 11:38 b1-00296582 6 12131 bia Clo C 0.0 23258 22405 21144 11:38 b2-00296582 6 12131 bia Clo C 0.0 22940 22728 22575 11:38 r1-00296582 6 12131 bia Clo C 0.0 23258 22405 21144 11:38 r2-00296582 6 12131 bia Clo C 0.0 22940 22728 22575 94 11:40 b1-00296583 6 12131 bia Clo C 0.0 23258 22405 21144 11:40 b2-00296583 6 12131 bia Clo C 0.0 22940 22728 22575 11:40 r1-00296583 6 12131 bia Clo C 0.0 23258 22405 21144 11:40 r2-00296583 6 12131 bia Clo C 0.0 22940 22728 22575 95 11:41 b1-00296584 6 12131 bia Clo C 0.0 23258 22405 21144 11:41 b2-00296584 6 12131 bia Clo C 0.0 22940 22728 22575 11:41 r1-00296584 6 12131 bia Clo C 0.0 23258 22405 21144 11:41 r2-00296584 6 12131 bia Clo C 0.0 22940 22728 22575 96 11:42 b1-00296585 6 12131 bia Clo C 0.0 23258 22405 21144 11:42 b2-00296585 6 12131 bia Clo C 0.0 22940 22728 22575 11:42 r1-00296585 6 12131 bia Clo C 0.0 23258 22405 21144 11:42 r2-00296585 6 12131 bia Clo C 0.0 22940 22728 22575 97 11:44 b1-00296586 6 12131 bia Clo C 0.0 23258 22405 21144 11:44 b2-00296586 6 12131 bia Clo C 0.0 22940 22728 22575 11:44 r1-00296586 6 12131 bia Clo C 0.0 23258 22405 21144 11:44 r2-00296586 6 12131 bia Clo C 0.0 22940 22728 22575 98 11:45 b1-00296587 6 12131 bia Clo C 0.0 23258 22405 21144 11:45 b2-00296587 6 12131 bia Clo C 0.0 22940 22728 22575 11:45 r1-00296587 6 12131 bia Clo C 0.0 23258 22405 21144 11:45 r2-00296587 6 12131 bia Clo C 0.0 22940 22728 22575 99 11:46 b1-00296588 6 12131 bia Clo C 0.0 23258 22405 21144 11:46 b2-00296588 6 12131 bia Clo C 0.0 22940 22728 22575 11:46 r1-00296588 6 12131 bia Clo C 0.0 23258 22405 21144 11:46 r2-00296588 6 12131 bia Clo C 0.0 22940 22728 22575 100 11:48 b1-00296589 6 12131 bia Clo C 0.0 23258 22405 21144 11:48 b2-00296589 6 12131 bia Clo C 0.0 22940 22728 22575 11:48 r1-00296589 6 12131 bia Clo C 0.0 23258 22405 21144 11:48 r2-00296589 6 12131 bia Clo C 0.0 22940 22728 22575 101 11:49 b1-00296590 6 12131 bia Clo C 0.0 23258 22405 21144 11:49 b2-00296590 6 12131 bia Clo C 0.0 22940 22728 22575 11:49 r1-00296590 6 12131 bia Clo C 0.0 23258 22405 21144 11:49 r2-00296590 6 12131 bia Clo C 0.0 22940 22728 22575 102 11:50 b1-00296591 6 12131 bia Clo C 0.0 23258 22405 21144 11:50 b2-00296591 6 12131 bia Clo C 0.0 22940 22728 22575 11:50 r1-00296591 6 12131 bia Clo C 0.0 23258 22405 21144 11:50 r2-00296591 6 12131 bia Clo C 0.0 22940 22728 22575 103 11:52 b1-00296592 6 12131 bia Clo C 0.0 23258 22405 21144 11:52 b2-00296592 6 12131 bia Clo C 0.0 22940 22728 22575 11:52 r1-00296592 6 12131 bia Clo C 0.0 23258 22405 21144 11:52 r2-00296592 6 12131 bia Clo C 0.0 22940 22728 22575 104 11:53 b1-00296593 6 12131 bia Clo C 0.0 23258 22405 21144 11:53 b2-00296593 6 12131 bia Clo C 0.0 22940 22728 22575 11:53 r1-00296593 6 12131 bia Clo C 0.0 23258 22405 21144 11:53 r2-00296593 6 12131 bia Clo C 0.0 22940 22728 22575 105 11:54 b1-00296594 6 12131 bia Clo C 0.0 23258 22405 21144 11:54 b2-00296594 6 12131 bia Clo C 0.0 22940 22728 22575 11:54 r1-00296594 6 12131 bia Clo C 0.0 23258 22405 21144 11:54 r2-00296594 6 12131 bia Clo C 0.0 22940 22728 22575 106 11:56 b1-00296595 6 12131 bia Clo C 0.0 23258 22405 21144 11:56 b2-00296595 6 12131 bia Clo C 0.0 22940 22728 22575 11:56 r1-00296595 6 12131 bia Clo C 0.0 23258 22405 21144 11:56 r2-00296595 6 12131 bia Clo C 0.0 22940 22728 22575 107 11:57 b1-00296596 6 12131 dar Clo C 900.0 23258 22405 21144 11:57 b2-00296596 6 12131 dar Clo C 900.0 22940 22728 22575 11:57 r1-00296596 6 12131 dar Clo C 900.0 23258 22405 21144 11:57 r2-00296596 6 12131 dar Clo C 900.0 22940 22728 22575 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - --- OK --- ---- Weather Log ---- Time Inst Temp DP Dif Humid Wind Dir Dust,1um IRSC IRSCm -------------------------------------------------------------------------------- 01:13 5-12162A 12.2 11.9 0.3 98 27 254 9999 3.0 331 1.5 apog 01:21 5-12162A 12.2 11.9 0.3 98 17 250 9999 2.9 331 1.6 apog 01:30 5-12162A 12.2 11.9 0.3 98 17 249 9999 3.0 330 1.5 apog 01:38 5-12162A 12.2 11.9 0.3 98 17 236 9999 4.5 331 1.4 apog 01:47 5-12162A 12.2 11.9 0.3 98 28 251 9999 3.8 330 1.7 apog 01:55 5-12162A 12.2 11.9 0.3 98 22 255 9999 2.7 330 1.5 apog 02:03 5-12162A 12.2 11.9 0.3 98 26 259 9999 4.6 331 1.3 apog 02:12 5-12162A 12.2 11.9 0.3 98 27 254 9999 4.5 331 1.2 apog 02:33 9-11421A 12.2 11.9 0.3 98 19 260 9999 5.8 331 1.2 apog 02:41 9-11421A 12.2 11.9 0.3 98 16 253 9999 3.7 331 1.3 apog 02:50 9-11421A 12.2 11.9 0.3 98 20 243 9999 3.1 331 1.3 apog 02:58 9-11421A 12.2 11.9 0.3 98 19 242 9999 3.2 331 1.1 apog 03:06 9-11421A 12.2 11.9 0.3 98 16 262 9999 4.3 331 1.3 apog 03:15 9-11421A 12.2 11.9 0.3 98 19 254 9999 3.2 331 1.1 apog 03:23 9-11421A 12.2 11.9 0.3 98 15 264 9999 2.7 331 1.3 apog 03:32 9-11421A 12.2 11.9 0.3 98 21 265 9999 4.2 331 1.1 apog 03:53 8-11480A 12.2 11.9 0.3 98 15 266 9999 4.8 316 1.3 apog 04:09 8-11480A 12.2 11.9 0.3 98 20 267 9999 3.9 331 1.4 apog 04:26 8-11480A 12.2 11.9 0.3 98 21 259 9999 2.8 331 1.6 apog 04:43 8-11480A 12.2 11.9 0.3 98 20 264 9999 2.6 331 1.3 apog 05:11 7-11466A 12.2 11.9 0.3 98 20 263 9999 4.2 331 1.3 apog 05:19 7-11466A 12.2 11.9 0.3 98 21 254 9999 2.7 331 1.3 apog 05:28 7-11466A 12.2 11.9 0.3 98 19 248 9999 2.8 331 1.3 apog 05:36 7-11466A 12.2 11.9 0.3 98 22 266 9999 3.1 331 1.4 apog 05:44 7-11466A 12.2 11.9 0.3 98 17 264 9999 2.9 331 1.4 apog 05:53 7-11466A 12.2 11.9 0.3 98 18 261 9999 3.2 331 1.4 apog 06:01 7-11466A 12.2 11.9 0.3 98 21 258 9999 4.6 331 1.4 apog 06:09 7-11466A 12.2 11.9 0.3 98 20 260 9999 3.8 331 1.5 apog 06:18 7-11466A 12.2 11.9 0.3 98 19 262 9999 3.1 331 1.3 apog 06:26 7-11466A 12.2 11.9 0.3 98 22 260 9999 3.1 331 1.4 apog 06:53 1-12132A 12.2 11.9 0.3 98 17 260 9999 3.0 331 1.6 apog 07:02 1-12132A 12.2 11.9 0.3 98 15 262 9999 3.9 331 1.7 apog 07:10 1-12132A 12.2 11.9 0.3 98 17 253 9999 3.7 330 1.5 apog 07:19 1-12132A 12.2 11.9 0.3 98 14 254 9999 3.5 331 1.4 apog 07:27 1-12132A 12.2 11.9 0.3 98 20 240 9999 9.3 269 1.4 apog 07:35 1-12132A 12.2 11.9 0.3 98 20 252 9999 2.8 331 1.2 apog 09:24 6-12131A 12.2 11.9 0.3 98 22 270 9999 2.8 331 1.1 apog 09:32 6-12131A 12.2 11.9 0.3 98 17 268 9999 3.4 331 1.2 apog 09:40 6-12131A 12.2 11.9 0.3 98 17 267 9999 2.6 331 1.2 apog 11:56 6-12131A 12.2 11.9 0.3 98 17 264 9999 6.3 498 1.1 boss ---- Problems in Detail ---- Information for the first incidence of MCP death. At 08:01, "gotoField" failed with an error message: 08:01:42 OBSERVER.kgrabow 155 sop f text="Failed to prepare for flats" 08:01:42 CmdDone: OBSERVER.kgrabow 155 sop gotoField arcTime=4.0 flatTime=25.0 guiderFlatTime=0.5 guiderTime=5.0 Then we noticed that MCP Menu showed that "MCP is unavailable", arc lamps are still on. Then we followed the instruction from Jeff to restart MCP process, mcp stop (it didn't do anything since mcp was not running). module load mcp/git (later on we tried module load mcp/1.2 and 1.1) mcp reset mcp start "mcp reset" returned lots of "bad file descriptor" messages, but eventually stopped. Then we entered "MCP start". Then the terminal is flooded with messages === ip480 write port : Bad file descriptor ip480 write port : Bad file descriptor ip480 write port : Bad file descriptor read_zpb : Bad file descriptor read_zpb : Bad file descriptor read_zpb : Bad file descriptor read_zpb : Bad file descriptor read_zpb : Bad file descriptor read_zpb : Bad file descriptor read_zpb : Bad file descriptor read_zpb : Bad file descriptor read_zpb : Bad file descriptor read_zpb : Bad file descriptor read_zpb : Bad file descriptor read_zpb : Bad file descriptor read_zpb : Bad file descriptor read_zpb : Bad file descriptor read_zpb : Bad file descriptor read_zpb : Bad file descriptor read_zpb : Bad file descriptor read_zpb : Bad file descriptor read_zpb : Bad file descriptor dsp_read_dm : Bad file descriptor read_zpb : Bad file descriptor read_zpb : Bad file descriptor Timeout waiting for Clear TX buffer read_zpb : Bad file descriptor read_zpb : Bad file descriptor read_zpb : Bad file descriptor ==== and kept going, never stop. control^C and control^Z would not stop the rolling message.At the mean time, "MCP is unavailable" disappeared on MCP Menu, "Restrict " came back, just appeared MCP is alive. We then "hub startNubs mcp" returned normally, "mcp info" returned correct information. However, we still could not turn lamp off. Apparently, MCP didn't come back correct. We tried to start mcp processes with different version of mcp, they all behaved the same. We called Conor but did reached him. Called Jeff, he restarted MCP with the previous version: 1.2-1-gab77|2019-10-09|17:04". Here is his message: === The MCP was rebooted after some trouble with the software. MCP was trying to use the new module load mcp feature, but did not reload the drivers correctly on reboot. Jeff was able to load the drivers and get the software started. The bootup script was reverted to always use the most recent software from the git directory. Instruction for restart MCP: The correct directions are: ssh mcp@10.25.1.200 Before you restart you could do a module load mcp/xxx to whatever revision you are on mcp status and put the status output in the log To revert: module load mcp/1.2 mcp reset mcp start To just restart the processes and not revert: mcp stop mcp start ********************************** Information for the 2nd MCP deadth. 09:48Z: MCP dead again. But this time, it behavors differently. We first noticely, we lost stars during the 3rd Apogee exposures, later on found warn messages: "09:48:36 .tcc 0 tcc w text='MOVE failed for axis: az'. "09:48:43 client.v1204.guider 89 tcc f Text="one or more critical axes cannot make it"; AxisNoSlew=TTT". However, MCP Menu still looked like MCP is alive, No "MCP is unavailable", the clock on MCP menu is running. We thought that MCP is disconnected from hub, did an "hub startNubs mcp" and "mcp info", both returned normally as showed below. ==== 09:49:24 CmdDone: OBSERVER.kpan 20 hub startNubs mcp 09:49:24 CmdStarted: 0 1384 mcp info 09:49:24 .mcp 0 mcp i yourUserNum=52786 09:49:24 .mcp 0 mcp i userId="User hub:52786, TID 0x308" 09:49:24 .hub 1384 mcp i ffsStatus=00,00,00,00,00,00,00,00 09:49:24 .hub 1384 mcp i ffsCommandedOpen=false 09:49:24 .hub 1384 mcp i ffsSelected=11 09:49:24 .hub 1384 mcp i ffLamp=0,0,0,0 09:49:24 .hub 1384 mcp i ffLampCommandedOn=false 09:49:24 .hub 1384 mcp i NeLamp=0,0,0,0 09:49:24 .hub 1384 mcp i NeLampCommandedOn=false 09:49:24 .hub 1384 mcp i HgCdLamp=0,0,0,0 09:49:24 .hub 1384 mcp i HgCdLampCommandedOn=false 09:49:24 .hub 1384 mcp i UVLampCommandedOn=false 09:49:24 .hub 1384 mcp i whtLampCommandedOn=false 09:49:24 .hub 1384 mcp i instrumentNumConsistent=true 09:49:24 .hub 1384 mcp i instrumentNum=-1 09:49:24 .hub 1384 mcp i apogeeGang=0 09:49:24 .hub 1384 mcp i azFiducialVersion="v1_140" 09:49:24 .hub 1384 mcp i altFiducialVersion="v1_140" 09:49:24 .hub 1384 mcp i rotFiducialVersion="v1_140" 09:49:24 .hub 1384 mcp i goodFiducialVersions=true 09:49:24 .hub 1384 mcp i msOnMaxCorrection=600,600,600 09:49:24 .hub 1384 mcp i minEncoderMismatch=1000,1000,1000 09:49:24 .hub 1384 mcp i plcVersion=34 09:49:24 .hub 1384 mcp i mcpVersion="1.3|2019-10-09|11:46" 09:49:24 .hub 1384 mcp i version="1.3|2019-10-09|11:46" 09:49:24 .hub 1384 mcp i cwPositions=328,332,324,311 09:49:24 .hub 1384 mcp i cwStatus=..,..,..,.. 09:49:24 .hub 1384 mcp i sp1Slithead=00,0,0 09:49:24 .hub 1384 mcp i sp2Slithead=00,0,0 09:49:24 .hub 1384 mcp i ab_status=0,0,0,14 09:49:24 .hub 1384 mcp i ab_I1_L0=0x00000000 09:49:24 .hub 1384 mcp i ab_I1_L1=0x00000000 09:49:24 .hub 1384 mcp i ab_I1_L2=0x00000000 09:49:24 .hub 1384 mcp i ab_I1_L3=0x00000000 09:49:24 .hub 1384 mcp i ab_I1_L4=0x00000000 09:49:24 .hub 1384 mcp i ab_I1_L5=0x00000000 09:49:24 .hub 1384 mcp i ab_I1_L6=0x00000000 09:49:24 .hub 1384 mcp i ab_I1_L7=0x00000000 09:49:24 .hub 1384 mcp i ab_I1_L8=0x00000000 09:49:24 .hub 1384 mcp i ab_I1_L9=0x00000000 09:49:24 .hub 1384 mcp i ab_I1_L10=0x00000000 09:49:24 .hub 1384 mcp i ab_I1_L11=0x00000000 09:49:24 .hub 1384 mcp i ab_I1_L12=0x00000000 09:49:24 .hub 1384 mcp i ab_I1_L13=0x00000000 09:49:24 .hub 1384 mcp i ab_I1_L14=0x00000000 09:49:24 .hub 1384 mcp i ab_I1_L15=0x00000000 09:49:24 .hub 1384 mcp i ab_I2_L0=0x00000000 09:49:24 .hub 1384 mcp i ab_I6_L0=0x00000000 09:49:24 .hub 1384 mcp i ab_I7_L0=0x00000000 09:49:24 .hub 1384 mcp i ab_I8_L0=0x00000000 09:49:24 .hub 1384 mcp i ab_I9_L0=0x00000000 09:49:24 .hub 1384 mcp i ab_I10_L0=0x0000cddd 09:49:24 .hub 1384 mcp i ab_O1_L1=0x00000000 09:49:24 .hub 1384 mcp i ab_O1_L5=0x00000000 09:49:24 .hub 1384 mcp i ab_O1_L6=0x00000000 09:49:24 .hub 1384 mcp i ab_O1_L9=0x00000000 09:49:24 .hub 1384 mcp i ab_O1_L10=0x00000000 09:49:24 .hub 1384 mcp i ab_O1_L14=0x00000000 09:49:24 .hub 1384 mcp i ab_O2_L0=0x00000000 09:49:24 .hub 1384 mcp i ab_O11_L0=0x00000000 09:49:24 .hub 1384 mcp i ab_O12_L0=0x00000000 09:49:24 .hub 1384 mcp : command="info" 09:49:24 CmdDone: 0 1384 mcp info 09:49:24 .mcp 0 mcp i haveSemaphore=false ======= 09:49:33 CmdStarted: OBSERVER.kpan 21 mcp info 09:49:33 OBSERVER.kpan 21 mcp i ffsStatus=00,00,00,00,00,00,00,00 09:49:33 OBSERVER.kpan 21 mcp i ffsCommandedOpen=false 09:49:33 OBSERVER.kpan 21 mcp i ffsSelected=11 09:49:33 OBSERVER.kpan 21 mcp i ffLamp=0,0,0,0 09:49:33 OBSERVER.kpan 21 mcp i ffLampCommandedOn=false 09:49:33 OBSERVER.kpan 21 mcp i NeLamp=0,0,0,0 09:49:33 OBSERVER.kpan 21 mcp i NeLampCommandedOn=false 09:49:33 OBSERVER.kpan 21 mcp i HgCdLamp=0,0,0,0 09:49:33 OBSERVER.kpan 21 mcp i HgCdLampCommandedOn=false 09:49:33 OBSERVER.kpan 21 mcp i UVLampCommandedOn=false 09:49:33 OBSERVER.kpan 21 mcp i whtLampCommandedOn=false 09:49:33 OBSERVER.kpan 21 mcp i instrumentNumConsistent=true 09:49:33 OBSERVER.kpan 21 mcp i instrumentNum=-1 09:49:33 OBSERVER.kpan 21 mcp i apogeeGang=0 09:49:33 OBSERVER.kpan 21 mcp i azFiducialVersion="v1_140" 09:49:33 OBSERVER.kpan 21 mcp i altFiducialVersion="v1_140" 09:49:33 OBSERVER.kpan 21 mcp i rotFiducialVersion="v1_140" 09:49:33 OBSERVER.kpan 21 mcp i goodFiducialVersions=true 09:49:33 OBSERVER.kpan 21 mcp i msOnMaxCorrection=600,600,600 09:49:33 OBSERVER.kpan 21 mcp i minEncoderMismatch=1000,1000,1000 09:49:33 OBSERVER.kpan 21 mcp i plcVersion=34 09:49:33 OBSERVER.kpan 21 mcp i mcpVersion="1.3|2019-10-09|11:46" 09:49:33 OBSERVER.kpan 21 mcp i version="1.3|2019-10-09|11:46" 09:49:33 OBSERVER.kpan 21 mcp i cwPositions=315,313,320,319 09:49:33 OBSERVER.kpan 21 mcp i cwStatus=..,..,..,.. 09:49:33 OBSERVER.kpan 21 mcp i sp1Slithead=00,0,0 09:49:33 OBSERVER.kpan 21 mcp i sp2Slithead=00,0,0 09:49:33 OBSERVER.kpan 21 mcp i ab_status=0,0,0,14 09:49:33 OBSERVER.kpan 21 mcp i ab_I1_L0=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_I1_L1=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_I1_L2=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_I1_L3=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_I1_L4=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_I1_L5=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_I1_L6=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_I1_L7=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_I1_L8=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_I1_L9=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_I1_L10=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_I1_L11=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_I1_L12=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_I1_L13=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_I1_L14=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_I1_L15=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_I2_L0=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_I6_L0=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_I7_L0=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_I8_L0=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_I9_L0=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_I10_L0=0x0000cddd 09:49:33 OBSERVER.kpan 21 mcp i ab_O1_L1=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_O1_L5=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_O1_L6=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_O1_L9=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_O1_L10=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_O1_L14=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_O2_L0=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_O11_L0=0x00000000 09:49:33 OBSERVER.kpan 21 mcp i ab_O12_L0=0x00000000 09:49:33 OBSERVER.kpan 21 mcp : command="info" 09:49:33 CmdDone: OBSERVER.kpan 21 mcp info 09:49:33 .mcp 0 mcp i haveSemaphore=false 09:49:33 .mcp 0 mcp i semaphoreOwner="None:0:0" ==================== Weired thing here is that: MCP 1.2 was running, but "hub startNubs mcp" without "mcp stop" neither "MCP start" before this command, we saw the MCP version is 09:49:24 .hub 1384 mcp i version="1.3|2019-10-09|11:46". Is version 1.3 still running in the background? Anyway, reconnecting MCP and tcc axis didn't work, so we went through Jeff's instruction. 1. getting status: sdss-obs2:~ kpan$ ssh mcp@10.25.1.200 mcp@10.25.1.200's password: Linux mcp 4.9.0-9-amd64 #1 SMP Debian 4.9.168-1+deb9u4 (2019-07-19) x86_64 The programs included with the Debian GNU/Linux system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. You have new mail. Last login: Thu Oct 10 03:06:18 2019 from 10.25.1.2 loaded default module in /home/mcp/sdss/tagged/1.3 mcp@mcp:~$ module load mcp/1.2 mcp@mcp:~$ mcp status comm pid 776 kill_req 0 verbose 0, running amp_mgt pid 777 kill_req 0 verbose 0, running tmove pid 778 kill_req 0 verbose 0, running tmtcc_az pid 779 kill_req 0 verbose 0, running tmtcc_alt pid 780 kill_req 0 verbose 0, running tmtcc_rot pid 781 kill_req 0 verbose 0, running tlatch pid 782 kill_req 0 verbose 0, running mei_dc pid 783 kill_req 0 verbose 0, running slc_dc pid 784 kill_req 0 verbose 0, running tbrakes pid 785 kill_req 0 verbose 0, running tcw pid 786 kill_req 0 verbose 0, running tlamps pid 787 kill_req 0 verbose 0, running tspec pid 788 kill_req 0 verbose 0, running tffs pid 789 kill_req 0 verbose 0, running tslithead pid 790 kill_req 0 verbose 0, running status of mcp message queues /STATUS mq_maxmsg: 200 mq_msgsize: 444 mq_curmsgs: 0 /TMOVE mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /TLATCH mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /AZIMUTH mq_maxmsg: 50 mq_msgsize: 408 mq_curmsgs: 0 /ALTITUDE mq_maxmsg: 50 mq_msgsize: 408 mq_curmsgs: 0 /INSTRUMENT mq_maxmsg: 50 mq_msgsize: 408 mq_curmsgs: 0 /BRAKE mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /LAMP mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /SPECTRO mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /FFS mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /COUNTER_WEIGHT mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /SLITHEAD mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 status of mcp semaphores /LATCH 1 /SDSSDC 1 /MEI 1 /SLC 0 active directory: /home/mcp/sdss/tagged/1.2/src slc_dc( 784) kill requested tbrakes( 785) kill requested tcw( 786) kill requested tlamps( 787) kill requested tspec( 788) kill requested tffs( 789) kill requested tslithead( 790) kill requested active directory: /home/mcp/sdss/tagged/1.2/src tried to stop and start MCP with the version of 1.2. mcp@mcp:~$ mcp start comm process starting amp_mgt process starting tmove process starting tmtcc_az process starting tmtcc_alt process starting tmtcc_rot process starting tlatch process starting mei_dc process starting slc_dc process starting tbrakes process starting tcw process starting tlamps process starting tspec process starting tffs process starting tslithead process starting comm pid 0 kill_req 0 no process amp_mgt pid 0 kill_req 0 no process tmove pid 0 kill_req 0 no process tmtcc_az pid 0 kill_req 0 no process tmtcc_alt pid 0 kill_req 0 no process tmtcc_rot pid 0 kill_req 0 no process tlatch pid 0 kill_req 0 no process mei_dc pid 0 kill_req 0 no process slc_dc pid 0 kill_req 0 no process tbrakes pid 0 kill_req 0 no process tcw pid 0 kill_req 0 no process tlamps pid 0 kill_req 0 no process tspec pid 0 kill_req 0 no process tffs pid 0 kill_req 0 no process tslithead pid 0 kill_req 0 no process active directory: /home/mcp/sdss/tagged/1.2/src waiting for connections on port 31013 waiting for connections on port 31011 waiting for connections on port 31012 mcp@mcp:~$ waiting for cmd pmc0 : No such file or directory Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer Timeout waiting for Clear TX buffer This timeout message scrolling by the screen, will not stop. At the same time, we even cloud not ssh to mcp@10.25.1.200. Connection refused. Then, we called JohnD, with his permission, we reset the MCP crate. After the reset, we checked the mcp status again, here is the info: 10:35:55 OBSERVER.kgrabow: mcp@mcp:~$ mcp status comm pid 783 kill_req 0 verbose 0, running amp_mgt pid 784 kill_req 0 verbose 0, running tmove pid 785 kill_req 0 verbose 0, running tmtcc_az pid 786 kill_req 0 verbose 0, running tmtcc_alt pid 787 kill_req 0 verbose 0, running tmtcc_rot pid 788 kill_req 0 verbose 0, running tlatch pid 789 kill_req 0 verbose 0, running mei_dc pid 790 kill_req 0 verbose 0, running slc_dc pid 791 kill_req 0 verbose 0, running tbrakes pid 792 kill_req 0 verbose 0, running tcw pid 793 kill_req 0 verbose 0, running tlamps pid 794 kill_req 0 verbose 0, running tspec pid 795 kill_req 0 verbose 0, running tffs pid 796 kill_req 0 verbose 0, running tslithead pid 797 kill_req 0 verbose 0, running status of mcp message queues /STATUS mq_maxmsg: 200 mq_msgsize: 444 mq_curmsgs: 0 /TMOVE mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /TLATCH mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /AZIMUTH mq_maxmsg: 50 mq_msgsize: 408 mq_curmsgs: 0 /ALTITUDE mq_maxmsg: 50 mq_msgsize: 408 mq_curmsgs: 0 /INSTRUMENT mq_maxmsg: 50 mq_msgsize: 408 mq_curmsgs: 0 /BRAKE mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /LAMP mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /SPECTRO mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /FFS mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /COUNTER_WEIGHT mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /SLITHEAD mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 status of mcp semaphores /LATCH 1 /SDSSDC 1 /MEI 1 /SLC 1 active directory: /home/mcp/sdss/tagged/1.3/src comm to stop Looks like the reset started MCP with the version 1.3. then we load the version 1.2 and restarted the processes. mcp@mcp:~$ mcp reset reinitialize the global mem and semaphores [sudo] password for mcp: /mcp_global change to uid 1000 pid 1000 create /STATUS max 200 len 444 /STATUS change to uid 1000 pid 1000 create /TMOVE max 50 len 384 /TMOVE change to uid 1000 pid 1000 create /TLATCH max 50 len 384 /TLATCH change to uid 1000 pid 1000 create /AZIMUTH max 50 len 408 /AZIMUTH change to uid 1000 pid 1000 create /ALTITUDE max 50 len 408 /ALTITUDE change to uid 1000 pid 1000 create /INSTRUMENT max 50 len 408 /INSTRUMENT change to uid 1000 pid 1000 create /BRAKE max 50 len 384 /BRAKE change to uid 1000 pid 1000 create /LAMP max 50 len 384 /LAMP change to uid 1000 pid 1000 create /SPECTRO max 50 len 384 /SPECTRO change to uid 1000 pid 1000 create /FFS max 50 len 384 /FFS change to uid 1000 pid 1000 create /SLITHEAD max 50 len 384 /SLITHEAD change to uid 1000 pid 1000 create /COUNTER_WEIGHT max 50 len 384 /COUNTER_WEIGHT change to uid 1000 pid 1000 creating semaphore /LATCH creating semaphore /SDSSDC creating semaphore /MEI creating semaphore /SLC loading config file success loading config file to global memory dsp_init Passed! linux mei driver already open dsp_reset Passed! old dsp_sample_rate 1250 old new dsp_sample_rate 160 now 5136-SD-VME: RAM window at 0E0000, registers at 1000 Data Highway Plus (DH+) / Peer Communications Link (PCL) Driver Copyright S-S Technologies Inc. 1989-1992 S-S Technologies 5136-SD/SD2/MCA card Ver. 5.43-03-05 Serial Number - 9008897 5136-SD-VME Adapter running diagnostics Loaded 7680 bytes from "../ab/sddhp.bin" Z180 ok zp=0xf00e8000, local station=0xa, local name=chasb, MOD_ID=0xa4 Data Highway Plus Diagnostic Utility Version 5.05 Copyright (c) 1989-1995 S-S Technologies Inc. Active Station List (Who's on) Node=01(Octal) 5/11 **** Node=12(Octal) TERM chasb initialized slc500 and mei-dsp loading az fit table az.dat fid_init axis 0 loading /home/mcp/az.dat loading alt fit table alt.dat fid_init axis 1 loading /home/mcp/alt.dat loading rot fit table rot.dat fid_init axis 2 loading /home/mcp/rot.dat done initializing, Run mcp start to start active directory: /home/mcp/sdss/tagged/1.2/src mcp@mcp:~$ mcp start comm process starting amp_mgt process starting tmove process starting tmtcc_az process starting tmtcc_alt process starting tmtcc_rot process starting tlatch process starting mei_dc process starting slc_dc process starting tbrakes process starting tcw process starting tlamps process starting tspec process starting tffs process starting tslithead process starting comm pid 0 kill_req 0 no process amp_mgt pid 0 kill_req 0 no process tmove pid 0 kill_req 0 no process tmtcc_az pid 0 kill_req 0 no process tmtcc_alt pid 0 kill_req 0 no process tmtcc_rot pid 0 kill_req 0 no process tlatch pid 0 kill_req 0 no process mei_dc pid 0 kill_req 0 no process slc_dc pid 0 kill_req 0 no process tbrakes pid 0 kill_req 0 no process tcw pid 0 kill_req 0 no process tlamps pid 0 kill_req 0 no process tspec pid 0 kill_req 0 no process tffs pid 0 kill_req 0 no process waiting for connections on port 31013 tslithead pid 0 kill_req 0 no process waiting for connections on port 31011 waiting for connections on port 31012 active directory: /home/mcp/sdss/tagged/1.2/src mcp@mcp:~$ waiting for cmd pmc0 : No such file or directory mcp@mcp:~$ mcp status comm pid 1007 kill_req 0 verbose 0, running amp_mgt pid 1008 kill_req 0 verbose 0, running tmove pid 1009 kill_req 0 verbose 0, running tmtcc_az pid 1010 kill_req 0 verbose 0, running tmtcc_alt pid 1011 kill_req 0 verbose 0, running tmtcc_rot pid 1012 kill_req 0 verbose 0, running tlatch pid 1013 kill_req 0 verbose 0, running mei_dc pid 1014 kill_req 0 verbose 0, running slc_dc pid 1015 kill_req 0 verbose 0, running tbrakes pid 1016 kill_req 0 verbose 0, running tcw pid 1017 kill_req 0 verbose 0, running tlamps pid 1018 kill_req 0 verbose 0, running tspec pid 1019 kill_req 0 verbose 0, running tffs pid 1020 kill_req 0 verbose 0, running tslithead pid 1021 kill_req 0 verbose 0, running status of mcp message queues /STATUS mq_maxmsg: 200 mq_msgsize: 444 mq_curmsgs: 0 /TMOVE mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /TLATCH mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /AZIMUTH mq_maxmsg: 50 mq_msgsize: 408 mq_curmsgs: 0 /ALTITUDE mq_maxmsg: 50 mq_msgsize: 408 mq_curmsgs: 0 /INSTRUMENT mq_maxmsg: 50 mq_msgsize: 408 mq_curmsgs: 0 /BRAKE mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /LAMP mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /SPECTRO mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /FFS mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /COUNTER_WEIGHT mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /SLITHEAD mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 status of mcp semaphores /LATCH 1 /SDSSDC 1 /MEI 1 /SLC 1 active directory: /home/mcp/sdss/tagged/1.2/src After restarted MCP 1.2, reconnect it to hub, reconnect axis, we found that we cloud not load cartridge. We cloud move the telescope but no fiducial crossing, no corrections. So we decided to try MCP 1.1. ====== mcp stop module load mcp/1.1 mcp@mcp:~$ mcp reset reinitialize the global mem and semaphores [sudo] password for mcp: /mcp_global change to uid 1000 pid 1000 create /STATUS max 200 len 444 /STATUS change to uid 1000 pid 1000 create /TMOVE max 50 len 384 /TMOVE change to uid 1000 pid 1000 create /TLATCH max 50 len 384 /TLATCH change to uid 1000 pid 1000 create /AZIMUTH max 50 len 408 /AZIMUTH change to uid 1000 pid 1000 create /ALTITUDE max 50 len 408 /ALTITUDE change to uid 1000 pid 1000 create /INSTRUMENT max 50 len 408 /INSTRUMENT change to uid 1000 pid 1000 create /BRAKE max 50 len 384 /BRAKE change to uid 1000 pid 1000 create /LAMP max 50 len 384 /LAMP change to uid 1000 pid 1000 create /SPECTRO max 50 len 384 /SPECTRO change to uid 1000 pid 1000 create /FFS max 50 len 384 /FFS change to uid 1000 pid 1000 create /SLITHEAD max 50 len 384 /SLITHEAD change to uid 1000 pid 1000 create /COUNTER_WEIGHT max 50 len 384 /COUNTER_WEIGHT change to uid 1000 pid 1000 creating semaphore /LATCH creating semaphore /SDSSDC creating semaphore /MEI creating semaphore /SLC loading config file start=home success loading config file to global memory dsp_init Passed! linux mei driver already open dsp_reset Passed! old dsp_sample_rate 1250 old new dsp_sample_rate 160 now 5136-SD-VME: RAM window at 0E0000, registers at 1000 Data Highway Plus (DH+) / Peer Communications Link (PCL) Driver Copyright S-S Technologies Inc. 1989-1992 S-S Technologies 5136-SD/SD2/MCA card Ver. 5.43-03-05 Serial Number - 9008897 5136-SD-VME Adapter running diagnostics Loaded 7680 bytes from "../ab/sddhp.bin" Z180 ok zp=0xf00e8000, local station=0xa, local name=chasb, MOD_ID=0xa4 Data Highway Plus Diagnostic Utility Version 5.05 Copyright (c) 1989-1995 S-S Technologies Inc. Active Station List (Who's on) Node=01(Octal) 5/11 **** Node=12(Octal) TERM chasb initialized slc500 and mei-dsp loading az fit table az.dat fid_init axis 0 loading /home/mcp/az.dat loading alt fit table alt.dat fid_init axis 1 loading /home/mcp/alt.dat loading rot fit table rot.dat fid_init axis 2 loading /home/mcp/rot.dat done initializing, Run mcp start to start active directory: /home/mcp/sdss/tagged/1.1/src mcp@mcp:~$ mcp status comm pid 1110 kill_req 0 verbose 0, running amp_mgt pid 1111 kill_req 0 verbose 0, running tmove pid 1112 kill_req 0 verbose 0, running tmtcc_az pid 1113 kill_req 0 verbose 0, running tmtcc_alt pid 1114 kill_req 0 verbose 0, running tmtcc_rot pid 1115 kill_req 0 verbose 0, running tlatch pid 1116 kill_req 0 verbose 0, running mei_dc pid 1117 kill_req 0 verbose 0, running slc_dc pid 1118 kill_req 0 verbose 0, running tbrakes pid 1119 kill_req 0 verbose 0, running tcw pid 1120 kill_req 0 verbose 0, running tlamps pid 1121 kill_req 0 verbose 0, running tspec pid 1122 kill_req 0 verbose 0, running tffs pid 1123 kill_req 0 verbose 0, running tslithead pid 1124 kill_req 0 verbose 0, running status of mcp message queues /STATUS mq_maxmsg: 200 mq_msgsize: 444 mq_curmsgs: 0 /TMOVE mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /TLATCH mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /AZIMUTH mq_maxmsg: 50 mq_msgsize: 408 mq_curmsgs: 0 /ALTITUDE mq_maxmsg: 50 mq_msgsize: 408 mq_curmsgs: 0 /INSTRUMENT mq_maxmsg: 50 mq_msgsize: 408 mq_curmsgs: 0 /BRAKE mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /LAMP mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /SPECTRO mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /FFS mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /COUNTER_WEIGHT mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 /SLITHEAD mq_maxmsg: 50 mq_msgsize: 384 mq_curmsgs: 0 status of mcp semaphores /LATCH 1 /SDSSDC 1 /MEI 1 /SLC 1 active directory: /home/mcp/sdss/tagged/1.1/src With MCP 1.1, we still cloud not load cartridge, didn't have fiducial crossing and corrections. Also, we tried to align up altitude with marks on windbaffle, MCP position was off by ~0.5 deg or so for 30 deg mark, then we slow to 55, MCP and mark aligned up there even MCP showed no fiducial crossing. Then we moved to 40, it aligned up there, then moved to 5 deg, it off by ~20 arc minutes or so. Even MCP showed the position is 5:01:00, the enclosure will not moved, Then we moved to 4:30:00, and were able to move the enclosure back on. When axis was initialed, MCP position changed by several arc minutes. I left MCP version 1.1 running at the end of night.