Subject: 2.5m obslog 20130704 (MJD 56478) From: doravetz@apo.nmsu.edu Date: Fri, 5 Jul 2013 11:38:43 +0000 (UTC) APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Thursday July 04, 2013 (MJD 56478) ---- Observing Team ---- Audrey Oravetz Dan Oravetz ---- Observing Plan ---- 4 Jul 2013 56478.0 0.18 03:37 10:39 15:26 22:30 BOSS 7.0 hours ---- Observing Summary ---- Partly cloudy skies for the first ~4 hours and then mostly clear with just a few high clouds thereafter; variable seeing, but then the seeing stabilized the last couple hours of the night. Observed the following: Cart. 12, 6985 Cart. 13, 6834 - DONE Cart. 14, 6799 - DONE Cart. 15, 6798 Cart. 17, 6299 - DONE Cart. 18, 6507 - DONE Cart. 9, 5551, 090+00_A - 2XABBA @ 500s each; 2 faint fibers in flat The handle on the Lindy cart that latches the cart in place for cartridge changes broke off. It's sitting on top of the Lindy cart in the shed. The SOS disk space is 96% full and needs attention: sdR-r1-00164224.fit.gz: WARNING: SOS disk /data/boss/sos/56478 is 96% full At 04:29Z there were three critical alerts relating to low SP1 10L (secondary) dewar pressure and temperature alerts for B1 and R1 (they were actually alerts that these cameras were too cold). Connie suspects these alerts were due to a problem with one of the SP1 power supplies: "So getting a slew of voltage errors on sp1 tells me that there was nothing wrong with the LN2 pressure or the CCD temperatures, but rather something wrong with one of the power supplies on SP1. Maybe something flaky, where a connection opened up or shorted when the telescope went to instrument change. And then fixed itself when you slewed to the field. Cables or the power supply boards or the modules themselves could do this." ---- Observing Log ---- Afternoon ========= Checked STUI actors, checked mirror numbers, checked upper and lower telescope enclosure, checked fiducials, cleaned APOGEE gang connector and took APOGEE evening cals, verified APOGEE archiving, took a few BOSS biases to verify aliveness, took guider dark sequence. Focused BOSS spectrographs using cart. 11 according to Observing Plan: Median spectrographs temperatures: Tsp1= 15.3, Tsp2= 14.5 Outside air temperature= 19.0 COLLIMATE: Camera b1 appears to be IN-FOCUS (|mean| 89.61 [above 0.50] Cart. 13, 6834 ************** 03:51Z - Applied +0.002 deg offset in altitude (calib offset) to acquire field. Partly cloudy. Seeing 1.5-2.0" FWHM. 03:52Z - Hartmanns. 03:52:55 boss exposure 4.0, file=164230 03:53:44 boss exposure 4.0, file=164231 03:54:43 hartSp2: r=1063, b=3.6, txt=OK, sp2Temp = 13.8 03:54:43 sos.sp2AverageMove = -2107 03:54:51 sp2.motor.move= -2105, -2106, -2107 03:54:51 hartSp1: r=-110, b=-0.4, txt=OK, sp1Temp = 14.4 03:54:51 sos.sp1AverageMove = -529 03:54:54 sp1.motor.move= -530, -529, -530 03:55Z - Cals. 03:55:24 boss exposure 4.0, file=164232 03:56:58 boss exposure 30.0, file=164233 04:00Z - Science. 900s exposures. 04:00:42 boss exposure 900.0, file=164234 04:17:10 boss exposure 900.0, file=164235 04:33:34 boss exposure 900.0, file=164236 MJD b1 r1 b2 r2 Texp Sum Plugfile --------------------------------------------------------------- 56471 2.88 8.01 2.57 7.10 900.8 20.56 6834-56455-01 56471 2.20 6.52 2.06 5.77 900.8 16.55 6834-56455-01 56471 1.91 5.71 1.83 4.73 900.8 14.18 6834-56455-01 56471 2.11 6.12 1.83 5.35 900.8 15.40 6834-56455-01 56478 0.83 3.14 0.74 2.88 900.8 7.59 6834-56455-01 56478 0.87 3.60 0.89 3.26 900.8 8.62 6834-56455-01 56478 1.02 3.78 0.86 3.26 900.8 8.91 6834-56455-01 --------------------------------------------------------------- s/n2= 11.82 36.87 10.77 32.36 6305.8 --> 91.82 [above 0.50] 04:49Z - During the cartridge change STUI alerted of three critical alerts. The first was issued at 04:49:27 and indicated that the SP1 10L (secondary) dewar was in a low-pressure state; STUI reported the SP1 10L gauge read 0 psi. We verified that an appropriate amount of LN2 had transferred from the 180L dewars during the force-fill after opening by checking the dewar weight tables and we are confident there was not a leak between the 180L==>10L transfer during the force-fill because the autofill cut off as expected (i.e., the 180L dewar did not continue to lose LN2 and have to be manually turned off as it would if the autofill connection weren't tight and we were losing LN2 to the atmosphere). In other words, we were certain there was LN2 in the secondary dewars. We issued a startfillseq and after its completion the critical alert disappeared. We thought perhaps the combination of the telescope at zenith along with a dewar that was actually too full (the force fill after opening occurred shortly after a 180L==>10L fill had already completed) resulted in an insufficient gas head for a pressure reading to be available, and by filling we relieved enough LN2 from the dewar to establish sufficient gas head? Of course, we didn't experience this alert during the previous cartridge change which occurred under the same set of circumstances. All told, we don't know the cause, but the alert is no longer a problem. See Problems Section for more details. The R1 and B1 critical alerts were issued at 04:50:29 and 04:50:30 and were actually due to the camera temperatures being about 20-25C _colder_ than their set point temperatures. There was a deluge of SP1 voltage warnings accompanying the R1 and B1 critical alerts. We did notice that the heater voltages went low for both R1 and B1 during the time when the cameras grew colder, but we don't know why this happened when the heaters should be on/higher voltage when LN2TEMP and cameras are colder than nominal. We suspect maybe there was some disruption in communications or something that affected the logic controlling things like the heaters turning on when they should be commanded to. See Problems Section for more details. Cart. 14, 6799 ************** 04:52Z - GSOGTF. Partly cloudy. Seeing 1.6-1.8" FWHM. 04:54Z - Hartmanns. 04:54:00 boss exposure 4.0, file=164237 04:54:50 boss exposure 4.0, file=164238 04:55:52 hartSp2: r=1183, b=4.1, txt=OK, sp2Temp = 13.7 04:55:52 sos.sp2AverageMove = 654 04:55:53 sp2.motor.move= 654, 654, 654 04:55:53 hartSp1: r=-270, b=-0.9, txt=OK, sp1Temp = 14.0 04:55:53 sos.sp1AverageMove = -689 04:55:57 sp1.motor.move= -689, -689, -687 04:56Z - Cals. 04:56:36 boss exposure 4.0, file=164239 04:58:11 boss exposure 30.0, file=164240 05:00Z - Science. 900s exposures. 05:00:35 boss exposure 900.0, file=164241 05:16:57 boss exposure 900.0, file=164242 MJD b1 r1 b2 r2 Texp Sum Plugfile --------------------------------------------------------------- 56456 1.80 6.42 1.84 6.21 900.8 16.27 6799-56456-01 56456 1.29 4.68 1.29 4.51 900.8 11.77 6799-56456-01 56477 1.89 6.21 1.96 6.01 900.8 16.07 6799-56456-01 56477 2.08 6.38 2.20 6.19 900.8 16.85 6799-56456-01 56477 1.69 5.29 1.78 5.16 900.8 13.92 6799-56456-01 56477 0.81 3.59 0.84 3.52 900.8 8.77 6799-56456-01 56478 1.03 3.77 0.99 3.63 900.8 9.42 6799-56456-01 56478 0.56 1.98 0.52 2.01 900.8 5.07 6799-56456-01 --------------------------------------------------------------- s/n2= 11.16 38.34 11.42 37.23 7206.7 --> 98.15 [above 0.50] Cart. 15, 6798 ************** 05:37Z - GSOGTF. Partly cloudy. Seeing 1.6-1.9" FWHM. 05:38Z - Cals. 05:38:42 boss exposure 4.0, file=164243 05:39:35 boss exposure 4.0, file=164244 05:40:33 hartSp1: r=-270, b=-0.9, txt=OK, sp1Temp = 13.8 05:40:33 sos.sp1AverageMove = -689 05:40:37 sp1.motor.move= -689, -689, -689 05:40:37 hartSp2: r=827, b=2.8, txt=OK, sp2Temp = 13.4 05:40:37 sos.sp2AverageMove = -1550 05:40:43 sp2.motor.move= -1551, -1550, -1550 05:45Z - Science. 900s exposures. 05:45:44 boss exposure 900.0, file=164247 06:02:06 boss exposure 900.0, file=164248 06:18:28 boss exposure 900.0, file=164249 06:34:51 boss exposure 900.0, file=164250 MJD b1 r1 b2 r2 Texp Sum Plugfile --------------------------------------------------------------- 56478 0.98 3.04 0.93 2.76 900.8 7.71 6798-56457-01 56478 2.16 6.90 2.06 6.18 900.8 17.29 6798-56457-01 56478 2.31 6.95 2.27 6.52 900.8 18.04 6798-56457-01 56478 2.66 7.24 2.51 6.79 900.8 19.19 6798-56457-01 --------------------------------------------------------------- s/n2= 8.10 24.13 7.77 22.23 3603.4 --> 62.23 [above 0.50] Cart. 17, 6299 ************** 06:52Z - GSOGTF. Mostly clear. Seeing 1.4-1.7" FWHM. 06:56Z - Hartmanns. 06:56:00 boss exposure 4.0, file=164251 06:56:50 boss exposure 4.0, file=164252 06:57:49 hartSp2: r=-117, b=-0.4, txt=OK, sp2Temp = 13.8 06:57:49 sos.sp2AverageMove = 674 06:57:52 sp2.motor.move= 677, 676, 674 06:57:53 hartSp1: r=849, b=2.9, txt=OK, sp1Temp = 14.0 06:57:53 sos.sp1AverageMove = 431 06:57:56 sp1.motor.move= 431, 434, 431 06:58Z - Cals. 06:58:17 boss exposure 4.0, file=164253 06:59:51 boss exposure 30.0, file=164254 07:02Z - Science. 900s. 07:02:18 boss exposure 900.0, file=164255 07:18:40 boss exposure 900.0, file=164256 07:35:07 boss exposure 900.0, file=164257 07:51:30 boss exposure 900.0, file=164258 MJD b1 r1 b2 r2 Texp Sum Plugfile --------------------------------------------------------------- 56455 1.48 6.45 2.17 6.64 900.8 16.74 6299-56448-01 56455 1.70 6.64 2.41 7.23 900.8 17.99 6299-56448-01 56456 0.56 2.89 0.75 2.51 900.8 6.71 6299-56448-01 56456 0.70 3.46 1.05 3.41 900.8 8.63 6299-56448-01 56456 0.81 3.85 1.20 3.96 900.8 9.82 6299-56448-01 56461 0.29 1.44 0.37 1.41 900.8 3.51 6299-56448-01 56461 0.34 1.75 0.50 1.72 900.9 4.31 6299-56448-01 56461 0.34 1.70 0.49 1.76 900.9 4.29 6299-56448-01 56477 0.12 0.93 0.17 0.90 900.8 2.11 6299-56448-01 56477 0.04 0.62 0.05 0.62 900.8 1.34 6299-56448-01 56477 0.18 1.94 0.26 2.09 900.9 4.47 6299-56448-01 56477 0.20 2.39 0.30 2.48 900.9 5.36 6299-56448-01 56478 1.08 4.12 1.36 4.20 900.8 10.77 6299-56448-01 56478 1.29 4.55 1.55 4.79 900.9 12.17 6299-56448-01 56478 1.15 4.28 1.59 4.70 900.8 11.71 6299-56448-01 56478 1.37 5.00 1.80 5.49 900.9 13.67 6299-56448-01 --------------------------------------------------------------- s/n2= 10.15 41.25 13.88 42.92 8107.6 --> 108.20 [above 0.50] Cart. 18, 6507 ************** 08:10Z - GSOGTF. Mostly clear; an occasional high cloud. Seeing 1.3-1.5" FWHM. 08:13Z - Hartmanns. 08:13:41 boss exposure 4.0, file=164259 08:14:31 boss exposure 4.0, file=164260 08:15:30 hartSp2: r=-618, b=-2.1, txt=OK, sp2Temp = 13.8 08:15:30 sos.sp2AverageMove = 1495 08:15:36 sp2.motor.move= 1495, 1495, 1495 08:15:36 hartSp1: r=820, b=2.8, txt=OK, sp1Temp = 14.4 08:15:36 sos.sp1AverageMove = -180 08:15:39 sp1.motor.move= -179, -185, -179 08:16Z - Cals. 08:16:00 boss exposure 4.0, file=164261 08:17:36 boss exposure 30.0, file=164262 08:20Z - Science. 900s exposures. 08:20:41 boss exposure 900.0, file=164263 08:37:03 boss exposure 900.0, file=164264 08:53:25 boss exposure 900.0, file=164265 09:09:50 boss exposure 900.0, file=164266 09:26:13 boss exposure 900.0, file=164267 09:42:36 boss exposure 900.0, file=164268 MJD b1 r1 b2 r2 Texp Sum Plugfile --------------------------------------------------------------- 56478 1.58 5.37 1.72 5.48 900.8 14.15 6507-56461-01 56478 1.99 6.72 2.10 7.01 900.9 17.82 6507-56461-01 56478 2.27 7.74 2.39 8.10 900.9 20.50 6507-56461-01 56478 2.40 8.34 2.49 8.77 900.9 22.00 6507-56461-01 56478 2.38 8.61 2.51 9.01 900.9 22.51 6507-56461-01 56478 2.51 9.29 2.73 9.64 900.8 24.17 6507-56461-01 --------------------------------------------------------------- s/n2= 13.13 46.07 13.94 48.02 5405.1 --> 121.16 [above 0.50] 09:59Z - APOGEE darks. Cart. 9, 5551, 090+00_A *********************** 10:05Z - GSOGTF. Clear skies. Seeing 1.3" FWHM. - Science. 2XABBA @ 500s each. 10:10:07 apogee.expose=apRaw-09160011, 47, A, Object 10:18:31 apogee.expose=apRaw-09160012, 47, B, Object 10:26:51 apogee.expose=apRaw-09160013, 47, B, Object 10:35:19 apogee.expose=apRaw-09160014, 47, A, Object 10:43:39 apogee.expose=apRaw-09160015, 47, A, Object 10:52:02 apogee.expose=apRaw-09160016, 47, B, Object 11:00:22 apogee.expose=apRaw-09160017, 47, B, Object 11:08:50 apogee.expose=apRaw-09160018, 47, A, Object 11:17Z - Dome flat. 11:17:30 apogee.expose=apRaw-09160019, 5, A, DomeFlat [observer@sos3 collimate]$ aptest 56478 09160019 flat= /data/apogee/utr_cdr/56478/apRaw-09160019.fits master= /home/observer/bin/apRaw-03720068.fits nfibers= 300 average ratio= 0.710121367436 missing fibers (flat/master -------------------------------------------------------------------------------- 03:30 12-6985A 1.000348 -3430 -1414 265 193.8 57.1 15.7 9 137 1.8 boss 04:00 13-6834A 1.000268 -2643 -748 272 212.2 53.6 15.8 10 132 2.0 boss 04:16 13-6834A 1.000277 -2736 -828 263 213.8 51.8 15.7 6 133 1.9 boss 04:32 13-6834A 1.000290 -2860 -917 268 215.1 49.8 15.6 14 142 1.5 boss 05:00 14-6799A 1.000283 -2793 -909 252 199.1 60.1 15.4 11 125 1.6 boss 05:16 14-6799A 1.000277 -2730 -845 260 202.3 58.9 15.5 4 128 1.8 boss 05:45 15-6798A 1.000335 -3306 -1339 242 202.0 58.4 15.7 0 180 1.9 boss 06:01 15-6798A 1.000336 -3312 -1365 219 204.8 57.0 15.5 3 118 1.8 boss 06:17 15-6798A 1.000324 -3198 -1268 215 207.2 55.5 15.7 1 236 1.6 boss 06:34 15-6798A 1.000328 -3233 -1301 207 209.3 53.9 16.2 2 206 1.6 boss 07:01 17-6299A 1.000332 -3277 -1298 208 93.2 42.3 15.4 5 216 1.7 boss 07:18 17-6299A 1.000310 -3061 -1141 197 91.1 45.7 16.5 6 214 1.5 boss 07:34 17-6299A 1.000298 -2940 -1052 197 88.8 49.2 16.3 7 256 1.5 boss 07:50 17-6299A 1.000284 -2809 -950 200 86.4 52.6 16.3 8 239 1.4 boss 08:20 18-6507A 1.000366 -3612 -1638 187 97.9 55.1 15.9 10 211 1.5 boss 08:36 18-6507A 1.000376 -3707 -1724 189 96.4 58.6 15.8 7 217 1.4 boss 08:52 18-6507A 1.000383 -3784 -1791 194 94.7 62.0 15.7 7 191 1.4 boss 09:09 18-6507A 1.000381 -3765 -1772 204 93.0 65.5 15.1 7 192 1.3 boss 09:25 18-6507A 1.000374 -3688 -1705 214 91.1 68.9 15.5 10 199 1.4 boss 09:42 18-6507A 1.000376 -3707 -1726 216 88.8 72.3 15.5 8 187 1.3 boss 10:09 9-5551A 1.000180 -1777 -117 225 206.4 72.0 15.3 10 180 1.3 apog 10:17 9-5551A 1.000175 -1732 -66 238 210.0 71.2 15.1 4 186 1.3 apog 10:26 9-5551A 1.000180 -1777 -104 237 213.1 70.2 15.2 11 179 1.2 apog 10:34 9-5551A 1.000182 -1796 -119 236 216.0 69.2 15.5 9 184 1.3 apog 10:43 9-5551A 1.000186 -1834 -151 232 218.4 68.2 15.3 6 190 1.3 apog 10:51 9-5551A 1.000190 -1876 -179 238 220.6 67.0 14.9 8 179 1.3 apog 10:59 9-5551A 1.000191 -1885 -172 250 222.5 65.9 14.6 10 180 1.3 apog 11:08 9-5551A 1.000198 -1949 -228 244 224.1 64.7 14.7 8 178 1.5 apog ---- Telescope Status ---- Telescope stowed at: 121.0, 30.0, 0.0 mount Instrument mounted: cartridge = 11 Counterweights at: 299 LN2 autofill systems: Connected and turned on 180L LN2 dewar scales: SP1 113.10 lbs, 19.83 psi SP2 118.60 lbs, 18.54 psi ---- Mirror Numbers ---- PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A: 5.36 5.362 Axial B: 5.206 5.212 Axial C: 5.514 5.517 Trans D: 10.357 10.353 Lateral E: 10.6426 10.6553 Lateral F: 12.2682 12.2809 GALILS Commanded: 11300 -650 5500 21900 13500 13050 Actual: 11300 -650 5500 21900 13500 13050 SETMIR VALUES PrimDesOrient: 0.00 0.00 0.00 612.35 1500.90 PrimOrient: 0.00 0.00 0.00 612.07 1500.38 SECONDARY: ---------- Focus: 0.00 Air temp: 18.5 Altitude: 30.0 MIGS TONIGHT NOMINAL Axial A: 7.042 7.036 Axial B: 7.053 7.041 Axial C: 7.04 7.031 Trans D: 0.331 0.334 GALILS Commanded: 1576550 1594600 1580000 -3550 -6650 Actual: 1576543 1594597 1580046 -3550 -6650 SETMIR VALUES SecDesOrient: 1257.00 3.00 5.00 0.00 115.81 SecOrient: 1257.01 2.98 5.01 -0.16 116.15 ---- Problems in Detail ---- STUI Errors during critical alerts: 04:49:27 .alerts 0 alerts e alert=boss.SP1SecondaryDewarPress,critical,"Reported by camCheck",enabled,noack,"" 04:49:30 .mcp 0 mcp w rotBadFiducial=-6,5.86 04:49:36 .hub 0 cmds d CmdQueued=1652685,1372999776.56,"APO.asimmons",79,"boss",1200,"sp1cam raw= startfillseq" 04:49:36 .hub 0 cmds d CmdQueued=1652686,1372999776.56,"APO.asimmons",80,"boss",1201,"sp2cam raw= startfillseq" 04:49:46 .tcc 0 tcc W AltStat=89.999625,0.000000,4879716586.53657,0x00001000; BadAltStatus; AltDTime=-0.1 04:49:48 .tcc 0 tcc W AltStat=89.999656,0.000000,4879716588.53575,0x00001800; BadAltStatus; AltDTime=-0.1 04:49:50 .tcc 0 tcc W AltStat=89.999660,0.000000,4879716590.53585,0x00001800; BadAltStatus; AltDTime=-0.1 04:49:52 .tcc 0 tcc W AltStat=89.999663,0.000000,4879716592.53602,0x00001800; BadAltStatus; AltDTime=-0.1 04:49:53 APO.asimmons 78 sop w text="in slew with badStat=2 halted=False slewing=False" 04:49:53 APO.asimmons 78 sop f text="Failed to slew to instrument change" 04:49:53 CmdDone: APO.asimmons 78 sop gotoInstrumentChange 04:49:54 .tcc 0 tcc W AltStat=89.999660,0.000000,4879716594.45827,0x00001800; BadAltStatus; AltDTime=-0.1 04:49:55 .hub 0 cmds d CmdQueued=1652697,1372999795.48,"APO.telescope",250,"tcc",18288,"axis stop" 04:49:56 .tcc 0 tcc W AltStat=89.999660,0.000000,4879716595.81581,0x00001800; BadAltStatus; AltDTime=-0.1 04:50:29 .boss 65681 boss w SP1SecondaryDewarPress=0; SP1R0CCDTempRead=-152.344; SP1R0TSetRead=-80.078; SP1R0TZeroRead=7.604; SP1R0INegTrim2Read=-5.488; SP1R0INegTrim1Read=-5.508; SP1R0IPosTrim2Read=-6.504; SP1R0IPosTrim1Read=-6.680; SP1R0VSWbRead=6.543; SP1R0VLG1Read=2.979; SP1R0VTcRead=-4.727; SP1R0VTbRead=1.953; SP1R0VPcBRead=-4.746; SP1R0VPcARead=-4.746; SP1R0VPbBRead=1.738; SP1R0VPbARead=1.836; SP1R0VSb1Read=9.082; SP1R0VSc1Read=2.012; SP1R0VRD2Read=-6.016; SP1R0VRD1Read=-5.898; SP1R0VDD2Read=-12.305; SP1R0VDD1Read=-12.129; SP1R1INegTrim4Read=-5.605; SP1R1INegTrim3Read=-5.547; SP1R1IPosTrim4Read=-6.660; SP1R1IPosTrim3Read=-6.582; SP1R1VRonRead=-7.363; SP1R1VLG2Read=2.998; SP1R1VLG3Read=2.979; SP1R1VLG4Read=0.713; SP1R1VRedPurgeRead=-9.941; SP1R1VPcCRead=-4.883; SP1R1VRedEraseRead=6.211; SP1R1VPbCRead=1.719; SP1R1VSb2Read=7.969; SP1R1VSc2Read=2.031; SP1R1VSubsRead=13.672; SP1R1VRD4Read=-5.898; SP1R1VRD3Read=-5.859; SP1R1VDD4Read=-12.070; SP1R1VDD3Read=-11.895; SP1B2CCDTempRead=-130.859; SP1B2TSetRead=47.656; SP1B2TZeroRead=10.027; SP1B2INegTrim2Read=-0.195; SP1B2INegTrim1Read=-0.176; SP1B2IPosTrim2Read=-6.523; SP1B2IPosTrim1Read=-6.172; SP1B2VSWbRead=-4.766; SP1B2VLG12Read=-2.832; SP1B2VTbRead=-5.645; SP1B2VTcRead=2.598; SP1B2VPbBRead=-5.625; SP1B2VPbARead=-5.605; SP1B2VPcBRead=2.754; SP1B2VPcARead=2.773; SP1B2VSb1Read=-4.746; SP1B2VSc1Read=-1.992; SP1B2VRonRead=1.992; SP1B2VRD2Read=6.445; SP1B2VRD1Read=6.406; SP1B2VDD2Read=15.879; SP1B2VDD1Read=15.879; SP1B3INegTrim4Read=-0.195; SP1B3INegTrim3Read=-0.176; SP1B3IPosTrim4Read=-6.270; SP1B3IPosTrim3Read=-6.191; SP1B3VRoffRead=-4.883; SP1B3VLG34Read=-2.812; SP1B3VPbDRead=-5.645; SP1B3VPbCRead=-5.723; SP1B3VPcDRead=2.695; SP1B3VPcCRead=2.715; SP1B3VSb2Read=-4.766; SP1B3VSc2Read=-1.973; SP1B3VRD4Read=6.289; SP1B3VRD3Read=5.820; SP1B3VDD4Read=15.820; SP1B3VDD3Read=15.703 04:50:29 .boss 65681 boss w camCheck=SP1SecondaryDewarPress,SP1R0CCDTempRead,SP1R0TSetRead,SP1R0TZeroRead,SP1R0INegTrim2Read,SP1R0INegTrim1Read,SP1R0IPosTrim2Read,SP1R0IPosTrim1Read,SP1R0VSWbRead,SP1R0VLG1Read,SP1R0VTcRead,SP1R0VTbRead,SP1R0VPcBRead,SP1R0VPcARead,SP1R0VPbBRead,SP1R0VPbARead,SP1R0VSb1Read,SP1R0VSc1Read,SP1R0VRD2Read,SP1R0VRD1Read,SP1R0VDD2Read,SP1R0VDD1Read,SP1R1INegTrim4Read,SP1R1INegTrim3Read,SP1R1IPosTrim4Read,SP1R1IPosTrim3Read,SP1R1VRonRead,SP1R1VLG2Read,SP1R1VLG3Read,SP1R1VLG4Read,SP1R1VRedPurgeRead,SP1R1VPcCRead,SP1R1VRedEraseRead,SP1R1VPbCRead,SP1R1VSb2Read,SP1R1VSc2Read,SP1R1VSubsRead,SP1R1VRD4Read,SP1R1VRD3Read,SP1R1VDD4Read,SP1R1VDD3Read,SP1B2CCDTempRead,SP1B2TSetRead,SP1B2TZeroRead,SP1B2INegTrim2Read,SP1B2INegTrim1Read,SP1B2IPosTrim2Read,SP1B2IPosTrim1Read,SP1B2VSWbRead,SP1B2VLG12Read,SP1B2VTbRead,SP1B2VTcRead,SP1B2VPbBRead,SP1B2VPbARead,SP1B2VPcBRead,SP1B2VPcARead,SP1B2VSb1Read,SP1B2VSc1Read,SP1B2VRonRead,SP1B2VRD2Read,SP1B2VRD1Read,SP1B2VDD2Read,SP1B2VDD1Read,SP1B3INegTrim4 Read,SP1B3INegTrim3Read,SP1B3IPosTrim4Read,SP1B3IPosTrim3Read,SP1B3VRoffRead,SP1B3VLG34Read,SP1B3VPbDRead,SP1B3VPbCRead,SP1B3VPcDRead,SP1B3VPcCRead,SP1B3VSb2Read,SP1B3VSc2Read,SP1B3VRD4Read,SP1B3VRD3Read,SP1B3VDD4Read,SP1B3VDD3Read 04:50:29 .alerts 0 alerts e alert=boss.SP1R0CCDTempRead,critical,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R0TSetRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R0TZeroRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R0INegTrim2Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R0INegTrim1Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R0IPosTrim2Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R0IPosTrim1Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R0VSWbRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R0VLG1Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R0VTcRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R0VTbRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R0VPcBRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R0VPcARead,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R0VPbBRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R0VPbARead,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R0VSb1Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R0VSc1Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R0VRD2Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R0VRD1Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R0VDD2Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R0VDD1Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R1INegTrim4Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R1INegTrim3Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R1IPosTrim4Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R1IPosTrim3Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R1VRonRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R1VLG2Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R1VLG3Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R1VLG4Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R1VRedPurgeRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R1VPcCRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:29 .alerts 0 alerts w alert=boss.SP1R1VRedEraseRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1R1VPbCRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1R1VSb2Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1R1VSc2Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1R1VSubsRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1R1VRD4Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1R1VRD3Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1R1VDD4Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1R1VDD3Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts e alert=boss.SP1B2CCDTempRead,critical,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B2TSetRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B2TZeroRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B2INegTrim2Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B2INegTrim1Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B2IPosTrim2Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B2IPosTrim1Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B2VSWbRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B2VLG12Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B2VTbRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B2VTcRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B2VPbBRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B2VPbARead,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B2VPcBRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B2VPcARead,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B2VSb1Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B2VSc1Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B2VRonRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B2VRD2Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B2VRD1Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B2VDD2Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B2VDD1Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B3INegTrim4Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B3INegTrim3Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B3IPosTrim4Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B3IPosTrim3Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B3VRoffRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B3VLG34Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B3VPbDRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B3VPbCRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B3VPcDRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B3VPcCRead,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B3VSb2Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B3VSc2Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B3VRD4Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B3VRD3Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B3VDD4Read,warn,"Reported by camCheck",enabled,noack,"" 04:50:30 .alerts 0 alerts w alert=boss.SP1B3VDD3Read,warn,"Reported by camCheck",enabled,noack,"" Timestamps for SP1 10L dewar pressure during critical alert: 2013-07-05T05:05:30.396693 10 2013-07-05T05:04:30.462909 10 2013-07-05T05:03:30.380494 8 2013-07-05T05:02:30.382552 7 2013-07-05T05:01:30.384939 2 2013-07-05T05:00:30.691549 0 2013-07-05T05:00:27.421629 0 2013-07-05T04:59:30.149737 0 2013-07-05T04:59:27.029708 0 2013-07-05T04:58:30.476640 0 2013-07-05T04:58:27.262805 0 2013-07-05T04:57:30.129933 0 2013-07-05T04:57:27.020180 0 2013-07-05T04:56:31.573765 0 2013-07-05T04:56:28.310021 0 2013-07-05T04:55:32.424575 0 2013-07-05T04:55:27.984147 0 2013-07-05T04:54:32.144990 0 2013-07-05T04:54:29.133400 0 2013-07-05T04:53:32.449380 0 2013-07-05T04:53:29.275790 0 2013-07-05T04:52:32.537990 0 2013-07-05T04:52:29.275240 0 2013-07-05T04:51:32.452690 0 2013-07-05T04:51:29.274100 0 2013-07-05T04:50:32.456250 0 2013-07-05T04:50:29.271600 0 2013-07-05T04:49:30.125470 0 2013-07-05T04:49:26.946030 0 2013-07-05T04:48:30.368930 10 2013-07-05T04:47:30.302800 10 Timestamps for R1 temperatures during critical alert: 2013-07-05T04:57:30.083910 -124.609 2013-07-05T04:56:31.530490 -124.609 2013-07-05T04:55:32.323611 -125.000 2013-07-05T04:54:32.091860 -135.156 2013-07-05T04:54:29.133400 -135.156 2013-07-05T04:53:32.400369 -139.844 2013-07-05T04:53:29.275790 -139.844 2013-07-05T04:52:32.479800 -143.750 2013-07-05T04:52:29.275240 -143.750 2013-07-05T04:51:32.398510 -149.219 2013-07-05T04:51:29.274100 -149.219 2013-07-05T04:50:32.403090 -152.344 2013-07-05T04:50:29.271600 -152.344 2013-07-05T04:49:30.079210 -151.172 2013-07-05T04:48:30.318280 -124.609 2013-07-05T04:47:30.243410 -124.609 Timestamps for B1 temperatures during critical alert: 2013-07-05T04:57:30.104760 -99.609 2013-07-05T04:56:31.551211 -99.609 2013-07-05T04:55:32.333810 -103.516 2013-07-05T04:54:32.112390 -113.281 2013-07-05T04:54:29.133400 -113.281 2013-07-05T04:53:32.420749 -115.234 2013-07-05T04:53:29.275790 -115.234 2013-07-05T04:52:32.500220 -121.094 2013-07-05T04:52:29.275240 -121.094 2013-07-05T04:51:32.419200 -124.609 2013-07-05T04:51:29.274100 -124.609 2013-07-05T04:50:32.423580 -131.641 2013-07-05T04:50:29.271600 -130.859 2013-07-05T04:49:30.100000 -99.609 2013-07-05T04:48:30.339120 -99.609 B1 Heater Volts: 2013-07-05T04:57:30.112830 7.266 2013-07-05T04:56:31.557130 7.422 2013-07-05T04:55:32.338620 7.207 2013-07-05T04:54:32.122450 5.664 2013-07-05T04:53:32.430670 6.094 2013-07-05T04:52:32.510219 5.723 2013-07-05T04:51:32.429290 4.688 2013-07-05T04:50:32.433630 4.258 2013-07-05T04:49:30.107700 5.195 2013-07-05T04:48:30.349340 7.461 2013-07-05T04:47:30.274230 7.480 2013-07-05T04:46:30.268270 7.441 2013-07-05T04:45:30.350870 7.441 R1 Heater Volts: 2013-07-05T04:59:30.106070 3.730 2013-07-05T04:58:30.358690 3.418 2013-07-05T04:57:30.094441 3.711 2013-07-05T04:56:31.540910 3.613 2013-07-05T04:55:32.328620 3.672 2013-07-05T04:54:32.102230 1.738 2013-07-05T04:53:32.410640 2.129 2013-07-05T04:52:32.490120 1.680 2013-07-05T04:51:32.408950 1.348 2013-07-05T04:50:32.413440 0.879 2013-07-05T04:49:30.089700 3.848 2013-07-05T04:48:30.328800 3.633 2013-07-05T04:47:30.253850 3.672 2013-07-05T04:46:30.252090 3.633 B1 LN2TEMP: 2013-07-05T04:57:30.104760 90.625 2013-07-05T04:56:31.551211 90.234 2013-07-05T04:55:32.333810 82.422 2013-07-05T04:54:32.112390 69.922 2013-07-05T04:53:32.420749 67.969 2013-07-05T04:52:32.500220 60.156 2013-07-05T04:51:32.419200 54.688 2013-07-05T04:50:32.423580 45.312 2013-07-05T04:49:30.100000 90.625 2013-07-05T04:48:30.339120 90.625 R1 LN2TEMP: 2013-07-05T04:57:30.083910 91.406 2013-07-05T04:56:31.530490 91.406 2013-07-05T04:55:32.323611 89.844 2013-07-05T04:54:32.091860 69.141 2013-07-05T04:53:32.400369 71.094 2013-07-05T04:52:32.479800 60.156 2013-07-05T04:51:32.398510 50.000 2013-07-05T04:50:32.403090 44.141 2013-07-05T04:49:30.079210 53.516 2013-07-05T04:48:30.318280 91.406 2013-07-05T04:47:30.243410 91.406