From: Connie Rockosi Date: Thu, 25 Oct 2018 13:08:29 -0700 Subject: Re: [sloan-25mlog 4240] 2.5m obslog 20181024 (SJD 58416) --0000000000006db702057913223d Content-Type: text/plain; charset="UTF-8" Thanks, Audrey. --Connie On Thu, Oct 25, 2018 at 1:02 PM wrote: > Afternoon check today: > > DOS reports all ok values for the 20 biases taken this morning. I just > took another bias (image 274579), r1 looks normal. Karen agrees. > Background values are around 2400. A hot column is present on the side, > but the camera looks overall healthy. I cannot explain what happened last > night, but it worked in our favor. > > Audrey > > > I am very hopeful after reading this log, and am cautiously relieved. > But > > has anyone looked at the frames to make sure they look normal, have > normal > > noise, etc? > > > > I am not really in a position to look at any frames right now, but will > > try > > as soon as I can. > > > > I'll send separate email about being soggy. > > > > --Connie > > > > On Thu, Oct 25, 2018 at 4:24 AM wrote: > > > >> > >> APACHE POINT OBSERVATORY SDSS 2.5M > >> > >> OBSERVING LOG > >> Wednesday October 24, 2018 > >> > >> (SJD 58416) > >> > >> ---- Observing Team ---- > >> > >> Elena Malanushenko > >> Audrey Oravetz > >> John Downey (support) > >> Karen Kinemuchi (support) > >> Kaike Pan (support) > >> > >> ---- Observing Plan ---- > >> > >> Wed 24 OCT 2018 58416.0 1.00 01:15 12:22 20:25 07:34 > >> Both > >> APOGEE-2 (11.1 h) > >> > >> > >> ---- Observing Summary ---- > >> > >> The night was lost due to humidity and clouds (APOGEE 11.1hrs). > >> > >> We battled with the B1 and R1 cameras warming the first few hours of the > >> night > >> while we were closed to weather. Power cycling the spectrogaphs helped > >> until > >> JohnD arrived back on site. He thoroughly cleaned the problematic > >> connector > >> (same one stated within the engineering log) that led into the R1 > >> saddlebag and > >> also wrapped additional insulation tape on the nitrogen pipes next to > >> the > >> saddlebag to help prevent more water seeping into the electronics. > >> > >> R1 was still showing unhealthy bias levels throughout the night, up > >> until > >> about > >> 10:00Z when we had a momentary warming of R1 while the boss-icc suddenly > >> died. > >> We restarted the boss-ICC, reinitialized the temperatures/voltages, and > >> issued a > >> primary fill and magically R1 bias levels returned to normal. See DOS > >> for > >> the > >> bias level progression. More investigation is needed as to what caused > >> this > >> issue in the first place and if it will happen again, but at the time of > >> this > >> writing, the camera seems to be operating normally. > >> > >> > >> ---- Observing Log ---- > >> > >> > ============================================================================ > >> > >> AFTERNOON CHECKOUT > >> > >> > ============================================================================ > >> The SP1 warming troubles this morning are not quite resolved yet. > >> John continue to work with it. > >> > >> All other systems are ok. > >> > >> John could fix temp for sp1 cameras. After he left, > >> b1 and r1 began to warm again. > >> I have applied "boss iack" and "boss meckIack", and it looks > >> helped. Audrey investigated how we can dry connectors our self. > >> > >> Restarted alerts actor. > >> > >> Sp1 starts to misbehavour again. > >> see sp1cam tcheck in problem section. > >> John recommended to power cycle sp1, we did. > >> Applied "boss iack" and boss mechIack. > >> The sp1 temp for b1 and r1 are up and down. > >> Disabled sp1 alerts while we are waiting for John for help. > >> > >> Critical boss.SP1B2CCDTempRead OBSERVER.elenam > >> Critical boss.SP1MicroNotTalking OBSERVER.elenam > >> Critical boss.SP1R0CCDTempRead OBSERVER.elenam > >> Serious boss.SP1ExecBoot OBSERVER.elenam > >> Serious boss.SP1PhaseBoot OBSERVER.elenam > >> > >> Boss exposure bias: > >> 01:24:18 boss exposure 0.0, file=274541, C > >> > >> sop doBossCalibs nbias=1 ndark=1 darktime=20 > >> 01:26:14 boss exposure 0.0, file=274542, C > >> 01:27:43 boss exposure 20.0, file=274543, C > >> And we got DOS response that these sp1 data for these > >> exposures are not in spec. > >> John was working with SP1, and at 02:15 it looks ok. > >> > >> Clouds, and high humidity. > >> Did not open on time. > >> > >> Night > >> ========== > >> sop collimateBoss > >> ========== > >> 03:10:00 OBSERVER.elenam.sop 18 hartmann e text="r1 had error: THERE > >> DOES > >> NOT > >> APPEAR TO BE ANY LIGHT FROM THE ARCS IN r1!!!" > >> 03:10:00 OBSERVER.elenam.sop 18 hartmann e text="Collimation calculation > >> failed! > >> Collimation calculation failed for r1." > >> > >> 03:10:00 OBSERVER.elenam.sop 18 hartmann i r2MeanOffset=0.10,"In Focus" > >> 03:10:00 OBSERVER.elenam.sop 18 hartmann i r2PistonMove=631 > >> 03:10:00 OBSERVER.elenam.sop 18 hartmann i b1MeanOffset=-0.09,"In Focus" > >> 03:10:00 OBSERVER.elenam.sop 18 hartmann i b1RingMove=1.8 > >> 03:10:00 OBSERVER.elenam.sop 18 hartmann i b2MeanOffset=0.00,"In Focus" > >> 03:10:00 OBSERVER.elenam.sop 18 hartmann i b2RingMove=0.0 > >> > >> Checked R1 - and all data zero. > >> > >> Called Kaike, got instruction to skip Manga during night if we have > >> chanse to open. And monitor SP1 temperature and vacuum BLU_ION_PUMP_LOGP > >> during the night, and call John for instruction in case we notice > >> vacuum drop during the night. > >> > >> Checked out the telescope. All ok. > >> > >> 09:43Z - 09:40:05 .alerts 0 alerts e alert=boss.heartbeat,critical,"The > >> boss > >> ICC's been dead for at least 791s",enabled,noack,"" > >> Restarted the boss-icc. Momentary warming of r1. boss iack. Restarted > >> alert actor. > >> > >> Magic! R1 is now reading appropriate bias levels all of the sudden.... > >> > >> Continuing with afternoon checkout... > >> > >> 09:53:38 boss exposure 4.0, file=274552, C > >> 09:54:49 boss exposure 4.0, file=274553, C > >> 09:56:13 sp1.motor.move= -128, -128, -128 > >> 09:56:17 sp2.motor.move= 157, 156, 157 > >> 09:56:17 Hartmann collimate output on cart #1 > >> sp1: offset: r= 0 (In Focus); b= 0.8 (In Focus) > >> sp2: offset: r= 315 (In Focus); b= 0.0 (In Focus) > >> sp1: pred. move: spAverageMove= -128 > >> sp2: pred. move: spAverageMove= 157 > >> sp1: pred. resid: r= 128, b= 0.4, txt= OK, spTemp = 10.0 > >> sp2: pred. resid: r= 157, b= 0.5, txt= OK, spTemp = 9.7 > >> > >> 11:11Z - The humidity began to get slightly better, but it was still too > >> variable to open safely. We will try again tomorrow night. We started > >> APOGEE and > >> BOSS cals. > >> > >> > >> ---- Mirror Numbers ---- > >> > >> > >> PRIMARY: > >> -------- > >> Scale: 1.000000; > >> > >> MIGS TONIGHT NOMINAL > >> Axial A: -5.315 -5.314 > >> Axial B: -5.074 -5.069 > >> Axial C: -5.338 -5.333 > >> Trans D: 10.738 10.738 > >> Lateral E: 9.0805 9.0805 > >> Lateral F: 14.224 14.2113 > >> > >> GALILS > >> Commanded: 10250 -1100 7100 16850 12400 12400 > >> Actual: 10250 -1100 7100 16850 12400 12400 > >> > >> SETMIR VALUES > >> PrimDesOrient: 0.0 3.27 2.1 115.0 -178.65 > >> PrimOrient: -1.06 3.29 2.19 115.07 -177.78 > >> > >> SECONDARY: > >> ---------- > >> > >> Focus: 0.0000 > >> Air temp: 6.6 > >> Altitude: 30.0 > >> > >> MIGS TONIGHT NOMINAL > >> Axial A: -0.702 -0.704 > >> Axial B: -0.7 -0.703 > >> Axial C: -0.09 -0.096 > >> Trans D: -0.11 -0.15 > >> > >> GALILS > >> Commanded: 864500 869150 826700 -5750 -7450 > >> Actual: 834083 855053 794435 -5750 -7450 > >> > >> SETMIR VALUES > >> SecDesOrient: 657.0 18.54 2.85 -17.2 138.98 > >> SecOrient: 656.98 18.55 2.84 -17.46 138.66 > >> > >> > >> > >> ---- Telescope Status ---- > >> > >> Status at: 11:12Z > >> Telescope stowed at: 121.0, 30.0, 0.0 mount > >> Instrument mounted: cartridge = 1 > >> Counterweights at: 425 > >> LN2 autofill systems: Connected and turned on > >> -180L LN2 dewar scales: > >> SP1 208.3 lbs, 22.6 psi > >> SP2 171.8 lbs, 22.5 ps > >> > >> ---- Software Used ---- > >> > >> Get Versions, 10/24/18, 23:02:34 > >> -- STUI software: > >> STUI : 1.7.3 > >> alerts : trunk > >> apo : trunk > >> boss : v4_0 > >> -boss.specMec : 3.0.4 > >> gcamera : v2_2 > >> guider : 3.9.0 > >> hartmann : 1.6.0 > >> hub : Tag: v4.2 > >> mcp : v7_0_4 > >> -mcp.plc : 34 > >> -mcp.azFiduci : v1_132 > >> -mcp.altFiduc : v1_132 > >> -mcp.rotFiduc : v1_132 > >> platedb : 2.8.0 > >> sop : 3.12.0 > >> tcc : 1.3.1 > >> apogee : not availble > >> apogeecal : not availble > >> apogeeql : v1_6_1 > >> APO-local : trunk > >> -- Other versions ---- > >> idlspec2d : v5_7_12 > >> plate-mapper3 : NOCVS:v6_0_11 > >> petunia : 1.2.0 > >> autoscheduler : 1.0.1/ > >> sdss_python_module: trunk > >> mangadrp : trunk exported > >> > >> ---- APOGEE Data Summary ---- > >> > >> APOGEE data list, mjd=58416, all > >> raw_data: /data/apogee/utr_cdr/58416/apRaw-*.fits > >> archive: /data/apogee/archive/58416/apR-[a,b,c]-*.apz > >> quickred: /data/apogee/archive/58416/apR-[a,b,c]-*.apz > >> > -------------------------------------------------------------------------------- > >> > >> mjd i UT File/Exp Imtype Nread A/B Dth Ct-Plate Arch Offset > >> %Flux Comm > >> > >> > -------------------------------------------------------------------------------- > >> 58416 1 22:59 28540001 Dark 3 A 13.0 1-11261 a-b-c - > >> - > >> 58416 2 23:05 28540002 Dark 60 A 13.0 1-11261 a-b-c - > >> - > >> 58416 3 23:16 28540003 QuarFlat 10 A 13.0 1-11261 a-b-c - > >> 46 > >> 58416 4 23:18 28540004 Arc-Thar 12 A 13.0 1-11261 a-b-c 0.04 > >> - > >> 58416 5 23:20 28540005 Arc-Une 40 A 13.0 1-11261 a-b-c - > >> - > >> 58416 6 23:28 28540006 Arc-Thar 12 B 13.5 1-11261 a-b-c 0.52 > >> - > >> 58416 7 23:30 28540007 Arc-Une 40 B 13.5 1-11261 a-b-c - > >> - > >> 58416 8 23:37 28540008 Dark 60 A 13.0 1-11261 a-b-c - > >> - > >> > -------------------------------------------------------------------------------- > >> > >> > >> > >> > >> ---- MaNGA Data Summary ---- > >> > >> Reading FITS from /data/spectro/58416/ > >> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > >> n time filename crt plt flav Shu Dth exp colA colB colC Hrt > >> b1 > >> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > >> 1 00:02 b1-00274537 1 11261 bia Clo C 0.0 23731 23307 22329 > >> 00:02 b2-00274537 1 11261 bia Clo C 0.0 18613 18973 18504 > >> 00:02 r1-00274537 1 11261 bia Clo C 0.0 23731 23307 22329 > >> 00:02 r2-00274537 1 11261 bia Clo C 0.0 18613 18973 18504 > >> 2 00:03 b1-00274538 1 11261 dar Clo C 20.0 23731 23307 22329 > >> 00:03 b2-00274538 1 11261 dar Clo C 20.0 18613 18973 18504 > >> 00:03 r1-00274538 1 11261 dar Clo C 20.0 23731 23307 22329 > >> 00:03 r2-00274538 1 11261 dar Clo C 20.0 18613 18973 18504 > >> 3 00:08 b1-00274539 1 11261 arc Lef C 4.1 23731 23307 22329 > >> HART > >> 00:08 b2-00274539 1 11261 arc Lef C 4.2 18613 18973 18504 > >> HART > >> 00:08 r1-00274539 1 11261 arc Lef C 4.1 23731 23307 22329 > >> HART > >> 00:08 r2-00274539 1 11261 arc Lef C 4.2 18613 18973 18504 > >> HART > >> 4 00:35 b1-00274540 1 11261 arc Lef C 4.1 23731 23307 22329 > >> HART > >> 00:35 b2-00274540 1 11261 arc Lef C 4.2 18613 18973 18504 > >> HART > >> 00:35 r1-00274540 1 11261 arc Lef C 4.1 23731 23307 22329 > >> HART > >> 00:35 r2-00274540 1 11261 arc Lef C 4.2 18613 18973 18504 > >> HART > >> 5 01:24 b1-00274541 1 11261 bia Clo C 0.0 23731 23307 22329 > >> 01:24 b2-00274541 1 11261 bia Clo C 0.0 18613 18973 18504 > >> 01:24 r1-00274541 1 11261 bia Clo C 0.0 23731 23307 22329 > >> 01:24 r2-00274541 1 11261 bia Clo C 0.0 18613 18973 18504 > >> 6 01:26 b1-00274542 1 11261 bia Clo C 0.0 23731 23307 22329 > >> 01:26 b2-00274542 1 11261 bia Clo C 0.0 18613 18973 18504 > >> 01:26 r1-00274542 1 11261 bia Clo C 0.0 23731 23307 22329 > >> 01:26 r2-00274542 1 11261 bia Clo C 0.0 18613 18973 18504 > >> 7 01:27 b1-00274543 1 11261 dar Clo C 20.0 23731 23307 22329 > >> 01:27 b2-00274543 1 11261 dar Clo C 20.0 18613 18973 18504 > >> 01:27 r1-00274543 1 11261 dar Clo C 20.0 23731 23307 22329 > >> 01:27 r2-00274543 1 11261 dar Clo C 20.0 18613 18973 18504 > >> 8 03:03 b1-00274544 1 11261 bia Clo C 0.0 23731 23307 22329 > >> 03:03 b2-00274544 1 11261 bia Clo C 0.0 18613 18973 18504 > >> 03:03 r1-00274544 1 11261 bia Clo C 0.0 23731 23307 22329 > >> 03:03 r2-00274544 1 11261 bia Clo C 0.0 18613 18973 18504 > >> 9 03:07 b1-00274545 1 11261 arc Lef C 4.1 23731 23307 22329 > >> HART > >> 03:07 b2-00274545 1 11261 arc Lef C 4.2 18613 18973 18504 > >> HART > >> 03:07 r1-00274545 1 11261 arc Lef C 4.1 23731 23307 22329 > >> HART > >> 03:07 r2-00274545 1 11261 arc Lef C 4.2 18613 18973 18504 > >> HART > >> 10 03:08 b1-00274546 1 11261 arc Rig C 4.1 23731 23307 22329 > >> HART > >> 03:08 b2-00274546 1 11261 arc Rig C 4.2 18613 18973 18504 > >> HART > >> 03:08 r1-00274546 1 11261 arc Rig C 4.1 23731 23307 22329 > >> HART > >> 03:08 r2-00274546 1 11261 arc Rig C 4.2 18613 18973 18504 > >> HART > >> 11 03:25 b1-00274547 1 11261 bia Clo C 0.0 23731 23307 22329 > >> 03:25 b2-00274547 1 11261 bia Clo C 0.0 18613 18973 18504 > >> 03:25 r1-00274547 1 11261 bia Clo C 0.0 23731 23307 22329 > >> 03:25 r2-00274547 1 11261 bia Clo C 0.0 18613 18973 18504 > >> 12 05:14 b1-00274548 1 11261 fla Out C 25.1 23731 23307 22329 > >> 05:14 b2-00274548 1 11261 fla Out C 25.2 18613 18973 18504 > >> 05:14 r1-00274548 1 11261 fla Out C 25.1 23731 23307 22329 > >> 05:14 r2-00274548 1 11261 fla Out C 25.2 18613 18973 18504 > >> 13 09:43 b1-00274549 1 11261 bia Clo C 0.0 23731 23307 22329 > >> 09:43 b2-00274549 1 11261 bia Clo C 0.0 18613 18973 18504 > >> 09:43 r1-00274549 1 11261 bia Clo C 0.0 23731 23307 22329 > >> 09:43 r2-00274549 1 11261 bia Clo C 0.0 18613 18973 18504 > >> 14 09:46 b1-00274550 1 11261 bia Clo C 0.0 23731 23307 22329 > >> 09:46 b2-00274550 1 11261 bia Clo C 0.0 18613 18973 18504 > >> 09:46 r1-00274550 1 11261 bia Clo C 0.0 23731 23307 22329 > >> 09:46 r2-00274550 1 11261 bia Clo C 0.0 18613 18973 18504 > >> 15 09:47 b1-00274551 1 11261 dar Clo C 20.0 23731 23307 22329 > >> 09:47 b2-00274551 1 11261 dar Clo C 20.0 18613 18973 18504 > >> 09:47 r1-00274551 1 11261 dar Clo C 20.0 23731 23307 22329 > >> 09:47 r2-00274551 1 11261 dar Clo C 20.0 18613 18973 18504 > >> 16 09:53 b1-00274552 1 11261 arc Lef C 4.1 23731 23307 22329 > >> HART > >> 09:53 b2-00274552 1 11261 arc Lef C 4.2 18613 18973 18504 > >> HART > >> 09:53 r1-00274552 1 11261 arc Lef C 4.1 23731 23307 22329 > >> HART > >> 09:53 r2-00274552 1 11261 arc Lef C 4.2 18613 18973 18504 > >> HART > >> 17 09:54 b1-00274553 1 11261 arc Rig C 4.1 23731 23307 22329 > >> HART > >> 09:54 b2-00274553 1 11261 arc Rig C 4.2 18613 18973 18504 > >> HART > >> 09:54 r1-00274553 1 11261 arc Rig C 4.1 23731 23307 22329 > >> HART > >> 09:54 r2-00274553 1 11261 arc Rig C 4.2 18613 18973 18504 > >> HART > >> 18 09:59 b1-00274554 1 11261 fla Out C 25.1 23603 23179 22201 > >> 09:59 b2-00274554 1 11261 fla Out C 25.2 18770 19129 18661 > >> 09:59 r1-00274554 1 11261 fla Out C 25.1 23603 23179 22201 > >> 09:59 r2-00274554 1 11261 fla Out C 25.2 18770 19129 18661 > >> 19 10:02 b1-00274555 1 11261 arc Out C 4.1 23603 23179 22201 > >> 10:02 b2-00274555 1 11261 arc Out C 4.2 18770 19129 18661 > >> 10:02 r1-00274555 1 11261 arc Out C 4.1 23603 23179 22201 > >> 10:02 r2-00274555 1 11261 arc Out C 4.2 18770 19129 18661 > >> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > >> --- OK --- > >> > >> > >> ---- Problems in Detail ---- > >> > >> Boss sp1cam tcheck: > >> 00:18:56 CmdStarted: OBSERVER.elenam 134 boss sp1cam raw=1 tcheck 2 > >> tcheck > >> > >> 00:18:56 OBSERVER.elenam 134 boss i text="Sending raw command 1 TCHECK 2 > >> TCHECK > >> to sp1cam" > >> 00:18:56 OBSERVER.elenam 135 boss i text="Sending raw command 1 TCHECK 2 > >> TCHECK > >> to sp2cam" > >> 00:18:58 OBSERVER.elenam 134 boss i sp1camRawText="SP1R0 LN2TEMP > >> 92.6K" > >> 00:18:58 OBSERVER.elenam 134 boss i sp1camRawText="CCD_TEMP: DAC SET AT > >> 160 = > >> 1.26 V = -126 C MEASURED -117.2C" > >> 00:18:58 OBSERVER.elenam 134 boss i sp1camRawText="HEATER_VOLTS -0.56 > >> V" > >> 00:18:58 OBSERVER.elenam 134 boss i sp1camRawText="" > >> 00:18:58 OBSERVER.elenam 134 boss i sp1camRawText="SP1B2 LN2TEMP > >> 94.9K" > >> 00:18:58 OBSERVER.elenam 134 boss i sp1camRawText="CCD_TEMP: DAC SET AT > >> 127 = > >> -0.04 V = -100 C MEASURED -95.3C" > >> 00:18:58 OBSERVER.elenam 134 boss i sp1camRawText="HEATER_VOLTS -0.54 > >> V" > >> 00:18:58 OBSERVER.elenam 134 boss i sp1camRawText=" ok" > >> --------------------------- > >> > >> > >> > >> > >> > >> ____________________________________________________________ > >> Logs of Nightly APO Activities > >> > https://trac.sdss.org/tracmailman/browser/private/sloan-25mlog/index.html > >> > > > > > --0000000000006db702057913223d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks, Audrey.--Connie= On Thu, Oct 25, 2018 at 1= :02 PM wrote:Afternoon check today= : DOS reports all ok values for the 20 biases taken this morning. I just took another bias (image 274579), r1 looks normal. Karen agrees. Background values are around 2400. A hot column is present on the side, but the camera looks overall healthy. I cannot explain what happened last night, but it worked in our favor. Audrey > I am very hopeful after reading this log, and am cautiously relieved.= =C2=A0 But > has anyone looked at the frames to make sure they look normal, have no= rmal > noise, etc? > > I am not really in a position to look at any frames right now, but wil= l > try > as soon as I can. > > I'll send separate email about being soggy. > > --Connie > > On Thu, Oct 25, 2018 at 4:24 AM wrote: > >> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 APACHE POINT OBSERVATORY SDSS 2.5M >> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 OBSERVING LOG >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Wednesday October 24, 2018 >> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(SJD 58416) >> >> ----=C2=A0 Observing Team ---- >> >> Elena Malanushenko >> Audrey Oravetz >> John Downey (support) >> Karen Kinemuchi (support) >> Kaike Pan (support) >> >> ---- Observing Plan ---- >> >> Wed 24 OCT 2018=C2=A0 =C2=A0 58416.0=C2=A0 =C2=A01.00=C2=A0 =C2=A0= 01:15=C2=A0 12:22=C2=A0 =C2=A0 20:25=C2=A0 07:34 >>=C2=A0 Both >> APOGEE-2 (11.1 h) >> >> >> ---- Observing Summary ---- >> >> The night was lost due to humidity and clouds (APOGEE 11.1hrs). >> >> We battled with the B1 and R1 cameras warming the first few hours = of the >> night >> while we were closed to weather. Power cycling the spectrogaphs he= lped >> until >> JohnD arrived back on site. He thoroughly cleaned the problematic >> connector >> (same one stated within the engineering log) that led into the R1 >> saddlebag and >> also wrapped additional insulation tape on the nitrogen pipes next= to >> the >> saddlebag to help prevent more water seeping into the electronics.= >> >> R1 was still showing unhealthy bias levels throughout the night, u= p >> until >> about >> 10:00Z when we had a momentary warming of R1 while the boss-icc su= ddenly >> died. >> We restarted the boss-ICC, reinitialized the temperatures/voltages= , and >> issued a >> primary fill and magically R1 bias levels returned to normal. See = DOS >> for >> the >> bias level progression. More investigation is needed as to what ca= used >> this >> issue in the first place and if it will happen again, but at the t= ime of >> this >> writing, the camera seems to be operating normally. >> >> >> ---- Observing Log ---- >> >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D >> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0AFTERNOON CHECKOUT >> >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D >> The SP1 warming troubles this morning are not quite resolved yet. >> John continue to work with it. >> >> All other systems are ok. >> >> John=C2=A0 could fix temp for sp1 cameras. After he left, >> b1 and r1 began to warm=C2=A0 again. >> I have applied "boss iack" and "boss meckIack"= , and it looks >> helped. Audrey investigated how we can dry connectors our self. >> >> Restarted alerts actor. >> >> Sp1 starts to misbehavour again. >> see sp1cam tcheck in problem section. >> John recommended to=C2=A0 power cycle sp1, we did. >> Applied "boss iack" and boss mechIack. >> The sp1 temp for b1 and r1 are up and down. >> Disabled sp1 alerts while we are waiting for John for help. >> >> Critical=C2=A0 =C2=A0 =C2=A0 =C2=A0 boss.SP1B2CCDTempRead=C2=A0 = =C2=A0OBSERVER.elenam >> Critical=C2=A0 =C2=A0 =C2=A0 =C2=A0 boss.SP1MicroNotTalking OBSERV= ER.elenam >> Critical=C2=A0 =C2=A0 =C2=A0 =C2=A0 boss.SP1R0CCDTempRead=C2=A0 = =C2=A0OBSERVER.elenam >> Serious=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0boss.SP1ExecBoot=C2=A0 = =C2=A0 =C2=A0 =C2=A0 OBSERVER.elenam >> Serious=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0boss.SP1PhaseBoot=C2=A0 = =C2=A0 =C2=A0 =C2=A0OBSERVER.elenam >> >> Boss exposure bias: >> 01:24:18=C2=A0 boss exposure=C2=A0 =C2=A0 0.0, file=3D274541, C >> >> sop doBossCalibs nbias=3D1 ndark=3D1 darktime=3D20 >> 01:26:14=C2=A0 boss exposure=C2=A0 =C2=A0 0.0, file=3D274542, C >> 01:27:43=C2=A0 boss exposure=C2=A0 =C2=A020.0, file=3D274543, C >> And we got DOS response that these sp1 data for these >> exposures are not in spec. >> John was working with SP1, and at 02:15 it looks ok. >> >> Clouds, and high humidity. >> Did not open on time. >> >> Night >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> sop collimateBoss >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> 03:10:00 OBSERVER.elenam.sop 18 hartmann e text=3D"r1 had err= or: THERE >> DOES >> NOT >> APPEAR TO BE ANY LIGHT FROM THE ARCS IN r1!!!" >> 03:10:00 OBSERVER.elenam.sop 18 hartmann e text=3D"Collimatio= n calculation >> failed! >> Collimation calculation failed for r1." >> >> 03:10:00 OBSERVER.elenam.sop 18 hartmann i r2MeanOffset=3D0.10,&qu= ot;In Focus" >> 03:10:00 OBSERVER.elenam.sop 18 hartmann i r2PistonMove=3D631 >> 03:10:00 OBSERVER.elenam.sop 18 hartmann i b1MeanOffset=3D-0.09,&q= uot;In Focus" >> 03:10:00 OBSERVER.elenam.sop 18 hartmann i b1RingMove=3D1.8 >> 03:10:00 OBSERVER.elenam.sop 18 hartmann i b2MeanOffset=3D0.00,&qu= ot;In Focus" >> 03:10:00 OBSERVER.elenam.sop 18 hartmann i b2RingMove=3D0.0 >> >> Checked R1 - and all data zero. >> >> Called Kaike, got instruction to skip Manga during night if we hav= e >> chanse to open. And monitor SP1 temperature and vacuum BLU_ION_PUM= P_LOGP >> during the night, and call John for instruction in case we notice >> vacuum drop during the night. >> >> Checked out the telescope. All ok. >> >> 09:43Z - 09:40:05 .alerts 0 alerts e alert=3Dboss.heartbeat,critic= al,"The >> boss >> ICC's been dead for at least 791s",enabled,noack,"&q= uot; >> Restarted the boss-icc. Momentary warming of r1. boss iack. Restar= ted >> alert actor. >> >> Magic! R1 is now reading appropriate bias levels all of the sudden= .... >> >> Continuing with afternoon checkout... >> >> 09:53:38=C2=A0 boss exposure=C2=A0 =C2=A0 4.0, file=3D274552, C >> 09:54:49=C2=A0 boss exposure=C2=A0 =C2=A0 4.0, file=3D274553, C >> 09:56:13=C2=A0 sp1.motor.move=3D -128, -128, -128 >> 09:56:17=C2=A0 sp2.motor.move=3D 157, 156, 157 >> 09:56:17 Hartmann collimate output on cart #1 >>=C2=A0 =C2=A0 sp1: offset: r=3D 0 (In Focus);=C2=A0 b=3D 0.8 (In Fo= cus) >>=C2=A0 =C2=A0 sp2: offset: r=3D 315 (In Focus);=C2=A0 b=3D 0.0 (In = Focus) >>=C2=A0 =C2=A0 sp1: pred. move: spAverageMove=3D -128 >>=C2=A0 =C2=A0 sp2: pred. move: spAverageMove=3D 157 >>=C2=A0 =C2=A0 sp1: pred. resid: r=3D 128, b=3D 0.4, txt=3D OK, spTe= mp =3D 10.0 >>=C2=A0 =C2=A0 sp2: pred. resid: r=3D 157, b=3D 0.5, txt=3D OK, spTe= mp =3D 9.7 >> >> 11:11Z - The humidity began to get slightly better, but it was sti= ll too >> variable to open safely. We will try again tomorrow night. We star= ted >> APOGEE and >> BOSS cals. >> >> >> ---- Mirror Numbers ---- >> >> >> PRIMARY: >> -------- >> Scale: 1.000000; >> >> MIGS=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 TONIGHT NOMINAL >> Axial A:=C2=A0 =C2=A0 =C2=A0 =C2=A0 -5.315=C2=A0 -5.314 >> Axial B:=C2=A0 =C2=A0 =C2=A0 =C2=A0 -5.074=C2=A0 -5.069 >> Axial C:=C2=A0 =C2=A0 =C2=A0 =C2=A0 -5.338=C2=A0 -5.333 >> Trans D:=C2=A0 =C2=A0 =C2=A0 =C2=A0 10.738=C2=A0 10.738 >> Lateral E:=C2=A0 =C2=A0 =C2=A0 9.0805=C2=A0 9.0805 >> Lateral F:=C2=A0 =C2=A0 =C2=A0 14.224=C2=A0 14.2113 >> >> GALILS >> Commanded:=C2=A0 10250 -1100 7100 16850 12400 12400 >> Actual:=C2=A0 =C2=A0 =C2=A010250 -1100 7100 16850 12400 12400 >> >> SETMIR VALUES >> PrimDesOrient: 0.0 3.27 2.1 115.0 -178.65 >> PrimOrient:=C2=A0 =C2=A0 -1.06 3.29 2.19 115.07 -177.78 >> >> SECONDARY: >> ---------- >> >> Focus:=C2=A0 =C2=A0 =C2=A0 0.0000 >> Air temp:=C2=A0 =C2=A06.6 >> Altitude:=C2=A0 =C2=A030.0 >> >> MIGS=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 TONIGHT NOMINAL >> Axial A:=C2=A0 =C2=A0 =C2=A0 =C2=A0 -0.702=C2=A0 -0.704 >> Axial B:=C2=A0 =C2=A0 =C2=A0 =C2=A0 -0.7=C2=A0 =C2=A0 -0.703 >> Axial C:=C2=A0 =C2=A0 =C2=A0 =C2=A0 -0.09=C2=A0 =C2=A0-0.096 >> Trans D:=C2=A0 =C2=A0 =C2=A0 =C2=A0 -0.11=C2=A0 =C2=A0-0.15 >> >> GALILS >> Commanded: 864500 869150 826700 -5750 -7450 >> Actual:=C2=A0 =C2=A0 834083 855053 794435 -5750 -7450 >> >> SETMIR VALUES >> SecDesOrient: 657.0 18.54 2.85 -17.2 138.98 >> SecOrient:=C2=A0 =C2=A0 656.98 18.55 2.84 -17.46 138.66 >> >> >> >> ---- Telescope Status ---- >> >> Status at: 11:12Z >> Telescope stowed at:=C2=A0 =C2=A0 121.0, 30.0,=C2=A0 =C2=A00.0=C2= =A0 mount >> Instrument mounted:=C2=A0 =C2=A0 =C2=A0cartridge =3D=C2=A0 =C2=A0 = =C2=A01 >> Counterweights at:=C2=A0 =C2=A0 =C2=A0 425 >> LN2 autofill systems:=C2=A0 =C2=A0Connected and turned on >> -180L LN2 dewar scales: >>=C2=A0 =C2=A0 =C2=A0 SP1=C2=A0 208.3 lbs, 22.6 psi >>=C2=A0 =C2=A0 =C2=A0 SP2=C2=A0 171.8 lbs, 22.5 ps >> >> ---- Software Used ---- >> >> Get Versions, 10/24/18, 23:02:34 >>=C2=A0 =C2=A0-- STUI software: >> STUI=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 : 1.7.3 >> alerts=C2=A0 =C2=A0 =C2=A0 =C2=A0 : trunk >> apo=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0: trunk >> boss=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 : v4_0 >> -boss.specMec : 3.0.4 >> gcamera=C2=A0 =C2=A0 =C2=A0 =C2=A0: v2_2 >> guider=C2=A0 =C2=A0 =C2=A0 =C2=A0 : 3.9.0 >> hartmann=C2=A0 =C2=A0 =C2=A0 : 1.6.0 >> hub=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0: Tag: v4.2 >> mcp=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0: v7_0_4 >> -mcp.plc=C2=A0 =C2=A0 =C2=A0 : 34 >> -mcp.azFiduci : v1_132 >> -mcp.altFiduc : v1_132 >> -mcp.rotFiduc : v1_132 >> platedb=C2=A0 =C2=A0 =C2=A0 =C2=A0: 2.8.0 >> sop=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0: 3.12.0 >> tcc=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0: 1.3.1 >> apogee=C2=A0 =C2=A0 =C2=A0 =C2=A0 : not availble >> apogeecal=C2=A0 =C2=A0 =C2=A0: not availble >> apogeeql=C2=A0 =C2=A0 =C2=A0 : v1_6_1 >> APO-local=C2=A0 =C2=A0 =C2=A0: trunk >> -- Other versions ---- >> idlspec2d=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0: v5_7_12 >> plate-mapper3=C2=A0 =C2=A0 =C2=A0: NOCVS:v6_0_11 >> petunia=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0: 1.2.0 >> autoscheduler=C2=A0 =C2=A0 =C2=A0: 1.0.1/ >> sdss_python_module: trunk >> mangadrp=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 : trunk exported >> >> ---- APOGEE Data Summary ---- >> >> APOGEE data list,=C2=A0 =C2=A0mjd=3D58416,=C2=A0 all >> raw_data:=C2=A0 /data/apogee/utr_cdr/58416/apRaw-*.fits >> archive:=C2=A0 /data/apogee/archive/58416/apR-[a,b,c]-*.apz >> quickred:=C2=A0 /data/apogee/archive/58416/apR-[a,b,c]-*.apz >> ------------------------------------------------------------------= -------------- >> >> mjd=C2=A0 =C2=A0 =C2=A0i=C2=A0 =C2=A0UT=C2=A0 =C2=A0File/Exp=C2=A0= Imtype Nread A/B Dth=C2=A0 Ct-Plate Arch Offset >> %Flux Comm >> >> ------------------------------------------------------------------= -------------- >> 58416=C2=A0 =C2=A01 22:59=C2=A0 28540001=C2=A0 =C2=A0 Dark=C2=A0 = =C2=A0 3=C2=A0 A=C2=A0 13.0=C2=A0 =C2=A01-11261=C2=A0 a-b-c=C2=A0 =C2=A0 - >> - >> 58416=C2=A0 =C2=A02 23:05=C2=A0 28540002=C2=A0 =C2=A0 Dark=C2=A0 = =C2=A060=C2=A0 A=C2=A0 13.0=C2=A0 =C2=A01-11261=C2=A0 a-b-c=C2=A0 =C2=A0 - >> - >> 58416=C2=A0 =C2=A03 23:16=C2=A0 28540003=C2=A0 QuarFlat 10=C2=A0 A= =C2=A0 13.0=C2=A0 =C2=A01-11261=C2=A0 a-b-c=C2=A0 =C2=A0 - >> 46 >> 58416=C2=A0 =C2=A04 23:18=C2=A0 28540004=C2=A0 Arc-Thar 12=C2=A0 A= =C2=A0 13.0=C2=A0 =C2=A01-11261=C2=A0 a-b-c=C2=A0 0.04 >> - >> 58416=C2=A0 =C2=A05 23:20=C2=A0 28540005=C2=A0 Arc-Une=C2=A0 40=C2= =A0 A=C2=A0 13.0=C2=A0 =C2=A01-11261=C2=A0 a-b-c=C2=A0 =C2=A0 - >> - >> 58416=C2=A0 =C2=A06 23:28=C2=A0 28540006=C2=A0 Arc-Thar 12=C2=A0 B= =C2=A0 13.5=C2=A0 =C2=A01-11261=C2=A0 a-b-c=C2=A0 0.52 >> - >> 58416=C2=A0 =C2=A07 23:30=C2=A0 28540007=C2=A0 Arc-Une=C2=A0 40=C2= =A0 B=C2=A0 13.5=C2=A0 =C2=A01-11261=C2=A0 a-b-c=C2=A0 =C2=A0 - >> - >> 58416=C2=A0 =C2=A08 23:37=C2=A0 28540008=C2=A0 =C2=A0 Dark=C2=A0 = =C2=A060=C2=A0 A=C2=A0 13.0=C2=A0 =C2=A01-11261=C2=A0 a-b-c=C2=A0 =C2=A0 - >> - >> ------------------------------------------------------------------= -------------- >> >> >> >> >> ---- MaNGA Data Summary ---- >> >> Reading FITS from /data/spectro/58416/ >> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - = - - - >>=C2=A0 =C2=A0n=C2=A0 time filename=C2=A0 =C2=A0crt plt flav Shu Dth= =C2=A0 exp=C2=A0 colA=C2=A0 =C2=A0colB=C2=A0 =C2=A0colC=C2=A0 =C2=A0Hrt >> b1 >> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - = - - - >>=C2=A0 =C2=A01 00:02 b1-00274537=C2=A0 1 11261 bia Clo C=C2=A0 =C2= =A0 0.0=C2=A0 23731=C2=A0 23307=C2=A0 22329 >>=C2=A0 =C2=A0 =C2=A000:02 b2-00274537=C2=A0 1 11261 bia Clo C=C2=A0= =C2=A0 0.0=C2=A0 18613=C2=A0 18973=C2=A0 18504 >>=C2=A0 =C2=A0 =C2=A000:02 r1-00274537=C2=A0 1 11261 bia Clo C=C2=A0= =C2=A0 0.0=C2=A0 23731=C2=A0 23307=C2=A0 22329 >>=C2=A0 =C2=A0 =C2=A000:02 r2-00274537=C2=A0 1 11261 bia Clo C=C2=A0= =C2=A0 0.0=C2=A0 18613=C2=A0 18973=C2=A0 18504 >>=C2=A0 =C2=A02 00:03 b1-00274538=C2=A0 1 11261 dar Clo C=C2=A0 =C2= =A020.0=C2=A0 23731=C2=A0 23307=C2=A0 22329 >>=C2=A0 =C2=A0 =C2=A000:03 b2-00274538=C2=A0 1 11261 dar Clo C=C2=A0= =C2=A020.0=C2=A0 18613=C2=A0 18973=C2=A0 18504 >>=C2=A0 =C2=A0 =C2=A000:03 r1-00274538=C2=A0 1 11261 dar Clo C=C2=A0= =C2=A020.0=C2=A0 23731=C2=A0 23307=C2=A0 22329 >>=C2=A0 =C2=A0 =C2=A000:03 r2-00274538=C2=A0 1 11261 dar Clo C=C2=A0= =C2=A020.0=C2=A0 18613=C2=A0 18973=C2=A0 18504 >>=C2=A0 =C2=A03 00:08 b1-00274539=C2=A0 1 11261 arc Lef C=C2=A0 =C2= =A0 4.1=C2=A0 23731=C2=A0 23307=C2=A0 22329 >> HART >>=C2=A0 =C2=A0 =C2=A000:08 b2-00274539=C2=A0 1 11261 arc Lef C=C2=A0= =C2=A0 4.2=C2=A0 18613=C2=A0 18973=C2=A0 18504 >> HART >>=C2=A0 =C2=A0 =C2=A000:08 r1-00274539=C2=A0 1 11261 arc Lef C=C2=A0= =C2=A0 4.1=C2=A0 23731=C2=A0 23307=C2=A0 22329 >> HART >>=C2=A0 =C2=A0 =C2=A000:08 r2-00274539=C2=A0 1 11261 arc Lef C=C2=A0= =C2=A0 4.2=C2=A0 18613=C2=A0 18973=C2=A0 18504 >> HART >>=C2=A0 =C2=A04 00:35 b1-00274540=C2=A0 1 11261 arc Lef C=C2=A0 =C2= =A0 4.1=C2=A0 23731=C2=A0 23307=C2=A0 22329 >> HART >>=C2=A0 =C2=A0 =C2=A000:35 b2-00274540=C2=A0 1 11261 arc Lef C=C2=A0= =C2=A0 4.2=C2=A0 18613=C2=A0 18973=C2=A0 18504 >> HART >>=C2=A0 =C2=A0 =C2=A000:35 r1-00274540=C2=A0 1 11261 arc Lef C=C2=A0= =C2=A0 4.1=C2=A0 23731=C2=A0 23307=C2=A0 22329 >> HART >>=C2=A0 =C2=A0 =C2=A000:35 r2-00274540=C2=A0 1 11261 arc Lef C=C2=A0= =C2=A0 4.2=C2=A0 18613=C2=A0 18973=C2=A0 18504 >> HART >>=C2=A0 =C2=A05 01:24 b1-00274541=C2=A0 1 11261 bia Clo C=C2=A0 =C2= =A0 0.0=C2=A0 23731=C2=A0 23307=C2=A0 22329 >>=C2=A0 =C2=A0 =C2=A001:24 b2-00274541=C2=A0 1 11261 bia Clo C=C2=A0= =C2=A0 0.0=C2=A0 18613=C2=A0 18973=C2=A0 18504 >>=C2=A0 =C2=A0 =C2=A001:24 r1-00274541=C2=A0 1 11261 bia Clo C=C2=A0= =C2=A0 0.0=C2=A0 23731=C2=A0 23307=C2=A0 22329 >>=C2=A0 =C2=A0 =C2=A001:24 r2-00274541=C2=A0 1 11261 bia Clo C=C2=A0= =C2=A0 0.0=C2=A0 18613=C2=A0 18973=C2=A0 18504 >>=C2=A0 =C2=A06 01:26 b1-00274542=C2=A0 1 11261 bia Clo C=C2=A0 =C2= =A0 0.0=C2=A0 23731=C2=A0 23307=C2=A0 22329 >>=C2=A0 =C2=A0 =C2=A001:26 b2-00274542=C2=A0 1 11261 bia Clo C=C2=A0= =C2=A0 0.0=C2=A0 18613=C2=A0 18973=C2=A0 18504 >>=C2=A0 =C2=A0 =C2=A001:26 r1-00274542=C2=A0 1 11261 bia Clo C=C2=A0= =C2=A0 0.0=C2=A0 23731=C2=A0 23307=C2=A0 22329 >>=C2=A0 =C2=A0 =C2=A001:26 r2-00274542=C2=A0 1 11261 bia Clo C=C2=A0= =C2=A0 0.0=C2=A0 18613=C2=A0 18973=C2=A0 18504 >>=C2=A0 =C2=A07 01:27 b1-00274543=C2=A0 1 11261 dar Clo C=C2=A0 =C2= =A020.0=C2=A0 23731=C2=A0 23307=C2=A0 22329 >>=C2=A0 =C2=A0 =C2=A001:27 b2-00274543=C2=A0 1 11261 dar Clo C=C2=A0= =C2=A020.0=C2=A0 18613=C2=A0 18973=C2=A0 18504 >>=C2=A0 =C2=A0 =C2=A001:27 r1-00274543=C2=A0 1 11261 dar Clo C=C2=A0= =C2=A020.0=C2=A0 23731=C2=A0 23307=C2=A0 22329 >>=C2=A0 =C2=A0 =C2=A001:27 r2-00274543=C2=A0 1 11261 dar Clo C=C2=A0= =C2=A020.0=C2=A0 18613=C2=A0 18973=C2=A0 18504 >>=C2=A0 =C2=A08 03:03 b1-00274544=C2=A0 1 11261 bia Clo C=C2=A0 =C2= =A0 0.0=C2=A0 23731=C2=A0 23307=C2=A0 22329 >>=C2=A0 =C2=A0 =C2=A003:03 b2-00274544=C2=A0 1 11261 bia Clo C=C2=A0= =C2=A0 0.0=C2=A0 18613=C2=A0 18973=C2=A0 18504 >>=C2=A0 =C2=A0 =C2=A003:03 r1-00274544=C2=A0 1 11261 bia Clo C=C2=A0= =C2=A0 0.0=C2=A0 23731=C2=A0 23307=C2=A0 22329 >>=C2=A0 =C2=A0 =C2=A003:03 r2-00274544=C2=A0 1 11261 bia Clo C=C2=A0= =C2=A0 0.0=C2=A0 18613=C2=A0 18973=C2=A0 18504 >>=C2=A0 =C2=A09 03:07 b1-00274545=C2=A0 1 11261 arc Lef C=C2=A0 =C2= =A0 4.1=C2=A0 23731=C2=A0 23307=C2=A0 22329 >> HART >>=C2=A0 =C2=A0 =C2=A003:07 b2-00274545=C2=A0 1 11261 arc Lef C=C2=A0= =C2=A0 4.2=C2=A0 18613=C2=A0 18973=C2=A0 18504 >> HART >>=C2=A0 =C2=A0 =C2=A003:07 r1-00274545=C2=A0 1 11261 arc Lef C=C2=A0= =C2=A0 4.1=C2=A0 23731=C2=A0 23307=C2=A0 22329 >> HART >>=C2=A0 =C2=A0 =C2=A003:07 r2-00274545=C2=A0 1 11261 arc Lef C=C2=A0= =C2=A0 4.2=C2=A0 18613=C2=A0 18973=C2=A0 18504 >> HART >>=C2=A0 10 03:08 b1-00274546=C2=A0 1 11261 arc Rig C=C2=A0 =C2=A0 4.= 1=C2=A0 23731=C2=A0 23307=C2=A0 22329 >> HART >>=C2=A0 =C2=A0 =C2=A003:08 b2-00274546=C2=A0 1 11261 arc Rig C=C2=A0= =C2=A0 4.2=C2=A0 18613=C2=A0 18973=C2=A0 18504 >> HART >>=C2=A0 =C2=A0 =C2=A003:08 r1-00274546=C2=A0 1 11261 arc Rig C=C2=A0= =C2=A0 4.1=C2=A0 23731=C2=A0 23307=C2=A0 22329 >> HART >>=C2=A0 =C2=A0 =C2=A003:08 r2-00274546=C2=A0 1 11261 arc Rig C=C2=A0= =C2=A0 4.2=C2=A0 18613=C2=A0 18973=C2=A0 18504 >> HART >>=C2=A0 11 03:25 b1-00274547=C2=A0 1 11261 bia Clo C=C2=A0 =C2=A0 0.= 0=C2=A0 23731=C2=A0 23307=C2=A0 22329 >>=C2=A0 =C2=A0 =C2=A003:25 b2-00274547=C2=A0 1 11261 bia Clo C=C2=A0= =C2=A0 0.0=C2=A0 18613=C2=A0 18973=C2=A0 18504 >>=C2=A0 =C2=A0 =C2=A003:25 r1-00274547=C2=A0 1 11261 bia Clo C=C2=A0= =C2=A0 0.0=C2=A0 23731=C2=A0 23307=C2=A0 22329 >>=C2=A0 =C2=A0 =C2=A003:25 r2-00274547=C2=A0 1 11261 bia Clo C=C2=A0= =C2=A0 0.0=C2=A0 18613=C2=A0 18973=C2=A0 18504 >>=C2=A0 12 05:14 b1-00274548=C2=A0 1 11261 fla Out C=C2=A0 =C2=A025.= 1=C2=A0 23731=C2=A0 23307=C2=A0 22329 >>=C2=A0 =C2=A0 =C2=A005:14 b2-00274548=C2=A0 1 11261 fla Out C=C2=A0= =C2=A025.2=C2=A0 18613=C2=A0 18973=C2=A0 18504 >>=C2=A0 =C2=A0 =C2=A005:14 r1-00274548=C2=A0 1 11261 fla Out C=C2=A0= =C2=A025.1=C2=A0 23731=C2=A0 23307=C2=A0 22329 >>=C2=A0 =C2=A0 =C2=A005:14 r2-00274548=C2=A0 1 11261 fla Out C=C2=A0= =C2=A025.2=C2=A0 18613=C2=A0 18973=C2=A0 18504 >>=C2=A0 13 09:43 b1-00274549=C2=A0 1 11261 bia Clo C=C2=A0 =C2=A0 0.= 0=C2=A0 23731=C2=A0 23307=C2=A0 22329 >>=C2=A0 =C2=A0 =C2=A009:43 b2-00274549=C2=A0 1 11261 bia Clo C=C2=A0= =C2=A0 0.0=C2=A0 18613=C2=A0 18973=C2=A0 18504 >>=C2=A0 =C2=A0 =C2=A009:43 r1-00274549=C2=A0 1 11261 bia Clo C=C2=A0= =C2=A0 0.0=C2=A0 23731=C2=A0 23307=C2=A0 22329 >>=C2=A0 =C2=A0 =C2=A009:43 r2-00274549=C2=A0 1 11261 bia Clo C=C2=A0= =C2=A0 0.0=C2=A0 18613=C2=A0 18973=C2=A0 18504 >>=C2=A0 14 09:46 b1-00274550=C2=A0 1 11261 bia Clo C=C2=A0 =C2=A0 0.= 0=C2=A0 23731=C2=A0 23307=C2=A0 22329 >>=C2=A0 =C2=A0 =C2=A009:46 b2-00274550=C2=A0 1 11261 bia Clo C=C2=A0= =C2=A0 0.0=C2=A0 18613=C2=A0 18973=C2=A0 18504 >>=C2=A0 =C2=A0 =C2=A009:46 r1-00274550=C2=A0 1 11261 bia Clo C=C2=A0= =C2=A0 0.0=C2=A0 23731=C2=A0 23307=C2=A0 22329 >>=C2=A0 =C2=A0 =C2=A009:46 r2-00274550=C2=A0 1 11261 bia Clo C=C2=A0= =C2=A0 0.0=C2=A0 18613=C2=A0 18973=C2=A0 18504 >>=C2=A0 15 09:47 b1-00274551=C2=A0 1 11261 dar Clo C=C2=A0 =C2=A020.= 0=C2=A0 23731=C2=A0 23307=C2=A0 22329 >>=C2=A0 =C2=A0 =C2=A009:47 b2-00274551=C2=A0 1 11261 dar Clo C=C2=A0= =C2=A020.0=C2=A0 18613=C2=A0 18973=C2=A0 18504 >>=C2=A0 =C2=A0 =C2=A009:47 r1-00274551=C2=A0 1 11261 dar Clo C=C2=A0= =C2=A020.0=C2=A0 23731=C2=A0 23307=C2=A0 22329 >>=C2=A0 =C2=A0 =C2=A009:47 r2-00274551=C2=A0 1 11261 dar Clo C=C2=A0= =C2=A020.0=C2=A0 18613=C2=A0 18973=C2=A0 18504 >>=C2=A0 16 09:53 b1-00274552=C2=A0 1 11261 arc Lef C=C2=A0 =C2=A0 4.= 1=C2=A0 23731=C2=A0 23307=C2=A0 22329 >> HART >>=C2=A0 =C2=A0 =C2=A009:53 b2-00274552=C2=A0 1 11261 arc Lef C=C2=A0= =C2=A0 4.2=C2=A0 18613=C2=A0 18973=C2=A0 18504 >> HART >>=C2=A0 =C2=A0 =C2=A009:53 r1-00274552=C2=A0 1 11261 arc Lef C=C2=A0= =C2=A0 4.1=C2=A0 23731=C2=A0 23307=C2=A0 22329 >> HART >>=C2=A0 =C2=A0 =C2=A009:53 r2-00274552=C2=A0 1 11261 arc Lef C=C2=A0= =C2=A0 4.2=C2=A0 18613=C2=A0 18973=C2=A0 18504 >> HART >>=C2=A0 17 09:54 b1-00274553=C2=A0 1 11261 arc Rig C=C2=A0 =C2=A0 4.= 1=C2=A0 23731=C2=A0 23307=C2=A0 22329 >> HART >>=C2=A0 =C2=A0 =C2=A009:54 b2-00274553=C2=A0 1 11261 arc Rig C=C2=A0= =C2=A0 4.2=C2=A0 18613=C2=A0 18973=C2=A0 18504 >> HART >>=C2=A0 =C2=A0 =C2=A009:54 r1-00274553=C2=A0 1 11261 arc Rig C=C2=A0= =C2=A0 4.1=C2=A0 23731=C2=A0 23307=C2=A0 22329 >> HART >>=C2=A0 =C2=A0 =C2=A009:54 r2-00274553=C2=A0 1 11261 arc Rig C=C2=A0= =C2=A0 4.2=C2=A0 18613=C2=A0 18973=C2=A0 18504 >> HART >>=C2=A0 18 09:59 b1-00274554=C2=A0 1 11261 fla Out C=C2=A0 =C2=A025.= 1=C2=A0 23603=C2=A0 23179=C2=A0 22201 >>=C2=A0 =C2=A0 =C2=A009:59 b2-00274554=C2=A0 1 11261 fla Out C=C2=A0= =C2=A025.2=C2=A0 18770=C2=A0 19129=C2=A0 18661 >>=C2=A0 =C2=A0 =C2=A009:59 r1-00274554=C2=A0 1 11261 fla Out C=C2=A0= =C2=A025.1=C2=A0 23603=C2=A0 23179=C2=A0 22201 >>=C2=A0 =C2=A0 =C2=A009:59 r2-00274554=C2=A0 1 11261 fla Out C=C2=A0= =C2=A025.2=C2=A0 18770=C2=A0 19129=C2=A0 18661 >>=C2=A0 19 10:02 b1-00274555=C2=A0 1 11261 arc Out C=C2=A0 =C2=A0 4.= 1=C2=A0 23603=C2=A0 23179=C2=A0 22201 >>=C2=A0 =C2=A0 =C2=A010:02 b2-00274555=C2=A0 1 11261 arc Out C=C2=A0= =C2=A0 4.2=C2=A0 18770=C2=A0 19129=C2=A0 18661 >>=C2=A0 =C2=A0 =C2=A010:02 r1-00274555=C2=A0 1 11261 arc Out C=C2=A0= =C2=A0 4.1=C2=A0 23603=C2=A0 23179=C2=A0 22201 >>=C2=A0 =C2=A0 =C2=A010:02 r2-00274555=C2=A0 1 11261 arc Out C=C2=A0= =C2=A0 4.2=C2=A0 18770=C2=A0 19129=C2=A0 18661 >> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - = - - - >> --- OK --- >> >> >> ---- Problems in Detail ---- >> >> Boss sp1cam tcheck: >> 00:18:56 CmdStarted: OBSERVER.elenam 134 boss sp1cam raw=3D1 tchec= k 2 >> tcheck >> >> 00:18:56 OBSERVER.elenam 134 boss i text=3D"Sending raw comma= nd 1 TCHECK 2 >> TCHECK >>=C2=A0 to sp1cam" >> 00:18:56 OBSERVER.elenam 135 boss i text=3D"Sending raw comma= nd 1 TCHECK 2 >> TCHECK >>=C2=A0 to sp2cam" >> 00:18:58 OBSERVER.elenam 134 boss i sp1camRawText=3D"SP1R0=C2= =A0 LN2TEMP >> 92.6K" >> 00:18:58 OBSERVER.elenam 134 boss i sp1camRawText=3D"CCD_TEMP= :=C2=A0 DAC SET AT >> 160 =3D >>=C2=A0 =C2=A01.26 V =3D -126 C=C2=A0 MEASURED=C2=A0 -117.2C" >> 00:18:58 OBSERVER.elenam 134 boss i sp1camRawText=3D"HEATER_V= OLTS=C2=A0 =C2=A0 -0.56 >> V" >> 00:18:58 OBSERVER.elenam 134 boss i sp1camRawText=3D"" >> 00:18:58 OBSERVER.elenam 134 boss i sp1camRawText=3D"SP1B2=C2= =A0 LN2TEMP >> 94.9K" >> 00:18:58 OBSERVER.elenam 134 boss i sp1camRawText=3D"CCD_TEMP= :=C2=A0 DAC SET AT >> 127 =3D >>=C2=A0 -0.04 V =3D -100 C=C2=A0 MEASURED=C2=A0 =C2=A0-95.3C" >> 00:18:58 OBSERVER.elenam 134 boss i sp1camRawText=3D"HEATER_V= OLTS=C2=A0 =C2=A0 -0.54 >> V" >> 00:18:58 OBSERVER.elenam 134 boss i sp1camRawText=3D" ok"= ; >> --------------------------- >> >> >> >> >> >> ____________________________________________________________ >> Logs of Nightly APO Activities >> https://trac.sdss.= org/tracmailman/browser/private/sloan-25mlog/index.html >> > --0000000000006db702057913223d--