Subject: 2.5m obslog 20120310 (MJD 55997) From: elenam@apo.nmsu.edu Date: Sun, 11 Mar 2012 10:35:02 +0000 (UTC) APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Saturday March 10, 2012 (MJD 55997) ---- Observing Team ---- Kaike Pan Elena Malanushenko Phone Support: Craig Loomis Jon Brinkman ---- Observing Plan ---- BOSS until 04:08 (1.7 h), then MARVELS/APOGEE (8.3 h) ---- Observing Summary ---- Note to engineers/developers: 1. BOSS spectro daqs lost their collections to bossICC. We has the problem for several times during the past week. Please check the power supply and network connections on the daqs (see afternoon section for error message and more details). 2. tcc lost its connections to the primary and secondary galil controllers (see afternoon section for error message and more details). Last night was lost because of snow, clouds, and high humidity. We did not open. ---- Observing Log ---- Afternoon: ---------- All actors and systems were checked. Three problem noticed. 1: We has DAQs problem with the following error message when we tried to take a BOSS bias: 00:14:09 APO.kpan.sop 142 boss f text="Need micros ([]) for all active DAQs (['sp1daq', 'sp2daq'])" 00:14:09 CmdDone: APO.kpan.sop 142 boss exposure bias 00:14:09 APO.kpan 8 sop f text="Failed to take bias exposure" 00:14:09 CmdDone: APO.kpan 8 sop doCalibs nbias=1 We know that "boss reloadControllers" can get daqs back to work status. But since we are having this problem very frequently, a few time a week. We are afraid that this could cost us an exposure if it happens during boss integrations. So we called Craig. Looks like that the micros has problems in their connections to bossICC. Craig said that it could be a power supple problem or internet problem. He will write a message to Jon and Fritz and ask them to the power supply and the network connections. 2. We saw the follow error message in STUI log. Consulate with Jon on this. Looks like that TCC has been disconnected from the primary and the secondary galil controls. 01:06:41 .tcc 0 tcc F Modu="prt_Read"; Text="read timed out on port LTA9:" 01:06:41 .tcc 0 tcc F Modu="mir_o_MoveOneGalil"; Text="could not specify new positions for Prim mirror actuators" 01:06:41 .tcc 0 tcc W Text="failed to adjust collimation and focus" 01:06:47 .tcc 0 tcc F Modu="prt_Read"; Text="read timed out on port LTA10:" 01:06:47 .tcc 0 tcc F Modu="mir_o_MoveOneGalil"; Text="could not specify new positions for Sec mirror actuators" Login into tcc@tcc25m, and issue a "telstop" followed by "telrun" and hub startNubs tcc25m (from STUI) to reestablish the connections. BOSS spectro collimation was checked on cartridge 16. No adjustment was made. motorPosition sp1: 20114, 19076, 10867; sp2: 26172, 26049, 27085; Median spectrographs temperatures: Tsp1= 4.8, Tsp2= 3.6; Outside air temperature= -6.2; When r1 and r2 are in focus, b1 and b2 are -6.5 degrees and 2.5 deg away from their perfect focus position. we probable should adjust b2 focus ring for -5 deg or so for the current temperature. Since we will most likely not able to open tonight, we left it as it was. 3. Guider temperature was noticed at -32 C with a work load of 80%, and chiller pressure was at almost zero. Bleeding air bubbles from glycol loop fixed the problem. Apogee evening calibs were taken, and frames checked normal. MARVELS TIO and ThAr were taken, exposures looks good. Weather, snowing, high humdity prevent us from ventilation. Night ----- Cannot open because of snow, clouds, and high humidity. The weather did not improve and less and less hope to get data. 07:49:04 We started apogee morning cals. during apogee calibration we got large amount of warnings: 07:58:27 .apogeeql 0 apogeeql w text="failed to make SPA card from None" ---- Spectroscopy Data Summary ---- Reading FITS from /data/spectro/55997/ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - n time filename cart plt flav hrt exp colA colB colC Hrt - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 1 00:34 b1-00140953 16 5430 bia 0.0 19047 18010 9807 00:34 b2-00140953 16 5430 bia 0.0 26172 26049 27085 00:34 r1-00140953 16 5430 bia 0.0 19047 18010 9807 00:34 r2-00140953 16 5430 bia 0.0 26172 26049 27085 2 00:38 b1-00140954 16 5430 arc Lef 4.9 19047 18010 9807 HART 00:38 b2-00140954 16 5430 arc Lef 4.9 26172 26049 27085 HART 00:38 r1-00140954 16 5430 arc Lef 4.9 19047 18010 9807 HART 00:38 r2-00140954 16 5430 arc Lef 4.9 26172 26049 27085 HART 3 00:40 b1-00140955 16 5430 arc Rig 4.9 19047 18010 9807 HART 00:40 b2-00140955 16 5430 arc Rig 4.9 26172 26049 27085 HART 00:40 r1-00140955 16 5430 arc Rig 4.9 19047 18010 9807 HART 00:40 r2-00140955 16 5430 arc Rig 4.9 26172 26049 27085 HART 4 00:46 b1-00140956 16 5430 arc Lef 4.9 20114 19076 10867 HART 00:46 b2-00140956 16 5430 arc Lef 4.9 26172 26049 27085 HART 00:46 r1-00140956 16 5430 arc Lef 4.9 20114 19076 10867 HART 00:46 r2-00140956 16 5430 arc Lef 4.9 26172 26049 27085 HART 5 00:48 b1-00140957 16 5430 arc Rig 4.9 20114 19076 10867 HART 00:48 b2-00140957 16 5430 arc Rig 4.9 26172 26049 27085 HART 00:48 r1-00140957 16 5430 arc Rig 4.9 20114 19076 10867 HART 00:48 r2-00140957 16 5430 arc Rig 4.9 26172 26049 27085 HART 6 00:54 b1-00140958 16 5430 fla Out 30.9 20114 19076 10867 00:54 b2-00140958 16 5430 fla Out 30.9 26172 26049 27085 00:54 r1-00140958 16 5430 fla Out 30.9 20114 19076 10867 00:54 r2-00140958 16 5430 fla Out 30.9 26172 26049 27085 7 00:58 b1-00140959 16 5430 arc Out 4.9 20114 19076 10867 00:58 b2-00140959 16 5430 arc Out 4.9 26172 26049 27085 00:58 r1-00140959 16 5430 arc Out 4.9 20114 19076 10867 00:58 r2-00140959 16 5430 arc Out 4.9 26172 26049 27085 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - --- OK --- Reading FITS from /data/apogee/quickred/55997/ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - N UT File/Exp Cart Plate Imagetype Dth Nread Comment - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 1 00:09 04350001 16 5430 Dark A 10 None 2 00:11 04350002 16 5430 Dark A 10 None 3 00:19 04350003 16 5430 Dark A 60 None 4 00:30 04350004 16 5430 QuartzFlat A 10 None 5 00:32 04350005 16 5430 ArcLamp A 12 None 6 00:34 04350006 16 5430 ArcLamp A 40 None 7 00:41 04350007 16 5430 ArcLamp B 12 None 8 00:44 04350008 16 5430 ArcLamp B 40 None 9 00:51 04350009 16 5430 Dark A 60 None 10 07:48 04350010 16 5430 Dark A 60 None 11 07:58 04350011 16 5430 Dark A 60 None 12 08:09 04350012 16 5430 Dark A 60 None 13 08:20 04350013 16 5430 QuartzFlat A 10 None 14 08:22 04350014 16 5430 QuartzFlat A 10 None 15 08:24 04350015 16 5430 QuartzFlat A 10 None 16 08:26 04350016 16 5430 ArcLamp A 12 None 17 08:28 04350017 16 5430 ArcLamp A 40 None 18 08:35 04350018 16 5430 ArcLamp B 12 None 19 08:37 04350019 16 5430 ArcLamp B 40 None 20 08:45 04350020 16 5430 Dark A 30 None 21 08:50 04350021 16 5430 InternalFl A 30 None 22 08:55 04350022 16 5430 InternalFl A 30 None 23 09:01 04350023 16 5430 InternalFl A 30 None - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - --- OK --- ---- Telescope Offsets and Scale ---- none ---- Focus Log ---- none ---- Weather Log ---- Wind 1 um DIMM IRSC Time Temp C Dewp C MPH Dir Dust fwhm mean sigma 19:00 -2.3 -3.8 2.8 177.1 6192 19:15 -2.1 -3.8 5.0 223.2 8135 19:30 -1.7 -3.3 2.5 195.6 5784 19:45 -1.4 -3.1 0.6 144.5 11775 20:00 -0.8 -2.8 10.0 188.7 4453 20:15 -1.0 -3.3 1.0 222.6 4662 20:30 -1.3 -3.6 11.1 210.4 3548 20:45 -1.8 -4.0 12.4 184.9 3015 21:00 -2.4 -4.4 11.8 188.4 2670 21:15 -2.8 -4.7 8.2 188.0 2048 21:30 -2.7 -4.7 9.8 148.0 4088 21:45 -2.1 -4.6 15.6 182.0 3593 22:00 -3.2 -5.3 16.1 198.3 2963 22:15 -3.4 -5.5 12.0 207.3 4061 22:30 -3.6 -5.8 15.0 213.3 1937 22:45 -3.7 -5.8 21.4 199.2 4876 23:00 -3.7 -5.7 16.0 210.7 6562 23:15 -3.6 -5.7 15.8 202.8 4715 23:30 -3.7 -5.7 15.2 211.8 4483 23:45 -3.9 -5.9 5.7 241.4 5270 00:00 -3.8 -5.7 16.9 208.1 6881 00:15 -4.0 -6.0 14.5 217.3 7230 00:30 -3.8 -5.9 13.3 217.3 4946 00:45 -3.9 -6.1 12.2 218.7 5983 01:00 -4.2 -6.4 12.2 239.0 6753 01:15 -4.1 -6.2 10.7 253.9 7288 01:30 -4.3 -6.3 11.7 255.9 2438 01:45 -4.2 -6.3 6.0 215.6 4931 02:00 -4.3 -6.5 11.7 243.8 2896 02:15 -4.6 -6.8 8.3 259.3 2055 02:30 -4.9 -7.1 9.2 261.7 1695 02:45 -5.2 -7.3 7.0 259.0 3441 03:00 -5.3 -7.4 7.5 283.2 2414 03:15 -5.3 -7.4 11.6 291.7 1780 03:30 -5.3 -7.4 10.7 294.0 2656 03:45 -5.4 -7.5 9.4 292.7 1396 04:00 -5.6 -7.6 11.0 291.6 2113 04:15 -5.4 -7.4 9.8 274.7 1000 04:30 -5.4 -7.4 13.8 273.1 801 04:45 -5.5 -7.6 14.8 307.1 2281 05:00 -5.3 -7.2 13.4 311.2 2669 05:15 -5.1 -6.8 8.7 270.1 1408 05:30 -4.9 -6.8 10.7 279.3 1815 05:45 -4.8 -6.6 10.2 272.9 1972 06:00 -4.6 -6.4 10.2 288.2 3729 06:15 -4.3 -6.3 13.4 290.1 1153 06:30 -4.1 -6.1 16.3 291.0 841 06:45 -4.2 -6.2 10.5 288.0 742 07:00 -4.2 -6.2 13.3 303.8 768 07:15 -4.1 -6.2 9.5 286.5 787 07:30 -4.2 -6.1 12.9 295.5 655 07:45 -4.2 -6.1 9.6 266.5 610 08:00 -3.9 -5.8 12.5 300.1 1688 08:15 -3.7 -5.8 13.6 292.3 815 08:30 -3.7 -5.8 13.0 275.0 916 08:45 -4.0 -5.9 9.7 281.3 1700 09:00 -4.0 -5.8 13.0 292.9 2096 09:15 -4.0 -5.8 8.9 277.3 1241 09:30 -4.1 -5.9 11.9 323.5 1491 09:45 -3.8 -5.8 8.7 324.6 1627 10:00 -3.8 -5.8 14.2 319.2 1839 10:15 -3.8 -5.9 5.8 316.8 1871 10:30 -4.2 -6.2 6.1 340.6 2057 ---- Telescope Status ---- _none___ Started ventilation _none___ encl25m : open _none___ encl25m : closed No interlocks bypassed Status at: 10:33:29 Telescope stowed at: 121.0, 30.3, 0.0 mount Instrument mounted: cartridge = 16 Counterweights at: 285, 285, 285, 287 LN2 autofill systems: Connected and turned on -180L LN2 dewar scales: SP1 210.6 lbs, 21.2 psi SP2 200.8 lbs, 20.4 psi ---- Software Used ---- ----STUI software: STUI: 1.1.2b1 2011-11-08; alerts: v1_2_0; apo: v1_7; boss: v2_5_4; boss.daq: 1.3.2; boss.specMech: 3.0.1; gcamera: v1_2_7; guider: v1_3_17; hub: Tag: sdss3_v1_1_8; mcp: v6_6_11; mcp.plc: 28; mcp.azFiducialVersion: v1_100; mcp.altFiducialVersion: v1_100; mcp.rotFiducialVersion: v1_100; platedb: v1_0_5; sop: v1_2_11; sos: v1_2_0; sos.idlspec2d: v5_5_3; tcc: 2.14.0; apogee: ? apogeecal: ? apogeeql: v1_4_1 -----Other software sdssProcedure(sdss3): v3_06; Setting up: idlspec2d Flavor: Linux64 Version: v5_5_7 Setting up: |elodie Flavor: Linux64 Version: v1_3 Setting up: |spinspect Flavor: Linux64 Version: start Setting up: |specflat Flavor: NULL Version: v1_18 Setting up: |speclog Flavor: Linux64 Version: trunk Setting up: |platelist Flavor: Linux64 Version: trunk Setting up: |platedb Flavor: Linux64 Version: v1_0_9 Setting up: |hooloovookit Flavor: Linux64 Version: v3.0b4 Setting up: |sqlalchemy Flavor: Linux64 Version: 0.5.5 Setting up: |psycopg Flavor: Linux64 Version: 2.0.11 Setting up: |scipy Flavor: Linux64 Version: 0.7.1 Setting up: |actorcore Flavor: Linux64 Version: v1_15_1 Setting up: |opscore Flavor: Linux64 Version: v1_15_1 Setting up: |external Flavor: Linux64 Version: v1_2_0 Setting up: |actorkeys Flavor: Linux64 Version: trunk Setting up: |plc Flavor: Linux64 Version: v28_1 Setting up: |sdss3tools Flavor: Linux64 Version: v1_7_10 Setting up: |RO Flavor: Linux64 Version: RO-2.8.0 Setting up: |tron Flavor: Linux64 Version: sdss3_v1_1_8 Setting up: |idlutils Flavor: Linux64 Version: v5_4_26 Setting up: |idl Flavor: Linux64 Version: 7.1 Setting up: |dust Flavor: Linux64 Version: v0_1 Setting up: actorcore Flavor: Linux64 Version: v1_15_1 Setting up: |opscore Flavor: Linux64 Version: v1_15_1 Setting up: |external Flavor: Linux64 Version: v1_2_0 Setting up: |actorkeys Flavor: Linux64 Version: trunk Setting up: |plc Flavor: Linux64 Version: v28_1 Setting up: |sdss3tools Flavor: Linux64 Version: v1_7_10 Setting up: |RO Flavor: Linux64 Version: RO-2.8.0 Setting up: |tron Flavor: Linux64 Version: sdss3_v1_1_8 ---- Mirror Numbers ---- PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A: -0.132 -0.127 Axial B: 0.069 0.08 Axial C: -0.01 -0.004 Trans D: 0.41005 0.40935 Lateral E: 10.6553 10.6553 Lateral F: 12.2682 12.2809 GALILS Commanded: 11300 -650 5500 19150 13500 13050 Actual: 11300 -650 5500 19150 13500 13050 SETMIR VALUES PrimDesOrient: 0.00 0.00 0.00 612.35 1325.90 PrimOrient: 0.00 0.00 0.00 612.43 1325.18 SECONDARY: ---------- Focus: 0.00 Air temp: -6.2 Altitude: 30.0 MIGS TONIGHT NOMINAL Axial A: -8.412 -8.408 Axial B: -8.701 -8.705 Axial C: -12.152 -12.158 Trans D: -10.025 -10.018 GALILS Commanded: 1577450 1611550 1562700 -9050 -6750 Actual: 1577444 1611435 1562704 -9050 -6750 SETMIR VALUES SecDesOrient: 1257.00 13.00 8.00 -60.00 174.63 SecOrient: 1256.97 12.98 7.98 -60.40 174.70 ---- Problems in Detail ---- Other weather problem - snow High Humidity