Subject: 2.5m obslog 20140827 (MJD 56897) From: dmbiz@apo.nmsu.edu Date: Thu, 28 Aug 2014 10:20:40 +0000 (UTC) APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Wednesday August 27, 2014 (MJD 56897) ---- Observing Team ---- Audrey Oravetz Dmitry Bizyaev Dan Long (over the phone) Joe Huehnerhoff (over the phone) Kyle Dawson (over the phone) ---- Observing Plan ---- MANGA: Plate 7991: Visibility window LST 14:46 - 19:55, UT: 23:26 - 4:35 Plate 7977: Visibility window LST ???, UT: 4:25 - 8:00 Plate 8069: Special all-sky plate, take two 15-minute exposures (no dither required) between LST 21:00-0:06, UT: 5:40-8:46 Plan: 1) obtain as many dithered exposures on plate 7991 as possible until UT 4:35 2) at UT 4:35, switch to plate 7977 and take as many dithered exposures as possible until UT 8:00 before switching to eBOSS 3) HOWEVER, if plate 8069 is available (depending on maintenance work), switch to this plate in time to observe 2x15 minute exposures just before UT 8:00 MaNGA until 07:57 ( 5.2 h), then eBOSS ( 3.5 h) ---- Observing Summary ---- We completed 1/2 of the long "Flexture test" (with cart 2). The sky was clear some part of the night, but two different hardware problems (see below), and then weather conditions, prevented us from getting science data. Many scripts initiated after login to eboss@sdss4-eboss show "failed", see "Software used" section. At 03:00:04 we got alert about ~ -70C temp at B2, while the nominal is -100C. ALL other temperatures were in spec. In a few minutes r2 started showing T = +0.7C, and all values around zero. Checked power supply boxes. b2 was ok. r2 all values close to zero. +5 FOXI had red LED at it. Got Joe on phone, power cycled the aluminum box (~3:40Z) on the r2,b2 power supply. Temperatures normalized after that. It did it again at 4:48, and we swapped r2 power supply block. We got bad AZ#1 amplifier functioning. Bypassed the amplifier and went to the sky on AZ#2 only. Had to close because of raising humidity. Numerous error reports as 09:52:16 .boss 0 boss w text="FAILED to write local BOSS file copy /export/home/boss/data/56897/sdR-b2-00183297.fit: [Errno 28] No space left on device" at the end of the night. A few times read-out of calibration exposures failed. Cured by reloadControllers, clearExposure. see log. ---- Observing Log ---- Afternoon: ========== Tested software. BOSS would not ping. Traced back to a netburner error: 21:53:03 APO.aoravetz 26 boss f text="command failed: error('Invalid response from Netburner.',) at /home/sdss4/products/Linux64/boss/trunk/python/boss/DAQInterface.py:1435" Reloaded the controllers. BOSS is happy now. Took biases and a dark. BOSS hung at flushing during the second bias exposure. 21:59:07 APO.aoravetz.sop 409 boss e text="Error while waiting for flush to complete on sp1cam. " 21:59:07 APO.aoravetz.sop 409 boss f text="Flush failed, exposure aborted." 21:59:07 CmdDone: APO.aoravetz.sop 409 boss exposure bias 21:59:07 .boss 436 boss w text="this command has already been finished!!!! (client.v4 436): exposure bias " 21:59:07 .boss 436 boss w text="this command has already been finished!!!! (client.v4 436): exposure bias " 21:59:07 .boss 436 boss f text="exposure aborted" 21:59:07 APO.aoravetz 47 sop e text="BOSS failed on exposure readout" 21:59:07 APO.aoravetz 47 sop f text="Failed to take bias exposure" 21:59:07 CmdDone: APO.aoravetz 47 sop doBossCalibs nBias=3 nDark=1 nFlat=0 nArc=0 darkTime=20.0 flatTime=30.0 guiderFlatTime=0.5 arcTime=4.0 Reloaded the controllers and cleared the exposure. BOSS is happy again. Collimated the spectrographs with cart 14: MotorPosition sp1: 23507, 23327, 22842; sp2: 27635, 26904, 28058; ---------- Median spectrographs temperatures: Tsp1= 15.0, Tsp2= 14.5; Outside air temperature= 14.7; 22:08:22 sp2.motor.move= -128, -127, -128 22:08:33 sp1.motor.move= -726, -725, -721 COLLIMATE: Camera b1 appears to be IN-FOCUS (|mean| Collimate-56897-b1-00183198.log Collimate-56897-b2-00183198.log Collimate-56897-r1-00183198.log Collimate-56897-r2-00183198.log >)." reloaded controllers, cleared exposure. 06:07:07 sop.gotoField = running; Calibs are done. GSOGTF, seeing ~ 1.5" (before fine tuning). Humidity was gradually raising, and reached 2.5C dew point difference. Given ~1C colder secondary, we are approaching a trouble. We managed to close not taking science exposures. 06:17Z enclosure closed, ventilation stopped. Dan L: I watched the voltages and see it is too high. It is dangerous to operate telescope with AZ#2 only. Do not re-open. We resumed the Flexture test: 07:10Z - Telescope at 90 alt and 0 rot with cart 2, plate 7991 All cameras in focus - Hartmanns 183251-52 Cals 183253-54 Alt Rot ExpNum Flavor == === ======== ===== 45 90 183255-56 Hartmanns (did not focus) 45 90 183257-58 Cals 30 0 183259-60 Hartmanns (did not focus) 30 0 183261-62 Cals 30 30 183263-64 Hartmanns (did not focus) 30 30 183265-66 Cals 30 45 183267-68 Hartmanns (did not focus) 30 45 183269-70 Cals 30 90 183271-72 Hartmanns (did not focus) 30 90 183273-74 Cals 30 135 183275-76 Hartmanns (did not focus) 183277 failed to read out 30 135 183278-79 Cals 30 180 183280-81 Hartmanns (did not focus) 30 180 183282-83 Cals 30 225 183284-85 Hartmanns (did not focus) 30 225 183286-87 Cals 30 270 183288-89 Hartmanns (did not focus) 30 270 183290-91 Cals 30 315 183292-93 Hartmanns (did not focus) 30 315 183294-95 Cals 30 360 183296-97 Hartmanns (did not focus) 30 360 183298-99 Cals NOTE: Around 08:21Z, we began receiving blue warning messages in STUI, but the images were visible on sdss4-eboss - 08:21:34 .boss 0 boss w text="FAILED to write local BOSS file copy /export/home/boss/data/56897/sdR-r1-00183261.fit: [Errno 28] No space left on device" etc etc... We see spectra is all files. 1/2 of the flexture test was done. 10:10Z started eBOSS calibrations. ---- Weather Log ---- Wind 1 um DIMM IRSC Time Temp C Dewp C MPH Dir Dust fwhm mean sigma 18:00 12.9 9.2 2.4 257.2 586 18:15 13.3 9.4 5.5 258.1 588 18:30 13.1 9.7 4.8 263.0 624 18:45 14.4 11.3 1.7 240.9 627 19:00 13.9 10.6 5.5 264.7 604 19:15 13.0 9.9 8.1 312.3 653 19:30 12.6 10.2 5.1 279.9 696 19:45 12.7 10.0 5.2 287.0 656 20:00 13.1 11.4 4.8 289.8 717 20:15 13.4 10.1 2.3 280.9 731 20:30 14.1 10.9 2.5 261.1 775 20:45 15.5 11.5 5.6 248.3 745 21:00 14.0 10.1 1.7 233.0 807 21:15 14.0 10.0 1.8 228.3 683 21:30 14.5 10.3 6.3 236.4 701 21:45 14.2 10.9 5.6 195.3 774 22:00 14.2 10.8 2.7 255.5 782 22:15 14.4 10.7 1.1 221.7 735 22:30 14.3 10.8 2.4 256.7 714 22:45 15.1 11.0 1.0 250.7 696 23:00 15.4 11.1 6.5 265.4 726 23:15 17.4 12.0 6.4 257.4 674 23:30 17.0 11.6 3.5 250.6 769 23:45 15.9 11.4 1.1 231.1 682 00:00 16.3 12.1 7.8 286.8 635 00:15 13.7 10.9 7.1 271.1 689 00:30 13.2 10.9 5.8 272.9 681 00:45 13.1 10.9 2.0 261.0 724 01:00 13.0 10.9 2.7 262.7 710 01:15 13.1 10.9 0.9 229.0 681 01:30 12.4 11.2 1.5 258.8 662 01:45 12.3 11.3 1.5 228.9 712 02:00 12.1 11.1 3.9 192.7 672 02:15 12.4 10.1 1.1 224.0 711 02:30 13.7 7.3 0.8 315.1 285 02:45 13.1 7.2 2.7 223.1 236 03:00 14.4 5.9 3.3 322.6 223 03:15 14.0 6.8 9.3 306.0 223 03:30 14.2 5.7 6.6 340.6 189 03:45 13.8 6.7 8.7 289.3 277 04:00 13.8 7.0 6.5 274.5 251 808 15.39 04:15 13.7 7.3 4.8 289.3 246 831 153.40 04:30 13.7 7.4 5.9 286.1 231 857 215.15 04:45 13.3 7.6 4.8 264.3 284 805 16.22 05:00 13.2 7.7 7.0 279.9 284 807 11.28 05:15 13.1 8.2 7.4 298.7 310 804 7.78 05:30 12.8 8.6 10.1 279.2 338 802 7.82 05:45 12.8 8.7 10.0 287.8 373 803 13.24 06:00 12.6 9.1 11.9 301.2 423 803 13.63 06:15 12.4 9.6 7.6 295.8 519 804 8.92 06:30 12.3 10.4 7.9 296.0 632 801 16.70 06:45 12.1 10.3 10.7 302.8 768 802 12.64 07:00 12.1 10.1 10.9 318.6 560 806 37.13 07:15 12.0 10.2 5.8 322.3 536 824 29.49 07:30 12.0 10.3 8.4 313.1 423 867 192.40 07:45 11.9 10.1 0.6 270.6 489 844 254.93 08:00 12.0 10.3 8.0 4.3 606 829 203.40 08:15 11.8 10.3 8.2 0.0 429 813 41.83 08:30 11.8 10.2 14.0 9.5 592 816 30.62 08:45 12.6 6.6 11.4 0.0 272 813 44.58 09:00 12.9 4.5 9.9 357.2 189 803 179.41 09:15 12.2 5.3 17.7 349.9 158 801 8.84 09:30 12.4 5.2 9.7 8.0 131 801 11.13 09:45 12.4 5.2 13.0 7.0 171 802 13.34 10:00 12.3 5.3 13.5 339.7 120 801 7.92 10:15 12.2 5.7 6.3 1.9 172 802 12.30 ---- Telescope Status ---- 4:43 Started ventilation 4:43 encl25m : open 6:17 encl25m : closed No interlocks bypassed Status at: 10:15Z Telescope stowed at: 121.0, 30.0, 0.0 mount Instrument mounted: cartridge = 2 Counterweights at: 423 LN2 autofill systems: Connected and turned on 180L LN2 dewar scales: SP1 112 lbs, 19 psi SP2 137 lbs, 17 psi ---- Software Used ---- Get Versions, 08/27/14, 21:55:33 -- STUI software: STUI : 1.4.1b4 alerts : trunk+svn149374 apo : trunk+svn153334 boss : trunk+svn157642 -boss.daq : 1.3.2 -boss.specMec : 3.0.4 gcamera : trunk+svn151798 guider : trunk+svn158013 hub : Tag: sdss3_v1_2_0 mcp : v6_6_18 -mcp.plc : 29 -mcp.azFiduci : v1_112 -mcp.altFiduc : v1_112 -mcp.rotFiduc : v1_112 platedb : branch_sdss4_platedb_removal+svn157983M sop : trunk+svn157845 sos : FAILED -sos.idlspec2 : FAILED tcc : 2.14.0 apogee : not availble apogeecal : not availble apogeeql : trunk+svn158024 -- Other software sdssProcedure : v3_06 APO-local : trunk idlspec2d (sos3) --- trunk 158074:158084 plate-mapper3 ------ Last login: Wed Aug 27 23:23:55 2014 from 10.25.1.252 Setting up: idlspec2d Flavor: Linux64 Version: trunk ... optional setup bbspec requested by idlspec2d trunk failed Setting up: |elodie Flavor: Linux64 Version: v1_3 Setting up: |spinspect Flavor: Linux64 Version: trunk Setting up: |specflat Flavor: Linux64 Version: v1_21 Setting up: |speclog Flavor: Linux64 Version: trunk Setting up: |platelist Flavor: Linux64 Version: trunk Setting up: |sdss_python_module Flavor: Linux64 Version: trunk Setting up: | sqlalchemy Flavor: Linux64 Version: 0.9.7 Setting up: | psycopg2 Flavor: Linux64 Version: 2.5.3 Setting up: |idlutils Flavor: Linux64 Version: v5_5_16 Setting up: | idl Flavor: Linux64 Version: 7.1 Setting up: | dust Flavor: Linux64 Version: v0_1 ... optional setup first requested by idlutils v5_5_16 failed Setting up: | photolog Flavor: Linux64 Version: trunk ... optional setup rc3 requested by idlutils v5_5_16 failed ... optional setup rass requested by idlutils v5_5_16 failed ... optional setup gsc requested by idlutils v5_5_16 failed ... optional setup twomass requested by idlutils v5_5_16 failed ... optional setup twomass_xsc requested by idlutils v5_5_16 failed ... optional setup tycho2 requested by idlutils v5_5_16 failed ... optional setup ucac requested by idlutils v5_5_16 failed ... optional setup usno requested by idlutils v5_5_16 failed ... optional setup ucac3 requested by idlutils v5_5_16 failed ... optional setup ucacr14 requested by idlutils v5_5_16 failed ... optional setup wise requested by idlutils v5_5_16 failed ---- Mirror Numbers ---- PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A: 0.035 0.037 Axial B: -0.021 -0.042 Axial C: -0.02 -0.022 Trans D: 3.21 3.21 Lateral E: 10.7188 10.7188 Lateral F: 12.3063 12.3063 GALILS Commanded: 10850 -450 5750 13050 14750 14300 Actual: 10850 -450 5750 13050 14750 14300 SETMIR VALUES PrimDesOrient: 0.00 0.00 0.00 651.00 937.40 PrimOrient: 0.00 0.00 0.00 651.08 936.54 SECONDARY: ---------- Focus: 0.00 Air temp: 14.6 Altitude: 30.0 MIGS TONIGHT NOMINAL Axial A: -7.06 -7.063 Axial B: -7.122 -7.128 Axial C: -7.032 -7.034 Trans D: -0.179 -0.179 GALILS Commanded: 1581300 1578850 1590700 1400 -2000 Actual: 1581306 1578862 1590767 1400 -2000 SETMIR VALUES SecDesOrient: 1257.00 -3.80 0.00 0.00 9.58 SecOrient: 1257.01 -3.82 0.01 0.01 9.45