Subject: 2.5m obslog 20190429 (SJD 58603) From: kgrabowski@apo.nmsu.edu Date: Tue, 30 Apr 2019 10:33:34 +0000 (UTC) APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Monday April 29, 2019 (SJD 58603) ---- Observing Team ---- Karen Kinemuchi Katie Grabowski Shane Thomas (shadowing) ---- Observing Plan ---- Mon 29 APR 2019 58603.0 0.25 02:59 11:03 10:26 18:33 ---- MaNGA ( 8.1 h) ---- Observing Summary ---- Thick clouds, radar all night kept us closed. Some rain. Lost 8.1 hr of MaNGA due to weather. ---- Observing Log ---- 23:58Z Cloudy on site. Nearby radar. AFTERNOON CHECKOUT: =================== 00:24Z boss ping failed, restarted the boss-icc, and everything came back. Started APOGEE evening cals. Grabbed mirror numbers - ok; gcamera/guider prep - done. 01:03Z Having problems with BOSS spectrograph sp2daq not responding. 00:53:48 .boss 0 boss w text="failed to start DAQ sp2daq, disconnecting myself, [Failure instance: Traceback (failure with no frames): : timed out" 00:53:48 .boss 0 boss w RawLine="]\"" 01:36:33 .boss 155 boss i text="Parsing CAMCHECK from sp2cam" 01:36:33 .boss 155 boss w text="Camcheck discovered unacknowledged state for sp2cam, which will need to be iacked" 01:36:33 .boss 155 boss i text="Parsed CAMCHECK on sp2cam" 01:36:33 .boss 155 boss d text="Completed message from sp1cam" 01:36:33 .boss 155 boss i text="Parsing CAMCHECK from sp1cam" 01:36:33 .boss 155 boss i text="Parsed CAMCHECK on sp1cam" 01:36:33 .boss 155 boss w SP2ExecBoot=UNACKNOWLEDGED; SP2PhaseBoot=UNACKNOWLEDGED; Issued boss iack and boss mechIack. Issued boss reloadControllers controllers=sp2daq Still receiving following warning message: 01:46:12 .boss 0 boss w text="failed to start DAQ sp2daq, disconnecting myself, [Failure instance: Traceback (failure with no frames): : timed out" 01:46:12 .boss 0 boss w RawLine="]\"" Power cycled SP2 and performed the above commands. sp2daq is not responding. We received a phase boot error and issued boss iack and boss mechIack. We checked the ethernet cord and found a dead moth near it. We reseated the cable and reloaded controllers. SP2DAQ returned. Took a test bias and short dark - these look fine. SP1Micro is now not talking, but this may be a stale alert from the previous troubleshooting/power cycling we did to find the sp2daq. Clearing alerts actor and resuming afternoon checkout tasks. We will be updating the boss troubleshooting wiki page to update on this particular problem. Resuming MANGA CALS: 01:55:24 boss exposure 0.0, file=288775, C 01:56:44 boss exposure 20.0, file=288776, C BOSS COLLIMATION: 02:08:22 boss exposure 4.0, file=288777, C 02:09:33 boss exposure 4.0, file=288778, C 02:10:58 sp1.motor.move= 346, 345, 343 02:11:04 sp2.motor.move= 1892, 1893, 1891 02:11:04 Hartmann collimate output on cart #6 sp1: offset: r= -659 (In Focus); b= -4.2 (Out of focus) sp2: offset: r= 2524 (Out of focus); b= -3.9 (Out of focus) sp1: pred. move: spAverageMove= 345 sp2: pred. move: spAverageMove= 1892 sp1: pred. resid: r= -1004, b= -3.1, txt= OK, spTemp = 12.8 sp2: pred. resid: r= 632, b= 2.0, txt= OK, spTemp = 11.1 Did not adjust the blue rings. According to last night's numbers, these are in spec. motorPosition sp1: 28092, 27740, 26757; sp2: 22090, 22309, 21917; ---------- Median spectrographs temperatures: Tsp1= 12.8, Tsp2= 11.1; Outside air temperature= 12.3; MANGA CALS: 02:12:55 boss exposure 25.0, file=288779, C 02:15:45 boss exposure 4.0, file=288780, C Inspected APOGEE evening cals - all ok. Disk check: df -kh Filesystem Size Used Avail Use% Mounted on /dev/vda2 7.7G 6.1G 1.3G 83% / tmpfs 3.4G 0 3.4G 0% /dev/shm /dev/vda1 194M 128M 57M 70% /boot /dev/vdb 19G 8.8G 9.1G 49% /exports/hub-local /dev/vdd 688G 623G 65G 91% /home /dev/vdc1 11T 10T 818G 93% /data Delayed ventilation and opening due to cloud cover and nearby radar. 02:30Z Cart bay and telescope ready. Thick clouds, radar all night kept us closed. Some rain. Lost 8.1 hr of MaNGA due to weather. 10:32Z-Started APOGEE and BOSS calibrations. ---- Mirror Numbers ---- PRIMARY: -------- Scale: 1.000000; MIGS TONIGHT NOMINAL Axial A: -5.314 -5.316 Axial B: -5.071 -5.072 Axial C: -5.33 -5.332 Trans D: 10.739 10.73 Lateral E: 9.0805 9.0805 Lateral F: 14.2113 14.1859 GALILS Commanded: 10250 -1100 7100 16850 12400 12400 Actual: 10250 -1100 7100 16850 12400 12400 SETMIR VALUES PrimDesOrient: 0.0 3.27 2.1 115.0 -178.65 PrimOrient: -1.06 3.29 2.19 115.07 -177.78 SECONDARY: ---------- Focus: 0.0000 Air temp: 13.5 Altitude: 30.0 MIGS TONIGHT NOMINAL Axial A: -0.905 -0.904 Axial B: -0.904 -0.903 Axial C: -0.294 -0.292 Trans D: -0.093 -0.1 GALILS Commanded: 1117850 1123500 1079800 -5700 -7450 Actual: 1085969 1107046 1046454 -5700 -7450 SETMIR VALUES SecDesOrient: 857.0 18.54 2.85 -17.2 138.98 SecOrient: 856.97 18.54 2.87 -16.95 138.78 ---- Telescope Status ---- No interlocks bypassed Status at: Telescope stowed at: 121.0, 30.0, 0.0 mount Instrument mounted: cartridge = 6 Counterweights at: 432, 431, 433, 433 LN2 autofill systems: Connected and turned on 180L LN2 dewar scales: SP1 206.8 lbs, 22.5 psi SP2 209.5 lbs, 21.8 psi ---- Software Used ---- Get Versions, 04/30/19, 00:20:30 -- STUI software: STUI : 1.7.7 alerts : trunk apo : trunk boss : v4_0 -boss.specMec : 3.0.4 gcamera : v2_2 guider : 3.8.1 hartmann : 1.6.0 hub : Tag: v4.2 mcp : v7_0_4 -mcp.plc : 34 -mcp.azFiduci : v1_136 -mcp.altFiduc : v1_136 -mcp.rotFiduc : v1_136 platedb : 2.8.0 sop : 3.12.3 tcc : 1.3.1 apogee : not availble apogeecal : not availble apogeeql : v1_6_1 APO-local : trunk -- done -- Home Disk Hogs (GB): muna 132.068 eboss 83.4139 manga 65.0463 sdss3 27.1813 apogee 17.5433 logging 13.6985 observer 10.1158 Filesystem Size Used Avail Use% Mounted on /dev/vdd 688G 623G 65G 91% /home /dev/vdc1 11T 10T 822G 93% /data Setting up: idlspec2d Flavor: Linux64 Version: v5_7_12 Setting up: |elodie Flavor: Linux64 Version: v1_3 Setting up: |spinspect Flavor: Linux64 Version: trunk Setting up: |specflat Flavor: Linux64 Version: v1_20 Setting up: |speclog Flavor: Linux64 Version: trunk Setting up: |platelist Flavor: Linux64 Version: trunk Setting up: |sdss_python_module Flavor: Linux64 Version: v1_6_7 Setting up: |sqlalchemy Flavor: Linux64 Version: 0.9.7 Setting up: |psycopg2 Flavor: Linux64 Version: 2.5.3 Setting up: |idlutils Flavor: Linux64 Version: v5_5_16 Setting up: |idl Flavor: Linux64 Version: 7.1 Setting up: |dust Flavor: Linux64 Version: v0_1 Setting up: |photolog Flavor: Linux64 Version: trunk Setting up: actorcore Flavor: Linux64 Version: v4_1_6 Setting up: |opscore Flavor: Linux64 Version: v2_5 Setting up: |external Flavor: Linux64 Version: v1_2_0 Setting up: |actorkeys Flavor: Linux64 Version: trunk Setting up: |plc Flavor: Linux64 Version: v30_2 Setting up: |sdsstools Flavor: Linux64 Version: v4_33 Setting up: |sdss3tools Flavor: Linux64 Version: v1_7_13 Setting up: |RO Flavor: Linux64 Version: 3.6.9 -- Other versions ---- idlspec2d : v5_7_12 plate-mapper3 : NOCVS:v6_0_12 petunia : 1.4.1 autoscheduler : 1.2.0 sdss_python_module: trunk mangadrp : trunk exported ---- APOGEE Data Summary ---- APOGEE data list, mjd=58603, all raw_data: /data/apogee/utr_cdr/58603/apRaw-*.fits archive: /data/apogee/archive/58603/apR-[a,b,c]-*.apz quickred: /data/apogee/quickred/58603/ap2D-a-*.fits.fz -------------------------------------------------------------------------------- mjd i UT File/Exp Imtype Nread A/B Dth Ct-Plate Arch Offset %Flux Comm -------------------------------------------------------------------------------- 58603 1 00:10 30410001 Dark 3 A 13.0 6-11454 a-b-c - - 58603 2 00:16 30410002 Dark 60 A 13.0 6-11454 a-b-c - - 58603 3 00:27 30410003 QuarFlat 10 A 13.0 6-11454 a-b-c - 48 58603 4 00:29 30410004 Arc-Thar 12 A 13.0 6-11454 a-b-c 0.41 - 58603 5 00:31 30410005 Arc-Une 40 A 13.0 6-11454 a-b-c - - 58603 6 00:38 30410006 Arc-Thar 12 B 13.5 6-11454 a-b-c 0.90 - 58603 7 00:40 30410007 Arc-Une 40 B 13.5 6-11454 a-b-c - - 58603 8 00:48 30410008 Dark 60 A 13.0 6-11454 a-b-c - - -------------------------------------------------------------------------------- ---- MaNGA Data Summary ---- Reading FITS from /data/spectro/58603/ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - n time filename crt plt flav Shu Dth exp colA colB colC Hrt b1 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 1 01:55 b1-00288775 6 11454 bia Clo C 0.0 27746 27395 26414 01:55 b2-00288775 6 11454 bia Clo C 0.0 20198 20416 20026 01:55 r1-00288775 6 11454 bia Clo C 0.0 27746 27395 26414 01:55 r2-00288775 6 11454 bia Clo C 0.0 20198 20416 20026 2 01:56 b1-00288776 6 11454 dar Clo C 20.0 27746 27395 26414 01:56 b2-00288776 6 11454 dar Clo C 20.0 20198 20416 20026 01:56 r1-00288776 6 11454 dar Clo C 20.0 27746 27395 26414 01:56 r2-00288776 6 11454 dar Clo C 20.0 20198 20416 20026 3 02:08 b1-00288777 6 11454 arc Lef C 4.1 27746 27395 26414 HART 02:08 b2-00288777 6 11454 arc Lef C 4.2 20198 20416 20026 HART 02:08 r1-00288777 6 11454 arc Lef C 4.1 27746 27395 26414 HART 02:08 r2-00288777 6 11454 arc Lef C 4.2 20198 20416 20026 HART 4 02:09 b1-00288778 6 11454 arc Rig C 4.1 27746 27395 26414 HART 02:09 b2-00288778 6 11454 arc Rig C 4.1 20198 20416 20026 HART 02:09 r1-00288778 6 11454 arc Rig C 4.1 27746 27395 26414 HART 02:09 r2-00288778 6 11454 arc Rig C 4.1 20198 20416 20026 HART 5 02:12 b1-00288779 6 11454 fla Out C 25.1 28092 27740 26757 02:12 b2-00288779 6 11454 fla Out C 25.2 22090 22309 21917 02:12 r1-00288779 6 11454 fla Out C 25.1 28092 27740 26757 02:12 r2-00288779 6 11454 fla Out C 25.2 22090 22309 21917 6 02:15 b1-00288780 6 11454 arc Out C 4.1 28092 27740 26757 02:15 b2-00288780 6 11454 arc Out C 4.2 22090 22309 21917 02:15 r1-00288780 6 11454 arc Out C 4.1 28092 27740 26757 02:15 r2-00288780 6 11454 arc Out C 4.2 22090 22309 21917 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - --- OK ---