Subject: 2.5m obslog 20250226 (SJD 60733) From: doravetz@apo.nmsu.edu Date: Thu, 27 Feb 2025 12:04:51 +0000 (UTC) APACHE POINT OBSERVATORY SDSS 2.5M OBSERVING LOG Wednesday February 26, 2025 (SJD 60733) ---- Observing Team ---- Dan Oravetz ---- Observing Plan ---- Wed 26 Feb 2025 60733 dark (10.3 h) plus 0.3 h for 12 deg design ---- Observing Summary ---- The night was lost to weather tonight. Clouds and radar returns materialized before sunset and persisted well into the second half of the night. The humidity, dust counts, and wind gusts also increased throughout the night, with wind gusts exceeding ~45 mph. While closed, I repeatedly commanded the arc lamps on and off to test the reliability of the circuit against the breaker tripping with the substitute transformer in place as of this afternoon. The first cycle tripped the breaker immediately, then the second cycle completed 100 on/off sequences successfully. However, the breaker tripped for each of the next eight attempted sequences, and in each of those eight cases the breaker tripped early on in each cycle -- all tripped within the first 8 on/off iterations, except for one instance which tripped on the 11th iteration. (Time stamps are in the log body in case they might be of use.) ---- Observing Log ---- ============================================================================ AFTERNOON CHECKOUT ============================================================================ All systems alive and responsive. Disk space: Filesystem Size Used Avail Use% Mounted on /dev/sdb1 2.0T 459G 1.6T 23% /home /dev/sdd1 22T 13T 8.5T 60% /data FPS powered on. Positioners are home/folded: 00:41:34 APO.doravetz 31 jaeger i folded=T 00:41:34 APO.doravetz 31 jaeger : text="All positioners are folded." Loaded a manual folded configuration: 00:43:01 CmdStarted: APO.doravetz 32 jaeger configuration load --from-positions 00:43:03 APO.doravetz 32 jaeger d text="Resetting cherno offsets." 00:43:03 APO.doravetz 32 jaeger d text="Focal plane scale: 1.0." 00:43:03 APO.doravetz 32 jaeger d configuration_loaded=19619,-999,-999,-999.0,-999.0,0,-999.0,-999.0,/home/sdss5/software/sdsscore/main/apo/summary_files/019XXX/0196XX/confSummary-19619.par,F 00:43:08 APO.doravetz 32 jaeger i configuration_snapshot=/data/logs/jaeger/configuration_snapshots/60733/configuration_snapshot_60733_19619.pdf 00:43:08 APO.doravetz 32 jaeger d design_preloaded=-999 00:43:08 APO.doravetz 32 jaeger : text="Configuration 19619 loaded." 00:43:08 CmdDone: APO.doravetz 32 jaeger configuration load --from-positions Took a '--no-apply' FVC image: FVC loop results: fvc_rms= 71.84 fvc_deltarms= 49.61 fvc_filename=/data/fcam/60733/fimg-fvc1n-0001.fits fvc_perc_90= 89.50 fvc_percent_reached= 1.20 fvc iterations= 1 Scattered-light check: 00:45:08 CmdStarted: APO.doravetz 34 fliswarm talk -n fvc expose 50 fimg-fvc1n-0002.fits shows no evidence of light artifacts. Exercised positioners: permanently_disabled=201,275,460,478,503,535,639,716,802,846,935,984,985,995,997,1026,1043,1049,1202,1231,1246,1290 manually_disabled=1187 00:48:56 CmdStarted: APO.doravetz 35 jaeger configuration random --danger 00:50:10 APO.doravetz 35 jaeger d trajectory_dump_file=/data/logs/jaeger/trajectories/60733/trajectory-60733-0001.json 00:50:11 CmdDone: APO.doravetz 35 jaeger configuration random --danger 00:50:16 .jaeger 0 jaeger i snapshot=/data/logs/jaeger/snapshots/60733/fps_snapshot_60733_0001.pdf 00:51:53 CmdStarted: APO.doravetz 36 jaeger configuration reverse GFA biases/darks: 00:53:48 CmdStarted: APO.doravetz 37 fliswarm talk -c gfa expose --bias --count 10 00:55:09 CmdStarted: APO.doravetz 38 fliswarm talk -c gfa expose --dark --count 5 5 BOSS arc and flat: 01:12:06 BOSS expNum=382201, expTime= 4.0s 01:13:57 BOSS expNum=382202, expTime= 25.0s Ventilation delayed; weather conditions were only getting worse and opening soon was not going to happen. ============================================================================ NIGHT ============================================================================ While closed I spent a couple of hours in the lower enclosure with a laptop in front of the circuit breaker running loops of arc lamps toggling on/off. The first attempt failed immediately: 03:08:33 CmdStarted: APO.doravetz 52 mcp hgcd.on Reset breaker #11 and then looped the following sequence 100 times from a STUI Loop Commands script, which completed successfully: mcp hgcd.on mcp ne.on tui wait 2 mcp hgcd.off mcp ne.off This time, for the sake of trying something different, I flip-flopped the order of the Ne and HgCd arc lamp commands. Breaker #11 tripped on the 6th iteration: 5 (100) --- 04:33:50 mcp ne.on mcp hgcd.on tui wait 2 mcp ne.off mcp hgcd.off 6 (100) --- 04:34:04 mcp ne.on After resetting the breaker I tried again, but it tripped on the first attempt: 1 (100) --- 04:35:37 mcp ne.on Reset breaker and tried again, but it failed on the third attempt: 3 (100) --- 04:36:39 mcp ne.on Reset breaker and switched order of lamps being commanded-on again to the same sequence used previously for the 100 successfully completed sequences. mcp hgcd.on mcp ne.on tui wait 2 mcp hgcd.off mcp ne.off 100 cycles completed successfully. Switched back to having Ne commanded on first again and breaker tripped on 7th cycle: 7 (100) --- 05:05:42 mcp ne.on This time I ran a sequence of only commanding the Ne lamps on and off, but the breaker tripped on the 8th iteration: 8 (100) --- 05:08:36 mcp ne.on Next, I tried a sequence of commanding only the HgCd lamps and the breaker tripped on the 11th iteration: 11 (100) --- 05:11:45 mcp hgcd.on Reset breaker and tried again, but it tripped on the third attempt of the next cycle: 3 (100) --- 05:20:25 mcp hgcd.on With the breaker tripping often for both Ne and HgCd I called it quits, reset the breaker, and tested one cycle of the arc lamps turning on and off successfully before returning to the ops building. 09:07Z - Wind gusts currently 43 mph, humidity 75%, and dust counts not currently reporting. 12:01Z - Weather is still not fit to open. Calling it a night. FPS is powered-off. Positioners are home/folded. APOGEE morning cals started. ---- Mirror Numbers ---- PRIMARY: -------- Scale: 1.000000 MIGS TONIGHT NOMINAL Axial A: 0.0030 0.0040 Axial B: -0.0008 -0.0010 Axial C: -0.0030 -0.0060 Trans D: -0.0090 -0.0100 Lateral E: 0.3380 0.3380 Lateral F: 0.5455 0.5455 GALILS Commanded: 9800 -300 6750 19150 -3550 -3250 Actual: 9800 -300 6750 19150 -3550 -3250 SETMIR VALUES PrimDesOrient: 0.0 -18.5 -7.8 -427.0 -998.0 PrimOrient: 0.0 -18.23 -7.9 -427.17 -999.0 SECONDARY: ---------- Focus: 0.0000 Air temp: 0.00 Altitude: 30.0 MIGS TONIGHT NOMINAL Axial A: -0.1020 -0.0970 Axial B: -0.0630 -0.0650 Axial C: -0.0430 -0.0420 Trans D: 0.0060 -0.0030 GALILS Commanded: 5887150 5829000 5957350 -1500 -7250 Actual: 5883548 5811926 5906521 -1500 -7250 SETMIR VALUES SecDesOrient: 4657.0 -24.5 -17.5 18.0 125.41 SecOrient: 4657.12 -24.58 -17.39 17.76 125.31 ---- Telescope Status ---- Status at: 10:00Z Telescope stowed at: 121.0, 30.0, 0.0 mount Instrument mounted: FPS Counterweights at: 652 654 651 650 LN2 autofill systems: Connected and turned on 180L LN2 dewar scales: SP1 215 lbs, 24 psi FPS Status ========== CP10 Loop = 7.73 L/min GFA Loop = 2.44 L/min Supply pressure = 44.1 psi Return pressure = 29.2 psi Chiller Status ============== Set temperarature = 1.4 C Flow = 4.9 Gpm ---- Software Used ---- STUI : 1.7.19 alerts : 2.3.0 boss : 6.0.2 -boss.specMec : 3.0.4 apo : 4.1.0 fliswarm : 0.5.1a0 jaeger : 1.9.1 kaiju : 1.4.0 coordio : 1.14.0 fps_calibrati : 2025.02.24 cherno : 1.0.1a0 astrometrynet : 0.93 hal : 1.4.1 hartmann : 2.0.2 hub : 5.0.1a0 mcp : 2.1 -mcp.plc : 35 -mcp.azFiduci : v1_150 -mcp.altFiduc : v1_150 -mcp.rotFiduc : v1_150 tcc : 1.4.0 apogee : not available apogeecal : not available apogeeql : v1_7_0dev STUIAdditions : 3.1.18 boson : 0.4.9 sdss-obstools : 3.9.3 TPM : 1.2-31-gd7eb MCP : 2.1-1-ge461 PLC : V35 Fiducials : v1_150 idlspec2d : v6_2_0 Kronos : 2.0.0 roboscheduler : 2.1.0 boson : 0.4.9 sdss-obstools : 3.9.3 TPM : 1.2-31-gd7eb MCP : 2.1-1-ge461 PLC : V35 Fiducials : v1_150 idlspec2d : v6_2_0 Kronos : 2.0.0 roboscheduler : 2.1.0 ---- APOGEE Data Summary ---- 60732 morning cals: num field design config exptype dithpix nread relflux okay dx dy sigx sigy fpi1 fpi2 -------- ------ ------ ------ ------------ ------- ----- ------- ----- ------- ------- ------ ------ ---- ---- 51700077 100138 711438 19618 DARK 13.5 60 -15 True -- -- -- -- -- -- 51700078 100138 711438 19618 DARK 13.5 60 -3 True -- -- -- -- -- -- 51700079 100138 711438 19618 DARK 13.5 60 -5 True -- -- -- -- -- -- 51700080 100138 711438 19618 QUARTZFLAT 13.5 10 105 True -- -0.18 -- 0.87 -- -- 51700081 100138 711438 19618 QUARTZFLAT 13.5 10 105 True -- -0.19 -- 0.88 -- -- 51700082 100138 711438 19618 QUARTZFLAT 13.5 10 104 True -- -0.18 -- 0.86 -- -- 51700083 100138 711438 19618 ARCLAMP-THAR 13.5 12 102 True 4.03 0.60 1.08 0.89 -- -- 51700084 100138 711438 19618 ARCLAMP-UNE 13.5 40 100 True 3.64 0.47 1.08 0.87 -- -- 51700085 100138 711438 19618 ARCLAMP-THAR 13.0 12 93 True 3.51 0.63 1.07 0.87 -- -- 51700086 100138 711438 19618 ARCLAMP-UNE 13.0 40 97 True 3.12 0.43 1.11 0.92 -- -- 51700087 100138 711438 19618 DARK 13.0 30 0 False -- -- -- -- -- -- 51700088 100138 711438 19618 INTERNALFLAT 13.0 30 99 True -- -- -- -- -- -- 51700089 100138 711438 19618 INTERNALFLAT 13.0 30 100 True -- -- -- -- -- -- 51700090 100138 711438 19618 INTERNALFLAT 13.0 30 100 True -- -- -- -- -- -- 51700091 100138 711438 19618 DARK 13.0 30 277 True -- -- -- -- -- -- 51700092 100138 711438 19618 FPI 13.0 30 199 True 14.97 0.39 1.16 0.89 -- -- 51700093 100138 711438 19618 FPI 13.5 30 202 True -9.16 0.42 1.19 0.88 -- -- num exptype nread mtp1 mtp2 mtp3 mtp4 mtp5 mtp6 mtp7 mtp8 mtp9 mtp10 -------- ---------- ----- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----- 51700080 QUARTZFLAT 10 66 70 81 90 90 85 87 89 89 82 51700081 QUARTZFLAT 10 69 73 82 91 90 85 88 89 89 83 51700082 QUARTZFLAT 10 69 74 82 91 91 85 88 89 89 83 (observers39) [observer@sdss5-hub: ~]$ check_ap 60733 num field design config exptype dithpix nread relflux okay dx dy sigx sigy fpi1 fpi2 -------- ------ ------ ------ ------------ ------- ----- ------- ----- ------- ------- ------ ------ ---- ---- 51710001 100138 711438 19618 ARCLAMP-THAR 13.0 10 89 True 3.51 0.60 1.06 0.86 -- -- 51710002 100138 711438 19618 ARCLAMP-THAR 10.5 10 97 True 1.00 0.61 1.08 0.90 -- -- 51710003 100138 711438 19618 ARCLAMP-THAR 13.5 10 85 False 1.47 0.58 1.06 0.88 -- -- 51710004 100138 711438 19618 ARCLAMP-THAR 13.0 10 83 False 0.99 0.62 1.08 0.90 -- -- 51710005 100138 711438 19618 DARK 13.0 3 -33 True -- -- -- -- -- -- 51710006 -999 -999 19619 DARK 13.0 60 -30 True -- -- -- -- -- -- 51710007 -999 -999 19619 QUARTZFLAT 13.0 10 104 True -- -0.26 -- 0.87 -- -- 51710008 -999 -999 19619 ARCLAMP-THAR 13.5 12 82 False 1.49 0.59 1.06 0.86 -- -- 51710009 -999 -999 19619 ARCLAMP-UNE 13.5 40 108 True 1.08 0.34 1.11 0.88 -- -- 51710010 -999 -999 19619 ARCLAMP-THAR 13.0 12 95 True 1.01 0.56 1.07 0.89 -- -- 51710011 -999 -999 19619 ARCLAMP-UNE 13.0 40 103 True 0.62 0.40 1.08 0.86 -- -- 51710012 -999 -999 19619 DARK 13.0 60 -2 True -- -- -- -- -- -- 51710013 -999 -999 19619 FPI 13.0 30 208 True -12.15 0.32 1.16 0.87 -- -- 51710014 -999 -999 19619 FPI 13.5 30 210 True 12.92 0.31 1.17 0.88 -- -- num exptype nread mtp1 mtp2 mtp3 mtp4 mtp5 mtp6 mtp7 mtp8 mtp9 mtp10 -------- ---------- ----- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----- 51710007 QUARTZFLAT 10 66 70 81 91 91 85 88 90 89 82 ---- BOSS Data Summary ---- Reading FITS from /data/spectro//60733 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - n Time Filename design-field cfg flav shu Exp colA colB colC Hrt - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 1 00:33 r1-00382196 ----------- ---- bia Clo 0.0 1070 1201 1184 2 00:33 b1-00382196 ----------- ---- bia Clo 0.0 1070 1201 1184 3 01:07 b1-00382197 -999--999 19619 arc Lef 4.1 1070 1201 1184 HART 4 01:07 r1-00382197 -999--999 19619 arc Lef 4.1 1070 1201 1184 HART 5 01:08 r1-00382198 -999--999 19619 arc Rig 4.1 1070 1201 1184 HART 6 01:08 b1-00382198 -999--999 19619 arc Rig 4.1 1070 1201 1184 HART 7 01:10 r1-00382199 -999--999 19619 arc Lef 4.1 1423 1554 1538 HART 8 01:10 b1-00382199 -999--999 19619 arc Lef 4.1 1423 1554 1538 HART 9 01:11 r1-00382200 -999--999 19619 arc Rig 4.1 1423 1554 1538 HART 10 01:11 b1-00382200 -999--999 19619 arc Rig 4.1 1423 1554 1538 HART 11 01:12 b1-00382201 -999--999 19619 arc Out 4.1 1615 1746 1729 12 01:12 r1-00382201 -999--999 19619 arc Out 4.1 1615 1746 1729 13 01:13 r1-00382202 -999--999 19619 fla Out 25.1 1615 1746 1729 14 01:13 b1-00382202 -999--999 19619 fla Out 25.1 1615 1746 1729 ---- BOSS Spectrographs Collimation ---- Afternoon focus check ===================== 01:10:23 BOSS expNum=382199, expTime= 4.0s 01:11:03 BOSS expNum=382200, expTime= 4.0s 01:11:44 sp1.motor.move= 192, 192, 191 01:11:44 Hartmann collimate output sp1: offset: r= 0 (In Focus); b= -1.2 (In Focus) sp1: pred. move: spAverageMove= 192 sp1: pred. resid: r= -192, b= -0.6, txt= OK, spTemp = 7.3 ---- Time Tracking ---- *=*=*= Observatory: APO *=*=*= MJD: 60733 Fill night time in Hours below *=*=*= Total bright time : 0.3 *=*=*= Bright time, queued observations : 0 *=*=*= Bright time, commissioning/engineering: 0 *=*=*= Bright time lost to weather : 0.3 *=*=*= Bright time lost to technical issues : 0 *=*=*= Total dark time : 10.3 *=*=*= Dark time, queued observations : 0 *=*=*= Dark time, commissioning/engineering : 0 *=*=*= Dark time lost to weather : 10.3 *=*=*= Dark time lost to technical issues : 0