Date: Tue, 4 Apr 2023 12:54:24 -0600 Subject: Re: [sloan-25mlog 6664] 2.5m obslog 20230403 (SJD 60038) From: "Daniel Oravetz" That one was taken when I had intentionally disconnected sp1mech, so I'm not at all surprised it didn't complete correctly. I think it would've been more surprising if it actually had looked like a normal exposure file in this case. > Just as a note, one of the bais frames (sdR-b1-00354483.fit.gz. > sdR-r1-00354483.fit.gz) taken during the "Checking the specMechProtocol > alert" was missing some of the header cards including EXPTIME. I don't > know if this has to do with how the data was taken or if it is a symptom > of something so I wanted to pass on the observation. > > Sean Morrison > > SDSS-V BHM BOSS Pipeline Team > Postdoctoral Research Associate > Department of Astronomy > University of Illinois at Urbana-Champaign > 1002 W Green Street > Urbana, IL 61801 > smorris0@illinois.edu > ________________________________ > From: sloan-25mlog on behalf of > doravetz@apo.nmsu.edu > Sent: Tuesday, April 4, 2023 6:38 AM > To: obs-report-25m@apo.nmsu.edu > Subject: [sloan-25mlog 6664] 2.5m obslog 20230403 (SJD 60038) > > > APACHE POINT OBSERVATORY SDSS 2.5M > > OBSERVING LOG > Monday April 03, 2023 > > (SJD 60038) > > ---- Observing Team ---- > > Dan Oravetz > > ---- Observing Plan ---- > > Mon 03 Apr 2023 60038 0.96 02:18 02:33 11:38 08:19 17:25 Day bright ( 9.3 > h, w/ 0.2 h for 12 deg design) > BOSS dithers if conditions warrant > > ---- Observing Summary ---- > > The night was lost to high winds. > > ---- Observing Log ---- > > ============================================================================ > AFTERNOON CHECKOUT > ============================================================================ > > All systems alive and responsive. Windy and dust is picking up. APOGEE > evening cals started. > > Afternoon disk space: > > Filesystem Size Used Avail Use% Mounted on > /dev/sdb1 2.0T 250G 1.8T 13% /home > /dev/sdd1 22T 15T 6.7T 69% /data > > FPS powered on. Positioners are home/folded: > > 01:25:41 APO.doravetz 115 jaeger i folded=T > 01:25:41 APO.doravetz 115 jaeger : text="All positioners are folded." > > Loaded a manual folded configuration: > > 01:25:56 CmdStarted: APO.doravetz 116 jaeger configuration load > --from-positions > 01:26:02 APO.doravetz 116 jaeger d text="Focal plane scale: 1.0." > 01:26:02 APO.doravetz 116 jaeger d > configuration_loaded=9324,-999,-999,-999.0,-999.0,0,-999.0,-999.0,/home/sdss5/software/sdsscore/main/apo/summary_files/0093XX/confSummary-9324.par,F > 01:26:07 APO.doravetz 116 jaeger i > configuration_snapshot=/data/logs/jaeger/configuration_snapshots/60038/configuration_snapshot_60038_9324.pdf > 01:26:07 APO.doravetz 116 jaeger d design_preloaded=-999 > 01:26:07 APO.doravetz 116 jaeger : text="Configuration 9324 loaded." > 01:26:07 CmdDone: APO.doravetz 116 jaeger configuration load > --from-positions > > Took a '--no-apply' FVC image: > > 01:26:28 CmdStarted: APO.doravetz 117 jaeger fvc loop --no-apply > 01:26:56 FVC Loop > fvc_rms= 34.85 > fvc_deltarms= 35.89 > fvc_filename=/data/fcam/60038/fimg-fvc1n-0001.fits > fvc_perc_90= 51.60 > fvc_percent_reached= 3.60 > fvc iterations= 1 > > Scattered-light check: > > 01:27:21 CmdStarted: APO.doravetz 118 fliswarm talk -n fvc expose 50 > 01:27:26 CmdStarted: APO.doravetz 119 boss exposure science itime=300 > > fimg-fvc1n-0002.fits shows no evidence of light artifacts. > > Ventilation delayed due to high winds and dust. > > ============================================================================ > NIGHT > ============================================================================ > > Did not open at sunset due to high winds - gusts > 45 mph. > > 03:35Z - Wind gusts have come down. Now hovering around 35 mph. Currently > 34.1 mph, but going to give it some time to make sure it's actually going > to hold steady. > > 04:27Z - Wind gusts back up to > 39 mph. > > 05:35Z - Wind gusts down to 35.2 mph at the moment. Sky is clear and dust > levels are acceptable. Still just waiting on the wind to calm down and > stabilize. > > 06:01Z - Wind gusts back up to 39 mph. > > 08:41Z - Wind gusts still above the closure limit, but sometimes it looks > like they could improve. Still keeping an eye on them. Force-filled BOSS > just in case there's a chance to open later. > > 09:45Z - Still too windy. > > 10:18Z - Wind gusts 39 mph. 1.0um dust counts are beginning to rise. > > Started APOGEE morning cals. FPS powered off and positioners are > home/folded. > > ---- Mirror Numbers ---- > > PRIMARY: > -------- > Scale: 1.000000 > > MIGS TONIGHT NOMINAL > Axial A: 0.0120 0.0090 > Axial B: 0.0050 0.0020 > Axial C: -0.0260 -0.0290 > Trans D: 0.0700 0.0580 > Lateral E: 0.3315 0.3315 > Lateral F: 0.5350 0.5345 > > GALILS > Commanded: 12500 -1600 5300 47050 -8800 -8400 > Actual: 12500 -1600 5300 47050 -8800 -8400 > > SETMIR VALUES > PrimDesOrient: 0.0 -59.5 -9.15 -512.0 1906.3 > PrimOrient: 1.07 -59.51 -9.21 -512.13 1906.31 > > SECONDARY: > ---------- > Focus: 0.0000 > Air temp: 0.00 > Altitude: 30.0 > > MIGS TONIGHT NOMINAL > Axial A: -0.0320 -0.0310 > Axial B: -0.0270 -0.0260 > Axial C: -0.0500 -0.0470 > Trans D: -0.0150 -0.0270 > > GALILS > Commanded: 4894200 4865450 4961950 -50 -3450 > Actual: 4859230 4859117 4859211 -50 -3450 > > SETMIR VALUES > SecDesOrient: 3857.0 0.0 0.0 0.0 42.22 > SecOrient: 3856.97 -0.01 -0.03 -0.02 42.87 > > ---- Telescope Status ---- > > Status at: 10:50Z > > Telescope stowed at: 121.0, 30.0, 0.0 mount > Instrument mounted: FPS > Counterweights at: 696 696 714 713 > LN2 autofill systems: Connected and turned on > 180L LN2 dewar scales: SP1 132 lbs, 22 psi > > FPS Status > ========== > CP10 Loop = 7.31 L/min > GFA Loop = 1.65 L/min > Supply pressure = 40.8 psi > Return pressure = 28.4 psi > > Chiller Status > ============== > Set temperarature = 5.5 C > Flow = 5.4 Gpm > > ---- Software Used ---- > > STUI : 1.7.19 > alerts : 2.3.0 > boss : 6.0.2-alpha.0 > -boss.specMec : 3.0.4 > apo : 3.0.1 > fliswarm : 0.4.0 > jaeger : 1.3.4a0 > kaiju : 1.2.4-alpha.0 > coordio : 1.6.2a0 > fps_calibrati : 2023.01.05 > cherno : 0.5.2 > astrometrynet : 0.93 > hal : 0.6.0b5 > hartmann : 2.0.2 > hub : 5.0.1a0 > mcp : 2.1 > -mcp.plc : 35 > -mcp.azFiduci : v1_147 > -mcp.altFiduc : v1_147 > -mcp.rotFiduc : v1_147 > tcc : 1.4.0 > apogee : not available > apogeecal : not available > apogeeql : v1_7_0dev > STUIAdditions : 3.1.20 > Boson : 0.3.2 > sdss-obstools : 3.9.3 > TPM : 1.2-31-gd7eb > MCP : 2.1-1-ge461 > PLC : V35 > Fiducials : v1_147 > idlspec2d : v6_0_9 > Kronos : 1.6.1 > roboscheduler : 1.3.2 > > ---- APOGEE Data Summary ---- > > 60037 morning cals: > > raw_data: /data/apogee/utr_cdr/60037/apRaw-*.fits > archive: /data/apogee/archive/60037/apR-[a,b,c]-*.apz > quickred: /data/apogee/quickred/60037/apq-*.fits > ------------------------------------------------------------------------------ > mjd i UT File/Exp Imtype Nread A/B Dth Design-Cfg Arch Offset > %Flux/FPI Comm > ------------------------------------------------------------------------------ > 60037 70 12:03 44750070 Dark 60 B 13.5 ---------- a-b-c - > - > 60037 71 12:13 44750071 Dark 60 B 13.5 ---------- a-b-c - > - > 60037 72 12:24 44750072 Dark 60 B 13.5 ---------- a-b-c - > - > 60037 73 12:26 44750073 QuarFlat 10 B 13.5 ---------- a-b-c - > - > 60037 74 12:28 44750074 QuarFlat 10 B 13.5 ---------- a-b-c - > - > 60037 75 12:29 44750075 QuarFlat 10 B 13.5 ---------- a-b-c - > - > 60037 76 12:32 44750076 Arc-Thar 12 B 13.5 ---------- a-b-c 0.87 > - > 60037 77 12:39 44750077 Arc-Une 40 B 13.5 ---------- a-b-c - > - > 60037 78 12:41 44750078 Arc-Thar 12 A 13.0 ---------- a-b-c 0.35 > - > 60037 79 12:48 44750079 Arc-Une 40 A 13.0 ---------- a-b-c - > - > 60037 80 12:54 44750080 Dark 30 A 13.0 ---------- a-b-c - > - > 60037 81 12:59 44750081 InteFlat 30 A 13.0 ---------- a-b-c - > - > 60037 82 13:04 44750082 InteFlat 30 A 13.0 ---------- a-b-c - > - > 60037 83 13:10 44750083 InteFlat 30 A 13.0 ---------- a-b-c - > - > 60037 84 13:15 44750084 Dark 30 A 13.0 ---------- a-b-c - > - > 60037 85 13:21 44750085 Arc---- 30 A 13.0 ---------- a-b-c - > 89 FPI > ------------------------------------------------------------------------------ > > APOGEE data list, mjd=60038, all > raw_data: /data/apogee/utr_cdr/60038/apRaw-*.fits > archive: /data/apogee/archive/60038/apR-[a,b,c]-*.apz > quickred: /data/apogee/quickred/60038/apq-*.fits > ------------------------------------------------------------------------------ > mjd i UT File/Exp Imtype Nread A/B Dth Design-Cfg Arch Offset > %Flux/FPI Comm > ------------------------------------------------------------------------------ > 60038 1 21:40 44760001 Dark 3 A 13.0 ---------- a-b-c - > - > 60038 2 21:53 44760002 Dark 60 A 13.0 ---------- a-b-c - > - > 60038 3 21:54 44760003 QuarFlat 10 A 13.0 ---------- a-b-c - > - > 60038 4 21:57 44760004 Arc-Thar 12 B 13.5 ---------- a-b-c 0.83 > - > 60038 5 22:04 44760005 Arc-Une 40 B 13.5 ---------- a-b-c - > - > 60038 6 22:06 44760006 Arc-Thar 12 A 13.0 ---------- a-b-c 0.31 > - > 60038 7 22:13 44760007 Arc-Une 40 A 13.0 ---------- a-b-c - > - > 60038 8 22:24 44760008 Dark 60 A 13.0 ---------- a-b-c - > - > 60038 9 22:30 44760009 Arc---- 30 A 13.0 ---------- a-b-c - > 92 FPI > ------------------------------------------------------------------------------ > > ---- BOSS Spectrographs Collimation ---- > > Afternoon focus check > ===================== > 03:44:15 BOSS expNum=354487, expTime= 4.0s > 03:44:57 BOSS expNum=354488, expTime= 4.0s > 03:45:44 sp1.motor.move= 18, 19, 18 > 03:45:44 Hartmann collimate output > sp1: offset: r= -989 (In Focus); b= -3.2 (In Focus) > sp1: pred. move: spAverageMove= 19 > sp1: pred. resid: r= -1008, b= -3.2, txt= OK, spTemp = 6.1 > 03:45:36 APO.doravetz 138 hartmann i r1MeanOffset=-0.15,"In Focus" > 03:45:36 APO.doravetz 138 hartmann i b1MeanOffset=0.16,"In Focus" > > ---- Time Tracking ---- > > *=*=*= Observatory: APO > *=*=*= MJD: 60038 > Fill night time in Hours below > *=*=*= Total bright time : 9.3 > *=*=*= Bright time, queued observations : 9.3 > *=*=*= Bright time, commissioning/engineering: 0 > *=*=*= Bright time lost to weather : 0 > *=*=*= Bright time lost to technical issues : 0 > > *=*=*= Total dark time : 0 > *=*=*= Dark time, queued observations : 0 > *=*=*= Dark time, commissioning/engineering : 0 > *=*=*= Dark time lost to weather : 0 > *=*=*= Dark time lost to technical issues : 0 > > ---- Problems in Detail ---- > > Checking the specMechProtocol alert > =================================== > All three BOSS controllers are alive and connected: > > 03:14:46 CmdStarted: APO.doravetz 121 boss ping > > 03:14:46 APO.doravetz 121 boss i text="sp1daq: OK" > 03:14:46 APO.doravetz 121 boss i text="sp1cam: OK" > 03:14:47 APO.doravetz 121 boss i text="sp1mech: OK" > > Disconnected sp1mech: > > 03:15:18 CmdStarted: APO.doravetz 122 boss disconnectControllers > controllers=sp1mech > 03:15:18 .boss 0 boss w text="Controller sp1mech disconnected." > 03:15:18 .boss 0 boss w text="sp1mech removed from available hardware." > > Tried 'boss exposure bias'. That spawned an alert: > > Critical boss.specMechProtocol at 2023-04-04 03:16:36 UT found 0. > > Reloaded sp1mech: > > 03:16:49 CmdDone: APO.doravetz 125 boss reloadControllers > controllers=sp1mech > > Issued a 'boss mechIack' and all was fine. > > Once again, I disconnected sp1mech for several minutes but didn't try to > initiate an exposure. No alert was raised during this time, however. > > 03:20:48 CmdStarted: APO.doravetz 132 boss disconnectControllers > controllers=sp1mech > 03:23:55 CmdStarted: APO.doravetz 134 boss reloadControllers > controllers=sp1mech > > FYI, if you issue a 'boss ping' when a BOSS exposure is reading-out, > sp1cam always replies with a "DEAD" status. When the readout completes the > sp1cam status is "OK". > > ____________________________________________________________ > Logs of Nightly APO Activities > https://urldefense.com/v3/__https://trac.sdss.org/tracmailman/browser/private/sloan-25mlog/index.html__;!!DZ3fjg!795s-PbNLHfdLDmU6MxKEM97Xy2gJsZeVgiYKt7VA_A6JHeiqmc3YqrrRRDR7NIBrXFMkFQP3VosaEBIrPSyqDtO$ >