Date: Wed, 29 Dec 2021 04:21:07 -0700 Subject: Re: [sloan-25mlog 5817] 2.5m obslog 20211228 (SJD 59577) From: "Audrey Oravetz" Also, would someone have a look at GRAFANA? It continuously hangs when trying to access the GFA page. Cerebro was restarted a couple of times, which helped with the FPS page, but did nothing with the GFA page. Thanks! > > APACHE POINT OBSERVATORY SDSS 2.5M > > OBSERVING LOG > Tuesday December 28, 2021 > > (SJD 59577) > > ---- Observing Team ---- > > Dmitry Bizyaev > Audrey Oravetz > > Zoom: > Kaike, Karen, DanO, Katie > > > ---- Observing Plan ---- > > FPS commissioning tasks > > > > ---- Observing Summary ---- > > Cloudy, but we gathered: > > 9 loops of science data WITH fvc loop --no-apply in between dither > configuration > 0.1 / reverse sequence for a BOSS field (35884). > > Focus test - BOSS exposures at varying secondary mirror values (design > 35914). > > 3x3 telescope offset test with an undithered BOSS configuration (design > 35914), > spaced by 1" > > Notes: > ====== > > ** Hal goto-field failed on us tonight on the guider 'acquire' step > because the > routine did not open the FF petals automatically. Note that some of the > commands > were purposely omitted (05:18:11 CmdStarted: OBSERVER.dmbiz 715 hal > goto-field -s > prepare,slew,boss_hartmann,boss_arcs,boss_flat,fvc,acquire) ** > > ** We had many instances where the coolant flow to the GFAs was > practically zero. > We tweaked the chiller flow rate by setting it down from 10 to 2, and > then back > up again to 10 L/min. It helped temporarily, but the flow returned to > negligible > values a few hours later and remained around 0. Engineers - please take a > look at > the coolant system tomorrow as the behavior is quite erratic. Also, > please > advise us the appropriate steps to follow if this is to happen again. > > ---- Observing Log ---- > > Afternoon > --------- > All systems checked, no problems found. > > 2:56:12 Hartmann collimate output on cart #19 > sp1: offset: r= -2308 (Out of focus); b= 4.8 (Out of focus) > sp1: pred. move: spAverageMove= -1925 > sp1: pred. resid: r= -383, b= -1.2, txt= OK, spTemp = 2.8 > > > 22:55Z Started ventilation > > Night: > ====== > > 00:26Z - Opened telescope and started a fill. > > 00:39Z - Fill is done. We are waiting until the sky gets clearer and less > bright > before proceeding. > > The coolant flow through the GFA's were showing 0.0 L/min starting at > 00:30Z. > Before this, the flow was stable at around 1.1L/min, except for mid-day > today > where the flow showed zero for hours. > > 01:04:53 CmdStarted: OBSERVER.dmbiz 555 jaeger ieb status > fbi_led=0.0,0.0,0.0,0.0; flow=6.217,0.001; humidity=21.216,39.674; > power_can=T,T,T,T,T,T; power_gfa=F,T,T,T,T,T; power_nuc=T,T,T,T,T,T; > power_sextant=T,T,T,T,T,T; power_sync=F; pressure=42.406,25.034; > > 01:06:21 CmdStarted: OBSERVER.dmbiz 557 jaeger chiller set flow 1.5 > 01:12:16 CmdDone: OBSERVER.dmbiz 564 jaeger chiller set flow 6.2 > 01:18:00 CmdStarted: OBSERVER.dmbiz 565 jaeger chiller set flow 10 > > Tweaking the set point seemed to seemed to help, because the flow is back > to > normal: > > 01:26:13 APO.aoravetz 275 jaeger : fbi_led=0.0,0.0,0.0,0.0; > flow=6.676,1.017; > humidity=22.34,39.821; power_can=T,T,T,T,T,T; power_gfa=F,T,T,T,T,T; > power_nuc=T,T,T,T,T,T; power_sextant=T,T,T,T,T,T; power_sync=F; > pressure=42.101,24.844; > temperature=3.888,-1.044,10.15,7.02,6.92,2.22,-0.48,- > 0.38,13.52,2.702,1.52,1.91,8.169; > voltage=24.105,24.308,24.175,23.972,24.151,24.065 > > **** > BOSS: > 35884 15.000000 40.000000 fps_position_fields_10_015_40_BOSS.fits > 398 Field cadence: bright_1x1 Calibration targets: sky_boss: 50/50 > standard_boss: 50/50 sky_apogee: 0/35 standard_apogee: 0/15 Science > targets: > BOSS targets assigned: 397 APOGEE targets assigned: 0 Targets per epoch: > 397 > Robots used: BOSS-only: 201 APOGEE-BOSS: 296 Carton completion: > manual_fps_position_stars_10: 398 of 3359 > > 02:00:18 CmdDone: OBSERVER.dmbiz 566 jaeger configuration load 35884 > Failed at execution many times: > fps_snapshot_59577_0001.pdf > jaeger unwind --force > fps_snapshot_59577_0005.pdf > > 02:11:25 CmdStarted: OBSERVER.dmbiz 582 jaeger configuration load 35884 > 02:13:10 CmdStarted: OBSERVER.dmbiz 583 jaeger configuration execute > 02:15:38 OBSERVER.dmbiz 611 jaeger i text="Configuration 835." > > 02:15:56 CmdStarted: OBSERVER.dmbiz 584 hal goto-field -s > prepare,slew,boss_hartmann,boss_arcs,boss_flat,fvc > > 02:17:55 sp1.motor.move= -1924, -1925, -1926 > 02:18:03 boss exposure 4.0, file=335752 > 02:18:44 boss exposure 4.0, file=335753 > 02:19:25 sp1.motor.move= -453, -452, -456 > 02:19:25 Hartmann collimate output on cart #19 > sp1: offset: r= -329 (In Focus); b= 1.8 (In Focus) > sp1: pred. move: spAverageMove= -453 > sp1: pred. resid: r= 124, b= 0.4, txt= OK, spTemp = 0.3 > 02:22:46 boss exposure 4.0, file=335754 > 02:24:19 boss exposure 25.0, file=335755 > > 02:19:48 jaeger.fliswarm 611 fliswarm i > filename_bundle=/data/fcam/59577/fimg-fvc1n-0001.fits > ... > 02:21:54 OBSERVER.dmbiz.jaeger 410 jaeger d > fvc_filename=/data/fcam/59577/fimg-fvc1n-0004.fits > > 02:19:51 OBSERVER.dmbiz.jaeger 410 jaeger i text="[FVC]: RMS full fit > 80.280 um." > 02:20:34 OBSERVER.dmbiz.jaeger 410 jaeger i text="[FVC]: RMS full fit > 11.330 um." > 02:21:16 OBSERVER.dmbiz.jaeger 410 jaeger i text="[FVC]: RMS full fit > 7.230 um." > 02:21:57 OBSERVER.dmbiz.jaeger 410 jaeger i text="[FVC]: RMS full fit > 6.690 um." > > Focus sweep. Best focus at 450. Seeing 1.2". > > 02:44:54 CmdStarted: OBSERVER.dmbiz 605 cherno acquire -c -t 30 > > Extremely cloudy. The cherno RMS never converged below 0.1 (optimum is > 0.07), but we started > science anyway. > > 03:00:12 boss exposure 300.0, file=335756 - BASE CONFIGURATION > > 03:06:58 CmdStarted: OBSERVER.dmbiz 611 jaeger configuration dither 0.1 > 03:06:58 OBSERVER.dmbiz 611 jaeger > > 03:06:58 OBSERVER.dmbiz 611 jaeger i text="Creating dithered configuration > from 835." > > (1) > 03:07:19 CmdDone: OBSERVER.dmbiz 611 jaeger configuration dither 0.1 > > 03:07:59 CmdStarted: OBSERVER.dmbiz 612 jaeger fvc loop --no-apply > 03:08:22 OBSERVER.dmbiz 612 jaeger d > fvc_filename=/data/fcam/59577/fimg-fvc1n-0005.fits > 03:08:24 OBSERVER.dmbiz 612 jaeger i text="[FVC]: RMS full fit 80.910 um." > > 03:10:00 CmdStarted: OBSERVER.dmbiz 614 cherno acquire -c -t 20 > > 03:19:19 boss exposure 300.0, file=335757, very cloudy. > > 03:24:43 CmdStarted: OBSERVER.dmbiz 621 jaeger configuration reverse > > 03:25:04 CmdStarted: OBSERVER.dmbiz 622 jaeger fvc loop --no-apply > 03:25:26 OBSERVER.dmbiz 622 jaeger d > fvc_filename=/data/fcam/59577/fimg-fvc1n-0006.fits > 03:25:29 OBSERVER.dmbiz 622 jaeger i text="[FVC]: RMS full fit 41.190 um." > > (2) > 03:26:10 CmdStarted: OBSERVER.dmbiz 623 jaeger configuration dither 0.1 > 03:26:28 OBSERVER.dmbiz 623 jaeger : text="Dithered configuration 837 > loaded and executed." > > 03:27:39 CmdStarted: OBSERVER.dmbiz 624 jaeger fvc loop --no-apply > 03:28:02 OBSERVER.dmbiz /data/fcam/59577/fimg-fvc1n-0007.fits" > 03:28:04 OBSERVER.dmbiz 624 jaeger i text="[FVC]: RMS full fit 78.550 um." > > 03:34:22 CmdStarted: OBSERVER.dmbiz 625 cherno acquire -c -t 20 > > 03:39:28 boss exposure 300.0, file=335758 > > 03:44:49 CmdStarted: OBSERVER.dmbiz 632 jaeger configuration reverse > > 03:45:18 CmdStarted: OBSERVER.dmbiz 633 jaeger fvc loop --no-apply > 03:45:41 OBSERVER.dmbiz 633 jaeger d > fvc_filename=/data/fcam/59577/fimg-fvc1n-0008.fits > 03:45:44 OBSERVER.dmbiz 633 jaeger i text="[FVC]: RMS full fit 19.240 um." > > (3) > 03:46:25 CmdStarted: OBSERVER.dmbiz 634 jaeger configuration dither 0.1 > 03:46:47 OBSERVER.dmbiz 634 jaeger : text="Dithered configuration 838 > loaded and executed." > > 03:47:04 CmdStarted: OBSERVER.dmbiz 635 jaeger fvc loop --no-apply > 03:47:27 OBSERVER.dmbiz 635 jaeger d > fvc_filename=/data/fcam/59577/fimg-fvc1n-0009.fits > 03:47:29 OBSERVER.dmbiz 635 jaeger i text="[FVC]: RMS full fit 75.060 um." > > 03:49:18 boss exposure 300.0, file=335759, extremely cloudy. > > 03:54:31 CmdStarted: OBSERVER.dmbiz 642 jaeger configuration reverse > > 03:55:06 CmdStarted: OBSERVER.dmbiz 643 jaeger fvc loop --no-apply > 03:55:29 OBSERVER.dmbiz 643 jaeger d > fvc_filename=/data/fcam/59577/fimg-fvc1n-0010.fits > 03:55:32 OBSERVER.dmbiz 643 jaeger i text="[FVC]: RMS full fit 12.870 um." > > (4) > 03:55:48 CmdStarted: OBSERVER.dmbiz 644 jaeger configuration dither 0.1 > 03:56:10 OBSERVER.dmbiz 644 jaeger : text="Dithered configuration 839 > loaded and executed." > > 03:56:42 CmdStarted: OBSERVER.dmbiz 645 jaeger fvc loop --no-apply > 03:57:04 OBSERVER.dmbiz 645 jaeger d > fvc_filename=/data/fcam/59577/fimg-fvc1n-0011.fits > 03:57:07 OBSERVER.dmbiz 645 jaeger i text="[FVC]: RMS full fit 78.070 um." > > 04:03:50 boss exposure 300.0, file=335760; A little better. Seeing 1.2" > > 04:09:02 CmdStarted: OBSERVER.dmbiz 651 jaeger configuration reverse > > 04:09:25 CmdStarted: OBSERVER.dmbiz 652 jaeger fvc loop --no-apply > 04:09:48 OBSERVER.dmbiz 652 jaeger d > fvc_filename=/data/fcam/59577/fimg-fvc1n-0012.fits > 04:09:51 OBSERVER.dmbiz 652 jaeger i text="[FVC]: RMS full fit 12.530 um." > > (5) > 04:10:09 CmdStarted: OBSERVER.dmbiz 653 jaeger configuration dither 0.1 > 04:10:30 OBSERVER.dmbiz 653 jaeger : text="Dithered configuration 840 > loaded and executed." > > 04:10:49 CmdStarted: OBSERVER.dmbiz 654 jaeger fvc loop --no-apply > 04:11:12 OBSERVER.dmbiz 654 jaeger d > fvc_filename=/data/fcam/59577/fimg-fvc1n-0013.fits > 04:11:15 OBSERVER.dmbiz 654 jaeger i text="[FVC]: RMS full fit 78.640 um." > > 04:15:46 boss exposure 300.0, file=335761 > > 04:20:59 CmdStarted: OBSERVER.dmbiz 660 jaeger configuration reverse > > 04:21:24 CmdStarted: OBSERVER.dmbiz 661 jaeger fvc loop --no-apply > 04:21:47 OBSERVER.dmbiz 661 jaeger d > fvc_filename=/data/fcam/59577/fimg-fvc1n-0014.fits > 04:21:50 OBSERVER.dmbiz 661 jaeger i text="[FVC]: RMS full fit 13.320 um." > > (6) > 04:22:37 CmdDone: OBSERVER.dmbiz 662 jaeger configuration dither 0.1 > 04:22:37 OBSERVER.dmbiz 662 jaeger : text="Dithered configuration 841 > loaded and executed." > > 04:24:05 CmdDone: OBSERVER.dmbiz 663 jaeger fvc loop --no-apply > 04:23:45 OBSERVER.dmbiz 663 jaeger d > fvc_filename=/data/fcam/59577/fimg-fvc1n-0015.fits > 04:23:48 OBSERVER.dmbiz 663 jaeger i text="[FVC]: RMS full fit 78.260 um." > > 04:28:18 boss exposure 300.0, file=335762. Cloudy. > > 04:33:30 CmdStarted: OBSERVER.dmbiz 670 jaeger configuration reverse > > 04:33:53 CmdStarted: OBSERVER.dmbiz 671 jaeger fvc loop --no-apply > 04:34:16 OBSERVER.dmbiz 671 jaeger d > fvc_filename=/data/fcam/59577/fimg-fvc1n-0016.fits > 04:34:18 OBSERVER.dmbiz 671 jaeger i text="[FVC]: RMS full fit 11.640 um." > > (7) > 04:34:39 CmdStarted: OBSERVER.dmbiz 672 jaeger configuration dither 0.1 > 04:35:00 OBSERVER.dmbiz 672 jaeger : text="Dithered configuration 842 > loaded and executed." > > 04:35:23 CmdStarted: OBSERVER.dmbiz 673 jaeger fvc loop --no-apply > 04:35:46 OBSERVER.dmbiz 673 jaeger d > fvc_filename=/data/fcam/59577/fimg-fvc1n-0017.fits > 04:35:49 OBSERVER.dmbiz 673 jaeger i text="[FVC]: RMS full fit 77.870 um." > > 04:37:14 boss exposure 300.0, file=335763 > > 04:42:47 CmdStarted: OBSERVER.dmbiz 679 jaeger configuration reverse > > 04:43:10 CmdStarted: OBSERVER.dmbiz 680 jaeger fvc loop --no-apply > 04:43:33 OBSERVER.dmbiz 680 jaeger d > fvc_filename=/data/fcam/59577/fimg-fvc1n-0018.fits > 04:43:36 OBSERVER.dmbiz 680 jaeger i text="[FVC]: RMS full fit 11.780 um." > > (8) > 04:43:52 CmdStarted: OBSERVER.dmbiz 681 jaeger configuration dither 0.1 > 04:44:12 OBSERVER.dmbiz 681 jaeger : text="Dithered configuration 843 > loaded and executed." > > 04:43:53 OBSERVER.dmbiz 680 jaeger f error="Command 'fvc loop --no-apply' > failed > because of an uncaught error 'JaegerError: Configuration needs to be set > and > loaded to the DB.'. See traceback in the log for more information." > 04:45:14 CmdStarted: OBSERVER.dmbiz 683 jaeger fvc loop --no-apply > 04:45:37 OBSERVER.dmbiz 683 jaeger d > fvc_filename=/data/fcam/59577/fimg-fvc1n-0019.fits > 04:45:40 OBSERVER.dmbiz 683 jaeger i text="[FVC]: RMS full fit 80.470 um." > > 04:47:29 boss exposure 300.0, file=335764 > > 04:52:42 CmdStarted: OBSERVER.dmbiz 689 jaeger configuration reverse > > 04:53:02 CmdStarted: OBSERVER.dmbiz 690 jaeger fvc loop --no-apply > 04:53:24 OBSERVER.dmbiz 690 jaeger d > fvc_filename=/data/fcam/59577/fimg-fvc1n-0020.fits > 04:53:27 OBSERVER.dmbiz 690 jaeger i text="[FVC]: RMS full fit 69.940 um." > > (9) > 04:53:48 CmdStarted: OBSERVER.dmbiz 691 jaeger configuration dither 0.1 > 04:54:08 OBSERVER.dmbiz 691 jaeger : text="Dithered configuration 845 > loaded and executed." > > 04:54:51 CmdStarted: OBSERVER.dmbiz 692 jaeger fvc loop --no-apply > 04:55:14 OBSERVER.dmbiz 692 jaeger d > fvc_filename=/data/fcam/59577/fimg-fvc1n-0021.fits > 04:55:16 OBSERVER.dmbiz 692 jaeger i text="[FVC]: RMS full fit 79.780 um." > > 04:56:36 boss exposure 300.0, file=335765 > > 05:02:03 CmdStarted: OBSERVER.dmbiz 698 jaeger configuration reverse > > 05:02:26 .hub 0 cmds d CmdIn="OBSERVER.dmbiz","jaeger","fvc loop > --no-apply" > 05:02:49 OBSERVER.dmbiz 699 jaeger d > fvc_filename=/data/fcam/59577/fimg-fvc1n-0022.fits > 05:02:52 OBSERVER.dmbiz 699 jaeger i text="[FVC]: RMS full fit 10.060 um." > > jaeger unwind > > 05:05:09 OBSERVER.dmbiz 700 jaeger f error="Trajectory failed with error: > Something went wrong during the trajectory: The FPS got locked during the > trajectory. Trajectory failed 11.89 seconds after start" > fps_snapshot_59577_0031.pdf > > **** > Focus test - take BOSS exposures in an undithered configuration at offset > focus > -100, -50, 50, and 100 microns of secondary motion. > > 35914 75.000000 20.000000 fps_position_fields_10_075_20_BOSS.fits > 433 Field cadence: bright_1x1 Calibration targets: sky_boss: 50/50 > standard_boss: 16/50 sky_apogee: 0/35 standard_apogee: 0/15 Science > targets: > BOSS targets assigned: 433 APOGEE targets assigned: 0 Targets per epoch: > 433 > Robots used: BOSS-only: 201 APOGEE-BOSS: 298 Carton completion: > manual_fps_position_stars_10: 433 of 2994 > > 05:14:25 CmdStarted: OBSERVER.dmbiz 713 jaeger configuration load 35914 > 05:15:36 OBSERVER.dmbiz 713 jaeger : text="Configuration 846 loaded." > 05:16:34 CmdDone: OBSERVER.dmbiz 714 jaeger configuration execute > > 05:18:11 CmdStarted: OBSERVER.dmbiz 715 hal goto-field -s > prepare,slew,boss_hartmann,boss_arcs,boss_flat,fvc,acquire > > 05:02:49 OBSERVER.dmbiz 699 jaeger d > fvc_filename=/data/fcam/59577/fimg-fvc1n-0022.fits > ... > 05:23:56 OBSERVER.dmbiz.jaeger 433 jaeger d > fvc_filename=/data/fcam/59577/fimg-fvc1n-0026.fits > > 05:21:53 OBSERVER.dmbiz.jaeger 433 jaeger i text="[FVC]: RMS full fit > 81.170 um." > 05:22:36 OBSERVER.dmbiz.jaeger 433 jaeger i text="[FVC]: RMS full fit > 23.670 um." > 05:23:17 OBSERVER.dmbiz.jaeger 433 jaeger i text="[FVC]: RMS full fit > 22.590 um." > 05:23:59 OBSERVER.dmbiz.jaeger 433 jaeger i text="[FVC]: RMS full fit > 22.350 um." > > 05:19:58 boss exposure 4.0, file=335766 > 05:20:38 boss exposure 4.0, file=335767 > 05:21:28 sp1.motor.move= -1099, -1100, -1096 > 05:21:28 Hartmann collimate output on cart #19 > sp1: offset: r= -659 (In Focus); b= 4.8 (Out of focus) > sp1: pred. move: spAverageMove= -1100 > sp1: pred. resid: r= 441, b= 1.4, txt= OK, spTemp = -0.1 > 05:24:49 boss exposure 4.0, file=335768 > 05:26:29 boss exposure 25.0, file=335769 > > Focus loop. Best focus at 380. Seeing 1.1". Still cloudy, but better than > before. > > 05:39:43 boss exposure 300.0, file=335770 > > OBSERVER.dmbiz 732 tcc set focus=280.0 (-100) > > 05:46:11 boss exposure 300.0, file=335771 > > OBSERVER.dmbiz 732 tcc set focus=330.0 (-50) > > 05:52:37 boss exposure 300.0, file=335772 > > OBSERVER.dmbiz 732 tcc set focus=430 (50) > > 05:59:07 boss exposure 300.0, file=335773 > > OBSERVER.dmbiz 732 tcc set focus=480 (100) > > 06:05:33 boss exposure 300.0, file=335774 > > OBSERVER.dmbiz 732 tcc set focus=380 (Best focus) > > 06:12:00 boss exposure 300.0, file=335775 > > *** > > 3x3 telescope dither test with an undithered BOSS configuration > (configuration 35914), spaced by 1". > > We were unsure as to what to do with the guider during these exposures. If > we > left the guiding alone, and offset the telescope, the guider will just > bring the > stars back to where they are supposed to be. Therefore, we decided to not > guide > at all during the offseted exposures. The other option would be to apply > the > cherno offset command (which if we have time, perhaps we will do later). > > IGNORE exposure 06:19:25 boss exposure 300.0, file=335776 since we were > figuring out the steps we should take going forward. > > boss exposure 300.0, file=335777 > [-1",-1"] > > boss exposure 300.0, file=335778 > [-1",0"] > > boss exposure 300.0, file=335779 > [-1",1"] > > boss exposure 300.0, file=335780 > [0",1"] > > boss exposure 300.0, file=335781 > [1",1"] > > boss exposure 300.0, file=335782 > [0",1"] > > boss exposure 300.0, file=335783 > [1",-1"] > > boss exposure 300.0, file=335784 > [0",-1"] > > boss exposure 300.0, file=335785 > [0",0"] > > ** Now, activate the guider and guide at the same offset as the telescope. > > boss exposure 300.0, file=335786 > [-1",-1"] > > boss exposure 300.0, file=335787 > [-1",0"] > > boss exposure 300.0, file=335788 > [-1",1"] > > boss exposure 300.0, file=335789 > [0",1"] > > boss exposure 300.0, file=335790 > [1",1"] > > boss exposure 300.0, file=335791 > [0",1"] > > boss exposure 300.0, file=335792 > [1",-1"] > > boss exposure 300.0, file=335793 > [0",-1"] > > boss exposure 300.0, file=335794 > [0",0"] > > However, it wasn't until after we finished the test that we noticed that > we only > implemented the cherno offset and not the telescope offset as well. > > Coolant flow gradually getting worse. Down to 0.2L/min. Going to try to do > the > same thing as before to get it to behave. > > 08:46:24 APO.aoravetz 277 jaeger : fbi_led=0.0,0.0,0.0,0.0; > flow=5.846,0.001; > humidity=25.294,54.079; power_can=T,T,T,T,T,T; power_gfa=F,T,T,T,T,T; > power_nuc=T,T,T,T,T,T; power_sextant=T,T,T,T,T,T; power_sync=F; > pressure=31.785,20.962; > temperature=4.962,-3.583,8.35,9.22,9.12,1.82,-1.48,- > 0.18,16.22,0.602,1.22,1.41,5.869; > > 08:37:02 CmdStarted: OBSERVER.dmbiz 791 jaeger chiller set flow 10 > > 08:45:28 CmdStarted: OBSERVER.dmbiz 792 jaeger chiller set flow 0.2 > > 08:47:04 CmdStarted: OBSERVER.dmbiz 794 jaeger chiller set flow 10 > > 08:47:28 CmdStarted: OBSERVER.dmbiz 799 jaeger ieb status > 08:47:28 OBSERVER.dmbiz 799 jaeger > > 08:47:28 OBSERVER.dmbiz 799 jaeger : fbi_led=0.0,0.0,0.0,0.0; > flow=6.794,0.485; > humidity=25.391,53.981; power_can=T,T,T,T,T,T; power_gfa=F,T,T,T,T,T; > power_nuc=T,T,T,T,T,T; power_sextant=T,T,T,T,T,T; power_sync=F; > pressure=37.071,22.769; > temperature=4.962,-3.583,8.65,9.22,9.12,1.72,-1.48,- > 0.18,16.22,0.702,0.92,1.11,5.769; > voltage=24.112,24.323,24.183,23.979,24.144,24.065 > > Better. > > 08:44Z - Waited for the clouds to move off to start another test. > > *** > > GRAFANA was hanging and was no longer responsive. > > Restarted the cerebro with systemcontol. It didn't help. We are still > having > coolant flow issues. > > 11:06Z - We closed due to clouds, humidity, and the FPS flow issue. > Started cals. > > > ---- Mirror Numbers ---- > > > PRIMARY: > -------- > Scale: 1.000000 > > MIGS TONIGHT NOMINAL > Axial A: 0.0020 0.0040 > Axial B: 0.0000 0.0030 > Axial C: -0.1110 -0.1150 > Trans D: -0.0340 -0.0338 > Lateral E: 0.3315 0.3315 > Lateral F: 0.5350 0.5350 > > GALILS > Commanded: 12500 -1600 5300 47050 -8800 -8400 > Actual: 12500 -1600 5300 47050 -8800 -8400 > > SETMIR VALUES > PrimDesOrient: 0.0 -59.5 -9.15 -512.0 1906.3 > PrimOrient: 1.07 -59.51 -9.21 -512.13 1906.31 > > SECONDARY: > ---------- > > Focus: 0.0000 > Air temp: -0.11 > Altitude: 30.0 > > MIGS TONIGHT NOMINAL > Axial A: -9.6560 -9.6570 > Axial B: -9.6710 -9.6710 > Axial C: -9.6950 -9.6970 > Trans D: 0.1130 0.1140 > > GALILS > Commanded: 4896650 4897450 4912650 -50 -3450 > Actual: 4859186 4859230 4859262 -50 -3450 > > SETMIR VALUES > SecDesOrient: 3857.0 0.0 0.0 0.0 42.22 > SecOrient: 3857.01 -0.01 0.02 0.01 42.87 > > > > ---- Telescope Status ---- > > 23:55Z Started ventilation > 01:10Z encl25m : open > 11:17Z encl25m : closed > > No interlocks bypassed > > Status at: > Telescope stowed at: 121.0, 30.0, 0.0 mount > Instrument mounted: cartridge = fps > Counterweights at: 680 680 705 705 > LN2 autofill systems: Connected and turned on > 180L LN2 dewar scales: SP1 175 lbs, 20 psi > > > ---- Software Used ---- > > > TUI : 1.7.13 > alerts : 2.2.0 > apo : 3.0.1 > boss : 6.0.1-alpha.0 > -boss.specMec : 3.0.4 > guider : 4.0.0 > ecamera : FAILED > fliswarm : FAILED > hartmann : 2.0.1-alpha.0 > hub : 5.0.0a0 > mcp : 2.1 > -mcp.plc : 34 > -mcp.azFiduci : v1_148 > -mcp.altFiduc : v1_148 > -mcp.rotFiduc : v1_148 > sop : FAILED > tcc : 1.4.0 > apogee : not availble > apogeecal : not availble > apogeeql : v1_7_0dev > STUIAdditions : 3.1.9 > > > /dev/sdb1 2.0T 244G 1.8T 12% /home > 10.25.1.230:/data 21T 17T 3.3T 84% /data > > > ---- APOGEE Data Summary ---- > > > last morning: > 59576 10 11:48 40140010 Dark 60 B 13.5 FPS- 0 a-b-c - - > 59576 11 11:58 40140011 Dark 60 B 13.5 FPS- 0 a-b-c - - > 59576 12 12:09 40140012 Dark 60 B 13.5 FPS- 0 a-b-c - - > 59576 13 12:11 40140013 QuarFlat 10 B 13.5 FPS- 0 a-b-c - 100 > 59576 14 12:13 40140014 QuarFlat 10 B 13.5 FPS- 0 a-b-c - 100 > 59576 15 12:15 40140015 QuarFlat 10 B 13.5 FPS- 0 a-b-c - 100 > 59576 16 12:17 40140016 Arc-Thar 12 A 13.0 FPS- 0 a-b-c -0.08 - > 59576 17 12:24 40140017 Arc-Une 40 A 13.0 FPS- 0 a-b-c - - > 59576 18 12:26 40140018 Arc-Thar 12 B 13.5 FPS- 0 a-b-c 0.51 - > 59576 19 12:34 40140019 Arc-Une 40 B 13.5 FPS- 0 a-b-c - - > 59576 20 12:39 40140020 Dark 30 A 13.0 FPS- 0 a-b-c - - > 59576 21 12:45 40140021 InteFlat 30 A 13.0 FPS- 0 a-b-c - 279 > 59576 22 12:50 40140022 InteFlat 30 A 13.0 FPS- 0 a-b-c - 280 > 59576 23 12:55 40140023 InteFlat 30 A 13.0 FPS- 0 a-b-c - 279 > 59576 24 13:01 40140024 Dark 30 A 13.0 FPS- 0 a-b-c - - > > > this evening: > ------------------------------------------------------------------------------ > mjd i UT File/Exp Imtype Nread A/B Dth Ct-Plate Arch Offset %Flux > Comm > ------------------------------------------------------------------------------ > 59577 1 22:47 40150001 Arc---- 3 A 13.0 FPS- 0 a-b-c - - > test > 59577 2 22:59 40150002 Dark 60 A 13.0 -------- a-b-c - - > 59577 3 23:01 40150003 QuarFlat 10 A 13.0 FPS- 0 a-b-c - 100 > 59577 4 23:03 40150004 Arc-Thar 12 B 13.5 FPS- 0 a-b-c 0.51 - > 59577 5 23:10 40150005 Arc-Une 40 B 13.5 FPS- 0 a-b-c - - > 59577 6 23:13 40150006 Arc-Thar 12 A 13.0 FPS- 0 a-b-c -0.11 - > 59577 7 23:20 40150007 Arc-Une 40 A 13.0 FPS- 0 a-b-c - - > 59577 8 23:31 40150008 Dark 60 A 13.0 -------- a-b-c - - > 59577 9 23:38 40150009 Arc---- 30 A 13.0 FPS- 0 x-x-x - - > FPI > ------------------------------------------------------------------------------ > > APOGEE data list, mjd=59577, all > raw_data: /data/apogee/utr_cdr/59577/apRaw-*.fits > archive: /data/apogee/archive/59577/apR-[a,b,c]-*.apz > quickred: /data/apogee/quickred/59577/apq-*.fits > ------------------------------------------------------------------------------ > mjd i UT File/Exp Imtype Nread A/B Dth Ct-Plate Arch Offset %Flux > Comm > ------------------------------------------------------------------------------ > 59577 1 22:47 40150001 Arc---- 3 A 13.0 FPS- 0 a-b-c - - > test > 59577 2 22:59 40150002 Dark 60 A 13.0 -------- a-b-c - - > 59577 3 23:01 40150003 QuarFlat 10 A 13.0 FPS- 0 a-b-c - 100 > 59577 4 23:03 40150004 Arc-Thar 12 B 13.5 FPS- 0 a-b-c 0.51 - > 59577 5 23:10 40150005 Arc-Une 40 B 13.5 FPS- 0 a-b-c - - > 59577 6 23:13 40150006 Arc-Thar 12 A 13.0 FPS- 0 a-b-c -0.11 - > 59577 7 23:20 40150007 Arc-Une 40 A 13.0 FPS- 0 a-b-c - - > 59577 8 23:31 40150008 Dark 60 A 13.0 -------- a-b-c - - > 59577 9 23:38 40150009 Arc---- 30 A 13.0 FPS- 0 a-b-c - - > FPI > ------------------------------------------------------------------------------ > > > > ---- eBOSS Data Summary ---- > > Reading FITS from /data/spectro//59577 > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > - - > n Time Filename crt plt flav shu Exp colA colB colC Hrt > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > - - > 1 22:47 b1-00335746 FPS-N None bia Clo 0.0 -4103 -4096 -4125 > 2 22:47 r1-00335746 FPS-N None bia Clo 0.0 -4103 -4096 -4125 > 3 22:51 b1-00335747 FPS-N None dar Clo 20.0 -4103 -4096 -4125 > 4 22:51 r1-00335747 FPS-N None dar Clo 20.0 -4103 -4096 -4125 > 5 22:54 b1-00335748 FPS-N None arc Lef 4.1 -4103 -4096 -4125 > HART > 6 22:54 r1-00335748 FPS-N None arc Lef 4.1 -4103 -4096 -4125 > HART > 7 22:55 b1-00335749 FPS-N None arc Rig 4.1 -4103 -4096 -4125 > HART > 8 22:55 r1-00335749 FPS-N None arc Rig 4.1 -4103 -4096 -4125 > HART > 9 22:58 b1-00335750 FPS-N None arc Out 4.1 -6027 -6021 -6051 > 10 22:58 r1-00335750 FPS-N None arc Out 4.1 -6027 -6021 -6051 > 11 22:59 b1-00335751 FPS-N None fla Out 25.1 -6027 -6021 -6051 > 12 22:59 r1-00335751 FPS-N None fla Out 25.1 -6027 -6021 -6051 > 13 02:18 r1-00335752 FPS-N None arc Lef 4.1 -6027 -6021 -6051 > HART > 14 02:18 b1-00335752 FPS-N None arc Lef 4.1 -6027 -6021 -6051 > HART > 15 02:18 r1-00335753 FPS-N None arc Rig 4.1 -6027 -6021 -6051 > HART > 16 02:18 b1-00335753 FPS-N None arc Rig 4.1 -6027 -6021 -6051 > HART > 17 02:22 b1-00335754 FPS-N None arc Out 4.1 -6480 -6473 -6507 > 18 02:22 r1-00335754 FPS-N None arc Out 4.1 -6480 -6473 -6507 > 19 02:24 b1-00335755 FPS-N None fla Out 25.1 -6480 -6473 -6507 > 20 02:24 r1-00335755 FPS-N None fla Out 25.1 -6480 -6473 -6507 > 21 03:00 b1-00335756 FPS-N None sci Out 300.1 -6480 -6473 -6507 > 22 03:00 r1-00335756 FPS-N None sci Out 300.1 -6480 -6473 -6507 > 23 03:19 b1-00335757 FPS-N None sci Out 300.1 -6480 -6473 -6507 > 24 03:19 r1-00335757 FPS-N None sci Out 300.1 -6480 -6473 -6507 > 25 03:39 b1-00335758 FPS-N None sci Out 300.1 -6480 -6473 -6507 > 26 03:39 r1-00335758 FPS-N None sci Out 300.1 -6480 -6473 -6507 > 27 03:49 b1-00335759 FPS-N None sci Out 300.1 -6480 -6473 -6507 > 28 03:49 r1-00335759 FPS-N None sci Out 300.1 -6480 -6473 -6507 > 29 04:03 b1-00335760 FPS-N None sci Out 300.1 -6480 -6473 -6507 > 30 04:03 r1-00335760 FPS-N None sci Out 300.1 -6480 -6473 -6507 > 31 04:15 b1-00335761 FPS-N None sci Out 300.1 -6480 -6473 -6507 > 32 04:15 r1-00335761 FPS-N None sci Out 300.1 -6480 -6473 -6507 > 33 04:28 b1-00335762 FPS-N None sci Out 300.1 -6480 -6473 -6507 > 34 04:28 r1-00335762 FPS-N None sci Out 300.1 -6480 -6473 -6507 > 35 04:37 b1-00335763 FPS-N None sci Out 300.1 -6480 -6473 -6507 > 36 04:37 r1-00335763 FPS-N None sci Out 300.1 -6480 -6473 -6507 > 37 04:47 b1-00335764 FPS-N None sci Out 300.1 -6480 -6473 -6507 > 38 04:47 r1-00335764 FPS-N None sci Out 300.1 -6480 -6473 -6507 > 39 04:56 b1-00335765 FPS-N None sci Out 300.1 -6480 -6473 -6507 > 40 04:56 r1-00335765 FPS-N None sci Out 300.1 -6480 -6473 -6507 > 41 05:19 r1-00335766 FPS-N None arc Lef 4.1 -6480 -6473 -6507 > HART > 42 05:19 b1-00335766 FPS-N None arc Lef 4.1 -6480 -6473 -6507 > HART > 43 05:20 r1-00335767 FPS-N None arc Rig 4.1 -6480 -6473 -6507 > HART > 44 05:20 b1-00335767 FPS-N None arc Rig 4.1 -6480 -6473 -6507 > HART > 45 05:24 r1-00335768 FPS-N None arc Out 4.1 -7579 -7573 -7603 > 46 05:24 b1-00335768 FPS-N None arc Out 4.1 -7579 -7573 -7603 > 47 05:26 b1-00335769 FPS-N None fla Out 25.1 -7579 -7573 -7603 > 48 05:26 r1-00335769 FPS-N None fla Out 25.1 -7579 -7573 -7603 > 49 05:39 r1-00335770 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 50 05:39 b1-00335770 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 51 05:46 r1-00335771 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 52 05:46 b1-00335771 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 53 05:52 r1-00335772 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 54 05:52 b1-00335772 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 55 05:59 b1-00335773 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 56 05:59 r1-00335773 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 57 06:05 r1-00335774 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 58 06:05 b1-00335774 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 59 06:12 b1-00335775 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 60 06:12 r1-00335775 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 61 06:19 r1-00335776 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 62 06:19 b1-00335776 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 63 06:26 r1-00335777 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 64 06:26 b1-00335777 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 65 06:32 b1-00335778 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 66 06:32 r1-00335778 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 67 06:44 b1-00335779 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 68 06:44 r1-00335779 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 69 06:50 b1-00335780 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 70 06:50 r1-00335780 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 71 06:57 b1-00335781 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 72 06:57 r1-00335781 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 73 07:03 r1-00335782 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 74 07:03 b1-00335782 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 75 07:10 r1-00335783 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 76 07:10 b1-00335783 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 77 07:16 r1-00335784 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 78 07:16 b1-00335784 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 79 07:23 r1-00335785 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 80 07:23 b1-00335785 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 81 07:38 r1-00335786 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 82 07:38 b1-00335786 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 83 07:45 r1-00335787 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 84 07:45 b1-00335787 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 85 07:51 r1-00335788 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 86 07:51 b1-00335788 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 87 07:58 r1-00335789 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 88 07:58 b1-00335789 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 89 08:04 b1-00335790 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 90 08:04 r1-00335790 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 91 08:13 b1-00335791 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 92 08:13 r1-00335791 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 93 08:20 b1-00335792 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 94 08:20 r1-00335792 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 95 08:27 b1-00335793 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 96 08:27 r1-00335793 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 97 08:33 b1-00335794 FPS-N None sci Out 300.1 -7579 -7573 -7603 > 98 08:33 r1-00335794 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > > ____________________________________________________________ > Logs of Nightly APO Activities > https://trac.sdss.org/tracmailman/browser/private/sloan-25mlog/index.html >