From: =?UTF-8?B?Sm9zw6kgU8OhbmNoZXotR2FsbGVnbw==?= Date: Sat, 1 Jan 2022 15:06:47 -0800 Subject: Re: [sloan-25mlog 5818] Re: 2.5m obslog 20211228 (SJD 59577) --0000000000008a34a805d48d5788 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Audrey, Shane, The GFA plots use a query that's not super optimal because it involves casting strings to floats. I'm planning to improve that but it requires some work under the hood. In that case restarting Cerebro won't help. My advise is that you try to avoid using too high a refresh rate (e.g. 5 or 15 seconds) when using Grafana, especially on the GFA/FVC page. Also, consider closing tabs with Grafana in them when not using them, especially if you're connecting remotely. Grafana is not really meant as a real-time tool (meaning seeing something just as it happens) so refresh rates of 1 minute or so would be more appropriate, but I understand that since there are no GUIs right now you use Grafana in that way. For those cases, I think it's better to have Grafana with a refresh rate of 30 seconds and if you need some quick feedback, you can always press the little circular arrow. Thanks, Jos=C3=A9 On Dec 29, 2021 at 03:21:07, Audrey Oravetz wrote: > Also, would someone have a look at GRAFANA? It continuously hangs when > trying to access the GFA page. Cerebro was restarted a couple of times, > which helped with the FPS page, but did nothing with the GFA page. Thanks= ! > > > > > APACHE POINT OBSERVATORY SDSS 2.5M > > > OBSERVING LOG > > Tuesday December 28, 2021 > > > (SJD 59577) > > > ---- Observing Team ---- > > > Dmitry Bizyaev > > Audrey Oravetz > > > Zoom: > > Kaike, Karen, DanO, Katie > > > > ---- Observing Plan ---- > > > FPS commissioning tasks > > > > > ---- Observing Summary ---- > > > Cloudy, but we gathered: > > > 9 loops of science data WITH fvc loop --no-apply in between dither > > configuration > > 0.1 / reverse sequence for a BOSS field (35884). > > > Focus test - BOSS exposures at varying secondary mirror values (design > > 35914). > > > 3x3 telescope offset test with an undithered BOSS configuration (design > > 35914), > > spaced by 1" > > > Notes: > > =3D=3D=3D=3D=3D=3D > > > ** Hal goto-field failed on us tonight on the guider 'acquire' step > > because the > > routine did not open the FF petals automatically. Note that some of the > > commands > > were purposely omitted (05:18:11 CmdStarted: OBSERVER.dmbiz 715 hal > > goto-field -s > > prepare,slew,boss_hartmann,boss_arcs,boss_flat,fvc,acquire) ** > > > ** We had many instances where the coolant flow to the GFAs was > > practically zero. > > We tweaked the chiller flow rate by setting it down from 10 to 2, and > > then back > > up again to 10 L/min. It helped temporarily, but the flow returned to > > negligible > > values a few hours later and remained around 0. Engineers - please take a > > look at > > the coolant system tomorrow as the behavior is quite erratic. Also, > > please > > advise us the appropriate steps to follow if this is to happen again. > > > ---- Observing Log ---- > > > Afternoon > > --------- > > All systems checked, no problems found. > > > 2:56:12 Hartmann collimate output on cart #19 > > sp1: offset: r=3D -2308 (Out of focus); b=3D 4.8 (Out of focus) > > sp1: pred. move: spAverageMove=3D -1925 > > sp1: pred. resid: r=3D -383, b=3D -1.2, txt=3D OK, spTemp =3D 2.8 > > > > 22:55Z Started ventilation > > > Night: > > =3D=3D=3D=3D=3D=3D > > > 00:26Z - Opened telescope and started a fill. > > > 00:39Z - Fill is done. We are waiting until the sky gets clearer and less > > bright > > before proceeding. > > > The coolant flow through the GFA's were showing 0.0 L/min starting at > > 00:30Z. > > Before this, the flow was stable at around 1.1L/min, except for mid-day > > today > > where the flow showed zero for hours. > > > 01:04:53 CmdStarted: OBSERVER.dmbiz 555 jaeger ieb status > > fbi_led=3D0.0,0.0,0.0,0.0; flow=3D6.217,0.001; humidity=3D21.216,39.674; > > power_can=3DT,T,T,T,T,T; power_gfa=3DF,T,T,T,T,T; power_nuc=3DT,T,T,T,T,T= ; > > power_sextant=3DT,T,T,T,T,T; power_sync=3DF; pressure=3D42.406,25.034; > > > 01:06:21 CmdStarted: OBSERVER.dmbiz 557 jaeger chiller set flow 1.5 > > 01:12:16 CmdDone: OBSERVER.dmbiz 564 jaeger chiller set flow 6.2 > > 01:18:00 CmdStarted: OBSERVER.dmbiz 565 jaeger chiller set flow 10 > > > Tweaking the set point seemed to seemed to help, because the flow is back > > to > > normal: > > > 01:26:13 APO.aoravetz 275 jaeger : fbi_led=3D0.0,0.0,0.0,0.0; > > flow=3D6.676,1.017; > > humidity=3D22.34,39.821; power_can=3DT,T,T,T,T,T; power_gfa=3DF,T,T,T,T,T= ; > > power_nuc=3DT,T,T,T,T,T; power_sextant=3DT,T,T,T,T,T; power_sync=3DF; > > pressure=3D42.101,24.844; > > temperature=3D3.888,-1.044,10.15,7.02,6.92,2.22,-0.48,- > > 0.38,13.52,2.702,1.52,1.91,8.169; > > voltage=3D24.105,24.308,24.175,23.972,24.151,24.065 > > > **** > > BOSS: > > 35884 15.000000 40.000000 fps_position_fields_10_015_40_BOSS.fits > > 398 Field cadence: bright_1x1 Calibration targets: sky_boss: 50/50 > > standard_boss: 50/50 sky_apogee: 0/35 standard_apogee: 0/15 Science > > targets: > > BOSS targets assigned: 397 APOGEE targets assigned: 0 Targets per epoch: > > 397 > > Robots used: BOSS-only: 201 APOGEE-BOSS: 296 Carton completion: > > manual_fps_position_stars_10: 398 of 3359 > > > 02:00:18 CmdDone: OBSERVER.dmbiz 566 jaeger configuration load 35884 > > Failed at execution many times: > > fps_snapshot_59577_0001.pdf > > jaeger unwind --force > > fps_snapshot_59577_0005.pdf > > > 02:11:25 CmdStarted: OBSERVER.dmbiz 582 jaeger configuration load 35884 > > 02:13:10 CmdStarted: OBSERVER.dmbiz 583 jaeger configuration execute > > 02:15:38 OBSERVER.dmbiz 611 jaeger i text=3D"Configuration 835." > > > 02:15:56 CmdStarted: OBSERVER.dmbiz 584 hal goto-field -s > > prepare,slew,boss_hartmann,boss_arcs,boss_flat,fvc > > > 02:17:55 sp1.motor.move=3D -1924, -1925, -1926 > > 02:18:03 boss exposure 4.0, file=3D335752 > > 02:18:44 boss exposure 4.0, file=3D335753 > > 02:19:25 sp1.motor.move=3D -453, -452, -456 > > 02:19:25 Hartmann collimate output on cart #19 > > sp1: offset: r=3D -329 (In Focus); b=3D 1.8 (In Focus) > > sp1: pred. move: spAverageMove=3D -453 > > sp1: pred. resid: r=3D 124, b=3D 0.4, txt=3D OK, spTemp =3D 0.3 > > 02:22:46 boss exposure 4.0, file=3D335754 > > 02:24:19 boss exposure 25.0, file=3D335755 > > > 02:19:48 jaeger.fliswarm 611 fliswarm i > > filename_bundle=3D/data/fcam/59577/fimg-fvc1n-0001.fits > > ... > > 02:21:54 OBSERVER.dmbiz.jaeger 410 jaeger d > > fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0004.fits > > > 02:19:51 OBSERVER.dmbiz.jaeger 410 jaeger i text=3D"[FVC]: RMS full fit > > 80.280 um." > > 02:20:34 OBSERVER.dmbiz.jaeger 410 jaeger i text=3D"[FVC]: RMS full fit > > 11.330 um." > > 02:21:16 OBSERVER.dmbiz.jaeger 410 jaeger i text=3D"[FVC]: RMS full fit > > 7.230 um." > > 02:21:57 OBSERVER.dmbiz.jaeger 410 jaeger i text=3D"[FVC]: RMS full fit > > 6.690 um." > > > Focus sweep. Best focus at 450. Seeing 1.2". > > > 02:44:54 CmdStarted: OBSERVER.dmbiz 605 cherno acquire -c -t 30 > > > Extremely cloudy. The cherno RMS never converged below 0.1 (optimum is > > 0.07), but we started > > science anyway. > > > 03:00:12 boss exposure 300.0, file=3D335756 - BASE CONFIGURATION > > > 03:06:58 CmdStarted: OBSERVER.dmbiz 611 jaeger configuration dither 0.1 > > 03:06:58 OBSERVER.dmbiz 611 jaeger > > > 03:06:58 OBSERVER.dmbiz 611 jaeger i text=3D"Creating dithered configurat= ion > > from 835." > > > (1) > > 03:07:19 CmdDone: OBSERVER.dmbiz 611 jaeger configuration dither 0.1 > > > 03:07:59 CmdStarted: OBSERVER.dmbiz 612 jaeger fvc loop --no-apply > > 03:08:22 OBSERVER.dmbiz 612 jaeger d > > fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0005.fits > > 03:08:24 OBSERVER.dmbiz 612 jaeger i text=3D"[FVC]: RMS full fit 80.910 u= m." > > > 03:10:00 CmdStarted: OBSERVER.dmbiz 614 cherno acquire -c -t 20 > > > 03:19:19 boss exposure 300.0, file=3D335757, very cloudy. > > > 03:24:43 CmdStarted: OBSERVER.dmbiz 621 jaeger configuration reverse > > > 03:25:04 CmdStarted: OBSERVER.dmbiz 622 jaeger fvc loop --no-apply > > 03:25:26 OBSERVER.dmbiz 622 jaeger d > > fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0006.fits > > 03:25:29 OBSERVER.dmbiz 622 jaeger i text=3D"[FVC]: RMS full fit 41.190 u= m." > > > (2) > > 03:26:10 CmdStarted: OBSERVER.dmbiz 623 jaeger configuration dither 0.1 > > 03:26:28 OBSERVER.dmbiz 623 jaeger : text=3D"Dithered configuration 837 > > loaded and executed." > > > 03:27:39 CmdStarted: OBSERVER.dmbiz 624 jaeger fvc loop --no-apply > > 03:28:02 OBSERVER.dmbiz /data/fcam/59577/fimg-fvc1n-0007.fits" > > 03:28:04 OBSERVER.dmbiz 624 jaeger i text=3D"[FVC]: RMS full fit 78.550 u= m." > > > 03:34:22 CmdStarted: OBSERVER.dmbiz 625 cherno acquire -c -t 20 > > > 03:39:28 boss exposure 300.0, file=3D335758 > > > 03:44:49 CmdStarted: OBSERVER.dmbiz 632 jaeger configuration reverse > > > 03:45:18 CmdStarted: OBSERVER.dmbiz 633 jaeger fvc loop --no-apply > > 03:45:41 OBSERVER.dmbiz 633 jaeger d > > fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0008.fits > > 03:45:44 OBSERVER.dmbiz 633 jaeger i text=3D"[FVC]: RMS full fit 19.240 u= m." > > > (3) > > 03:46:25 CmdStarted: OBSERVER.dmbiz 634 jaeger configuration dither 0.1 > > 03:46:47 OBSERVER.dmbiz 634 jaeger : text=3D"Dithered configuration 838 > > loaded and executed." > > > 03:47:04 CmdStarted: OBSERVER.dmbiz 635 jaeger fvc loop --no-apply > > 03:47:27 OBSERVER.dmbiz 635 jaeger d > > fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0009.fits > > 03:47:29 OBSERVER.dmbiz 635 jaeger i text=3D"[FVC]: RMS full fit 75.060 u= m." > > > 03:49:18 boss exposure 300.0, file=3D335759, extremely cloudy. > > > 03:54:31 CmdStarted: OBSERVER.dmbiz 642 jaeger configuration reverse > > > 03:55:06 CmdStarted: OBSERVER.dmbiz 643 jaeger fvc loop --no-apply > > 03:55:29 OBSERVER.dmbiz 643 jaeger d > > fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0010.fits > > 03:55:32 OBSERVER.dmbiz 643 jaeger i text=3D"[FVC]: RMS full fit 12.870 u= m." > > > (4) > > 03:55:48 CmdStarted: OBSERVER.dmbiz 644 jaeger configuration dither 0.1 > > 03:56:10 OBSERVER.dmbiz 644 jaeger : text=3D"Dithered configuration 839 > > loaded and executed." > > > 03:56:42 CmdStarted: OBSERVER.dmbiz 645 jaeger fvc loop --no-apply > > 03:57:04 OBSERVER.dmbiz 645 jaeger d > > fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0011.fits > > 03:57:07 OBSERVER.dmbiz 645 jaeger i text=3D"[FVC]: RMS full fit 78.070 u= m." > > > 04:03:50 boss exposure 300.0, file=3D335760; A little better. Seeing 1.= 2" > > > 04:09:02 CmdStarted: OBSERVER.dmbiz 651 jaeger configuration reverse > > > 04:09:25 CmdStarted: OBSERVER.dmbiz 652 jaeger fvc loop --no-apply > > 04:09:48 OBSERVER.dmbiz 652 jaeger d > > fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0012.fits > > 04:09:51 OBSERVER.dmbiz 652 jaeger i text=3D"[FVC]: RMS full fit 12.530 u= m." > > > (5) > > 04:10:09 CmdStarted: OBSERVER.dmbiz 653 jaeger configuration dither 0.1 > > 04:10:30 OBSERVER.dmbiz 653 jaeger : text=3D"Dithered configuration 840 > > loaded and executed." > > > 04:10:49 CmdStarted: OBSERVER.dmbiz 654 jaeger fvc loop --no-apply > > 04:11:12 OBSERVER.dmbiz 654 jaeger d > > fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0013.fits > > 04:11:15 OBSERVER.dmbiz 654 jaeger i text=3D"[FVC]: RMS full fit 78.640 u= m." > > > 04:15:46 boss exposure 300.0, file=3D335761 > > > 04:20:59 CmdStarted: OBSERVER.dmbiz 660 jaeger configuration reverse > > > 04:21:24 CmdStarted: OBSERVER.dmbiz 661 jaeger fvc loop --no-apply > > 04:21:47 OBSERVER.dmbiz 661 jaeger d > > fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0014.fits > > 04:21:50 OBSERVER.dmbiz 661 jaeger i text=3D"[FVC]: RMS full fit 13.320 u= m." > > > (6) > > 04:22:37 CmdDone: OBSERVER.dmbiz 662 jaeger configuration dither 0.1 > > 04:22:37 OBSERVER.dmbiz 662 jaeger : text=3D"Dithered configuration 841 > > loaded and executed." > > > 04:24:05 CmdDone: OBSERVER.dmbiz 663 jaeger fvc loop --no-apply > > 04:23:45 OBSERVER.dmbiz 663 jaeger d > > fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0015.fits > > 04:23:48 OBSERVER.dmbiz 663 jaeger i text=3D"[FVC]: RMS full fit 78.260 u= m." > > > 04:28:18 boss exposure 300.0, file=3D335762. Cloudy. > > > 04:33:30 CmdStarted: OBSERVER.dmbiz 670 jaeger configuration reverse > > > 04:33:53 CmdStarted: OBSERVER.dmbiz 671 jaeger fvc loop --no-apply > > 04:34:16 OBSERVER.dmbiz 671 jaeger d > > fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0016.fits > > 04:34:18 OBSERVER.dmbiz 671 jaeger i text=3D"[FVC]: RMS full fit 11.640 u= m." > > > (7) > > 04:34:39 CmdStarted: OBSERVER.dmbiz 672 jaeger configuration dither 0.1 > > 04:35:00 OBSERVER.dmbiz 672 jaeger : text=3D"Dithered configuration 842 > > loaded and executed." > > > 04:35:23 CmdStarted: OBSERVER.dmbiz 673 jaeger fvc loop --no-apply > > 04:35:46 OBSERVER.dmbiz 673 jaeger d > > fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0017.fits > > 04:35:49 OBSERVER.dmbiz 673 jaeger i text=3D"[FVC]: RMS full fit 77.870 u= m." > > > 04:37:14 boss exposure 300.0, file=3D335763 > > > 04:42:47 CmdStarted: OBSERVER.dmbiz 679 jaeger configuration reverse > > > 04:43:10 CmdStarted: OBSERVER.dmbiz 680 jaeger fvc loop --no-apply > > 04:43:33 OBSERVER.dmbiz 680 jaeger d > > fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0018.fits > > 04:43:36 OBSERVER.dmbiz 680 jaeger i text=3D"[FVC]: RMS full fit 11.780 u= m." > > > (8) > > 04:43:52 CmdStarted: OBSERVER.dmbiz 681 jaeger configuration dither 0.1 > > 04:44:12 OBSERVER.dmbiz 681 jaeger : text=3D"Dithered configuration 843 > > loaded and executed." > > > 04:43:53 OBSERVER.dmbiz 680 jaeger f error=3D"Command 'fvc loop --no-appl= y' > > failed > > because of an uncaught error 'JaegerError: Configuration needs to be set > > and > > loaded to the DB.'. See traceback in the log for more information." > > 04:45:14 CmdStarted: OBSERVER.dmbiz 683 jaeger fvc loop --no-apply > > 04:45:37 OBSERVER.dmbiz 683 jaeger d > > fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0019.fits > > 04:45:40 OBSERVER.dmbiz 683 jaeger i text=3D"[FVC]: RMS full fit 80.470 u= m." > > > 04:47:29 boss exposure 300.0, file=3D335764 > > > 04:52:42 CmdStarted: OBSERVER.dmbiz 689 jaeger configuration reverse > > > 04:53:02 CmdStarted: OBSERVER.dmbiz 690 jaeger fvc loop --no-apply > > 04:53:24 OBSERVER.dmbiz 690 jaeger d > > fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0020.fits > > 04:53:27 OBSERVER.dmbiz 690 jaeger i text=3D"[FVC]: RMS full fit 69.940 u= m." > > > (9) > > 04:53:48 CmdStarted: OBSERVER.dmbiz 691 jaeger configuration dither 0.1 > > 04:54:08 OBSERVER.dmbiz 691 jaeger : text=3D"Dithered configuration 845 > > loaded and executed." > > > 04:54:51 CmdStarted: OBSERVER.dmbiz 692 jaeger fvc loop --no-apply > > 04:55:14 OBSERVER.dmbiz 692 jaeger d > > fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0021.fits > > 04:55:16 OBSERVER.dmbiz 692 jaeger i text=3D"[FVC]: RMS full fit 79.780 u= m." > > > 04:56:36 boss exposure 300.0, file=3D335765 > > > 05:02:03 CmdStarted: OBSERVER.dmbiz 698 jaeger configuration reverse > > > 05:02:26 .hub 0 cmds d CmdIn=3D"OBSERVER.dmbiz","jaeger","fvc loop > > --no-apply" > > 05:02:49 OBSERVER.dmbiz 699 jaeger d > > fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0022.fits > > 05:02:52 OBSERVER.dmbiz 699 jaeger i text=3D"[FVC]: RMS full fit 10.060 u= m." > > > jaeger unwind > > > 05:05:09 OBSERVER.dmbiz 700 jaeger f error=3D"Trajectory failed with erro= r: > > Something went wrong during the trajectory: The FPS got locked during the > > trajectory. Trajectory failed 11.89 seconds after start" > > fps_snapshot_59577_0031.pdf > > > **** > > Focus test - take BOSS exposures in an undithered configuration at offset > > focus > > -100, -50, 50, and 100 microns of secondary motion. > > > 35914 75.000000 20.000000 fps_position_fields_10_075_20_BOSS.fits > > 433 Field cadence: bright_1x1 Calibration targets: sky_boss: 50/50 > > standard_boss: 16/50 sky_apogee: 0/35 standard_apogee: 0/15 Science > > targets: > > BOSS targets assigned: 433 APOGEE targets assigned: 0 Targets per epoch: > > 433 > > Robots used: BOSS-only: 201 APOGEE-BOSS: 298 Carton completion: > > manual_fps_position_stars_10: 433 of 2994 > > > 05:14:25 CmdStarted: OBSERVER.dmbiz 713 jaeger configuration load 35914 > > 05:15:36 OBSERVER.dmbiz 713 jaeger : text=3D"Configuration 846 loaded." > > 05:16:34 CmdDone: OBSERVER.dmbiz 714 jaeger configuration execute > > > 05:18:11 CmdStarted: OBSERVER.dmbiz 715 hal goto-field -s > > prepare,slew,boss_hartmann,boss_arcs,boss_flat,fvc,acquire > > > 05:02:49 OBSERVER.dmbiz 699 jaeger d > > fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0022.fits > > ... > > 05:23:56 OBSERVER.dmbiz.jaeger 433 jaeger d > > fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0026.fits > > > 05:21:53 OBSERVER.dmbiz.jaeger 433 jaeger i text=3D"[FVC]: RMS full fit > > 81.170 um." > > 05:22:36 OBSERVER.dmbiz.jaeger 433 jaeger i text=3D"[FVC]: RMS full fit > > 23.670 um." > > 05:23:17 OBSERVER.dmbiz.jaeger 433 jaeger i text=3D"[FVC]: RMS full fit > > 22.590 um." > > 05:23:59 OBSERVER.dmbiz.jaeger 433 jaeger i text=3D"[FVC]: RMS full fit > > 22.350 um." > > > 05:19:58 boss exposure 4.0, file=3D335766 > > 05:20:38 boss exposure 4.0, file=3D335767 > > 05:21:28 sp1.motor.move=3D -1099, -1100, -1096 > > 05:21:28 Hartmann collimate output on cart #19 > > sp1: offset: r=3D -659 (In Focus); b=3D 4.8 (Out of focus) > > sp1: pred. move: spAverageMove=3D -1100 > > sp1: pred. resid: r=3D 441, b=3D 1.4, txt=3D OK, spTemp =3D -0.1 > > 05:24:49 boss exposure 4.0, file=3D335768 > > 05:26:29 boss exposure 25.0, file=3D335769 > > > Focus loop. Best focus at 380. Seeing 1.1". Still cloudy, but better than > > before. > > > 05:39:43 boss exposure 300.0, file=3D335770 > > > OBSERVER.dmbiz 732 tcc set focus=3D280.0 (-100) > > > 05:46:11 boss exposure 300.0, file=3D335771 > > > OBSERVER.dmbiz 732 tcc set focus=3D330.0 (-50) > > > 05:52:37 boss exposure 300.0, file=3D335772 > > > OBSERVER.dmbiz 732 tcc set focus=3D430 (50) > > > 05:59:07 boss exposure 300.0, file=3D335773 > > > OBSERVER.dmbiz 732 tcc set focus=3D480 (100) > > > 06:05:33 boss exposure 300.0, file=3D335774 > > > OBSERVER.dmbiz 732 tcc set focus=3D380 (Best focus) > > > 06:12:00 boss exposure 300.0, file=3D335775 > > > *** > > > 3x3 telescope dither test with an undithered BOSS configuration > > (configuration 35914), spaced by 1". > > > We were unsure as to what to do with the guider during these exposures. I= f > > we > > left the guiding alone, and offset the telescope, the guider will just > > bring the > > stars back to where they are supposed to be. Therefore, we decided to not > > guide > > at all during the offseted exposures. The other option would be to apply > > the > > cherno offset command (which if we have time, perhaps we will do later). > > > IGNORE exposure 06:19:25 boss exposure 300.0, file=3D335776 since we we= re > > figuring out the steps we should take going forward. > > > boss exposure 300.0, file=3D335777 > > [-1",-1"] > > > boss exposure 300.0, file=3D335778 > > [-1",0"] > > > boss exposure 300.0, file=3D335779 > > [-1",1"] > > > boss exposure 300.0, file=3D335780 > > [0",1"] > > > boss exposure 300.0, file=3D335781 > > [1",1"] > > > boss exposure 300.0, file=3D335782 > > [0",1"] > > > boss exposure 300.0, file=3D335783 > > [1",-1"] > > > boss exposure 300.0, file=3D335784 > > [0",-1"] > > > boss exposure 300.0, file=3D335785 > > [0",0"] > > > ** Now, activate the guider and guide at the same offset as the telescope= . > > > boss exposure 300.0, file=3D335786 > > [-1",-1"] > > > boss exposure 300.0, file=3D335787 > > [-1",0"] > > > boss exposure 300.0, file=3D335788 > > [-1",1"] > > > boss exposure 300.0, file=3D335789 > > [0",1"] > > > boss exposure 300.0, file=3D335790 > > [1",1"] > > > boss exposure 300.0, file=3D335791 > > [0",1"] > > > boss exposure 300.0, file=3D335792 > > [1",-1"] > > > boss exposure 300.0, file=3D335793 > > [0",-1"] > > > boss exposure 300.0, file=3D335794 > > [0",0"] > > > However, it wasn't until after we finished the test that we noticed that > > we only > > implemented the cherno offset and not the telescope offset as well. > > > Coolant flow gradually getting worse. Down to 0.2L/min. Going to try to d= o > > the > > same thing as before to get it to behave. > > > 08:46:24 APO.aoravetz 277 jaeger : fbi_led=3D0.0,0.0,0.0,0.0; > > flow=3D5.846,0.001; > > humidity=3D25.294,54.079; power_can=3DT,T,T,T,T,T; power_gfa=3DF,T,T,T,T,= T; > > power_nuc=3DT,T,T,T,T,T; power_sextant=3DT,T,T,T,T,T; power_sync=3DF; > > pressure=3D31.785,20.962; > > temperature=3D4.962,-3.583,8.35,9.22,9.12,1.82,-1.48,- > > 0.18,16.22,0.602,1.22,1.41,5.869; > > > 08:37:02 CmdStarted: OBSERVER.dmbiz 791 jaeger chiller set flow 10 > > > 08:45:28 CmdStarted: OBSERVER.dmbiz 792 jaeger chiller set flow 0.2 > > > 08:47:04 CmdStarted: OBSERVER.dmbiz 794 jaeger chiller set flow 10 > > > 08:47:28 CmdStarted: OBSERVER.dmbiz 799 jaeger ieb status > > 08:47:28 OBSERVER.dmbiz 799 jaeger > > > 08:47:28 OBSERVER.dmbiz 799 jaeger : fbi_led=3D0.0,0.0,0.0,0.0; > > flow=3D6.794,0.485; > > humidity=3D25.391,53.981; power_can=3DT,T,T,T,T,T; power_gfa=3DF,T,T,T,T= ,T; > > power_nuc=3DT,T,T,T,T,T; power_sextant=3DT,T,T,T,T,T; power_sync=3DF; > > pressure=3D37.071,22.769; > > temperature=3D4.962,-3.583,8.65,9.22,9.12,1.72,-1.48,- > > 0.18,16.22,0.702,0.92,1.11,5.769; > > voltage=3D24.112,24.323,24.183,23.979,24.144,24.065 > > > Better. > > > 08:44Z - Waited for the clouds to move off to start another test. > > > *** > > > GRAFANA was hanging and was no longer responsive. > > > Restarted the cerebro with systemcontol. It didn't help. We are still > > having > > coolant flow issues. > > > 11:06Z - We closed due to clouds, humidity, and the FPS flow issue. > > Started cals. > > > > ---- Mirror Numbers ---- > > > > PRIMARY: > > -------- > > Scale: 1.000000 > > > MIGS TONIGHT NOMINAL > > Axial A: 0.0020 0.0040 > > Axial B: 0.0000 0.0030 > > Axial C: -0.1110 -0.1150 > > Trans D: -0.0340 -0.0338 > > Lateral E: 0.3315 0.3315 > > Lateral F: 0.5350 0.5350 > > > GALILS > > Commanded: 12500 -1600 5300 47050 -8800 -8400 > > Actual: 12500 -1600 5300 47050 -8800 -8400 > > > SETMIR VALUES > > PrimDesOrient: 0.0 -59.5 -9.15 -512.0 1906.3 > > PrimOrient: 1.07 -59.51 -9.21 -512.13 1906.31 > > > SECONDARY: > > ---------- > > > Focus: 0.0000 > > Air temp: -0.11 > > Altitude: 30.0 > > > MIGS TONIGHT NOMINAL > > Axial A: -9.6560 -9.6570 > > Axial B: -9.6710 -9.6710 > > Axial C: -9.6950 -9.6970 > > Trans D: 0.1130 0.1140 > > > GALILS > > Commanded: 4896650 4897450 4912650 -50 -3450 > > Actual: 4859186 4859230 4859262 -50 -3450 > > > SETMIR VALUES > > SecDesOrient: 3857.0 0.0 0.0 0.0 42.22 > > SecOrient: 3857.01 -0.01 0.02 0.01 42.87 > > > > > ---- Telescope Status ---- > > > 23:55Z Started ventilation > > 01:10Z encl25m : open > > 11:17Z encl25m : closed > > > No interlocks bypassed > > > Status at: > > Telescope stowed at: 121.0, 30.0, 0.0 mount > > Instrument mounted: cartridge =3D fps > > Counterweights at: 680 680 705 705 > > LN2 autofill systems: Connected and turned on > > 180L LN2 dewar scales: SP1 175 lbs, 20 psi > > > > ---- Software Used ---- > > > > TUI : 1.7.13 > > alerts : 2.2.0 > > apo : 3.0.1 > > boss : 6.0.1-alpha.0 > > -boss.specMec : 3.0.4 > > guider : 4.0.0 > > ecamera : FAILED > > fliswarm : FAILED > > hartmann : 2.0.1-alpha.0 > > hub : 5.0.0a0 > > mcp : 2.1 > > -mcp.plc : 34 > > -mcp.azFiduci : v1_148 > > -mcp.altFiduc : v1_148 > > -mcp.rotFiduc : v1_148 > > sop : FAILED > > tcc : 1.4.0 > > apogee : not availble > > apogeecal : not availble > > apogeeql : v1_7_0dev > > STUIAdditions : 3.1.9 > > > > /dev/sdb1 2.0T 244G 1.8T 12% /home > > 10.25.1.230:/data 21T 17T 3.3T 84% /data > > > > ---- APOGEE Data Summary ---- > > > > last morning: > > 59576 10 11:48 40140010 Dark 60 B 13.5 FPS- 0 a-b-c - - > > 59576 11 11:58 40140011 Dark 60 B 13.5 FPS- 0 a-b-c - - > > 59576 12 12:09 40140012 Dark 60 B 13.5 FPS- 0 a-b-c - - > > 59576 13 12:11 40140013 QuarFlat 10 B 13.5 FPS- 0 a-b-c - 10= 0 > > 59576 14 12:13 40140014 QuarFlat 10 B 13.5 FPS- 0 a-b-c - 10= 0 > > 59576 15 12:15 40140015 QuarFlat 10 B 13.5 FPS- 0 a-b-c - 10= 0 > > 59576 16 12:17 40140016 Arc-Thar 12 A 13.0 FPS- 0 a-b-c -0.08 - > > 59576 17 12:24 40140017 Arc-Une 40 A 13.0 FPS- 0 a-b-c - - > > 59576 18 12:26 40140018 Arc-Thar 12 B 13.5 FPS- 0 a-b-c 0.51 - > > 59576 19 12:34 40140019 Arc-Une 40 B 13.5 FPS- 0 a-b-c - - > > 59576 20 12:39 40140020 Dark 30 A 13.0 FPS- 0 a-b-c - - > > 59576 21 12:45 40140021 InteFlat 30 A 13.0 FPS- 0 a-b-c - 27= 9 > > 59576 22 12:50 40140022 InteFlat 30 A 13.0 FPS- 0 a-b-c - 28= 0 > > 59576 23 12:55 40140023 InteFlat 30 A 13.0 FPS- 0 a-b-c - 27= 9 > > 59576 24 13:01 40140024 Dark 30 A 13.0 FPS- 0 a-b-c - - > > > > this evening: > > > -------------------------------------------------------------------------= ----- > > mjd i UT File/Exp Imtype Nread A/B Dth Ct-Plate Arch Offset %Flu= x > > Comm > > > ------------------------------------------------------------------------= ------ > > 59577 1 22:47 40150001 Arc---- 3 A 13.0 FPS- 0 a-b-c - - > > test > > 59577 2 22:59 40150002 Dark 60 A 13.0 -------- a-b-c - - > > 59577 3 23:01 40150003 QuarFlat 10 A 13.0 FPS- 0 a-b-c - 10= 0 > > 59577 4 23:03 40150004 Arc-Thar 12 B 13.5 FPS- 0 a-b-c 0.51 - > > 59577 5 23:10 40150005 Arc-Une 40 B 13.5 FPS- 0 a-b-c - - > > 59577 6 23:13 40150006 Arc-Thar 12 A 13.0 FPS- 0 a-b-c -0.11 - > > 59577 7 23:20 40150007 Arc-Une 40 A 13.0 FPS- 0 a-b-c - - > > 59577 8 23:31 40150008 Dark 60 A 13.0 -------- a-b-c - - > > 59577 9 23:38 40150009 Arc---- 30 A 13.0 FPS- 0 x-x-x - - > > FPI > > > -------------------------------------------------------------------------= ----- > > > APOGEE data list, mjd=3D59577, all > > raw_data: /data/apogee/utr_cdr/59577/apRaw-*.fits > > archive: /data/apogee/archive/59577/apR-[a,b,c]-*.apz > > quickred: /data/apogee/quickred/59577/apq-*.fits > > > -------------------------------------------------------------------------= ----- > > mjd i UT File/Exp Imtype Nread A/B Dth Ct-Plate Arch Offset %Flu= x > > Comm > > > ------------------------------------------------------------------------= ------ > > 59577 1 22:47 40150001 Arc---- 3 A 13.0 FPS- 0 a-b-c - - > > test > > 59577 2 22:59 40150002 Dark 60 A 13.0 -------- a-b-c - - > > 59577 3 23:01 40150003 QuarFlat 10 A 13.0 FPS- 0 a-b-c - 10= 0 > > 59577 4 23:03 40150004 Arc-Thar 12 B 13.5 FPS- 0 a-b-c 0.51 - > > 59577 5 23:10 40150005 Arc-Une 40 B 13.5 FPS- 0 a-b-c - - > > 59577 6 23:13 40150006 Arc-Thar 12 A 13.0 FPS- 0 a-b-c -0.11 - > > 59577 7 23:20 40150007 Arc-Une 40 A 13.0 FPS- 0 a-b-c - - > > 59577 8 23:31 40150008 Dark 60 A 13.0 -------- a-b-c - - > > 59577 9 23:38 40150009 Arc---- 30 A 13.0 FPS- 0 a-b-c - - > > FPI > > > -------------------------------------------------------------------------= ----- > > > > > ---- eBOSS Data Summary ---- > > > Reading FITS from /data/spectro//59577 > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > > - - > > n Time Filename crt plt flav shu Exp colA colB colC Hr= t > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > > - - > > 1 22:47 b1-00335746 FPS-N None bia Clo 0.0 -4103 -4096 -4125 > > 2 22:47 r1-00335746 FPS-N None bia Clo 0.0 -4103 -4096 -4125 > > 3 22:51 b1-00335747 FPS-N None dar Clo 20.0 -4103 -4096 -4125 > > 4 22:51 r1-00335747 FPS-N None dar Clo 20.0 -4103 -4096 -4125 > > 5 22:54 b1-00335748 FPS-N None arc Lef 4.1 -4103 -4096 -4125 > > HART > > 6 22:54 r1-00335748 FPS-N None arc Lef 4.1 -4103 -4096 -4125 > > HART > > 7 22:55 b1-00335749 FPS-N None arc Rig 4.1 -4103 -4096 -4125 > > HART > > 8 22:55 r1-00335749 FPS-N None arc Rig 4.1 -4103 -4096 -4125 > > HART > > 9 22:58 b1-00335750 FPS-N None arc Out 4.1 -6027 -6021 -6051 > > 10 22:58 r1-00335750 FPS-N None arc Out 4.1 -6027 -6021 -6051 > > 11 22:59 b1-00335751 FPS-N None fla Out 25.1 -6027 -6021 -6051 > > 12 22:59 r1-00335751 FPS-N None fla Out 25.1 -6027 -6021 -6051 > > 13 02:18 r1-00335752 FPS-N None arc Lef 4.1 -6027 -6021 -6051 > > HART > > 14 02:18 b1-00335752 FPS-N None arc Lef 4.1 -6027 -6021 -6051 > > HART > > 15 02:18 r1-00335753 FPS-N None arc Rig 4.1 -6027 -6021 -6051 > > HART > > 16 02:18 b1-00335753 FPS-N None arc Rig 4.1 -6027 -6021 -6051 > > HART > > 17 02:22 b1-00335754 FPS-N None arc Out 4.1 -6480 -6473 -6507 > > 18 02:22 r1-00335754 FPS-N None arc Out 4.1 -6480 -6473 -6507 > > 19 02:24 b1-00335755 FPS-N None fla Out 25.1 -6480 -6473 -6507 > > 20 02:24 r1-00335755 FPS-N None fla Out 25.1 -6480 -6473 -6507 > > 21 03:00 b1-00335756 FPS-N None sci Out 300.1 -6480 -6473 -6507 > > 22 03:00 r1-00335756 FPS-N None sci Out 300.1 -6480 -6473 -6507 > > 23 03:19 b1-00335757 FPS-N None sci Out 300.1 -6480 -6473 -6507 > > 24 03:19 r1-00335757 FPS-N None sci Out 300.1 -6480 -6473 -6507 > > 25 03:39 b1-00335758 FPS-N None sci Out 300.1 -6480 -6473 -6507 > > 26 03:39 r1-00335758 FPS-N None sci Out 300.1 -6480 -6473 -6507 > > 27 03:49 b1-00335759 FPS-N None sci Out 300.1 -6480 -6473 -6507 > > 28 03:49 r1-00335759 FPS-N None sci Out 300.1 -6480 -6473 -6507 > > 29 04:03 b1-00335760 FPS-N None sci Out 300.1 -6480 -6473 -6507 > > 30 04:03 r1-00335760 FPS-N None sci Out 300.1 -6480 -6473 -6507 > > 31 04:15 b1-00335761 FPS-N None sci Out 300.1 -6480 -6473 -6507 > > 32 04:15 r1-00335761 FPS-N None sci Out 300.1 -6480 -6473 -6507 > > 33 04:28 b1-00335762 FPS-N None sci Out 300.1 -6480 -6473 -6507 > > 34 04:28 r1-00335762 FPS-N None sci Out 300.1 -6480 -6473 -6507 > > 35 04:37 b1-00335763 FPS-N None sci Out 300.1 -6480 -6473 -6507 > > 36 04:37 r1-00335763 FPS-N None sci Out 300.1 -6480 -6473 -6507 > > 37 04:47 b1-00335764 FPS-N None sci Out 300.1 -6480 -6473 -6507 > > 38 04:47 r1-00335764 FPS-N None sci Out 300.1 -6480 -6473 -6507 > > 39 04:56 b1-00335765 FPS-N None sci Out 300.1 -6480 -6473 -6507 > > 40 04:56 r1-00335765 FPS-N None sci Out 300.1 -6480 -6473 -6507 > > 41 05:19 r1-00335766 FPS-N None arc Lef 4.1 -6480 -6473 -6507 > > HART > > 42 05:19 b1-00335766 FPS-N None arc Lef 4.1 -6480 -6473 -6507 > > HART > > 43 05:20 r1-00335767 FPS-N None arc Rig 4.1 -6480 -6473 -6507 > > HART > > 44 05:20 b1-00335767 FPS-N None arc Rig 4.1 -6480 -6473 -6507 > > HART > > 45 05:24 r1-00335768 FPS-N None arc Out 4.1 -7579 -7573 -7603 > > 46 05:24 b1-00335768 FPS-N None arc Out 4.1 -7579 -7573 -7603 > > 47 05:26 b1-00335769 FPS-N None fla Out 25.1 -7579 -7573 -7603 > > 48 05:26 r1-00335769 FPS-N None fla Out 25.1 -7579 -7573 -7603 > > 49 05:39 r1-00335770 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 50 05:39 b1-00335770 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 51 05:46 r1-00335771 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 52 05:46 b1-00335771 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 53 05:52 r1-00335772 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 54 05:52 b1-00335772 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 55 05:59 b1-00335773 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 56 05:59 r1-00335773 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 57 06:05 r1-00335774 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 58 06:05 b1-00335774 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 59 06:12 b1-00335775 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 60 06:12 r1-00335775 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 61 06:19 r1-00335776 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 62 06:19 b1-00335776 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 63 06:26 r1-00335777 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 64 06:26 b1-00335777 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 65 06:32 b1-00335778 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 66 06:32 r1-00335778 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 67 06:44 b1-00335779 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 68 06:44 r1-00335779 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 69 06:50 b1-00335780 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 70 06:50 r1-00335780 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 71 06:57 b1-00335781 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 72 06:57 r1-00335781 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 73 07:03 r1-00335782 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 74 07:03 b1-00335782 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 75 07:10 r1-00335783 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 76 07:10 b1-00335783 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 77 07:16 r1-00335784 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 78 07:16 b1-00335784 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 79 07:23 r1-00335785 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 80 07:23 b1-00335785 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 81 07:38 r1-00335786 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 82 07:38 b1-00335786 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 83 07:45 r1-00335787 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 84 07:45 b1-00335787 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 85 07:51 r1-00335788 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 86 07:51 b1-00335788 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 87 07:58 r1-00335789 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 88 07:58 b1-00335789 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 89 08:04 b1-00335790 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 90 08:04 r1-00335790 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 91 08:13 b1-00335791 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 92 08:13 r1-00335791 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 93 08:20 b1-00335792 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 94 08:20 r1-00335792 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 95 08:27 b1-00335793 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 96 08:27 r1-00335793 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 97 08:33 b1-00335794 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > 98 08:33 r1-00335794 FPS-N None sci Out 300.1 -7579 -7573 -7603 > > > > ____________________________________________________________ > > Logs of Nightly APO Activities > > https://trac.sdss.org/tracmailman/browser/private/sloan-25mlog/index.html > > > > > ____________________________________________________________ > Logs of Nightly APO Activities > https://trac.sdss.org/tracmailman/browser/private/sloan-25mlog/index.html > --0000000000008a34a805d48d5788 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Audrey, Shane,The GFA plots use a query that's not super optimal because it involv= es casting strings to floats. I'm planning to improve that but it requi= res some work under the hood.In that case restarting Cerebro won't help. My advise is that you t= ry to avoid using too high a refresh rate (e.g. 5 or 15 seconds) when using= Grafana, especially on the GFA/FVC page. Also, consider closing tabs with = Grafana in them when not using them, especially if you're connecting re= motely.Grafana is not rea= lly meant as a real-time tool (meaning seeing something just as it happens)= so refresh rates of 1 minute or so would be more appropriate, but I unders= tand that since there are no GUIs right now you use Grafana in that way. Fo= r those cases, I think it's better to have Grafana with a refresh rate = of 30 seconds and if you need some quick feedback, you can always press the= little circular arrow.Thanks,Jos=C3=A9 On Dec 29, 2021 at= 03:21:07, Audrey Oravetz wrote: =20 Also, would someone have a look at GRAFANA? It continuously hangs whentrying to access the GFA page. Cerebro was restarted a couple of times,which helped with the FPS page, but did nothing with the GFA page. Thank= s! =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0APACHE POINT OBSERVATORY SDSS 2.5M= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0OBSERVING= LOG =C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0Tuesday D= ecember 28, 2021 =C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0(SJD 59577)= ---- =C2=A0Observing Team ---- Dmitry Biz= yaev Audrey Oravetz Zoom: Kaike, Karen, DanO, Katie= ---- Observing Plan = ----= FPS commissioning tasks= ---- Observing Summa= ry ---- Cloudy, but we gathered: 9 loops of science data WITH fvc loop --no-apply in= between dither confi= guration 0.1 / revers= e sequence for a BOSS field (35884). Focus= test - BOSS exposures at varying secondary mirror values (design 35914). 3x3 telescope offset test with an undithered BOSS configuration = (design 35914), spaced by 1" Notes: =3D=3D=3D=3D=3D=3D ** Hal goto-= field failed on us tonight on the guider 'acquire' step because the routine did not open the FF petals automa= tically. Note that some of the commands we= re purposely omitted (05:18:11 CmdStarted: OBSERVER.dmbiz 715 hal goto-field -s= prepare,slew,boss_hartmann,boss_arcs,= boss_flat,fvc,acquire) ** ** We had many= instances where the coolant flow to the GFAs was practically zero. =C2=A0We tweaked the chiller flow rate by setting= it down from 10 to 2, and then back up a= gain to 10 L/min. It helped temporarily, but the flow returned to negligible values a few hours later and remained ar= ound 0. Engineers - please take a look at = =C2=A0the coolant system tomorrow as the behavior is quite erratic. Also, please advise us the appropriate steps to f= ollow if this is to happen again. ---- Obs= erving Log ---- Afternoon= --------- All systems checked, no problems found. 2:56:12 Hartmann collimate output on cart #19 =C2=A0=C2=A0=C2=A0sp1: of= fset: r=3D -2308 (Out of focus); =C2=A0b=3D 4.8 (Out of focus) =C2=A0=C2=A0=C2=A0sp1: pred. move= : spAverageMove=3D -1925 =C2=A0=C2=A0=C2=A0sp1: pred. resid: r=3D -383, b=3D -1.2, txt=3D OK, sp= Temp =3D 2.8 22:55Z Started ventilation Night: =3D= =3D=3D=3D=3D=3D 00:26Z - Opened telescope = and started a fill. 00:39Z - Fill is done.= We are waiting until the sky gets clearer and less bright =C2=A0before proceeding.= The coolant flow through the GFA's were showing 0.0 L/min starting at 00:30Z. Before this, the flow was stable a= t around 1.1L/min, except for mid-day today where the flow showed zero for hours. 01= :04:53 CmdStarted: OBSERVER.dmbiz 555 jaeger ieb status fbi_led=3D0.0,0.0,0.0,0.0; flow=3D6.217,= 0.001; humidity=3D21.216,39.674; power_can=3DT,T,T,T,T,T; power_gfa=3DF,T,T,T,T,T; power_nuc=3DT= ,T,T,T,T,T; power_sex= tant=3DT,T,T,T,T,T; power_sync=3DF; pressure=3D42.406,25.034; 01:06:21 CmdStarted: OBSERVER.dmbiz 557 jaeger chille= r set flow 1.5 01:12:= 16 CmdDone: OBSERVER.dmbiz 564 jaeger chiller set flow 6.2= 01:18:00 CmdStarted: OBSERVER.dmbiz 5= 65 jaeger chiller set flow 10 Tweaking the= set point seemed to seemed to help, because the flow is back to normal: 01:26:13 AP= O.aoravetz 275 jaeger : fbi_led=3D0.0,0.0,0.0,0.0; flow=3D6.676,1.017; humidity=3D22.34,39.821; power_can=3DT,T,T,T,T= ,T; power_gfa=3DF,T,T,T,T,T; power_nuc=3DT,T,T,T,T,T; power_sextant=3DT,T,T,T,T,T; power_sync=3D= F; pressure=3D42.101,= 24.844; temperature= =3D3.888,-1.044,10.15,7.02,6.92,2.22,-0.48,- 0.38,13.52,2.702,1.52,1.91,8.169; voltage=3D24.105,24.308,24.175,23.972,= 24.151,24.065 **** BOSS: 35884 15.000000 40.000000 fps_position_fields_10_015_40_BOS= S.fits 398 Field cade= nce: bright_1x1 Calibration targets: sky_boss: 50/50 standard_boss: 50/50 sky_apogee: 0/35 stand= ard_apogee: 0/15 Science targets: BOSS tar= gets assigned: 397 APOGEE targets assigned: 0 Targets per epoch: 397 Robots used: BOSS-only: 201 APOGEE-BOSS: 296 Car= ton completion: manua= l_fps_position_stars_10: 398 of 3359 02:00= :18 CmdDone: OBSERVER.dmbiz 566 jaeger configuration load 35884 Failed at execution many times: fps_snapshot_59577_00= 01.pdf jaeger unwind = --force fps_snapshot_= 59577_0005.pdf 02:11:25 CmdStarted: OBSERV= ER.dmbiz 582 jaeger configuration load 35884 02:13:10 CmdStarted: OBSERVER.dmbiz 583 jaeger conf= iguration execute 02:= 15:38 OBSERVER.dmbiz 611 jaeger i text=3D"Configuration 835."= 02:15:56 CmdStarted: OBSERVER.dmbiz 584 hal g= oto-field -s prepare,= slew,boss_hartmann,boss_arcs,boss_flat,fvc= 02:17:55 =C2=A0sp1.motor.move=3D -1924, -1925, -1926 02:18:03 =C2=A0boss exposure =C2=A0=C2=A0= =C2=A04.0, file=3D335752 02:18:44 =C2=A0boss exposure =C2=A0=C2=A0=C2=A04.0, file=3D335753 02:19:25 =C2=A0sp1.motor.m= ove=3D -453, -452, -456 02:19:25 Hartmann collimate output on cart #19 =C2=A0=C2=A0=C2=A0sp1: offset: r=3D -329 (In F= ocus); =C2=A0b=3D 1.8 (In Focus) =C2=A0=C2=A0=C2=A0sp1: pred. move: spAverageMove=3D -453 =C2=A0=C2=A0=C2=A0sp1: pred= . resid: r=3D 124, b=3D 0.4, txt=3D OK, spTemp =3D 0.3 02:22:46 =C2=A0boss exposure =C2=A0=C2=A0= =C2=A04.0, file=3D335754 02:24:19 =C2=A0boss exposure =C2=A0=C2=A025.0, file=3D335755 02:19:48 jaeger.fliswarm 611 fliswarm i filename_bundle=3D/data/fcam/5957= 7/fimg-fvc1n-0001.fits ... 02:21:54 OBSERV= ER.dmbiz.jaeger 410 jaeger d fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0004.fits 02:19:51 OBSERVER.dmbiz.jaeger 410 jaeger i text=3D"= ;[FVC]: RMS full fit = 80.280 um." 02:2= 0:34 OBSERVER.dmbiz.jaeger 410 jaeger i text=3D"[FVC]: RMS full fit 11.330 um." 02:21:16 OBSERVER.dmbiz.jae= ger 410 jaeger i text=3D"[FVC]: RMS full fit 7.230 um." 02:21:57 OBSERVER.dmbiz.jaeger 410 jaeger i text=3D= "[FVC]: RMS full fit 6.690 um." Focus sweep. Best f= ocus at 450. Seeing 1.2". 02:44:54 Cm= dStarted: OBSERVER.dmbiz 605 cherno acquire -c -t 30 Extremely cloudy. The cherno RMS never converged below 0.1 (opti= mum is 0.07), but we = started science anywa= y. 03:00:12 =C2=A0boss exposure =C2=A0300.= 0, file=3D335756 - BASE CONFIGURATION 03:0= 6:58 CmdStarted: OBSERVER.dmbiz 611 jaeger configuration dither 0.1 03:06:58 OBSERVER.dmbiz 611 = jaeger > 03:06:58 = OBSERVER.dmbiz 611 jaeger i text=3D"Creating dithered configuration from 835." (1) 03:07:19 CmdDone: OBSERVER.dmbiz 611 jaeger configuration dither 0= .1 03:07:59 CmdStarted: OBSERVER.dmbiz 612= jaeger fvc loop --no-apply 03:08:22 OBSERVER.dmbiz 612 jaeger d fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0005.fits= 03:08:24 OBSERVER.dm= biz 612 jaeger i text=3D"[FVC]: RMS full fit 80.910 um." 03:10:00 CmdStarted: OBSERVER.dmbiz 614 cherno acq= uire -c -t 20 03:19:19 =C2=A0boss exposure= =C2=A0300.0, file=3D335757, very cloudy.= 03:24:43 CmdStarted: OBSERVER.dmbiz 621 jaeger configuration reverse 03:25:04 CmdStarted: OBSERVER.dmbiz 622 jaeger = fvc loop --no-apply 0= 3:25:26 OBSERVER.dmbiz 622 jaeger d fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0006.fits 03:25:29 OBSERVER.dmbiz 622 = jaeger i text=3D"[FVC]: RMS full fit 41.190 um."= (2) = 03:26:10 CmdStarted: OBSERVER.dmbiz 623 jaeger configuration dither 0.1= 03:26:28 OBSERVER.dmbiz = 623 jaeger : text=3D"Dithered configuration 837 loaded and executed."= 03:27:39 CmdStarted: OBSERVER.dmbiz 624 jaeger fvc loop --= no-apply 03:28:02 OBS= ERVER.dmbiz /data/fcam/59577/fimg-fvc1n-0007.fits" 03:28:04 OBSERVER.dmbiz 624 jaeger i tex= t=3D"[FVC]: RMS full fit 78.550 um." 03:34:22 CmdStarted: OBSERVER.dmbiz 625 cherno acquire -c -t 20 03:39:28 =C2=A0boss exposure =C2=A0300.0, file= =3D335758 03:44:49 CmdStarted: OBSERVER.dm= biz 632 jaeger configuration reverse 03:45= :18 CmdStarted: OBSERVER.dmbiz 633 jaeger fvc loop --no-apply 03:45:41 OBSERVER.dmbiz 633 jaeger= d fvc_filename=3D/da= ta/fcam/59577/fimg-fvc1n-0008.fits 03:45:44 OBSERVER.dmbiz 633 jaeger i text=3D"[FVC]: RMS = full fit 19.240 um." (3) 03:46:25 CmdStarted: OBSERVER.d= mbiz 634 jaeger configuration dither 0.1 03:46:47 OBSERVER.dmbiz 634 jaeger : text=3D"Dit= hered configuration 838 loaded and executed." 03:47:04 Cmd= Started: OBSERVER.dmbiz 635 jaeger fvc loop --no-apply 03:47:27 OBSERVER.dmbiz 635 jaeger d fvc_filename=3D/data/fcam= /59577/fimg-fvc1n-0009.fits 03:47:29 OBSERVER.dmbiz 635 jaeger i text=3D"[FVC]: RMS full fi= t 75.060 um." 03:49:18 =C2=A0boss exp= osure =C2=A0300.0, file=3D335759, extremely cloudy. 03:54:31 CmdStarted: OBSERVER.dmbiz 642 jaeger configuration reve= rse 03:55:06 CmdStarted: OBSERVER.dmbiz 64= 3 jaeger fvc loop --no-apply 03:55:29 OBSERVER.dmbiz 643 jaeger d fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0010.fit= s 03:55:32 OBSERVER.d= mbiz 643 jaeger i text=3D"[FVC]: RMS full fit 12.870 um." (4) 03:55:48 CmdStarted: OBSERVER.dmbiz 644 jaeger configuration dithe= r 0.1 03:56:10 OBSERV= ER.dmbiz 644 jaeger : text=3D"Dithered configuration 839 loaded and executed." 03:56:42 CmdStarted: OBSERVER.dmbiz 645 jaeger fv= c loop --no-apply 03:= 57:04 OBSERVER.dmbiz 645 jaeger d fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0011.fits 03:57:07 OBSERVER.dmbiz 645 ja= eger i text=3D"[FVC]: RMS full fit 78.070 um." 04:03:50 =C2=A0boss exposure =C2=A0300.0, file=3D335760; A l= ittle better. Seeing 1.2" 04:09:02 Cm= dStarted: OBSERVER.dmbiz 651 jaeger configuration reverse 04:09:25 CmdStarted: OBSERVER.dmbiz 652 jaeger fvc loop --n= o-apply 04:09:48 OBSE= RVER.dmbiz 652 jaeger d fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0012.fits 04:09:51 OBSERVER.dmbiz 652 jaeger i tex= t=3D"[FVC]: RMS full fit 12.530 um." (5) 04:10:09 C= mdStarted: OBSERVER.dmbiz 653 jaeger configuration dither 0.1 04:10:30 OBSERVER.dmbiz 653 jaeger= : text=3D"Dithered configuration 840 loaded and executed." 04:10:49 CmdStarted: OBSERVER.dmbiz 654 jaeger fvc loop --no-apply 04:11:12 OBSERVER.dmbi= z 654 jaeger d fvc_fi= lename=3D/data/fcam/59577/fimg-fvc1n-0013.fits 04:11:15 OBSERVER.dmbiz 654 jaeger i text=3D"= ;[FVC]: RMS full fit 78.640 um." 04:1= 5:46 =C2=A0boss exposure =C2=A0300.0, file=3D335761 04:20:59 CmdStarted: OBSERVER.dmbiz 660 jaeger configuration reve= rse 04:21:24 CmdStarted: OBSERVER.dmbiz 66= 1 jaeger fvc loop --no-apply 04:21:47 OBSERVER.dmbiz 661 jaeger d fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0014.fit= s 04:21:50 OBSERVER.d= mbiz 661 jaeger i text=3D"[FVC]: RMS full fit 13.320 um." (6) 04:22:37 CmdDone: OBSERVER.dmbiz 662 jaeger configuration dither 0= .1 04:22:37 OBSERVER.= dmbiz 662 jaeger : text=3D"Dithered configuration 841= loaded and executed." 04:24:05 CmdDone: OBSERVER.dmbiz 663 jaeger fvc loop= --no-apply 04:23:45 = OBSERVER.dmbiz 663 jaeger d fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0015.fits 04:23:48 OBSERVER.dmbiz 663 jaeger i= text=3D"[FVC]: RMS full fit 78.260 um." 04:28:18 =C2=A0boss exposure =C2=A0300.0, file=3D335762. Cloudy. 04:33:30 CmdStarted: OBSERVER.dmbiz 670 jae= ger configuration reverse 04:33:53 CmdSt= arted: OBSERVER.dmbiz 671 jaeger fvc loop --no-apply 04:34:16 OBSERVER.dmbiz 671 jaeger d fvc_filename=3D/data/fcam/5= 9577/fimg-fvc1n-0016.fits 04:34:18 OBSERVER.dmbiz 671 jaeger i text=3D"[FVC]: RMS full fi= t 11.640 um." (7) 04:34:39 CmdStarted: OBSERVER.dmbiz 672 = jaeger configuration dither 0.1 04:35:00 OBSERVER.dmbiz 672 jaeger : text=3D"Dithered confi= guration 842 loaded a= nd executed." 04:35:23 CmdStarted: OB= SERVER.dmbiz 673 jaeger fvc loop --no-apply 04:35:46 OBSERVER.dmbiz 673 jaeger d fvc_filename=3D/data/fcam/59577/fimg= -fvc1n-0017.fits 04:3= 5:49 OBSERVER.dmbiz 673 jaeger i text=3D"[FVC]: RMS full fit 77.870 um= ." 04:37:14 =C2=A0boss exposure =C2= =A0300.0, file=3D335763 04:42:47 CmdStarte= d: OBSERVER.dmbiz 679 jaeger configuration reverse 04:43:10 CmdStarted: OBSERVER.dmbiz 680 jaeger fvc loop --no-apply= 04:43:33 OBSERVER.dm= biz 680 jaeger d fvc_= filename=3D/data/fcam/59577/fimg-fvc1n-0018.fits 04:43:36 OBSERVER.dmbiz 680 jaeger i text=3D&qu= ot;[FVC]: RMS full fit 11.780 um." (8= ) 04:43:52 CmdStarted= : OBSERVER.dmbiz 681 jaeger configuration dither 0.1 04:44:12 OBSERVER.dmbiz 681 jaeger : text= =3D"Dithered configuration 843 loaded and executed."= 04:43:53 OBSERVER.dmbiz 680 jaeger f error=3D"Command 'fvc loop -= -no-apply' failed= =C2=A0because of an = uncaught error 'JaegerError: Configuration needs to be set and loaded to the DB.'. See traceback in the log f= or more information." 04:45:14 CmdStarted: OBSERVER.dmbiz 683 jaeger fvc loop --no-apply 04:45:37 OBSERVER.dmbi= z 683 jaeger d fvc_fi= lename=3D/data/fcam/59577/fimg-fvc1n-0019.fits 04:45:40 OBSERVER.dmbiz 683 jaeger i text=3D"= ;[FVC]: RMS full fit 80.470 um." 04:4= 7:29 =C2=A0boss exposure =C2=A0300.0, file=3D335764 04:52:42 CmdStarted: OBSERVER.dmbiz 689 jaeger configuration reve= rse 04:53:02 CmdStarted: OBSERVER.dmbiz 69= 0 jaeger fvc loop --no-apply 04:53:24 OBSERVER.dmbiz 690 jaeger d fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0020.fit= s 04:53:27 OBSERVER.d= mbiz 690 jaeger i text=3D"[FVC]: RMS full fit 69.940 um." (9) 04:53:48 CmdStarted: OBSERVER.dmbiz 691 jaeger configuration dithe= r 0.1 04:54:08 OBSERV= ER.dmbiz 691 jaeger : text=3D"Dithered configuration 845 loaded and executed." 04:54:51 CmdStarted: OBSERVER.dmbiz 692 jaeger fv= c loop --no-apply 04:= 55:14 OBSERVER.dmbiz 692 jaeger d fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0021.fits 04:55:16 OBSERVER.dmbiz 692 ja= eger i text=3D"[FVC]: RMS full fit 79.780 um." 04:56:36 =C2=A0boss exposure =C2=A0300.0, file=3D335765 05:02:03 CmdStarted: OBSERVER.dmbiz 698 jaeger= configuration reverse 05:02:26 .hub 0 cmd= s d CmdIn=3D"OBSERVER.dmbiz","jaeger","fvc loop --no-apply" 05:02:49 OBSERVER.dmbiz 699= jaeger d fvc_filenam= e=3D/data/fcam/59577/fimg-fvc1n-0022.fits 05:02:52 OBSERVER.dmbiz 699 jaeger i text=3D"[FV= C]: RMS full fit 10.060 um." jaeger u= nwind 05:05:09 OBSERVER.dmbiz 700 jaeger f= error=3D"Trajectory failed with error: Something went wrong during the trajectory: The FPS= got locked during the trajectory. Trajectory failed 11.89 seconds after start" fps_snapshot_59577_0031.pdf **** Focus test - take BOSS exposures in an undithered configuratio= n at offset focus= -100, -50, 50, and 100 m= icrons of secondary motion. 35914 75.00000= 0 20.000000 fps_position_fields_10_075_20_BOSS.fits 433 Field cadence: bright_1x1 Calibration ta= rgets: sky_boss: 50/50 standard_boss: 16/50 sky_apogee: 0/35 standard_apogee: 0/15 Science targets: BOSS targets assigned: 433 APOGEE targ= ets assigned: 0 Targets per epoch: 433 Rob= ots used: BOSS-only: 201 APOGEE-BOSS: 298 Carton completion: manual_fps_position_stars_10: 433 o= f 2994 05:14:25 CmdStarted: OBSERVER.dmbiz= 713 jaeger configuration load 35914 05:15:36 OBSERVER.dmbiz 713 jaeger : text=3D"Configura= tion 846 loaded." 05:16:34 CmdDone: OBSERVER.dmbiz 714 jaeger configuration execute 05:18:11 CmdStarted: OBSERVER.dmbiz 715 hal goto-= field -s prepare,slew= ,boss_hartmann,boss_arcs,boss_flat,fvc,acquire 05:02:49 OBSERVER.dmbiz 699 jaeger d fvc_filename=3D/data/fcam/59577/fimg-fvc1n-0022.fits= ...= 05:23:56 OBSERVER.dmbiz.jaeger 433 ja= eger d fvc_filename= =3D/data/fcam/59577/fimg-fvc1n-0026.fits= 05:21:53 OBSERVER.dmbiz.jaeger 433 jaeger i text=3D"[FVC]: RMS full f= it 81.170 um." 05:22:36 OBSERVER.dmbi= z.jaeger 433 jaeger i text=3D"[FVC]: RMS full fit 23.670 um." 05:23:17 OBSERVER.dmbiz.jaeger 433 jaeger i t= ext=3D"[FVC]: RMS full fit 22.590 um." 05:23:59 OBSERVER.dmbiz.jaeger 433 jaeger i text=3D"[FVC]: RMS = full fit 22.350 um.&q= uot;= 05:19:58 =C2=A0boss exposure =C2=A0= =C2=A0=C2=A04.0, file=3D335766 05:20:38 =C2=A0boss exposure =C2=A0=C2=A0=C2=A04.0, file=3D335767= 05:21:28 =C2=A0sp1.m= otor.move=3D -1099, -1100, -1096 05:21:28 Hartmann collimate output on cart #19= =C2=A0=C2=A0=C2=A0sp1: offset: r=3D -= 659 (In Focus); =C2=A0b=3D 4.8 (Out of focus) =C2=A0=C2=A0=C2=A0sp1: pred. move: spAverageMove= =3D -1100 =C2=A0=C2= =A0=C2=A0sp1: pred. resid: r=3D 441, b=3D 1.4, txt=3D OK, spTemp =3D -0.1 05:24:49 =C2=A0boss ex= posure =C2=A0=C2=A0=C2=A04.0, file=3D335768 05:26:29 =C2=A0boss exposure =C2=A0=C2=A025.0, file= =3D335769 Focus loop. Best focus at 380. S= eeing 1.1". Still cloudy, but better than before. 05:39:43= =C2=A0boss exposure =C2=A0300.0, file=3D335770 OBSERVER.dmbiz 732 tcc set focus=3D280.0 =C2=A0(-100) 05:46:11 =C2=A0boss exposure =C2=A0300.0, file=3D335771 OBSERVER.dmbiz 732 tcc set focus=3D330.0 = =C2=A0(-50) 05:52:37 =C2=A0boss exposure = =C2=A0300.0, file=3D335772 OBSERVER.dmbiz= 732 tcc set focus=3D430 =C2=A0=C2=A0=C2=A0(50) 05:59:07 =C2=A0boss exposure =C2=A0300.0, file=3D335773 OBSERVER.dmbiz 732 tcc set focus=3D480 =C2=A0=C2=A0(1= 00) 06:05:33 =C2=A0boss exposure =C2=A0300= .0, file=3D335774= OBSERVER.dmbiz 732 tcc s= et focus=3D380 =C2=A0=C2=A0=C2=A0(Best focus) 06:12:00 =C2=A0boss exposure =C2=A0300.0, file=3D335775 ***= 3x3 telescope dither= test with an undithered BOSS configuration (configuration 35914), spaced by 1". We were unsure as to what to do with the guider duri= ng these exposures. If we left the guiding= alone, and offset the telescope, the guider will just bring the stars back to where they are supposed to be. Therefo= re, we decided to not= guide at all during = the offseted exposures. The other option would be to apply= the cherno offset command (which if we have time, perhaps= we will do later). IGNORE exposure 06:19:= 25 =C2=A0boss exposure =C2=A0300.0, file=3D335776 since we were figuring out the steps we should= take going forward. boss exposure =C2=A03= 00.0, file=3D335777 [= -1",-1"] boss exposure =C2=A0300= .0, file=3D335778 [-1= ",0"] boss exposure =C2=A0300.0,= file=3D335779 [-1&qu= ot;,1"] boss exposure =C2=A0300.0, fi= le=3D335780 [0",= 1"] boss exposure =C2=A0300.0, file= =3D335781 [1",1&= quot;] boss exposure =C2=A0300.0, file=3D3= 35782 [0",1"= ;] boss exposure =C2=A0300.0, file=3D33578= 3 [1",-1"] boss exposure =C2=A0300.0, file=3D335784 [0",-1"]= boss exposure =C2=A0300.0, file=3D335785 [0",0"] ** Now, activate the guider and guide at the same= offset as the telescope. boss exposure = =C2=A0300.0, file=3D335786 [-1",-1"] boss exposure = =C2=A0300.0, file=3D335787 [-1",0"] boss exposure = =C2=A0300.0, file=3D335788 [-1",1"] boss exposure = =C2=A0300.0, file=3D335789 [0",1"] boss exposure =C2= =A0300.0, file=3D335790 [1",1"] boss exposure =C2=A03= 00.0, file=3D335791 [= 0",1"] boss exposure =C2=A0300.0= , file=3D335792 [1&qu= ot;,-1"] boss exposure =C2=A0300.0, f= ile=3D335793 [0"= ,-1"] boss exposure =C2=A0300.0, file= =3D335794 [0",0&= quot;] However, it wasn't until after = we finished the test that we noticed that we only implemented the cherno offset and not the telescope offset as well= . Coolant flow gradually getting worse. Do= wn to 0.2L/min. Going to try to do the sam= e thing as before to get it to behave. 08:= 46:24 APO.aoravetz 277 jaeger : fbi_led=3D0.0,0.0,0.0,0.0;= flow=3D5.846,0.001; humidity=3D25.294,54.079; power_can=3D= T,T,T,T,T,T; power_gfa=3DF,T,T,T,T,T; power_nuc=3DT,T,T,T,T,T; power_sextant=3DT,T,T,T,T,T; powe= r_sync=3DF; pressure= =3D31.785,20.962; tem= perature=3D4.962,-3.583,8.35,9.22,9.12,1.82,-1.48,- 0.18,16.22,0.602,1.22,1.41,5.869; 08:37:02 CmdStarted: OBSERVER.dmbiz 791 jaeger chill= er set flow 10 08:45:28 CmdStarted: OBSERV= ER.dmbiz 792 jaeger chiller set flow 0.2= 08:47:04 CmdStarted: OBSERVER.dmbiz 794 jaeger chiller set flow 10 08:47:28 CmdStarted: OBSERVER.dmbiz 799 jaeger ie= b status 08:47:28 OBS= ERVER.dmbiz 799 jaeger > 08:47:28 OBSERVER.dmbiz 799 jaeger : fbi_led=3D0.0,0.0,0.0,0.0; flow=3D6.794,0.485; =C2=A0humidity=3D25.391,53= .981; power_can=3DT,T,T,T,T,T; power_gfa=3DF,T,T,T,T,T; power_nuc=3DT,T,T,T,T,T; power_sextant= =3DT,T,T,T,T,T; power_sync=3DF; pressure=3D37.071,22.769; temperature=3D4.962,-3.583,8.65,9.22,9.12,1.72,-1.48,- 0.18,16.22,0.702,0.92,1.1= 1,5.769; voltage=3D24= .112,24.323,24.183,23.979,24.144,24.065 Be= tter. 08:44Z - Waited for the clouds to mo= ve off to start another test. *** GRAFANA was hanging and was no longer responsive. Restarted the cerebro with systemcontol. I= t didn't help. We are still having coo= lant flow issues.= 11:06Z - We closed due t= o clouds, humidity, and the FPS flow issue. Started cals. ---- Mirror Numbers ---- PRIMARY: -= ------- Scale: 1.0000= 00 MIGS TONIGHT NOMINAL Axial A: 0.0020 0.0040 Axial B: 0.0000 0.0030 Axial C: -0.1110 -0.1150 Trans D: -0.0340 -0.0338 Lateral E: 0.3315 0.= 3315 Lateral F: 0.53= 50 0.5350 GALILS Commanded: =C2=A012500 -1600 5300 47050 -8800= -8400 Actual: =C2=A0= =C2=A0=C2=A0=C2=A012500 -1600 5300 47050 -8800 -8400 SETMIR VALUES PrimDesOrient: 0.0 -59.5 -9.15 -512.0 1906.3 PrimOrient: =C2=A0=C2=A0=C2=A01.07 -59.51 -9.21= -512.13 1906.31 SECONDARY: ---------- Focus: =C2=A0=C2=A0=C2=A0=C2=A0=C2=A00.0000 Air temp: =C2=A0=C2=A0=C2=A0-0.11 Altitude: =C2=A0=C2=A030.0= MIGS TONIGHT NOMINAL Axial A: -9.6560 -9.6570 Axial B: -9.6710 -9.6710 Axial C: -9.6950 -9.6970 Trans D: 0.1130 0.1140 GALILS Commanded: 4896650 4897450 4912650 -50 -3450 Actual: =C2=A0=C2=A0=C2=A04859186 48592= 30 4859262 -50 -3450 SETMIR VALUES SecDesOrient: 3857.0 0.0 0.0= 0.0 42.22 SecOrient:= =C2=A0=C2=A0=C2=A03857.01 -0.01 0.02 0.01 42.87 ---- Telescope Status ---- 23:55Z =C2=A0=C2=A0=C2=A0=C2=A0Started ventilation= 01:10Z =C2=A0=C2=A0= =C2=A0=C2=A0encl25m : open 11:17Z =C2=A0=C2=A0=C2=A0=C2=A0encl25m : closed No interlocks bypassed Status= at: Telescope stowed= at: =C2=A0=C2=A0=C2=A0121.0, 30.0, =C2=A0=C2=A00.0 =C2=A0mount Instrument mounted: =C2=A0=C2=A0= =C2=A0=C2=A0cartridge =3D =C2=A0=C2=A0=C2=A0=C2=A0fps Counterweights at: =C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0680 680 705 705 LN2 autofill systems: =C2=A0=C2=A0Connected and turned on 180L LN2 dewar scales: =C2=A0SP1 = =C2=A0=C2=A0175 =C2=A0=C2=A0=C2=A0=C2=A0lbs, =C2=A020 =C2=A0=C2=A0psi ---- Software Used ---- TUI =C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0: 1.7.13 alerts =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0: 2.2.0= apo =C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0: 3.0.1 boss =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0: 6.0.1-alpha.0 -boss.specMec : 3.0.4 guider =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0: 4.0.0 ecamera =C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0: FAILED fliswarm =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0: FAILED hartmann =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0: = 2.0.1-alpha.0 hub =C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0: 5.0.0a0 mcp =C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0: 2.1 -mcp.plc =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0: 34 -mcp.azFiduci : v1_148 -mcp.altFiduc : v1_148 -mcp.rotFiduc : v1_148= sop =C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0: FAILED tcc =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0: 1.4.0 apogee =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0: not availble= apogeecal =C2=A0=C2=A0= =C2=A0=C2=A0: not availble apogeeql =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0: v1_7_0dev STUIAdditions : 3.1.9= /dev= /sdb1 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A02.0T =C2=A0244G =C2=A01.8T =C2=A012% /home 10.25.1.230:/data =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A021T =C2=A0=C2=A017T =C2=A03.3T =C2=A084= % /data ---- APOGEE Data Summary ---- last mornin= g: 59576 =C2=A010 11:= 48 =C2=A040140010 =C2=A0=C2=A0=C2=A0Dark =C2=A0=C2=A060 =C2=A0B =C2=A013.5 = =C2=A0FPS- =C2=A0=C2=A00 =C2=A0a-b-c =C2=A0=C2=A0=C2=A0- =C2=A0=C2=A0- 59576 =C2=A011 11:58 =C2= =A040140011 =C2=A0=C2=A0=C2=A0Dark =C2=A0=C2=A060 =C2=A0B =C2=A013.5 =C2=A0= FPS- =C2=A0=C2=A00 =C2=A0a-b-c =C2=A0=C2=A0=C2=A0- =C2=A0=C2=A0- 59576 =C2=A012 12:09 =C2=A04014= 0012 =C2=A0=C2=A0=C2=A0Dark =C2=A0=C2=A060 =C2=A0B =C2=A013.5 =C2=A0FPS- = =C2=A0=C2=A00 =C2=A0a-b-c =C2=A0=C2=A0=C2=A0- =C2=A0=C2=A0- 59576 =C2=A013 12:11 =C2=A040140013 = =C2=A0QuarFlat 10 =C2=A0B =C2=A013.5 =C2=A0FPS- =C2=A0=C2=A00 =C2=A0a-b-c = =C2=A0=C2=A0=C2=A0- =C2=A0100 59576 =C2=A014 12:13 =C2=A040140014 =C2=A0QuarFlat 10 =C2=A0B =C2= =A013.5 =C2=A0FPS- =C2=A0=C2=A00 =C2=A0a-b-c =C2=A0=C2=A0=C2=A0- =C2=A0100 59576 =C2=A015 12:15 = =C2=A040140015 =C2=A0QuarFlat 10 =C2=A0B =C2=A013.5 =C2=A0FPS- =C2=A0=C2=A0= 0 =C2=A0a-b-c =C2=A0=C2=A0=C2=A0- =C2=A0100 59576 =C2=A016 12:17 =C2=A040140016 =C2=A0Arc-Thar 1= 2 =C2=A0A =C2=A013.0 =C2=A0FPS- =C2=A0=C2=A00 =C2=A0a-b-c -0.08 =C2=A0-= 59576 =C2=A017 12:24 =C2= =A040140017 =C2=A0Arc-Une =C2=A040 =C2=A0A =C2=A013.0 =C2=A0FPS- =C2=A0=C2= =A00 =C2=A0a-b-c =C2=A0=C2=A0=C2=A0- =C2=A0=C2=A0- 59576 =C2=A018 12:26 =C2=A040140018 =C2=A0Arc= -Thar 12 =C2=A0B =C2=A013.5 =C2=A0FPS- =C2=A0=C2=A00 =C2=A0a-b-c =C2=A00.51= =C2=A0- 59576 =C2=A0= 19 12:34 =C2=A040140019 =C2=A0Arc-Une =C2=A040 =C2=A0B =C2=A013.5 =C2=A0FPS= - =C2=A0=C2=A00 =C2=A0a-b-c =C2=A0=C2=A0=C2=A0- =C2=A0=C2=A0- 59576 =C2=A020 12:39 =C2=A04014002= 0 =C2=A0=C2=A0=C2=A0Dark =C2=A0=C2=A030 =C2=A0A =C2=A013.0 =C2=A0FPS- =C2= =A0=C2=A00 =C2=A0a-b-c =C2=A0=C2=A0=C2=A0- =C2=A0=C2=A0- 59576 =C2=A021 12:45 =C2=A040140021 =C2= =A0InteFlat 30 =C2=A0A =C2=A013.0 =C2=A0FPS- =C2=A0=C2=A00 =C2=A0a-b-c =C2= =A0=C2=A0=C2=A0- =C2=A0279 59576 =C2=A022 12:50 =C2=A040140022 =C2=A0InteFlat 30 =C2=A0A =C2=A0= 13.0 =C2=A0FPS- =C2=A0=C2=A00 =C2=A0a-b-c =C2=A0=C2=A0=C2=A0- =C2=A0280= 59576 =C2=A023 12:55 =C2= =A040140023 =C2=A0InteFlat 30 =C2=A0A =C2=A013.0 =C2=A0FPS- =C2=A0=C2=A00 = =C2=A0a-b-c =C2=A0=C2=A0=C2=A0- =C2=A0279 59576 =C2=A024 13:01 =C2=A040140024 =C2=A0=C2=A0=C2= =A0Dark =C2=A0=C2=A030 =C2=A0A =C2=A013.0 =C2=A0FPS- =C2=A0=C2=A00 =C2=A0a-= b-c =C2=A0=C2=A0=C2=A0- =C2=A0=C2=A0- this evening: --------------------------------= ---------------------------------------------- =C2=A0mjd =C2=A0=C2=A0i =C2=A0=C2=A0UT =C2=A0=C2= =A0File/Exp =C2=A0Imtype Nread A/B Dth =C2=A0Ct-Plate Arch Offset %Flux= Comm =C2=A0----------------------------------= -------------------------------------------- 59577 =C2=A0=C2=A01 22:47 =C2=A040150001 =C2=A0Arc-= --- =C2=A0=C2=A03 =C2=A0A =C2=A013.0 =C2=A0FPS- =C2=A0=C2=A00 =C2=A0a-b-c = =C2=A0=C2=A0=C2=A0- =C2=A0=C2=A0- =C2=A0test 59577 =C2=A0=C2=A02 22:59 =C2=A040150002 =C2=A0=C2=A0=C2=A0Dark =C2=A0= =C2=A060 =C2=A0A =C2=A013.0 =C2=A0-------- =C2=A0a-b-c =C2=A0=C2=A0=C2=A0- = =C2=A0=C2=A0- 59577 = =C2=A0=C2=A03 23:01 =C2=A040150003 =C2=A0QuarFlat 10 =C2=A0A =C2=A013.0 =C2= =A0FPS- =C2=A0=C2=A00 =C2=A0a-b-c =C2=A0=C2=A0=C2=A0- =C2=A0100 59577 =C2=A0=C2=A04 23:03 =C2=A0= 40150004 =C2=A0Arc-Thar 12 =C2=A0B =C2=A013.5 =C2=A0FPS- =C2=A0=C2=A00 =C2= =A0a-b-c =C2=A00.51 =C2=A0- 59577 =C2=A0=C2=A05 23:10 =C2=A040150005 =C2=A0Arc-Une =C2=A040 =C2= =A0B =C2=A013.5 =C2=A0FPS- =C2=A0=C2=A00 =C2=A0a-b-c =C2=A0=C2=A0=C2=A0- = =C2=A0=C2=A0- 59577 = =C2=A0=C2=A06 23:13 =C2=A040150006 =C2=A0Arc-Thar 12 =C2=A0A =C2=A013.0 =C2= =A0FPS- =C2=A0=C2=A00 =C2=A0a-b-c -0.11 =C2=A0- 59577 =C2=A0=C2=A07 23:20 =C2=A040150007 =C2=A0A= rc-Une =C2=A040 =C2=A0A =C2=A013.0 =C2=A0FPS- =C2=A0=C2=A00 =C2=A0a-b-c =C2= =A0=C2=A0=C2=A0- =C2=A0=C2=A0- 59577 =C2=A0=C2=A08 23:31 =C2=A040150008 =C2=A0=C2=A0=C2=A0Dark = =C2=A0=C2=A060 =C2=A0A =C2=A013.0 =C2=A0-------- =C2=A0a-b-c =C2=A0=C2=A0= =C2=A0- =C2=A0=C2=A0-= 59577 =C2=A0=C2=A09 23:38 =C2=A040150009 =C2=A0Arc---- =C2=A030 =C2=A0A = =C2=A013.0 =C2=A0FPS- =C2=A0=C2=A00 =C2=A0x-x-x =C2=A0=C2=A0=C2=A0- =C2=A0= =C2=A0- =C2=A0FPI= ------------------------= ------------------------------------------------------ APOGEE data list, =C2=A0=C2=A0mjd=3D59577, =C2=A0all raw_data: =C2=A0/data/apogee/u= tr_cdr/59577/apRaw-*.fits archive: =C2=A0/data/apogee/archive/59577/apR-[a,b,c]-*.apz quickred: =C2=A0/data/apogee/= quickred/59577/apq-*.fits --------------------------------------------------------------------= ---------- =C2=A0mjd = =C2=A0=C2=A0i =C2=A0=C2=A0UT =C2=A0=C2=A0File/Exp =C2=A0Imtype Nread A/B Dt= h =C2=A0Ct-Plate Arch Offset %Flux Comm = =C2=A0---------------------------------------------------------------------= --------- 59577 =C2= =A0=C2=A01 22:47 =C2=A040150001 =C2=A0Arc---- =C2=A0=C2=A03 =C2=A0A =C2=A01= 3.0 =C2=A0FPS- =C2=A0=C2=A00 =C2=A0a-b-c =C2=A0=C2=A0=C2=A0- =C2=A0=C2=A0- =C2=A0test 59577 =C2=A0=C2=A02 22:59 =C2= =A040150002 =C2=A0=C2=A0=C2=A0Dark =C2=A0=C2=A060 =C2=A0A =C2=A013.0 =C2=A0= -------- =C2=A0a-b-c =C2=A0=C2=A0=C2=A0- =C2=A0=C2=A0- 59577 =C2=A0=C2=A03 23:01 =C2=A040150003 = =C2=A0QuarFlat 10 =C2=A0A =C2=A013.0 =C2=A0FPS- =C2=A0=C2=A00 =C2=A0a-b-c = =C2=A0=C2=A0=C2=A0- =C2=A0100 59577 =C2=A0=C2=A04 23:03 =C2=A040150004 =C2=A0Arc-Thar 12 =C2=A0B= =C2=A013.5 =C2=A0FPS- =C2=A0=C2=A00 =C2=A0a-b-c =C2=A00.51 =C2=A0- 59577 =C2=A0=C2=A05 23:10 = =C2=A040150005 =C2=A0Arc-Une =C2=A040 =C2=A0B =C2=A013.5 =C2=A0FPS- =C2=A0= =C2=A00 =C2=A0a-b-c =C2=A0=C2=A0=C2=A0- =C2=A0=C2=A0- 59577 =C2=A0=C2=A06 23:13 =C2=A040150006 = =C2=A0Arc-Thar 12 =C2=A0A =C2=A013.0 =C2=A0FPS- =C2=A0=C2=A00 =C2=A0a-b-c -= 0.11 =C2=A0- 59577 = =C2=A0=C2=A07 23:20 =C2=A040150007 =C2=A0Arc-Une =C2=A040 =C2=A0A =C2=A013.= 0 =C2=A0FPS- =C2=A0=C2=A00 =C2=A0a-b-c =C2=A0=C2=A0=C2=A0- =C2=A0=C2=A0- 59577 =C2=A0=C2=A08 23:= 31 =C2=A040150008 =C2=A0=C2=A0=C2=A0Dark =C2=A0=C2=A060 =C2=A0A =C2=A013.0 = =C2=A0-------- =C2=A0a-b-c =C2=A0=C2=A0=C2=A0- =C2=A0=C2=A0- 59577 =C2=A0=C2=A09 23:38 =C2=A0401= 50009 =C2=A0Arc---- =C2=A030 =C2=A0A =C2=A013.0 =C2=A0FPS- =C2=A0=C2=A00 = =C2=A0a-b-c =C2=A0=C2=A0=C2=A0- =C2=A0=C2=A0- =C2=A0FPI -------------------------------------------------------------= -----------------= ---- eBOSS Data Summary ---- Read= ing FITS from /data/spectro//59577 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -= - - - - - - - - =C2=A0=C2=A0n =C2=A0Time = =C2=A0Filename =C2=A0=C2=A0crt =C2=A0plt =C2=A0flav =C2=A0shu =C2=A0=C2=A0E= xp =C2=A0=C2=A0=C2=A0=C2=A0colA =C2=A0=C2=A0colB =C2=A0=C2=A0colC Hrt - - - - - - - - - - - - - = - - - - - - - - - - - - - - - - - - - - - - - - - - =C2=A0=C2=A01 22:47 b1-00335746 =C2=A0FPS-N =C2=A0None =C2=A0bia = =C2=A0Clo =C2=A0=C2=A0=C2=A0=C2=A00.0 =C2=A0-4103 =C2=A0-4096 =C2=A0-4125 =C2=A0=C2=A02 22:47 r1= -00335746 =C2=A0FPS-N =C2=A0None =C2=A0bia =C2=A0Clo =C2=A0=C2=A0=C2=A0=C2= =A00.0 =C2=A0-4103 =C2=A0-4096 =C2=A0-4125 =C2=A0=C2=A03 22:51 b1-00335747 =C2=A0FPS-N =C2=A0Non= e =C2=A0dar =C2=A0Clo =C2=A0=C2=A0=C2=A020.0 =C2=A0-4103 =C2=A0-4096 =C2=A0= -4125 =C2=A0=C2=A04 2= 2:51 r1-00335747 =C2=A0FPS-N =C2=A0None =C2=A0dar =C2=A0Clo =C2=A0=C2=A0=C2= =A020.0 =C2=A0-4103 =C2=A0-4096 =C2=A0-4125 =C2=A0=C2=A05 22:54 b1-00335748 =C2=A0FPS-N =C2=A0No= ne =C2=A0arc =C2=A0Lef =C2=A0=C2=A0=C2=A0=C2=A04.1 =C2=A0-4103 =C2=A0-4096 = =C2=A0-4125 HART =C2=A0=C2=A06 22:54 r1-00= 335748 =C2=A0FPS-N =C2=A0None =C2=A0arc =C2=A0Lef =C2=A0=C2=A0=C2=A0=C2=A04= .1 =C2=A0-4103 =C2=A0-4096 =C2=A0-4125 HART =C2=A0=C2=A07 22:55 b1-00335749 =C2=A0FPS-N =C2=A0None =C2=A0arc =C2=A0Ri= g =C2=A0=C2=A0=C2=A0=C2=A04.1 =C2=A0-4103 =C2=A0-4096 =C2=A0-4125 HART =C2=A0=C2=A08 22:55 r1-00335749 =C2=A0FPS-N = =C2=A0None =C2=A0arc =C2=A0Rig =C2=A0=C2=A0=C2=A0=C2=A04.1 =C2=A0-4103 =C2= =A0-4096 =C2=A0-4125 = HART =C2=A0=C2=A09 22= :58 b1-00335750 =C2=A0FPS-N =C2=A0None =C2=A0arc =C2=A0Out =C2=A0=C2=A0=C2= =A0=C2=A04.1 =C2=A0-6027 =C2=A0-6021 =C2=A0-6051 =C2=A010 22:58 r1-00335750 =C2=A0FPS-N =C2=A0No= ne =C2=A0arc =C2=A0Out =C2=A0=C2=A0=C2=A0=C2=A04.1 =C2=A0-6027 =C2=A0-6021 = =C2=A0-6051 =C2=A011 = 22:59 b1-00335751 =C2=A0FPS-N =C2=A0None =C2=A0fla =C2=A0Out =C2=A0=C2=A0= =C2=A025.1 =C2=A0-6027 =C2=A0-6021 =C2=A0-6051 =C2=A012 22:59 r1-00335751 =C2=A0FPS-N =C2=A0None= =C2=A0fla =C2=A0Out =C2=A0=C2=A0=C2=A025.1 =C2=A0-6027 =C2=A0-6021 =C2=A0-= 6051 =C2=A013 02:18 r= 1-00335752 =C2=A0FPS-N =C2=A0None =C2=A0arc =C2=A0Lef =C2=A0=C2=A0=C2=A0=C2= =A04.1 =C2=A0-6027 =C2=A0-6021 =C2=A0-6051 HART =C2=A014 02:18 b1-00335752 =C2=A0FPS-N =C2=A0None =C2=A0arc =C2=A0Le= f =C2=A0=C2=A0=C2=A0=C2=A04.1 =C2=A0-6027 =C2=A0-6021 =C2=A0-6051 HART =C2=A015 02:18 r1-00335753 =C2=A0FPS-N =C2=A0N= one =C2=A0arc =C2=A0Rig =C2=A0=C2=A0=C2=A0=C2=A04.1 =C2=A0-6027 =C2=A0-6021= =C2=A0-6051 HART= =C2=A016 02:18 b1-003357= 53 =C2=A0FPS-N =C2=A0None =C2=A0arc =C2=A0Rig =C2=A0=C2=A0=C2=A0=C2=A04.1 = =C2=A0-6027 =C2=A0-6021 =C2=A0-6051 HART = =C2=A017 02:22 b1-00335754 =C2=A0FPS-N =C2=A0None =C2=A0arc =C2=A0Out =C2= =A0=C2=A0=C2=A0=C2=A04.1 =C2=A0-6480 =C2=A0-6473 =C2=A0-6507 =C2=A018 02:22 r1-00335754 =C2=A0FP= S-N =C2=A0None =C2=A0arc =C2=A0Out =C2=A0=C2=A0=C2=A0=C2=A04.1 =C2=A0-6480 = =C2=A0-6473 =C2=A0-6507 =C2=A019 02:24 b1-00335755 =C2=A0FPS-N =C2=A0None =C2=A0fla =C2=A0Out = =C2=A0=C2=A0=C2=A025.1 =C2=A0-6480 =C2=A0-6473 =C2=A0-6507= =C2=A020 02:24 r1-00335755 =C2=A0FPS-= N =C2=A0None =C2=A0fla =C2=A0Out =C2=A0=C2=A0=C2=A025.1 =C2=A0-6480 =C2=A0-= 6473 =C2=A0-6507 =C2= =A021 03:00 b1-00335756 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0= =C2=A0300.1 =C2=A0-6480 =C2=A0-6473 =C2=A0-6507 =C2=A022 03:00 r1-00335756 =C2=A0FPS-N =C2=A0Non= e =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-6480 =C2=A0-6473 =C2=A0-6507= =C2=A023 03:19 b1-00= 335757 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-= 6480 =C2=A0-6473 =C2=A0-6507 =C2=A024 03:19 r1-00335757 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0O= ut =C2=A0=C2=A0300.1 =C2=A0-6480 =C2=A0-6473 =C2=A0-6507 =C2=A025 03:39 b1-00335758 =C2=A0FPS-N = =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-6480 =C2=A0-6473 = =C2=A0-6507 =C2=A026 = 03:39 r1-00335758 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A030= 0.1 =C2=A0-6480 =C2=A0-6473 =C2=A0-6507 =C2=A027 03:49 b1-00335759 =C2=A0FPS-N =C2=A0None =C2=A0= sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-6480 =C2=A0-6473 =C2=A0-6507 =C2=A028 03:49 r1-00335759 = =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-6480 = =C2=A0-6473 =C2=A0-6507 =C2=A029 04:03 b1-00335760 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out = =C2=A0=C2=A0300.1 =C2=A0-6480 =C2=A0-6473 =C2=A0-6507 =C2=A030 04:03 r1-00335760 =C2=A0FPS-N =C2= =A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-6480 =C2=A0-6473 =C2= =A0-6507 =C2=A031 04:= 15 b1-00335761 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1= =C2=A0-6480 =C2=A0-6473 =C2=A0-6507 =C2=A032 04:15 r1-00335761 =C2=A0FPS-N =C2=A0None =C2=A0sci= =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-6480 =C2=A0-6473 =C2=A0-6507 =C2=A033 04:28 b1-00335762 =C2= =A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-6480 =C2= =A0-6473 =C2=A0-6507 = =C2=A034 04:28 r1-00335762 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2= =A0=C2=A0300.1 =C2=A0-6480 =C2=A0-6473 =C2=A0-6507 =C2=A035 04:37 b1-00335763 =C2=A0FPS-N =C2=A0= None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-6480 =C2=A0-6473 =C2=A0-6= 507 =C2=A036 04:37 r1= -00335763 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2= =A0-6480 =C2=A0-6473 =C2=A0-6507 =C2=A037 04:47 b1-00335764 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2= =A0Out =C2=A0=C2=A0300.1 =C2=A0-6480 =C2=A0-6473 =C2=A0-6507 =C2=A038 04:47 r1-00335764 =C2=A0FP= S-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-6480 =C2=A0-647= 3 =C2=A0-6507 =C2=A03= 9 04:56 b1-00335765 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0= 300.1 =C2=A0-6480 =C2=A0-6473 =C2=A0-6507 =C2=A040 04:56 r1-00335765 =C2=A0FPS-N =C2=A0None =C2= =A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-6480 =C2=A0-6473 =C2=A0-6507 =C2=A041 05:19 r1-0033576= 6 =C2=A0FPS-N =C2=A0None =C2=A0arc =C2=A0Lef =C2=A0=C2=A0=C2=A0=C2=A04.1 = =C2=A0-6480 =C2=A0-6473 =C2=A0-6507 HART = =C2=A042 05:19 b1-00335766 =C2=A0FPS-N =C2=A0None =C2=A0arc =C2=A0Lef =C2= =A0=C2=A0=C2=A0=C2=A04.1 =C2=A0-6480 =C2=A0-6473 =C2=A0-6507 HART =C2=A043 05:20 r1-00335767 =C2=A0FPS-N =C2=A0None = =C2=A0arc =C2=A0Rig =C2=A0=C2=A0=C2=A0=C2=A04.1 =C2=A0-6480 =C2=A0-6473 =C2= =A0-6507 HART =C2=A044 05:20 b1-00335767 = =C2=A0FPS-N =C2=A0None =C2=A0arc =C2=A0Rig =C2=A0=C2=A0=C2=A0=C2=A04.1 =C2= =A0-6480 =C2=A0-6473 =C2=A0-6507 HART =C2= =A045 05:24 r1-00335768 =C2=A0FPS-N =C2=A0None =C2=A0arc =C2=A0Out =C2=A0= =C2=A0=C2=A0=C2=A04.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A046 05:24 b1-00335768 =C2=A0FPS-N= =C2=A0None =C2=A0arc =C2=A0Out =C2=A0=C2=A0=C2=A0=C2=A04.1 =C2=A0-7579 =C2= =A0-7573 =C2=A0-7603 = =C2=A047 05:26 b1-00335769 =C2=A0FPS-N =C2=A0None =C2=A0fla =C2=A0Out =C2= =A0=C2=A0=C2=A025.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A048 05:26 r1-00335769 =C2=A0FPS-N = =C2=A0None =C2=A0fla =C2=A0Out =C2=A0=C2=A0=C2=A025.1 =C2=A0-7579 =C2=A0-75= 73 =C2=A0-7603 =C2=A0= 49 05:39 r1-00335770 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2= =A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A050 05:39 b1-00335770 =C2=A0FPS-N =C2=A0None = =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A051 05:46 r1-0033= 5771 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-75= 79 =C2=A0-7573 =C2=A0-7603 =C2=A052 05:46 b1-00335771 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Ou= t =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A053 05:52 r1-00335772 =C2=A0FPS-N = =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 = =C2=A0-7603 =C2=A054 = 05:52 b1-00335772 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A030= 0.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A055 05:59 b1-00335773 =C2=A0FPS-N =C2=A0None =C2=A0= sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A056 05:59 r1-00335773 = =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 = =C2=A0-7573 =C2=A0-7603 =C2=A057 06:05 r1-00335774 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out = =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A058 06:05 b1-00335774 =C2=A0FPS-N =C2= =A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2= =A0-7603 =C2=A059 06:= 12 b1-00335775 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1= =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A060 06:12 r1-00335775 =C2=A0FPS-N =C2=A0None =C2=A0sci= =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A061 06:19 r1-00335776 =C2= =A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2= =A0-7573 =C2=A0-7603 = =C2=A062 06:19 b1-00335776 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2= =A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A063 06:26 r1-00335777 =C2=A0FPS-N =C2=A0= None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7= 603 =C2=A064 06:26 b1= -00335777 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2= =A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A065 06:32 b1-00335778 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2= =A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A066 06:32 r1-00335778 =C2=A0FP= S-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-757= 3 =C2=A0-7603 =C2=A06= 7 06:44 b1-00335779 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0= 300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A068 06:44 r1-00335779 =C2=A0FPS-N =C2=A0None =C2= =A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A069 06:50 b1-0033578= 0 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 = =C2=A0-7573 =C2=A0-7603 =C2=A070 06:50 r1-00335780 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out = =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A071 06:57 b1-00335781 =C2=A0FPS-N =C2= =A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2= =A0-7603 =C2=A072 06:= 57 r1-00335781 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1= =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A073 07:03 r1-00335782 =C2=A0FPS-N =C2=A0None =C2=A0sci= =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A074 07:03 b1-00335782 =C2= =A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2= =A0-7573 =C2=A0-7603 = =C2=A075 07:10 r1-00335783 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2= =A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A076 07:10 b1-00335783 =C2=A0FPS-N =C2=A0= None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7= 603 =C2=A077 07:16 r1= -00335784 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2= =A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A078 07:16 b1-00335784 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2= =A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A079 07:23 r1-00335785 =C2=A0FP= S-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-757= 3 =C2=A0-7603 =C2=A08= 0 07:23 b1-00335785 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0= 300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A081 07:38 r1-00335786 =C2=A0FPS-N =C2=A0None =C2= =A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A082 07:38 b1-0033578= 6 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 = =C2=A0-7573 =C2=A0-7603 =C2=A083 07:45 r1-00335787 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out = =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A084 07:45 b1-00335787 =C2=A0FPS-N =C2= =A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2= =A0-7603 =C2=A085 07:= 51 r1-00335788 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1= =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A086 07:51 b1-00335788 =C2=A0FPS-N =C2=A0None =C2=A0sci= =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A087 07:58 r1-00335789 =C2= =A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2= =A0-7573 =C2=A0-7603 = =C2=A088 07:58 b1-00335789 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2= =A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A089 08:04 b1-00335790 =C2=A0FPS-N =C2=A0= None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7= 603 =C2=A090 08:04 r1= -00335790 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2= =A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A091 08:13 b1-00335791 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2= =A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A092 08:13 r1-00335791 =C2=A0FP= S-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-757= 3 =C2=A0-7603 =C2=A09= 3 08:20 b1-00335792 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0= 300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A094 08:20 r1-00335792 =C2=A0FPS-N =C2=A0None =C2= =A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A095 08:27 b1-0033579= 3 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 = =C2=A0-7573 =C2=A0-7603 =C2=A096 08:27 r1-00335793 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out = =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 =C2=A097 08:33 b1-00335794 =C2=A0FPS-N =C2= =A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1 =C2=A0-7579 =C2=A0-7573 =C2= =A0-7603 =C2=A098 08:= 33 r1-00335794 =C2=A0FPS-N =C2=A0None =C2=A0sci =C2=A0Out =C2=A0=C2=A0300.1= =C2=A0-7579 =C2=A0-7573 =C2=A0-7603 __________________________= __________________________________ Logs of Nightly APO Activities https://trac.sdss.org/tracmailman/browse= r/private/sloan-25mlog/index.html__________________________________= __________________________Logs of Nightly APO Activities = https://trac.sdss.org/tracmailman/browser/private/sloan-25mlog/index.html =20 --0000000000008a34a805d48d5788--