Subject: Re: [sloan-25mlog 4700] 2.5m obslog 20191011 (SJD 58768)
From: Kaike Pan
Date: Sat, 12 Oct 2019 06:58:27 -0600
In previous two tonights, we observed 14 plates, only experienced one =
unexpected speedup slew.
Tonight, we had three runaway slews on six plates. Looks like that the =
runaway rate increases
significantly, should we revert MCP  back to the previous version?=20
We didn=E2=80=99t experience runaway issue when we didn=E2=80=99t use =
gotoGangChange (but with a different version
of MCP). On the other hand, no recent run away slew happened on =
gotoGangChange. It=E2=80=99s a puzzle.
Should we use our work around, not use sop commands (goto gangChange, =
goto Instruction Change,
etc.) to move the telescope except for gotoField and see what happens?=20=
It seems that the problem on SP2 hartmann door is on the right one. I =
could command left door
to in/out, but not the right door (time out).
The instruction on how to start tcc with the test version was in the =
restartActor page. I think
that we should tagged the test version, and declare it current now.
I found that axes are usually disconnected after restarting MCP =
processes, one needs to
issue =E2=80=9Ctcc axis connect=E2=80=9D, don=E2=80=99t have to restart =
tcc.=20
> On Oct 12, 2019, at 6:05 AM, dmbiz@apo.nmsu.edu wrote:
>=20
>=20
>                       APACHE POINT OBSERVATORY SDSS 2.5M
>=20
>                                 OBSERVING LOG
>                          Friday October 11, 2019
>=20
>                                  (SJD 58768)
>=20
> ----  Observing Team ----
>=20
> Elena Malanushenko
> Dmitry Bizyaev
>=20
> Jeff Hagen (in the evening, over the phone)
> Conor Sayres (in the evening, over the phone)
> John Downey (over the phone, during the first exposure)
> Kaike Pan (in STUI)
>=20
> ---- Observing Summary ----
>=20
> Clear night, APOGEE got data, MaNGA/MaStar did not get any data
> because of the Right Hartmann door problem.=20
> Since the night is and Engineering night, and MaStar is=20
> piggybacking, we don't count it as lost time.=20
>=20
> APOGEE lost 0.7 h due to weather (dew point diff dropped)
> APOGEE exposures
> ----------------
>=20
> 2 12165 CVZTILE_089+27_btx  A: 5xAB      DONE
>  Missing fibers: [38, 57, 198, 211, 259, 273]
>  Faint fibers:   [43, 121, 123, 127, 136, 148]
>=20
> 5 11238 K2_C3_051-46 A: 2xAB(94)         DONE
>  Missing fibers: [211, 273]
>  Faint fibers:   [123, '126 - 127', 147, 154]
>=20
> 9 11619 ANDR3        A: 2xAB(94)   not finished
>  Missing fibers: [36, 211, 245, 273]
>  Faint fibers:   [123, 127, 205]
>=20
> 3 11214 138-26_btx   A: 4xAB       not finished
>  Missing fibers: [211, 273]
>  Faint fibers:   [123, 134]
>=20
> 6 12131 K2_C4_168-21_btx  A:  A: 4xAB     DONE
>  Missing fibers: [181, 211, 273]
>  Faint fibers:   [15, 34, 105, 123, '152 - 163', '166 - 173', '175 - =
180']
>=20
> 1 12130 K2_C4_167-30_btx A: 4xAB         not finished  =20
>  Missing fibers: [211, 273]
>  Faint fibers:   ['1 - 30', 123, 148, 227]
>=20
>=20
>=20
> Notes
> -----
> 1. Despite the smooth checkout with hartmanns etc, SP2 got stuck, =
mechanically,
> during the first hartmann exposure (may be skightly before that). As a =
result, the
> cameras' shutter was shown as stuck in the Open position, and Hartmann =
as Left=20
> in.=20
>=20
> We were able to close the shutter, but the right hartmann door on SP2 =
doesn't=20
> move. Notified John D. He will look at it tomorrow.=20
> During the night we bypassed all MaStar exposures.=20
>=20
>=20
> 2. Runaway slews tonight:
> a) 00:42Z; was experienced when we moved telescope from Zenith to Stow=20=
>  after the day cartridge change, via STUI script ("stow" there), which =
calls =20
>  corresponding SOP command.=20
>  Caught the fast motion with the stop button. See log.=20
> b) 03:28Z, when telescope was sent from Zenith to Gang Change.The =
runaway
>  slew started immediately.=20
>  Caught telescope on the fly, and also got huge fiducial errors.=20
>  Corrected a few degrees (~5 deg) ALT position mismatch successfully.=20=
>  During the runaway Elena noticed message "Cannot compute" for the=20
>  ALT axis status in the STUI Status display.
> c) 09:03 - again, from Zenith to the gang change position.=20
>  Interesting that this time we first cleared axes, and then moved=20
>  telescope with "tcc track 121,45 mount" command in command line of =
STUI.
>  Interesting to notice that when telescope started moving, the log =
reported
>  that "tcc axix init" command is finished - in two seconds after the
>  start of the slew. May be the interference of commands track and init=20=
>  may cause the slew glitches?=20
>=20
> 3. Connected to #2, we've got huge ALT fiducial errors after the =
runaway=20
>  slews and spent time on their correction. First time it was in =
twilight=20
>  (no time lost). Next time (3:28Z) we lost 10 min on correction of =
large
>  fiducials. Next time (09:03) we spent some 5 min on fixing the ALT =
fiducials.=20
> 4. Elena was unable to move telescope during the afternoon checkout.
>   It complained about not connected axes (see log, afternoon's part).=20=
>  Called Jeff and he confirmed that the mcp is OK, and recommended=20
>  to restart tcc. We tried, but got errors. Called Conor.
>  Conor said that we have to run the test version of the=20
>  tcc, not a regular. After we switched to the test version,
>  it worked fine.=20
>  We updated Obs Tips with instruction on how to start TCC
>  in the test node. White board in the control room is updated
>  with a notice that we run the test mode.
>=20
> 5. Altitude Fiducials more often that usual show huge corrections.=20
>  For the same checkmarks it may show adjacent good and bad reading.=20
>  See Problem in Detail section for an excerpt from the fiducial
>  log window. Note that the repprted errors are not real and we=20
>  continued to see stars in fibers with large errors reported.
>=20
>=20
> 6. Dust accumulated: ~ 5.5K
>   Weather page dust counters are all off.=20
>   We measured the dust with the hand-help Met One counter.=20
> 7. Eastern enclosure door's motor started sounding loud , the door =
moved=20
>  down jerking, and then stopped half way down to closed position. We =
were able  =20
>  to close the door manually.=20
>=20
>=20