From: Kaike Subject: Re: [sloan-25mlog 7556] 2.5m obslog 20241214 (SJD 60659) Date: Sun, 15 Dec 2024 14:57:02 -0700 --Apple-Mail=_57D26CD5-F839-4DD7-9536-38AE1E4147D1 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Dec 15, 2024, at 2:34=E2=80=AFPM, John Downey = wrote: >=20 >=20 >=20 > On 12/15/24 13:22, Kaike wrote: >>=20 >>=20 >>> On Dec 15, 2024, at 8:57=E2=80=AFAM, John Downey = wrote: >>>=20 >>>>=20 >>>> APACHE POINT OBSERVATORY SDSS 2.5M >>>>=20 >>>> OBSERVING LOG >>>> Saturday December 14, 2024 >>>>=20 >>>> (SJD 60659) >>>>=20 >>>> ---- Observing Team ---- >>>>=20 >>>> Kaike Pan >>>>=20 >>>>=20 >>>>=20 >>>> Notes: >>>>=20 >>>> 2. After incidence of "wind_alt_ready" problem when move the = telescope from 5 to 30 deg after open the enclosure. >>>=20 >>> I'm skeptical that this is a true wind_alt_ready problem. >>>=20 >>> The primary reason for my skepticism is that in historical instances = of this problem it was not easy to clear. Our best understanding of the = problem is that it is related to the west rail stop button or it's = accompanying opto-isolator board. For the present time, the button has = been defeated at the interlock level because it is not a critical stop = button and the problem was causing lost observing time. >>>=20 >>> A secondary reason for my skepticism is that both instances of the = problem you've been having seem to be related to moving away from 5 = degrees via a button script, which have been problematic before. If the = script makes an impossible command to the MCP, the axis goes out of = closed loop. >>=20 >>>=20 >>> Thirdly, wind_alt_ready will be red when the ALT axis is out of = closed loop, that is normal. When all the other ALT axis drive = interlocks are green and wind_alt_ready remains red, that is the = wind_alt_ready problem. It is typically not clearable by cycling a stop = button, but if I remember correctly an imitation of the problem can be = produced with a rapid button cycle and cleared through a normal stop = button cycle. Perhaps that is a hint to what is going on if what you = are seeing is all the ALT drive interlocks green except for = wind_alt_ready. >>=20 >> In both case, I intended to move the telescope via =E2=80=9Cstow=E2=80=9D= button in the "telescope control script=E2=80=9D, which is a =E2=80=9Ctcc= track=E2=80=9D command. In the 1st case, the telescope moved from 5 to = 19.5 or so, then altitude went =E2=80=9Cout-closed-loop=E2=80=9D. At = that point, all axis could be initialized without any issue, but as soon = as a motion command via tcc track command or via a =E2=80=9Cmove" = command in MCP altitude axes is entered x altitude axes went = out-closed-loop before an actual motion is observed. I checked things, = powered cycled altitude amplifiers and wind-baffle switch etc, tried a = couple of time, no luck. Then I went back to control room to check the = interlock screen. While =E2=80=9Ctcc axis init=E2=80=9D was issued, MCP = menu show all three axes are successfully initialized with green **** in = all axes, but the interlock GUI show red =E2=80=9Cwind-alt_ready=E2=80=9D= (all other bits are green). Then I cycled the e-stop in the control = room, issue =E2=80=9Ctcc axis init=E2=80=9D and see the = =E2=80=9Cwind_alt_ready=E2=80=9D in the interlock GUI became green too = with all other bits. I was able to move the telescope to stow without = other issues for the night. > Interesting - the control room stop button is a bit like the rail stop = buttons, having a different path to each of the relevant PLC machines = than those built in to the telescope. Stops like the podium button = interact only with the telescope PLC, but serve to interrupt windbaffle = function by controlling motion permits that then affect the windbaffle = PLC permits. The rail stops have parallel paths to each of the two = PLC's, which is what prevents using the windbaffle's manual pendant when = we have the wind_alt_ready problem. We never considered the control = room stop - which goes through a fiber link to the communications rack = in the lower enclosure, and then to either the telescope PLC or to both = PLC's. That makes me wonder if cycling a non-telescope stop button = (like one of the rail stops) would clear the problem even when we know = that cycling a telescope stop button does not. >=20 Thanks for the info on how different e-stop have different paths.=20 > One feature that has always been suspicious is that this only seems to = affect the ALT axis. ALT has a wholly different windbaffle amplifier = that was added in 2016. There were other problems with it's integration = to the existing system, I've always suspected that the wind_alt_ready = problem could be a very subtle and intermittent one related to the new = windbaffle ALT amplifier - such as an incorrect ground somewhere. >=20 I recall that we else occasionally had =E2=80=9Cwind_AZ_ready=E2=80=9D = problem although it was not as often as =E2=80=9Cwind_Alt_ready=E2=80=9D = issue some years ago. We have not seen either one for years.=20 --Apple-Mail=_57D26CD5-F839-4DD7-9536-38AE1E4147D1 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 On Dec 15, 2024, at 2:34=E2=80=AFPM, John Downey = wrote: =20 =20 On 12/15/24 13:22, Kaike wrote: On Dec 15, 2024, at 8:57=E2=80=AFAM, John Downey = wrote: = &n= bsp; APACHE = POINT OBSERVATORY SDSS 2.5M = &n= bsp; &nbs= p; OBSERVING LOG = &n= bsp; &nbs= p; Saturday December 14, 2024 = &n= bsp; &nbs= p; (SJD 60659) ---- Observing Team ---- Kaike Pan Notes: 2. After incidence of "wind_alt_ready" problem when move the telescope from 5 to 30 deg after open the = enclosure. I'm skeptical that this is a true wind_alt_ready problem. The primary reason for my skepticism is that in historical instances of this problem it was not easy to clear. Our best understanding of the problem is that = it is related to the west rail stop button or it's accompanying opto-isolator board. For the present = time, the button has been defeated at the interlock level because it is not a critical stop button and the problem was causing lost observing time. A secondary reason for my skepticism is that both instances of the problem you've been having seem to be related to moving away from 5 degrees via a button script, which have been problematic before. If the = script makes an impossible command to the MCP, the axis goes out of closed loop. Thirdly, wind_alt_ready will be red when the ALT axis is out of closed loop, that is normal. When = all the other ALT axis drive interlocks are green and wind_alt_ready remains red, that is the wind_alt_ready problem. It is typically not clearable by cycling a = stop button, but if I remember correctly an imitation of the problem can be produced with a rapid button cycle and cleared through a normal stop button cycle. Perhaps = that is a hint to what is going on if what you are seeing is all the ALT drive interlocks green except for wind_alt_ready. In both case, I intended to move the telescope via =E2=80=9Csto= w=E2=80=9D button in the "telescope control script=E2=80=9D, which is a = =E2=80=9Ctcc track=E2=80=9D command. In the 1st case, the telescope moved from 5 to 19.5 or so, then altitude went =E2=80=9Cout-closed-loop=E2=80=9D. At = that point, all axis could be initialized without any issue, but as soon as a motion command via tcc track command or via a =E2=80=9Cmove" = command in MCP altitude axes is entered x altitude axes went out-closed-loop before an actual motion is observed. I checked things, powered cycled altitude amplifiers and wind-baffle switch etc, tried a couple of time, no luck. Then I went back to control room to check the interlock screen. While =E2=80=9Ctcc = axis init=E2=80=9D was issued, MCP menu show all three axes are = successfully initialized with green **** in all axes, but the interlock = GUI show red =E2=80=9Cwind-alt_ready=E2=80=9D (all other bits are = green). Then I cycled the e-stop in the control room, issue =E2=80=9Ctcc axis = init=E2=80=9D and see the =E2=80=9Cwind_alt_ready=E2=80=9D in the interlock GUI = became green too with all other bits. I was able to move the telescope to stow without other issues for the night. Interesting - the control room stop button is a bit = like the rail stop buttons, having a different path to each of the relevant PLC machines than those built in to the telescope. Stops like = the podium button interact only with the telescope PLC, but serve to interrupt windbaffle function by controlling motion permits that then affect the windbaffle PLC permits. The rail stops have parallel paths to each of the two PLC's, which is what prevents using the windbaffle's manual pendant when we have the wind_alt_ready problem. We never considered the control room = stop - which goes through a fiber link to the communications rack in the lower enclosure, and then to either the telescope PLC or to both PLC's. That makes me wonder if cycling a non-telescope = stop button (like one of the rail stops) would clear the problem even when we know that cycling a telescope stop button does = not.Thanks for the info on how = different e-stop have different paths. One feature that has always been suspicious = is that this only seems to affect the ALT axis. ALT has a wholly different windbaffle amplifier that was added in 2016. There were = other problems with it's integration to the existing system, I've always suspected that the wind_alt_ready problem could be a very subtle and intermittent one related to the new windbaffle ALT amplifier - such as an incorrect ground somewhere. I recall that we else occasionally had = =E2=80=9Cwind_AZ_ready=E2=80=9D problem although it was not as often as = =E2=80=9Cwind_Alt_ready=E2=80=9D issue some years ago. We have not seen = either one for years. = --Apple-Mail=_57D26CD5-F839-4DD7-9536-38AE1E4147D1--