Subject: Re: [sloan-25mlog 4771] 2.5m obslog 20191113 (SJD 58801)
From: John Downey
Date: Fri, 15 Nov 2019 12:03:42 -0700
We're planning on having a flag for this in the MCP log file.  Jeff =
spotted the timing problem, where a slew was commanded after the =
commanded arrival time using a console log that shows more detail than =
the already long MCP log.
This one was the result of what used to be a reposition command that the =
system does when the axis is more than 0.01 degrees off when a PVT is =
requested.  I believe the problem is caused by the TCC not knowing about =
the reposition command (to get the PVT starting point right), so the =
time it takes to reposition would cause the PVT to start after the =
commanded T, resulting in a runaway... What Jeff changed it to do is =
command a frame break instead when this condition happens, and from the =
observer's perspective, a frame break just looks like a commanded slew =
that never started or stopped without OCL partway in.
My grasp of this is not up to Jeff's level though, so he may want to =
correct what I just wrote :D
Maybe the thing to do is have any frame break trigger some kind of red =
"frame break" message in the STUI log tool.
> On Nov 14, 2019, at 5:53 PM, Kaike Pan  wrote:
>=20
>=20
>> On Nov 14, 2019, at 2:51 PM, Daniel Oravetz  =
wrote:
>>=20
>> If one or both of these events turn about to be the result of a
>> TCC-supplied slew path that would've resulted in a runaway slew in
>> previous MCP versions, then I would agree that we would prefer to =
have
>> this "new" behavior where the telescope does *NOT* actually act upon =
such
>> a slew trajectory; if it receives a bad slew path, just stay put, =
don't
>> run away and slew wildly.
>>=20
>> However, I do think we should probably implement some sort of louder, =
more
>> immediate, handling of such a case -- let us know the slew will fail =
and
>> don't even let the TCC timer run as if the slew is actually =
happening. For
>> example, the old out-of-closed-loop event was a clear sign that the
>> telescope did not, or would not, get to where it was commanded and it
>> demanded immediate intervention to correct the axis state and =
reattempt
>> the slew. The way these failed slews behaved last night, they =
basically
>> acted as though they completed based on the TCC timer and the =
axis-state
>> (Slewing, then Tracking).
>=20
> The happened another day with a =E2=80=9Ctcc track=E2=80=9D command =
too. Yes, it would be better
> if the slew fails with an error, so that we know the requested =
doesn=E2=80=99t start.=20
>=20
>>=20
>> Under the current conditions, I wonder how a "bad" slew would be =
handled
>> during a pointing model, for example. We used to get on the order of =
~6
>> out-of-closed-loop events during a typical pointing model and we =
would
>> correct the axis condition and re-attempt the slew. If the telescope
>> doesn't actually fail loudly enough now when a given axis does =
not/will
>> not make it to its commanded position, then it seems plausible the
>> exposure will just proceed as though the telescope actually *is* =
where it
>> should be. It should be clear during the TPOINT reduction to reject =
bad
>> exposures in these hypothetical cases, but it would be better to fail =
a
>> potential bad slew quickly and loudly so we could re-issue the slew
>> attempt, for example.
>=20
> Fortunately, no such slews in the pointing model process on Tuesday.
> If this happened on the data collection of a pointing model, it will =
result two possibility:
> 1) the script will report =E2=80=9Cno star=E2=80=9D in the field and =
pause for observer=E2=80=99s action: retry or skip=20
> the current star and continue to execute the rest of the script, =
hopefully observer can catch
> the incomplete slew and retry it.
> 2) a bad situation: a star happen to be in the view, the script will =
make a measure on that star (which
> we need to reject in the later data analysis). A even worse case would =
be, the measurements afterward
> will be messed up, cannot be used. Because a later slew uses =
calibration offset measured from the=20
> previous star, the measurement from the wrong star will affect later =
slews until a correct star is measured
> again. I think that we have a ticket on this. New slew should not use =
calibration offsets measured from
> the previous star, i.e. slew to new stars with no any offsets.=20
>=20
>=20
>>=20
>> Dan
>>=20
>>>>=20
>>>> =3D=3D=3D=3D=3D
>>>> Notes
>>>> =3D=3D=3D=3D=3D
>>>>=20
>>>>=20
>>>> (2) 10:47Z-10:49Z - During the commanded slew from 121,45,135 to
>>>> instChange
>>>> (121,90,00), altitude stayed at 45 deg. The MCP showed that the =
altitude
>>>> axis
>>>> was live and initialized, but altitude never moved from 45 deg. The =
next
>>>> attempt
>>>> at slewing to instChange resulted in altitude moving as commanded.
>>>> Please see
>>>> the Problems section for details.
>>>>=20
>>>> During afternoon checkout there was a similar missed slew at 23:15Z =
in
>>>> which the
>>>> telescope was at 121,90,00 and commanded to 121,30,00, but it =
stayed at
>>>> 121,90,00 despite STUI showing the status of the axes as 'Slewing.' =
See
>>>> Problems
>>>> section for details.
>>>>=20
>>>=20
>>> We'll look at this in more detail today, but at first glance it =
sounds
>>> like it is doing what it is supposed to under conditions that might =
have
>>> previously resulted in a wild slew.
>>>=20
>>> The afternoon one is a bit more suspicious to me due to it being a =
regular
>>> track command, those have been less troublesome that sop commands.
>>=20
>>=20
>> ____________________________________________________________
>> Logs of Nightly APO Activities
>> =
https://trac.sdss.org/tracmailman/browser/private/sloan-25mlog/index.html
>=20