Date: Sun, 28 Feb 2021 07:53:02 -0700 Subject: Re: [sloan-25mlog 5414] 2.5m obslog 20210227 (SJD 59273) From: Fred Mrozek "we can sing a wake up song aloud for the sleepy microswitch when replace c= artridges. Any choice of the song? Please recommend :)" Victor,=20 Ryan and I had some ideas. Like the sound track from the scene in the movi= e Jaws when Quint scratched his fingers on the chalkboard. But, out of an = abundance of caution, we decided that might cause other operational systems = to malfunction or a plague of moths to awaken... =20 Maxim: "First, do no harm." --Fred =EF=BB=BFOn 2/28/21, 7:01 AM, "Kaike Pan" wrote: >=20 > - Fred reported in the day log that Ryan needs spare parts to fix the= lift > problem. While the search for spares is going on, here is his sugges= tion > for observers: > "If observers encounter hesitancy-to-latch, use the thumbwheel to jo= g the=20 > cart up and down a few inches, returning to a correct upward pressur= e=20 > each time. It may take a few cycles, but you need not move the Lind= e=20 > Cart. Just short jogs up and down to activate a sleepy microswitch.= " > Not from Fred's email: we can sing a wake up song aloud for the slee= py =20 > microswitch when replace cartridges. Any choice of the song? Please=20 > recommend :) >=20 > - During cart 1 mounting we've got the same problem with latching.=20 > We look at the interlock keywords and noticed that the conditions=20 > become "green" after the keyword "inst_lift_sw2" turned green.=20 > Same problem was on cart 8 (did not see changes of keywords clearly)= . > - Upon checkout it was found that LN2 dewar was replaced and connecte= d, > but the valve was turned off. It did not make any problems though.=20 >=20 > - STUI script "Telescope Controls" is found inoperable on the telesco= pe laptop. > It doesn't react on buttons. Impossible to check memory, or to send = to=20 > stow or to inst change, etc. Restarted STUI - same. Tried to close a= ll > scripts, then to restart - such that scripts don't show up by defaul= t. Then > tried to open only the the "Telescope controls". Same result - butto= ns are not=20 > responsible. At the same time, sdss-obs1 runs this script normally > (STUIAdditions : 3.1.4), and sdss-obs2 runs it ok, too (STUIAddition= s : 3.1.5) > Checked telescope's version: STUIAdditions : dgatlin > We did git pull , then git checkout 3.1.5 . > "Telescope Controls" started working after that. STUI scripts should be returned to the previous table version after a t= est. I will sent out a separated email on this.=20 > - We checked APOGEE evening cals and found that they are all darks.=20 > Note that they were started from STUI/SOP, on a regular way. > The APOGEE control window did not show any alerts or red values.=20 > Tried to take one Arc exposure manually - via the APOGEE menu.=20 > Good spectrum. We tried an arc via the "inp"-like lines of script -=20 > it got signal. > We restarted evening calibrations again via the STUI/SOP. The first=20 > frame is 37110011 (60 SUTR dark) - this time the calibrations are al= l > good.=20 >=20 Sounds like that lamps were not turned on. We experienced this intermit= ted issue issue several time at a very low rate. The current procedure is to stop the c= alibration sequence and re-run it.=20 > - We performed test slews in order to understand reasons for overshoo= t, and > to see if Jeff's modifications to the logging help see the problem b= etter. > At a glance, we were able to get the ALT overshoot=20 > -- with both SEM TAKE and SEM STEAL options, > -- with and without ALT MOVE command prior to the move via MCP > -- not each time - one of 2 to 4 slews.=20 Thanks for the test slews. Sound like that MCP could get the semaphore = with =E2=80=9Csem_take=E2=80=9D while the telescope is in tracking. I think that MCP sho= uld only get the semaphore via =E2=80=9Csem_steal=E2=80=9D in this case.=20 >=20 >=20 > ---- Problems in Detail ---- >=20 > Started moving about 00:25Z > [1] > First group of ALT moves was done between ALT 45 and 20 in the follow= ing manner: > tcc track 121,45 mount > tcc track 121,46 mount > tcc axis stop > tcc axix init > tcc track 121,45 mount > [the axes are tracking] > SEM TAKE > TPM GUI -> "Move" to 20 > It moved as expected. No problems.=20 > repeated 3 times.=20 >=20 > [2] > Then the same performed from ALT 35 to ALT 10. > Same result, moved as expected.=20 >=20 > After the evening LN2 fill we continued. >=20 >=20 > *** Overshoots: >=20 > [1] Moved to ALT=3D46, then to 45, then to 20. Overshoot from 4th attem= pt. > tcc track 121,46 mount > tcc axis stop > tcc axis init > tcc track 121,45 mount > MCP: SEM STEAL > MCP: MOVE to 20 (speed 300,000) > Result: overshoot - went to ALT 19 (actually to 19:04) >=20 > 2021-02-28 02:47:35.862 frTCC:ALT STATUS > 2021-02-28 02:47:35.873 toTCC:44.999992 0.000000 10055.861631 0 0.000= 000 OK > 2021-02-28 02:47:35.877 frOP:SEM_STEAL > ... > 2021-02-28 02:47:40.826 altitude move pos 19.999997 vel 1.166669 acc = 0.077778 > 2021-02-28 02:47:40.826 frOP:ALT GOTO_POS_VA 5142846.655123 300000.00= 0000 20000. > ...the next 3 commands are unclear, because telescope has to move, no= t to stop > 2021-02-28 02:47:40.835 altitude cmd frame break > 2021-02-28 02:47:40.840 altitude frame_break: frames left=3D15 > 2021-02-28 02:47:40.841 altitude stp_frame >=20 > Then there is chain of commands > 2021-02-28 02:47:43.451 azimuth CALC FRAME: next frame required to fi= nish > 2021-02-28 02:47:43.468 azimuth CALC FRAME: next frame required to fi= nish > 2021-02-28 02:47:43.483 rotator CALC FRAME: next frame required to fi= nish > 2021-02-28 02:47:43.485 azimuth CALC FRAME: next frame required to fi= nish > ... >=20 > 2021-02-28 02:47:43.748 rotator frame_break: frames left=3D4 > 2021-02-28 02:47:43.749 rotator stp_frame > 2021-02-28 02:47:43.749 azimuth frame_break: frames left=3D4 > 2021-02-28 02:47:43.750 azimuth stp_frame > ... > 2021-02-28 02:48:18.782 MGT: bad alt state ABORT_EVENT: ID_ERROR_LIMI= T > 2021-02-28 02:48:18.782 MGT: bad alt state 14 > 2021-02-28 02:48:18.782 amp_mgt set altitude brake > 2021-02-28 02:48:18.782 MGT: wait 2 sec > 2021-02-28 02:48:18.997 altitude cmd frame break > 2021-02-28 02:48:18.997 altitude stp_frame > 2021-02-28 02:48:19.496 tbrakes altitude controller idle > 2021-02-28 02:48:19.496 axis altitude brake_on >=20 >=20 >=20 > [2] Moved to ALT=3D46, then to 45, then to 20. Overshoot from 2nd attem= pt. > tcc track 121,46 mount > tcc axis stop > tcc axis init > tcc track 121,45 mount > MCP: SEM TAKE > MCP: MOVE to 20 (speed 300,000) > Result: overshoot - went to ALT 19 (actually to 19:04) >=20 > [3] Moved to ALT=3D47, then to 45, then to 20. Overshoot from 2nd attem= pt. > tcc track 121,47 mount > tcc axis stop > tcc axis init > tcc track 121,45 mount > MCP: SEM TAKE > MCP: MOVE to 20 (speed 300,000) > Result: overshoot - went to ALT 18 (actually to 18:04) >=20 > [4] Moved to ALT=3D47, then to 45, then to 20. Overshoot from 3rd attem= pt. > Added ALT MOVE - to clear axis better > tcc track 121,47 mount > tcc axis stop > tcc axis init > tcc track 121,45 mount > MCP: SEM TAKE > MCP: ALT MOVE > MCP: MOVE to 20 (speed 300,000) > Result: overshoot - went to ALT 18 (actually to 18:04) >=20 > An impression is that we have a queue of move commands > that should be cleared, but they are not cleared in some cases > because the system is busy with some other, interferring tasks. >=20 >=20 >=20 > ____________________________________________________________ > Logs of Nightly APO Activities > https://trac.sdss.org/tracmailman/browser/private/sloan-25mlog/index.= html