From: John Subject: Re: [sloan-25mlog 6257] 2.5m obslog 20220914 (SJD 59837) Date: Thu, 15 Sep 2022 15:03:18 -0600 --Apple-Mail=_B3B27454-1550-43FE-991F-1779C9C784C6 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Thanks Bill, I just did that and it's now different at least. We=E2=80=99= ll see how it does overnight. John > On Sep 15, 2022, at 2:29 PM, William Ketzeback = wrote: >=20 > The processes on the TPM need to be restarted if the unit has been = power cycled.=20 >=20 >> If the temperature and humidity sensors that go through the TPM are = down, you can try to apply a procedure proposed by Jeff Hagen. >>=20 >> ssh tpm@10.25.1.204 = >> [pw as the mcp] >> cd sdss/tpm_linux/src >> ./proc >> ./proc -stop optidew >> ./proc to see that it stopped >> ./proc -start optidew >>=20 >> Also try restarting the 'humid' task if the weather info isn't = refreshing after restarting optidew. It's the same procedure as = restarting optidew but with humid instead: >>=20 >> ./proc -stop humid >>=20 >> ./proc (to check status) >>=20 >> ./proc -start humid >>=20 >=20 >=20 > Bill >=20 >=20 >> On Sep 15, 2022, at 2:27 PM, John > wrote: >>=20 >> At this point I am favoring a communications issue - the values on = the weather page have not changed all day, in spite of a simple power = off for a couple hours, followed by a more complex procedure that Bill = suggested. Either one of those should (I think) have shown variation in = the reported values if not sensible values. >>=20 >> I don=E2=80=99t mind having a look at the mirror just to be familiar = with getting to it even if we don=E2=80=99t clean it. >>=20 >> There was a manual somewhere in my office when I left in 2020, I = believe it was a duplicate because it was either buried in a bookcase or = found in an old box. It had been there a while. Bill told me he has = one too so its not a huge deal to find mine now. >>=20 >> John >>=20 >> (Thanks Dan :D ) >>=20 >>> On Sep 15, 2022, at 1:46 PM, Dan Long > wrote: >>>=20 >>> When they were new this was a rare to non-existent occurrence. As = they got older they got more prone to hanging at the transition between = frost and condensation. It seems some of this is because of aging = electronics not getting the servo right. >>>=20 >>> Once water becomes clear ice it becomes harder for the system to = tell a shiny mirror from an ice covered one. Another thing that can = exacerbate the problem is if the mirror gets dirty (which often happens = if it gets wet rather than just barely condensing e.g. when it spends a = lot of time below the dew point when the servo gets it wrong). >>>=20 >>> There is a cleaning procedure for the mirror that involves taking = the sensing unit out of the =E2=80=9Cmushroom=E2=80=9D and cleaning the = mirror with an alcohol swab or something similar. The TSL documentation = was once upon a time in one of the file drawers in the OPS building. >>>=20 >>> Cheers, >>>=20 >>> Dan >>>=20 >>> P.S. Welcome back John! >>>=20 >>>=20 >>>> On Sep 15, 2022, at 8:22 AM, Russet McMillan > wrote: >>>>=20 >>>> Both TSL sensors do this in the summertime, when the dewpoint is = above 0 C. This does not happen when the dewpoint is well below 0 C, = which is pretty much true all winter long. >>>>=20 >>>> In the winter, the Optidews are less accurate than the TSLs. In = the summer, the TSLs are prone to freezing. >>>>=20 >>>>> On Sep 15, 2022, at 8:11 AM, John > wrote: >>>>>=20 >>>>>=20 >>>>>=20 >>>>>> On Sep 15, 2022, at 5:48 AM, kpan@apo.nmsu.edu = wrote: >>>>>>=20 >>>>>>=20 >>>>>> Notes: PT weather station's dew point is not reliable. >>>>>>=20 >>>>>>=20 >>>>>> ---- Engineering Issues ---- >>>>>>=20 >>>>>> The dew point from PT weather station is incorrect. >>>>>>=20 >>>>>=20 >>>>> Unfortunately, this is a feature with that very old = hygrothermometer. The way it works is it cools a mirror that will fog = over at the dew point temperature. It has some variety of optical = emitter/detector looking at the little mirror and the fog of moisture at = the dew point causes a great deal of scatter, which causes a large drop = in signal from the detector. The temperature at which the signal drop = happens is reported as the dew point temperature. The electronics then = warms up the mirror and the cycle repeats. >>>>>=20 >>>>> As far as I know, the opti-dew units are much better at keeping = the cooling/warming cycles within the appropriate range, this old TSL = (Technical Services Laboratory) unit tends to freeze the mirror when = there is a sudden drop in temperature coupled with a sudden rise in = humidity. Once the mirror is frozen it behaves as you described in = another communication. >>>>>=20 >>>>> Going forward, it is good to note when this happens as we may be = in discussions of replacing the unit. However, we should not expect a = repair of the TSL hygrothermometer by engineering staff. It has always = done this, and we have always dealt with it by turning the unit off for = maybe half an hour during the day to let the mirror thaw. Night staff = is welcome to take this approach as well, though it may require more = time at night. I will do that this morning once ambient temp warms up a = little. >>>>>=20 >>>>> I will look for the manual to see if there are adjustments that = could be made. I doubt there are however, I think Ed would have done = that if there were (he used to look after this piece of equipment). >>>>>=20 >>>>> John >>>>>=20 >>>>> ____________________________________________________________ >>>>> Logs of Nightly APO Activities > >>>>> = https://trac.sdss.org/tracmailman/browser/private/sloan-25mlog/index.html = >>>>=20 >>>> ____________________________________________________________ >>>> Logs of Nightly APO Activities > >>>> = https://trac.sdss.org/tracmailman/browser/private/sloan-25mlog/index.html = >>>=20 >>=20 >> ____________________________________________________________ >> Logs of Nightly APO Activities > >> = https://trac.sdss.org/tracmailman/browser/private/sloan-25mlog/index.html = >=20 > Bill Ketzeback > Chief Telescope Engineer > ARC 3.5-meter Telescope >=20 > Apache Point Observatory > 2001 Apache Point Rd > Sunspot, NM 88349 >=20 > bketzeback@apo.nmsu.edu > V: 575-437-6822 (use extension 119) > V: 575-437-0954 (direct to desk) > C: 575-430-1832 > F: 575-434-5555 >=20 >=20 --Apple-Mail=_B3B27454-1550-43FE-991F-1779C9C784C6 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Thanks Bill, I just did that and it's now different at least. = We=E2=80=99ll see how it does overnight.JohnOn Sep = 15, 2022, at 2:29 PM, William Ketzeback wrote:The processes on the = TPM need to be restarted if the unit has been power cycled. If the temperature and = humidity sensors that go through the TPM are down, you can try to apply = a procedure proposed by Jeff Hagen.ssh tpm@10.25.1.204 [pw as the = mcp]cd sdss/tpm_linux/src./proc./proc -stop optidew./proc = to see that it stopped./proc -start optidewAlso try restarting the 'humid' task if the weather info = isn't refreshing after restarting optidew. It's the same procedure = as restarting optidew but with humid instead:./proc = -stop humid./proc (to check status)./proc -start humidBillOn Sep 15, 2022, at 2:27 PM, John wrote:At = this point I am favoring a communications issue - the values on the = weather page have not changed all day, in spite of a simple power off = for a couple hours, followed by a more complex procedure that Bill = suggested. Either one of those should (I think) have shown = variation in the reported values if not sensible values.I don=E2=80=99t mind having a look at the mirror just to be = familiar with getting to it even if we don=E2=80=99t clean it.There was a manual somewhere in my office when = I left in 2020, I believe it was a duplicate because it was either = buried in a bookcase or found in an old box. It had been there a = while. Bill told me he has one too so its not a huge deal to find = mine now.John(Thanks Dan :D )On Sep 15, 2022, at 1:46 PM, Dan Long = wrote: When they were new this was a rare = to non-existent occurrence. As they got older they got more = prone to hanging at the transition between frost and condensation. = It seems some of this is because of aging electronics not getting = the servo right. Once water becomes clear = ice it becomes harder for the system to tell a shiny mirror from an ice = covered one. Another thing that can exacerbate the problem is if = the mirror gets dirty (which often happens if it gets wet rather than = just barely condensing e.g. when it spends a lot of time below the dew = point when the servo gets it wrong). There = is a cleaning procedure for the mirror that involves taking the sensing = unit out of the =E2=80=9Cmushroom=E2=80=9D and cleaning the mirror with = an alcohol swab or something similar. The TSL documentation was = once upon a time in one of the file drawers in the OPS building. Cheers, = DanP.S. Welcome back = John!On Sep 15, 2022, at 8:22 AM, Russet McMillan = wrote:Both TSL sensors do this in the summertime, when the dewpoint = is above 0 C. This does not happen when the dewpoint is well below = 0 C, which is pretty much true all winter long.In the winter, the Optidews are less accurate than the TSLs. = In the summer, the TSLs are prone to freezing.On Sep 15, 2022, at 8:11 = AM, John wrote:On Sep 15, 2022, at 5:48 AM, kpan@apo.nmsu.edu = wrote:Notes: PT weather = station's dew point is not reliable.---- Engineering Issues ----The = dew point from PT weather station is incorrect.Unfortunately, this is a feature = with that very old hygrothermometer. The way it works is it cools = a mirror that will fog over at the dew point temperature. It has = some variety of optical emitter/detector looking at the little mirror = and the fog of moisture at the dew point causes a great deal of scatter, = which causes a large drop in signal from the detector. The = temperature at which the signal drop happens is reported as the dew = point temperature. The electronics then warms up the mirror and = the cycle repeats.As far as I know, the = opti-dew units are much better at keeping the cooling/warming cycles = within the appropriate range, this old TSL (Technical Services = Laboratory) unit tends to freeze the mirror when there is a sudden drop = in temperature coupled with a sudden rise in humidity. Once the = mirror is frozen it behaves as you described in another = communication.Going forward, it is good to = note when this happens as we may be in discussions of replacing the = unit. However, we should not expect a repair of the TSL = hygrothermometer by engineering staff. It has always done this, = and we have always dealt with it by turning the unit off for maybe half = an hour during the day to let the mirror thaw. Night staff is = welcome to take this approach as well, though it may require more time = at night. I will do that this morning once ambient temp warms up a = little.I will look for the manual to see = if there are adjustments that could be made. I doubt there are = however, I think Ed would have done that if there were (he used to look = after this piece of equipment).John____________________________________________________________Logs of Nightly APO Activities https://trac.sdss.org/tracmailman/browser/private/sloan-25mlog/= index.html____________________________________________________________Logs of Nightly APO Activities https://trac.sdss.org/tracmailman/browser/private/sloan-25mlog/= index.html____________________________________________________________Logs of Nightly APO Activities https://trac.sdss.org/tracmailman/browser/private/sloan-25mlog/= index.html Bill = KetzebackChief Telescope EngineerARC = 3.5-meter TelescopeApache Point = Observatory2001 Apache Point RdSunspot, NM = 88349bketzeback@apo.nmsu.eduV: 575-437-6822 = (use extension 119)V: 575-437-0954 (direct to = desk)C: 575-430-1832F: = 575-434-5555 = --Apple-Mail=_B3B27454-1550-43FE-991F-1779C9C784C6--