Subject: Re: [sloan-25mlog 3282] 2.5m obslog 20161217 (SJD 57740)
From: John Downey
Date: Sun, 18 Dec 2016 08:30:16 -0700
>=20
> Three out of five Glenteks amplifiers (3 bottom ones) were burned=20
> out, most probably die to power failures during the day time
> (site ran on generator power until the suntset.)=20
>=20
> At 23:50Z site turned back on commercial power.=20
> We waited to be sure that it is stable and try to turned on the =
amplifiers.
>=20
>=20
>=20
> Notes: we moved gang connector to 1-m port after sunset, and=20
> started 1m script, but we did not see any activity on APOGEE
> spectrograph connected to 1-m telescope. =20
>=20
>=20
> =3D=3D=3D Questions to address by engineering crew:
>=20
> 1. Why amplifiers were damaged when they were TURNED OFF (!) by power =
button?
> 2. Why they were damaged when 2 of 3 fuses were good (as initial =
inspection showed)?
> 3. Why it never happened during the last 17 years, and just happened =
how?
> 4. What should we do to prevent this in the future? Pull out power =
plugs in the
> morning? Add surge protection power stripes to the amplifiers?
>=20
> Adding to 1 and 2, do we have new problems with bad grounding, phases =
cross
> talks, etc.? who can check it?
>=20
We will know more after James has a look at the amps today, but I can =
provide some initial comments.
1.  The power button does not completely interrupt all incoming power to =
the chassis, more likely it controls much larger contactors that are =
appropriately sized for the power consumption of the amplifiers.  Damage =
is most likely in the circuits that drive the contactors.  Turned off =
does not necessarily mean completely powered down.
2.  Whether fuses blow or not depends on what they are intended to =
protect, and complicated units can have several fuses for different =
purposes.  =46rom what I understand Tracy also replaced additional fuses =
before finding that all three amps behaved similarly when plugged in.
3.  Most likely, just age and use.  Although moving parts wear more =
quickly, electrical characteristics of parts can change with age as =
well.  It is possible also that this is related to the last time there =
was a brownout and one or both of the ALT amps were affected and =
behaving oddly.  Since a power cycle fixed that, I don't really think =
they are related, but I can't rule it out.
4.  If we do anything to isolate the amps, probably the easiest is to =
turn off the circuit breakers providing power to the two circuits that =
run them (assuming nothing else is on those circuits).  Its another step =
to turning on/off, but it requires no wiring changes and is something =
that can easily be included in the routine if desired.  Surge protection =
is certainly something we can look into.  As the technology develops =
they get cheaper, and it may be cost effective to include now when it =
was not 20 years ago.