Subject: Re: [sloan-25mlog 5156] Re: 2.5m obslog 20200809 (SJD 59071) From: John Downey Date: Tue, 11 Aug 2020 13:41:26 -0600 --Apple-Mail=_B67AE4D7-05CD-48F8-892F-3597BA252EA2 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 James is the one who will want to reseat cables. We always power the control electronics off before messing with cables, = and that will cycle the analog power to the CCD's. If the cabling is to be unplugged and re-seated, it would be very = prudent to try to clean a little while they are unplugged. > On Aug 11, 2020, at 1:14 PM, Karen Kinemuchi = wrote: >=20 > Hi all, > I wrote to Alan and Connie to get their thoughts and advice on = this electronic noise issue. I have heard back from both of them, and = here are their response: >=20 > Alan says: >=20 > Connie is the person to ask. Generically I look for bad cable = connections then reboot the CCD micro to reload stuff (or does IACK do = this?). But that's probably been done already (?). >=20 > And Connie=E2=80=99s response: >=20 > My best guess is that this is a cable seating problem, yes. It looks = like the lines are getting out of sync. All the data seem to be there, = but some of it is in the wrong place. >=20 > But here are some thoughts: >=20 > An iack will not help. IACK just sets the values of variables that = control things like the best operating temperature or maybe the voltage = level on a CCD gate.=20 > It doesn't have anything to do with timing, and it doesn't give = anything a "kick" that might jolt a confused microprocessor into good = behavior.=20 >=20 > And we have several failure modes, but timing confusion on the part of = the microprocessor is not one of them. This looks like a cable seating = problem -- there are some=20 > synchronization and timing signals that go with the pixel data, and if = one of them is near the end of a connector and it is not seated = correctly,this kind of thing can happen. > It could be a version of a seating problem where a pin on the cable or = connector is bad, or something like that. The fact that it fixes itself = after a few exposures probably means=20 > this is going to be difficult to find, unfortunately. >=20 > I would not recommend power-cycling the microprocessor, as that will = also cycle the analog power to the CCDs and might make the blowtorch = worse. > =09 > I hope this helps in fixing the electronic noise issue that is = problematic to the MaNGA data. =20 > Cheers, > Karen >=20 >=20 >> On Aug 10, 2020, at 2:19 PM, Lazarz, Daniel > wrote: >>=20 >> Hey all, >>=20 >> We wanted to notify you all about an issue we encountered this = morning that seems to be related to the first set of Hartmanns attempted = on cart 6 failing. We noticed that a considerable number of exposures = taken on cart 6 (exp 317138-317147 in SP1) suffered from the same = electronics issue that we encountered about a month ago (see MJD 59035), = which can be seen from some of the DOS warnings on that cart. All of = these frames came out looking heavily pixelated, as shown in the example = below: >>=20 >> >>=20 >> Unfortunately, since this problem began in the calibration frames, = all of the subsequent science frames taken on b1 and r1 aren't really = salvageable, even though the problem appears to have subsided by = exposure 317148. In future cases of this happening, if the observers see = any of the DOS warnings indicating the electronics problem (or the = massive pixel bias RMS warnings that seem to accompany them), it'd be = great if we could get some post-cal frames to make up for the lack of = good pre-cal frames. >>=20 >> We'll probably bring this up in tomorrow's Operations telecon, but in = the meantime, Renbin should feel free to chime in if he has any = clarifying details. >>=20 >> Thanks, >> Dan >>=20 >=20 --Apple-Mail=_B67AE4D7-05CD-48F8-892F-3597BA252EA2 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 James is the one who will want to reseat cables.We always power the = control electronics off before messing with cables, and that will cycle = the analog power to the CCD's.If the cabling is to be unplugged and = re-seated, it would be very prudent to try to clean a little while they = are unplugged.On Aug 11, 2020, at 1:14 PM, = Karen Kinemuchi wrote:Hi all, = I wrote to Alan and Connie to get their thoughts and advice on = this electronic noise issue. I have heard back from both of them, = and here are their response:Alan says:Connie is the person to ask. Generically I look for bad cable = connections then reboot the CCD micro to reload stuff (or does = IACK do this?). But that's probably been done already = (?).And Connie=E2=80=99s = response:My best guess is = that this is a cable seating problem, yes. It looks like the lines = are getting out of sync. All the data seem to be there, but some of it = is in the wrong place.But here are some thoughts:An iack will not help. IACK just sets = the values of variables that control things like the best operating = temperature or maybe the voltage level on a CCD gate. It doesn't have anything to do with timing, and it doesn't = give anything a "kick" that might jolt a confused microprocessor into = good behavior. And we have several failure modes, but timing = confusion on the part of the microprocessor is not one of them. = This looks like a cable seating problem -- there are = some synchronization and timing signals that = go with the pixel data, and if one of them is near the end of a = connector and it is not seated correctly,this kind of thing can = happen.It could be a version of a seating problem = where a pin on the cable or connector is bad, or something like = that. The fact that it fixes itself after a few exposures probably = means this is going to be difficult to find, = unfortunately.I= would not recommend power-cycling the microprocessor, as that will also = cycle the analog power to the CCDs and might make the blowtorch = worse. I hope = this helps in fixing the electronic noise issue that is problematic to = the MaNGA data. Cheers, = KarenOn Aug 10, 2020, at 2:19 PM, Lazarz, Daniel = wrote:Hey all,We wanted to = notify you all about an issue we encountered this morning that seems to = be related to the first set of Hartmanns attempted on cart 6 failing. We = noticed that a considerable number of exposures taken on cart 6 (exp = 317138-317147 in SP1) suffered from the same electronics issue = that we encountered about a month ago (see MJD 59035), which can be seen = from some of the DOS warnings on that cart. All of these frames came out = looking heavily pixelated, as shown in the example below:Unfortunately, since this problem = began in the calibration frames, all = of the subsequent science frames taken on b1 and r1 aren't really = salvageable, even though the problem appears to have subsided by = exposure 317148. In future cases of this happening, if the observers see = any of the DOS warnings indicating the electronics problem (or the = massive pixel bias RMS warnings that seem to accompany them), it'd be = great if we could get some post-cal frames to make up for the lack of = good pre-cal frames.We'll probably bring this up in = tomorrow's Operations telecon, but in the meantime, Renbin should feel = free to chime in if he has any clarifying details.Thanks,Dan= --Apple-Mail=_B67AE4D7-05CD-48F8-892F-3597BA252EA2--