Subject: Re: [sloan-25mlog 2476] Re: 2.5m obslog 20150816 (SJD 57251) From: Dan Long Date: Mon, 17 Aug 2015 15:11:21 -0600 --Apple-Mail=_84230B42-4DBB-40A7-A0E6-0372E4DC7EC6 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 I just looked at some arcs taken on 57249 (frame 202480). They are = brighter than normal arcs and I can see this loss of flux above a = certain threshold in different quadrants of r1, and on r2 as well. So = this appears to be common to both red detectors. The thing that is = different is that the threshold is around 65k for this behavior in most = quadrants, but in the range of 45k on the left side of r1 (as seen in = ds9).=20 -Dan On Aug 17, 2015, at 2:10 PM, Dan Long wrote: > The frame Renbin included in his email from last October looks just = like the one from last night. The same area of the same quadrant is = exhibiting the odd behavior. So if there is an issue starting at 40,000 = counts, it seems to have been with us since the new r1 chip was = installed. >=20 > -Dan >=20 > On Aug 17, 2015, at 1:31 PM, Kyle Dawson = wrote: >=20 >> I don't understand the explanation. This appears to be suppressed = full well depth and poor isolation between the sense node and output = gate. It does not seem to be light on the serial register. The effect = is localized in the active region of the CCD, and saturation is occuring = at only ~40,000 counts. The saturation appears as a loss of flux, = whereby I believe that charge is spilling onto the sense node in the = first stage of the double correlated sampling. There is additional = bleeding of charge in the serial direction >>=20 >> The fact the effect appears localized in the CCD argues that it is = not due to light on the serial register. I also don't really understand = why the CCD sees light during the lossy fibers that it does not see in = normal exposures. A broader region of the CCD is likely exposed in the = fiber direction, but the light still needs to go through the dispersive = elements and should not be reaching any further in the spectral = direction. >>=20 >> I worry that there is a problem on the CCD that is leading to poor = transfer in the final stage of readout and possible charge injection (we = see a 10% increase in readnoise as well). It may not be fixable, and = maybe we should just treat it as a non-linearity and avoid counts above = 20 or 25. But it makes me worried. >>=20 >>=20 >>=20 >> On 08/17/2015 01:20 PM, Connie Rockosi wrote: >>> Thanks, Dan and Renbin. >>>=20 >>> I thought this sounded familiar, and yes, it does sound like another = instance of what Renbin described from the previous lossy fiber tests.=20= >>>=20 >>> I posted the noise measurements from when the CCDs were first cooled = back down on the telescope. Were there biases taken yesterday and do the = noise measurements match? >>>=20 >>> --Connie >>>=20 >>> On Mon, Aug 17, 2015 at 11:44 AM, Dan Long = wrote: >>> Hi Connie, >>>=20 >>> I think Renbin has seen this before soon after the latest r1 chip = was installed. See his message to infrastructure on October 16, 2014 = [10513]. It sounds like this is peculiar to lossy fiber data since = light can shine on parts of the detector that don=92t normally see it = and that is causing the early saturation. However, please take a look = and confirm. >>>=20 >>> Cheers, >>>=20 >>> Dan >>>=20 >>>=20 >>> On Aug 17, 2015, at 12:36 PM, ucolick wrote: >>>=20 >>> > What is "high" Vivek? >>> > >>> > I'll try to grab some data but my connection is slow so it will = take me a bit. >>> > >>> > Connie >>> > >>> >> On Aug 17, 2015, at 11:01 AM, VIVEK MARIAPPAN = wrote: >>> >> >>> >> HI Dan (cc'ing Renbin and Kai), >>> >> >>> >> This looks to me a serious problem. Reducing the integration time = might not be a solution The full well capacity should be well beyond = 40k. >>> >> counts are bleeding in one direction. I don't understand this. >>> >> Are charges getting dumped in the fake read of double correlated = sampling stage? >>> >> We are also noticing high read out noise for r1 camera after the = summer shutdown. >>> >> I think we have to address this asap. (Should we recede the = cables? ) >>> >> >>> >> regards, >>> >> vivek >>> >> >>> >> ________________________________________ >>> >> From: Dan Long [long@apo.nmsu.edu] >>> >> Sent: Monday, August 17, 2015 11:21 AM >>> >> To: VIVEK MARIAPPAN; Connie_Rockosi Rockosi >>> >> Cc: Dan Long; 2.5m_log >>> >> Subject: Re: [sloan-25mlog 2468] 2.5m obslog 20150816 (SJD 57251) >>> >> >>> >> The first frame mentioned (and presumably the others) clearly = show saturation effects as the counts exceed about 40k (Connie, is that = what one should expect?). Regular calibration frames (flats, arcs) look = fine and have lower counts. >>> >> >>> >> It looks like the lossy fiber exposure time is too long for the = red. So should we leave the lossy fiber setup where it is and have the = test repeated with shorter exposure times, or go ahead and move it to = the sp2 side? >>> >> >>> >> Cheers, >>> >> >>> >> Dan >>> >> >>> >> >>> >>> On Aug 17, 2015, at 6:20 AM, kinemuchi@apo.nmsu.edu wrote: >>> >>> >>> >>> >>> >>> APACHE POINT OBSERVATORY SDSS 2.5M >>> >>> >>> >>> OBSERVING LOG >>> >>> Sunday August 16, 2015 >>> >>> >>> >>> (SJD 57251) >>> >>> >>> >>> ---- Observing Team ---- >>> >>> >>> >>> Kaike Pan >>> >>> Karen Kinemuchi >>> >>> >>> >>> ---- Observing Summary ---- >>> >>> >>> >>> Most of the night lost to high humidity and clouds, during that = time, we >>> >>> finished the lossy fiber test on SP1. >>> >>> >>> >>> We open for last 2 hours of the night. Had hard time to find = star. It turned out >>> >>> that that was due to a big offset: -30" in RA, 100" in Dec. We = will continue the >>> >>> optical collimation. >>> >>> >>> >>> Lossy fiber test done for SP1. >>> >>> >>> >>> Note to engineers and developers: >>> >>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>> >>> During the lossy fiber test on SP1, we found a weird feature on = R1, >>> >>> possibly a saturation feature, starting from file # 202669 until = the end of the >>> >>> test, file # 202698. In the first quadrant, we think saturation = occurs at 30k >>> >>> counts. Should the test be re-done with shorter exposure time = but more exposures >>> >>> at each position. Please see log for more details. >>> >> >>> >> >>> >> >>> >> >>> >> !DSPAM:10028,55d223b612938719317755! >>> >> >>>=20 >>>=20 >>=20 >> ____________________________________________________________ >> Logs of Nightly APO Activities >> = https://trac.sdss.org/tracmailman/browser/private/sloan-25mlog/index.html >=20 --Apple-Mail=_84230B42-4DBB-40A7-A0E6-0372E4DC7EC6 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=windows-1252 I just = looked at some arcs taken on 57249 (frame 202480). They are = brighter than normal arcs and I can see this loss of flux above a = certain threshold in different quadrants of r1, and on r2 as well. = So this appears to be common to both red detectors. The = thing that is different is that the threshold is around 65k for this = behavior in most quadrants, but in the range of 45k on the left side of = r1 (as seen in = ds9). -DanOn Aug 17, = 2015, at 2:10 PM, Dan Long = wrote: = The frame Renbin included in his email from last October looks = just like the one from last night. The same area of the same = quadrant is exhibiting the odd behavior. So if there is an issue = starting at 40,000 counts, it seems to have been with us since the new = r1 chip was = installed.-DanOn Aug 17, = 2015, at 1:31 PM, Kyle Dawson = wrote: I don't understand the explanation. = This appears to be suppressed full well depth and poor isolation between = the sense node and output gate. It does not seem to be light on = the serial register. The effect is localized in the active region of the CCD, and saturation is occuring at only ~40,000 = counts. The saturation appears as a loss of flux, whereby I = believe that charge is spilling onto the sense node in the first stage = of the double correlated sampling. There is additional bleeding of charge in the serial direction The fact the effect appears localized in the CCD argues that it is not = due to light on the serial register. I also don't really = understand why the CCD sees light during the lossy fibers that it does = not see in normal exposures. A broader region of the CCD is likely exposed in the fiber direction, but the light still needs to = go through the dispersive elements and should not be reaching any = further in the spectral direction. I worry that there is a problem on the CCD that is leading to poor = transfer in the final stage of readout and possible charge injection (we = see a 10% increase in readnoise as well). It may not be fixable, = and maybe we should just treat it as a non-linearity and avoid counts above 20 or 25. But it makes me worried. On 08/17/2015 01:20 PM, Connie Rockosi wrote: Thanks, Dan and Renbin. I thought this sounded familiar, and yes, it does sound like = another instance of what Renbin described from the previous lossy fiber = tests. I posted the noise measurements from when the CCDs were first = cooled back down on the telescope. Were there biases taken yesterday and = do the noise measurements match? --Connie On Mon, Aug 17, 2015 at 11:44 AM, Dan Long = wrote: Hi Connie, I think Renbin has seen this before soon after the latest = r1 chip was installed. See his message to infrastructure on = October 16, 2014 [10513]. It sounds like this is peculiar to lossy = fiber data since light can shine on parts of the detector that don=92t normally see it and that is causing the early saturation. = However, please take a look and confirm. Cheers, = Dan On Aug 17, 2015, at 12:36 PM, ucolick = wrote: > What is "high" Vivek? > > I'll try to grab some data but my connection is slow so it will = take me a bit. > > Connie > >> On Aug 17, 2015, at 11:01 AM, VIVEK MARIAPPAN = wrote: >> >> HI Dan (cc'ing Renbin and Kai), >> >> This looks to me a serious problem. Reducing the integration = time might not be a solution The full well capacity should be well = beyond 40k. >> counts are bleeding in one direction. I don't understand = this. >> Are charges getting dumped in the fake read of double = correlated sampling stage? >> We are also noticing high read out noise for r1 camera after = the summer shutdown. >> I think we have to address this asap. (Should we recede = the cables? ) >> >> regards, >> vivek >> >> ________________________________________ >> From: Dan Long [long@apo.nmsu.edu] >> Sent: Monday, August 17, 2015 11:21 AM >> To: VIVEK MARIAPPAN; Connie_Rockosi Rockosi >> Cc: Dan Long; 2.5m_log >> Subject: Re: [sloan-25mlog 2468] 2.5m obslog 20150816 (SJD = 57251) >> >> The first frame mentioned (and presumably the others) = clearly show saturation effects as the counts exceed about 40k (Connie, = is that what one should expect?). Regular calibration frames = (flats, arcs) look fine and have lower counts. >> >> It looks like the lossy fiber exposure time is too long = for the red. So should we leave the lossy fiber setup where it is = and have the test repeated with shorter exposure times, or go ahead and = move it to the sp2 side? >> >> = Cheers, >> >> = Dan >> >> >>> On Aug 17, 2015, at 6:20 AM, kinemuchi@apo.nmsu.edu wrote: >>> >>> >>> = APACHE POINT OBSERVATORY SDSS 2.5M >>> >>> = OBSERVING LOG >>> = Sunday August 16, 2015 >>> >>> = (SJD 57251) >>> >>> ---- Observing Team ---- >>> >>> Kaike Pan >>> Karen Kinemuchi >>> >>> ---- Observing Summary ---- >>> >>> Most of the night lost to high humidity and clouds, during = that time, we >>> finished the lossy fiber test on SP1. >>> >>> We open for last 2 hours of the night. Had hard time to = find star. It turned out >>> that that was due to a big offset: -30" in RA, 100" in Dec. = We will continue the >>> optical collimation. >>> >>> Lossy fiber test done for SP1. >>> >>> Note to engineers and developers: >>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>> During the lossy fiber test on SP1, we found a weird = feature on R1, >>> possibly a saturation feature, starting from file # 202669 = until the end of the >>> test, file # 202698. In the first quadrant, we think = saturation occurs at 30k >>> counts. Should the test be re-done with shorter = exposure time but more exposures >>> at each position. Please see log for more details. >> >> >> >> >> !DSPAM:10028,55d223b612938719317755! >> ____________________________________________________________Logs of = Nightly APO Activities https://trac.sdss.org/tracmailman/browser/private/sloan-25mlog/in= dex.html= --Apple-Mail=_84230B42-4DBB-40A7-A0E6-0372E4DC7EC6--