Apache Point Observatory
New Mexico, USA
ARC 3.5-m Instrument News
Note: Comments in bold are to catch the readers attention as the most relevant for their upcoming science observations
News Worthy Items
- ARCES is available. See below for more information.
- DIS is available. See below for more information.
- ARCTIC is available but is experiencing some issues with darks. See below for more information.
Instrument issues which our users should be aware of before taking data. For historical information go here. For future instrumentation or planned upgrades go here.
- 2017-06-19: The instrument has been operating nominally.
- 2012-01-24: APO still does not recommend taking dome flats with AGILE. More information can be found here.
- 2017-09-13: Interorder light which is a measure of scattered light in the instrument from contamination is continuing to worsen. Users are advised to report any effect to their science to the APO staff. See the link below to the most recent IOL plots.
- 2017-08-31: It is back! Echelle is ready for on sky testing and science. We are slowly lowering temperature to the operating value. Contact the observing specialist for more information.
- 2017-08-22: Update: The camera readout is still intermittent. We have the saddlebag on the workbench in the lab and are attempting to track down the problem but with it being intermittent progress is slow. Ecamera is working however. Over shutdown we changed out the electronics boards effectively giving us a brand new slitviewer camera.
- 2017-07-08: Update: We are working with Dale Sanford and Connie Roskie to troubleshoot the readout electronics failure. We have experienced and found two additional failed components so far but the root cause is still ellusive. We appreciate users patience. The intermittent failure of the slit viewr camera is being discussed with Technicians at ANDOR. IT is possible it will need to be returned for evaluation and repair.
- 2017-06-22: The science camera has had a failure where it will not properly read out images. We are troubleshooting but until corrected the instrument is down.
- 2017-06-21: Further investigation of the slitview camera mentioned on Monday has revealed that bias levels for the camera are zero. The camera is also running a few degrees warmer than its temperature setpoint. Darks are linear and the CCD is seeing light. It can be used for guiding. We are investigating if the lower than expected bias level is adjustable on the board from the manufacturer. There has not been a repeat of the corrupted (or partially elevated background levels on the left side of the chip) that was reported intermittently Thursday - Sunday nights. Report any occurances to the Observing Specialist.
- 2017-06-19: No change with the main instrument other than the IOL is slowly worsening as seen in the forementioned plots. There was an issue last Thursday night where ecamera (Slitviewer) was producing strange images. Likely due to a low level in the liquid cooling reservoir although that has not been determined as the actual cause. The reservoir was filled this morning. We will continue to monitor. Please report any weird performance issues to the observing specialists.
- 2017-06-12: IOL still dropping but it should not affect data according to Russet. The Interorderlight prior to 2014 was typically in the mid-teens and had little effect on science as far as was reported. The sudden drop in the measured value has caught our attention because it can change quite rapidly and we want to minimize the amount of time where the levels are down to the 'teens' in the red or blue. We will continue to monitor the IOL levels going into shutdown. The purpose of asking users to take 30 minute darks is to verify that there is no dark current present. So far even with the CCD running at -90C (-120C is the setpoint), we see neglible to no dark current.
Most recent interorder ratio plots can be found here.
- 2017-05-30: Still running the CCD heater at full power. The blue IOL continues to be in a slow lowering trend but far enough away from what we have determine to be detrimental to science. Advise users to take 30 minute darks at the end of their run if possible.
- 2017-05-18: The instrument is available but running at a warmer temperature to slow contamination forming. Recommend, minimum of 1 dark exposure @ at least 15 minutes duration, 2-3 dark exposures to remove cosmic ray removal desired.
- 2017-05-03: Last night the observing specialists reported lower than normal IOL measurements indicating evidence that contamination was forming on the or near the CCD. After regular measurements throughout the night showing the problem was degrading rapidly it was decided to take the instrument offline and start to warm the camera. The instrument will be offline for another week to mitgate this disappointing development after servicing.
- 2017-05-01: Vacuum servicing complete. Ion pump was changed as part of the servicing work. IOL measurements look nominal after the CCD reached normal operating temperatures.
- 2017-04-19: Echelle is undergoing vacuum servicing. It is unavailable until May 1. This work includes installing a new ion pump.
- 2017-03-03: Most recent interorder ratio plot can be found here.
- 2017-06-19: The instrument temperature has purposesly been raised to alleviate a 2 hour recovery time from frozen preamp boards and the dropped quadrant problem on instrument resets. Because of this dark current is noticable. The dark current scales linearly with exposure times for exposures longer than 10 seconds and can be removed due to traditional methods. Users should be taking darks. For short exposures shorter than 10 seconds, seek the advice of Russet McMillan.
- 2017-06-12: Investigating increased dark current, new preamps that might be able to run colder, and image transformations based on amplifier choosen. More on this later.
- 2017-03-11: Diffuser installed (stationary only)
- 2017-02-23: Replaced the filterwheel hall effect sensor which was damaged when too long of screws were inserted in the filterwheel. Downtime minimal (less than one half night).
- 2017-02-16: After some experiments running the detector colder to increase performance it was decided that running it at 186K was a more proper setpoint to minimize downtime on loss of a dropped quadrant if an init needs to be preformed.
- 2017-02-10: More problems with the diffuser. Not spinning properly. HAve decided to ship it to UW to see if Jason Lozo can identify any troubles. It is not available until further notice.
- 2017-02-02: The diffuser is operational and has been tested extensively. We have implemented improvements to hardware and software to make it somewhat more reliable.
- 2017-01-15: The cold strap removal was successful and the instrument is running stabily at 186K.
- 2017-09-18: Russet reports that there is a rapid rise in contamination on both cameras since the servicing. Users are asked to report any effect to their science to the observatory staff.
- 2017-09-13: The instrument is once again available for use. The instrument servicing went well. We added 0.010" shims to the front of both CCDs to bring the focus adjustments closer to the middle of its range of travel. We replaced the ion pump on the blue camera. We installed the new red field flattener lens holder. We replaced all o-rings on both cameras and dewars. Inspected but id not clean the field lenses (they both appeared spotless). Verified the full range of the rotary micrometers for focus on each camera for documentation purposes. Last night Russet was able to redo the instrument blocks (x3) on the instrument as well as measuring scattered light and getting throughput data.
- 2017-09-01: DIS is unavailable. It is being warmed over the weekend in preparation for vacuum and contamination servicing next week. It is scheduled to return on Sept 12.
- 2017-08-22: UPDATE: The red ion pump still is having periodic dropouts but once restarted the vacuum recovers rather quickly. New High Voltage cables are on order but have a long lead time for mfg. As long as vacuum remains good the ccd temp will not raise effecting darks and science.
- 2017-07-12: Ion pump is still dropping out. Track problem to high voltage cable. Parts will be on order.
- 2017-07-11: DIS is back up.
- 2017-07-10: Cooling red camera. Will verify ion pump operation and vacuum stability before released back for science.
- 2017-07-08: On June 21, we had a failure of the ion pump on DIS red. The vacuum gauge and ion pumps were incorrectly reading good vacuums at this time and the cause went unnoticed, attributed to a problem with warm ambient temperatures or the compressors on the cryotigers. Gas pressure was checked and adjusted multiple times without any affect. It was guessed that there likely was a partial blockage of the capillies of the cold head. Plans to warm the camera and flush the cold head was made. While setting up for this procedure the ion pump failure was discovered. The camera was put on the vacuum pump and reached excellent pressures in a few hours and the camera temps dropped 20K. Unfortunately once the vacuum valve was closed pressures did not hold. Even with pressures in the 10^-2 Torr the ion pump was reading 10^-7. This morning I changed out the ion pump and put the camera back on the vacuum pump. Plans to check for a vacuum leak and cool early the coming week are in place.
- 2017-06-22: Due to very warm ambient temperatures the cryocoolers that chill the CCDs are failing to maintain temperatures. Suggest users monitor the dark current.
- 2017-06-19: Contamination is increasing at the same rate as this time last year. Users should expect to see light scattering from the main psf into the wings of emmision line spectra. Servicing of the instrument is expected in September of 2017. For more information speak with Russet McMillan.
- 2017-04-17: Engineering data last week shows the contamination is holding steady to only a slight increase on both cameras.
- 2017-02-27: Russet took some additional pinhole data over the weekend. No big changes even though the ion pump for Red appears to have been working harder over the past week. Only a marginal addition in the contamination.
- 2017-01:30: Ran several tests of opening and closing the turret bay doors with internal metrology. No indication that the contamination got any worse from these tests.
- 2016-11-18: Russet reports that the contamination is still present in pinhole spectra however nearly unmeasurable with slit spectra.
- 2016-11-01: Russet reports that the red camera is just starting to show a halo around line spectra. It is still too weak to be measurable but it is starting. It became visible on October 30 but not on October 29th
- 2017-07-10: The instrument was used last night in many different orientations. This morning when it was dismunted from the telescope the ion pump was reading high load and vacuum appeared to be soft (7e-6T). Tapping on the ion pump itself brought the controller off High Load and within 30 - 40 minutes vacuum readings returned to nominal values.
- 2017-07-08: The instrument has been operating nominally.
- 2017-06-19: The instrument has been operating nominally.
- 2017-04-17: There were three reported instances of the instrument needing resets over the weekend due to corrupted images. This may be note worthy because it has been very rare since 2012 that this problem has occured. I want to aanotate this as the time this has happened. However it is also noteworthy that Nicfps use has made gone up since triplespec has been unavailable for science and we are just seeing more of these occurances as a result rather than there being a sudden increase in their frequency. We will continue to monitor them.
- 2013-01-17: The instrument is currently not usable for Fowler Sampling modes > 8. NFS 8 and below still work fine.
- 2012-01-24: NICFPS still experiences intermittent failures since the fall of 2010. Please coordinate with the observing specialist for monitoring of failures and recovery modes as you take data. More information can be found here.
- 2017-08-22: Instrument is operating nominally
- 2017-07-10: On sky engineering this weekend was brief but was enough to release the instrument for science use. Focus had changed a small amount but possibly for the better. It will be able to return to science for scheduled observations this Wednesday.
- 2017-07-08: The instrument was cooled down this past week and is taking good images. On sky engineering tonight to determine if array focus and tip-tilt need adjustments before returning to science.
- 2017-06-22: circuit boards installed in the aux box. Instrument telemetry verified however it is still not displayable in TUI. Instrument will remain on the vacuum pump while a baseline of temperature tlemetry is maintained.
- 2017-06-21: Some delay in checking out the affected circuit boards in the Aux Box. This is delaying cooling down the instrument because we desire working temperature metrology before adding LN2.
- 2017-06-19: Instrument is on the vacuum pump on the observing floor. There were issues recabling it that required ordering parts. Hopefully cooldown later this week and on sky testing on the 24th.
- 2017-06-12: Brought instrument up to the observing floor for recabling and have on the vacuum pump for a few days.
- 2017-05-30: On the vacuum pump in the lab.
- 2017-05-26: Instrument is on the vacuum pump in the lab.
- 2017-05-19: UVa team left for what they feel is a successful trip. The found an issue with one of the clock lines leaving the leach board. Corrected. Found and corrected the slit wheel rotation issue with the stepper motor control. Found and corrected several cracked and damaged o-rings. Upgraded the temperature metrology boards and power supplies. They did leave with a few parts on order and uncompleted items for the APO staff to finish. Waiting on those items before we can start pumping on the dewar.
- 2017-04-17: UVa team is scheduled to service the instrument May 15, 2017.
- 2017-30-20: The tcamera has reached ambient temperature (290K), the rest of the dewar and the spectragram camera (speccam) is still warming and are a few degrees about freezing (274.3K). Power was removed from the Aux Tank resister this past Friday. The instrument will most likely be lowered off the floor the middle of May so the UVa team can investigate for the cause of the problem.
- 2017-03-13: Instrument is still warming.
- 2017-03-03: The thermal cycle was not successful in correcting the problem. We have started warming the instrument today. THe UVa team are currently planning a trip to work on the instrument. No dates have been decided but it will likely not be until mid May or mid June of 2017. Until then the instrument is warm and offline.
- 2017-02-24: Slitviewer and spectragraph cameras are both below 90K. Powered up. The slitviewer camera looks good after the first exposures. The spectragraph camera initial images still showed the same corrupted 'L' pattern. Tried a few resets to see if it would come back. On the 4th try the detector went blank and started producing zero's for all pixels. Tried one additional time, no change, all zeros. Informed UVa team. Will wait for their input before proceeding further.
- 2017-02-23: Cooled main tank. Will try powering up detectors when they cool to below 90K
- 2017-02-02: Started warming Triplespec for a thermal cycle. The hopes is this might be a possible correction for the failed state of the instrument.
- 2017-09-13: The software upgrade is complete and the version has been tagged.
- 2017-08-22: The guider focus mechanism software needed an upgrade due to some other communications hardware upgrades over the shutdown. It appears to be behaving since the rewrite but may require some additional testing. The hub needs to run in test mode until all the bugs are worked out and this version can be tagged.
- 2017-06-19: There was an issue of increased dark current and inability to maintain operating temperature due to high ambient temperatures and the need to turn off the axillary cooling fan for a visiing instrumet. That fan is back on and cooling seems to be back in normal range.
The information on this page is up to date as of September 13, 2017. This page is maintained by the 3.5-m Telescope Engineer .