}

Agile Procedures

Contents

Startup, Shutdown and Reset

Recommended Procedure

To reduce the danger of corrupt images and failed exposure sequences I recommend the following procedure for every night that Agile is to be used. Do this least an hour before the camera is to be used (to give the camera time to cool):

Start the ICC

  • Log into nimble as user ccd.
  • Execute the following (if log.txt exists then please rename the old log):
    • runAgileICC start
  • To connect or reconnect the hub to AgileICC, issue these command to the hub:
    • hub startNubs agile
    • hub startNubs agileExpose
  • The ICC listens for TCP connections on port 1025. It will accept multiple connections (one of them the hub). So you can talk directly the Agile ICC by typing the following from newton, nimble or another suitable computer:
    • telnet nimble 1025
  • Note that AgileICC is not responsive to commands while it is busy connecting to the camera.

If runAgileICC start fails, here are some things to check from your login on nimble:

  • If it fails with a complaint that the port is in use, then another copy of AgileICC is already running. You can either use the ICC that is already running or else stop the existing ICC and then try again.
  • If runAgileICC fails with error "couldn't connect to display" then start Xvfb using the following (-fp /usr... avoids a warning message) and then try again:
    • Xvfb :1 -fp /usr/X11R6/lib/X11/fonts/misc &
  • Make sure $DISPLAY is set to 0:1 (this should never be a problem for the ccd acount).
  • If runAgileICC fails with complaints about the python syntax or missing libraries then you are probably running the wrong python (this should never happen for the ccd account). Make sure your $PATH has /usr/local/bin before /usr/bin.

Stop the ICC

The best way to shut down the ICC is to issue ICC command shutdown.

If you prefer you can stop the ICC from a nimble login by issuing the command runAgileICC stop.

Disconnect/Connect the Camera

To disconnect from the camera (e.g. to reset the camera or allow aacq to use it), issue ICC command disconnDev camera. To connect to the camera again, issue ICC command connDev camera.

Note: AgileICC turns on the camera's thermoelectric cooler when it connect to the camera. The thermoelectric cooler then stays on until the camera is reset. or power cycled. (In theory, software can turn off cooling, but as of 2009-02-02 none of our software does this.)

Reset the Camera

If Agile starts writing corrupt images then you must reset the camera (and you may end up having to reboot nimble as well). To do this:

  • Make sure you know the password to Agile's power controller .Warning: if you issue the wrong password a few times then the power controller will lock out users for 2 minutes.
  • Disconnect the camera by issuing ICC command disconnDev camera.
  • Run the nimble command-line command resetAgile.py and enter the password for Agile's power controller when requested.
  • If Agile is really messed up then this will hang nimble. If that happens:
    • Power cycle both agile and nimble: log into newton as user agile and run the command-line command powerCycleAgile.py and then powerCycleNimble.py (in that order).
    • Once nimble reboots, start AgileICC again.
  • If agile reset without hanging nimble then reconnect the camera by issuing ICC command connDev.
  • If nimble hangs then see reboot nimble for instructions on how to power cycle it.

Reboot Nimble

Agile ICC runs on the computer "nimble". Sometimes it needs a reboot (the camera's usb driver seems to corrupt the kernel).

From the ccd account issue the command: reboot

If nimble hangs (as it sometimes does when the kernel is corrupted and you try to reboot it) then you must power cycle it. Log into agile@newton and run powerCycleNimble.py. You will need the password for crps01, the power controller nimble is plugged into. Or you can telnet to the power controller directly; nimble is on port 7.

Build and Install the Software

Check out the source code from its svn repository: svn://svn.apo.nmsu.edu/agile/ICC (subdirectory tags/tagname or trunk).

Read the ReadMe.txt file in the root level of the source code.

Known Issues

Corrupt Images

We have had serious problems with images being corrupted, typically by having a band of 0s along the top. The problem appears to be a bug in the camera control software from Princeton Instruments (one of many such bugs); see PVCam Gotchas for more information. We have tried to get Princeton Instruments to fix the problem but so far they have ignored all such requests. If you wish to have a go at it feel free to contact Princeton Instruments 609-587-9797. Rob Allen RAllen@piacton.com knows our system best but is not reliable about returning phone calls or responding to emails. The problem is described in detail in the Serious Bugs section of PVCam Gotchas.

Workaround: the ICC automatically disconnects from the camera and reconnects after every exposure sequence. This takes roughly 20 seconds and the ICC will be unresponsive for that time. This seems to dramatically reduce the frequency of the problem, but we're not yet sure if it is a complete cure.

Also, if an image is taken whose upper left pixel has value 0 the ICC will output keyword ImageIsBad=path and the FITs file will contain the keyword BADIMAGE. This is a crude test that could fail for several reasons: an image might actually end with 0 (false positive) or it might be corrupted in some other way (false negative).

We have seen two forms of corruption that this test would miss:

  • After a corrupt image that ends with a band of 0s, the next image may have a region with a strange background or gain, typically roughly the same pixels that were 0 in the previous image.
  • An image may be shifted (rolled horizontally and/or vertically). Fortunately this is rare.

Note: the ImageIsBad keyword is a temporary hack while we wait for a fix from Princeton Instruments. Please do not rely on it. It is intentionally not documented in the standard set of keywords below.

Overscan Issues

There are two things to be aware of regarding overscan:

  • For a usable bias region you must request more than 18 unbinned pixels of X overscan. The first 18 pixels are not usable for bias; we're not entirely sure what they are, but they may be masked pixels as they appear to include read noise as well.
  • We strongly recommend a Y overscan of 0. Anything else will corrupt data: if you request N rows of Y overscan, you will find the top N or so rows of your data are oddly offset.

In addition to Y overscan corrupting data, we're not sure what the pixels in the Y overscan region actually mean. The Y overscan regions we've seen do not look like bias regions. Perhaps this is because we've not read out enough rows to get to the bias region (like X overscan, that requires 18 unbinned columns to get to the bias region). Perhaps the fact that it is a frame transfer camera affects the Y overscan region.

Timing Errors

Agile ICC occasionally aborts an exposure sequence with a complaint about timing (sometimes when you first start the sequence, sometimes well into it). Our best guess at this point is that the kernel is getting corrupted and confusing the camera or the timing card.

The best bet is to reboot nimble.

Sending an ICC Command

To send an ICC Command to Agile ICC you need a connection, which you can get by logging into newton or nimble and typing: 

    telnet nimble 1025

If this fails then the ICC is probably not running; read Startup and Shutdown for instructions on starting the ICC.

If AgileICC is running and connected to the hub, then a simple way to send an ICC command is to type: 

    agile desired command

for example: 

    agile disconnDev camera

into the command bar along the bottom of TUI's Log window.