From: stephanie snedden
Subject: Re: 2.5m obslog 20051107 (MJD 53682)
Date: Tue, 8 Nov 2005 11:23:40 -0700
We usually use start_servers -K. However, this hasn't been working
lately because of the UDP port problem. We've been doing
start_servers -k and waiting for the spectro UDP port to time out.
That's why we were using the cron job autostart.
Steph
On Nov 8, 2005, at 8:27 AM, Craig Loomis wrote:
>>
>>  We killed all servers, and waited for 20 min. But servers did not
>> come
>> back (they are supposed to do so within 15 min),
>
>   I confess that this one is my fault: I had disabled the 15-minute
> cron job, and had left it disabled. The I was under the mistaken
> impression that you were manually restarting with the iop
> "start_servers" script in any case. [Do you really wait up to 15
> minutes for the servers to restart if they fail in the night?]
>
>> then we forced servers
>> to restart, spectro server still did not come back (with warning:
>> port 30307
>> appears to be in use; window may die) while the other two did.
>> Then we
>> rebooted the spectro crates, but servers and spectroscopic display
>> 1 &2 never come back, and restarted servers also failed. Then, we
>> tried to
>> take bias without grabInst, failed. Second attempt on rebooting
>> the crates
>> successfully got sectroscopic displays back but servers still did
>> not.
>> Without servers, we were able to take bias. Then Dan managed to
>> taking data
>> without relay server. About 50 min was lost due to the problem.
>>
>>   At about 03:40Z, servers restarted. 15 min ago, steph killed all
>> servers,
>> then she restarted servers by "start_servers -n -i -s". This time,
>> all
>> servers restarted. Since we called Kurt about this problem and he
>> might
>> work on it at the moment. It is possible that Kurt did something
>> so that
>> Steph could restart the servers. Later, Kurt called back and told
>> us that
>> he issued several commands to see what were going on about servers
>> but
>> he did not tried to restarted the servers.
>
>   There are two scripts currently in use: start_servers, which is
> part of iop, and startServers, which is a wrapper around
> start_servers and lives in ~observer/bin.  I will strongly urge you
> to use start_servers directly in order to remove one level of
> confusion and for you to get a better sense of what is actually
> happening. Note that the "servers" in this case are _only_ the
> three iop/watcher servers.
>   By the end of the afternoon, we should have a prescription for
> properly (re-)starting the three iop/watcher servers. I could guess
> now, but should test first.
>
>  - craig