Totally unresponsive server

[Guest Cabbagepatchkid]

Hi everyone,

 

I'm out of ideas so I hope someone can give me an idea. I'd appreciate any

help.

 

I have a small server hosting services for 3 people in a home office. Specs

are:

Acer Altos G520 server

4GB RAM

500GB effective storage via SATA disks running RAID 5

Windows 2003 Server x64 Standard Edition Service Pack 2

Exchange Server 2007

SQL Server 2005

Symantec Mail Security 6.0.0.177

Symantec Corporate Edition 10.1.5

 

 

About 4 weeks ago the server started behaving erratically overnight. Some

nights, it was OK. On other nights, it would totally freeze and become

unresponsive. The server fails to respond to:

- any RPC call (Outlook, SQL, Comp Managemment)

- DNS,

- FTP

- console is totally frozen

 

Interestingly it continues to respond to PING !! The only way to resolve

the issue is to reboot the server (cold start).

 

I checked the logs and found (for example):

 

EVENTLOG - 6008 - "The previous system shutdown at 3:49:00 AM on 23/09/2007

was unexpected." The log entries show the server can fail anytime from 1am

through 9am, but has never failed outside those times (ie: never frozen

during the day).

 

This indicated to me some process was involved that ran overnight, but I

have been unable to resolve the issue.

 

I have checked:

- RAM test with memtest and Windows Diag CD. Clean

- RAID integrity and disk (media) errors. Clean

- Windows patches - patched fully

- NO windows patches were installed in the month before the issue began.

Updates only installed AFTER instability in an attempt to fix it.

- rebooting server in case it is a memory leak however the server crashed 1

hour after rebooting

 

 

--

Regards,

Mark

[Guest Pegasus \(MVP\)]

Re: Totally unresponsive server

 

 

"Cabbagepatchkid" <Cabbagepatchkid@discussions.microsoft.com> wrote in

message news:5DBB387A-5322-49E8-A3DA-21FD733C1F28@microsoft.com...

> Hi everyone,

>

> I'm out of ideas so I hope someone can give me an idea. I'd appreciate

> any

> help.

>

> I have a small server hosting services for 3 people in a home office.

> Specs

> are:

> Acer Altos G520 server

> 4GB RAM

> 500GB effective storage via SATA disks running RAID 5

> Windows 2003 Server x64 Standard Edition Service Pack 2

> Exchange Server 2007

> SQL Server 2005

> Symantec Mail Security 6.0.0.177

> Symantec Corporate Edition 10.1.5

>

>

> About 4 weeks ago the server started behaving erratically overnight. Some

> nights, it was OK. On other nights, it would totally freeze and become

> unresponsive. The server fails to respond to:

> - any RPC call (Outlook, SQL, Comp Managemment)

> - DNS,

> - FTP

> - console is totally frozen

>

> Interestingly it continues to respond to PING !! The only way to resolve

> the issue is to reboot the server (cold start).

>

> I checked the logs and found (for example):

>

> EVENTLOG - 6008 - "The previous system shutdown at 3:49:00 AM on

> 23/09/2007

> was unexpected." The log entries show the server can fail anytime from

> 1am

> through 9am, but has never failed outside those times (ie: never frozen

> during the day).

>

> This indicated to me some process was involved that ran overnight, but I

> have been unable to resolve the issue.

>

> I have checked:

> - RAM test with memtest and Windows Diag CD. Clean

> - RAID integrity and disk (media) errors. Clean

> - Windows patches - patched fully

> - NO windows patches were installed in the month before the issue began.

> Updates only installed AFTER instability in an attempt to fix it.

> - rebooting server in case it is a memory leak however the server crashed

> 1

> hour after rebooting

>

>

> --

> Regards,

> Mark

>

 

You must narrow down the time when the server fails so that

you can relate it to the task that causes the problem. Use the

Task Scheduler to run this batch file once every five minutes:

 

@echo off

if not exist c:\Logs md c:\Logs

echo %date% %time% >> c:\Logs\Heartbeat.txt

 

Next time the server fails, check what other task might start

at this time, possibly invoked by the Task Scheduler.

 

It is also possible that you have a memory leak. See if rebooting

the server every night at 5am (for example) will prevent further

seizures.

[Guest Cabbagepatchkid]

Re: Totally unresponsive server

 

Hi Pegasus,

 

The system log clearly already shows the time it was last responsive in the

"eventlog" entries, and these actual entries are anywhere in the timeframe

mentioned. ie: it has crashed anywhere between those times. I know this

because I work late sometimes and my email no longer responds.

 

Further, I made the comment the server crashed 1 hour after being rebooted

hence that would be one massive memory leak, so I have discounted that at

this stage.

 

 

--

Regards,

Mark

 

 

 

"Pegasus (MVP)" wrote:

>

> "Cabbagepatchkid" <Cabbagepatchkid@discussions.microsoft.com> wrote in

> message news:5DBB387A-5322-49E8-A3DA-21FD733C1F28@microsoft.com...

> > Hi everyone,

> >

> > I'm out of ideas so I hope someone can give me an idea. I'd appreciate

> > any

> > help.

> >

> > I have a small server hosting services for 3 people in a home office.

> > Specs

> > are:

> > Acer Altos G520 server

> > 4GB RAM

> > 500GB effective storage via SATA disks running RAID 5

> > Windows 2003 Server x64 Standard Edition Service Pack 2

> > Exchange Server 2007

> > SQL Server 2005

> > Symantec Mail Security 6.0.0.177

> > Symantec Corporate Edition 10.1.5

> >

> >

> > About 4 weeks ago the server started behaving erratically overnight. Some

> > nights, it was OK. On other nights, it would totally freeze and become

> > unresponsive. The server fails to respond to:

> > - any RPC call (Outlook, SQL, Comp Managemment)

> > - DNS,

> > - FTP

> > - console is totally frozen

> >

> > Interestingly it continues to respond to PING !! The only way to resolve

> > the issue is to reboot the server (cold start).

> >

> > I checked the logs and found (for example):

> >

> > EVENTLOG - 6008 - "The previous system shutdown at 3:49:00 AM on

> > 23/09/2007

> > was unexpected." The log entries show the server can fail anytime from

> > 1am

> > through 9am, but has never failed outside those times (ie: never frozen

> > during the day).

> >

> > This indicated to me some process was involved that ran overnight, but I

> > have been unable to resolve the issue.

> >

> > I have checked:

> > - RAM test with memtest and Windows Diag CD. Clean

> > - RAID integrity and disk (media) errors. Clean

> > - Windows patches - patched fully

> > - NO windows patches were installed in the month before the issue began.

> > Updates only installed AFTER instability in an attempt to fix it.

> > - rebooting server in case it is a memory leak however the server crashed

> > 1

> > hour after rebooting

> >

> >

> > --

> > Regards,

> > Mark

> >

>

> You must narrow down the time when the server fails so that

> you can relate it to the task that causes the problem. Use the

> Task Scheduler to run this batch file once every five minutes:

>

> @echo off

> if not exist c:\Logs md c:\Logs

> echo %date% %time% >> c:\Logs\Heartbeat.txt

>

> Next time the server fails, check what other task might start

> at this time, possibly invoked by the Task Scheduler.

>

> It is also possible that you have a memory leak. See if rebooting

> the server every night at 5am (for example) will prevent further

> seizures.

>

>

>

[Guest Leythos]

Re: Totally unresponsive server

 

In article <5DBB387A-5322-49E8-A3DA-21FD733C1F28@microsoft.com>,

Cabbagepatchkid@discussions.microsoft.com says...

> Symantec Corporate Edition 10.1.5

>

>

> About 4 weeks ago the server started behaving erratically overnight. Some

> nights, it was OK. On other nights, it would totally freeze and become

> unresponsive. The server fails to respond to:

> - any RPC call (Outlook, SQL, Comp Managemment)

> - DNS,

> - FTP

> - console is totally frozen

>

> Interestingly it continues to respond to PING !! The only way to resolve

> the issue is to reboot the server (cold start).

 

 

I've seen this several times, about 20 different servers. It's been

Symantec Corp for W/S 10.2 (shows 10.1.5 on 2003/XP machines) each time.

Uninstall 10.2 from your server, install 10.0 or 10.1 and updates, no

problems after that.

 

In about 50% of cases, this started, in hindsight, as problems loading

profiles, then server lockups. Could be 1 hour or 2 days, but since 10.2

(10.1.5) was removed and 10.0/10.1 installed it's not happened again.

 

--

 

Leythos

- Igitur qui desiderat pacem, praeparet bellum.

- Calling an illegal alien an "undocumented worker" is like calling a

drug dealer an "unlicensed pharmacist"

spam999free@rrohio.com (remove 999 for proper email address)

[Guest Cabbagepatchkid]

Re: Totally unresponsive server

 

Thanks Leythos I'll certainly give that a try.

 

Overnight I disabled Symantec Mail Security background scanning of the store

however several hours later it crashed anyway.

 

I rebooted and turned off Windows Updates totally as I read some articles

describing how the process can get locked and freeze a server or PC.

Previously it was on NOTIFY only, but that would still involve the server

checking itself.

 

I also installed a hotfix from Microsoft 932762. While not specific to my

issue, I read a thread somewhere that mentioned it helped.

 

As soon as the server crashes next I will know the changes above have not

helped, so I will follow your recommendation to uninstall SAV CE 10.2

--

Regards,

Mark

 

 

 

"Leythos" wrote:

> In article <5DBB387A-5322-49E8-A3DA-21FD733C1F28@microsoft.com>,

> Cabbagepatchkid@discussions.microsoft.com says...

> > Symantec Corporate Edition 10.1.5

> >

> >

> > About 4 weeks ago the server started behaving erratically overnight. Some

> > nights, it was OK. On other nights, it would totally freeze and become

> > unresponsive. The server fails to respond to:

> > - any RPC call (Outlook, SQL, Comp Managemment)

> > - DNS,

> > - FTP

> > - console is totally frozen

> >

> > Interestingly it continues to respond to PING !! The only way to resolve

> > the issue is to reboot the server (cold start).

>

>

> I've seen this several times, about 20 different servers. It's been

> Symantec Corp for W/S 10.2 (shows 10.1.5 on 2003/XP machines) each time.

> Uninstall 10.2 from your server, install 10.0 or 10.1 and updates, no

> problems after that.

>

> In about 50% of cases, this started, in hindsight, as problems loading

> profiles, then server lockups. Could be 1 hour or 2 days, but since 10.2

> (10.1.5) was removed and 10.0/10.1 installed it's not happened again.

>

> --

>

> Leythos

> - Igitur qui desiderat pacem, praeparet bellum.

> - Calling an illegal alien an "undocumented worker" is like calling a

> drug dealer an "unlicensed pharmacist"

> spam999free@rrohio.com (remove 999 for proper email address)

>

[Guest Leythos]

Re: Totally unresponsive server

 

In article <CE7BB18A-9972-4A9E-BD3B-85CCCEC04499@microsoft.com>,

Cabbagepatchkid@discussions.microsoft.com says...

> Thanks Leythos I'll certainly give that a try.

>

> Overnight I disabled Symantec Mail Security background scanning of the store

> however several hours later it crashed anyway.

>

> I rebooted and turned off Windows Updates totally as I read some articles

> describing how the process can get locked and freeze a server or PC.

> Previously it was on NOTIFY only, but that would still involve the server

> checking itself.

>

> I also installed a hotfix from Microsoft 932762. While not specific to my

> issue, I read a thread somewhere that mentioned it helped.

>

> As soon as the server crashes next I will know the changes above have not

> helped, so I will follow your recommendation to uninstall SAV CE 10.2

 

SMSE didn't cause the problem on any of the servers, although we've only

got it still installed on 2, we've moved to GFI Mail Security and GFI

Mail Essentials for all Exchange/SBS servers and it's proven to be a LOT

better for all customers than SMSE even thought of being.

 

I tried everything, even the things you tried, and spent about 3 weeks

trying to get to the bottom of it - a last ditch guess was the fix.

 

--

 

Leythos

- Igitur qui desiderat pacem, praeparet bellum.

- Calling an illegal alien an "undocumented worker" is like calling a

drug dealer an "unlicensed pharmacist"

spam999free@rrohio.com (remove 999 for proper email address)