Jump to content

Trying to identify memory pool tags CM44, CMpa, and CMpb in Windows cluster.


Recommended Posts

Posted

We currently have a two node, Windows 2003 R2 Enterprise cluster (32 bit)

using shared SAN resources. Over the last several months we have

occasionally been getting a situation where the active node slows down and

becomes almost non-responsive. It doesn't quite stop, so the cluster

failover doesn't automatically happen. We've set the registry key (I don't

remember the KB article) that is supposed to cause the failover to happen

when the higher level Windows functions stop, but we've one case since then

where the failover didn't happen automatically. The event log entries seem

to indicate a loss of system resources, so I'm suspecting a memory leak in a

driver or program. About two months ago I updated all the drivers, BIOS,

hotfixes, etc. on the systems. So far we've had one case of a failover, but

that was related to a network issue, so I don't know if the updates took

care of the issue or not.

 

Since the update, I've been monitoring the system using the MemSnap utility

on a scheduled task a few times a day. I've been analyzing the results and

there are three "tags" that seem like they might be showing a memory leak.

I'm trying to find out what they are from. I've used the "memsnap /c"

function to build the system tag list, and these don't appear on the list,

so I'm assuming they're from a third party driver, and I'm hoping that

someone can give me some ideas.

 

The three tags in question are "CM44" (nonpaged), "CMpa" (nonpaged) and

"CMpb" (paged). Over the last several weeks they've slowly gone up in

memory usage. Currently they're at 398K, 796K, and 597K each. Memsnap

shows them increasing about 800 bytes/hour. Our current system has 4G of

RAM (I'm NOT using the /3gb switch) and shows about 92M non paged and 260M

paged in use.

 

I've done some internet searching, as well as Microsoft Technet, and I can't

find anything that would indicate where these are used. If anyone can give

me some information on them, I would appreciate it.

 

Mike O.

Guest Edwin vMierlo [MVP]
Posted

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Windows cluster.

 

did you search google ?

http://www.google.ie/search?hl=en&q=CM44+%2Btag&meta=

 

second hit will bring you here

https://forums.symantec.com/syment/board/message?board.id=103&message.id=37954

 

It is a discussion about the CMpa and CM44 tags, belonging to

"rxWriterSvc.exe"

 

Not sure if this is applicable for your situation

 

rgsd,

Edwin.

 

 

 

 

"Mike O" <put_the_spam@the.can> wrote in message

news:uz0nDGBIIHA.2064@TK2MSFTNGP06.phx.gbl...

> We currently have a two node, Windows 2003 R2 Enterprise cluster (32 bit)

> using shared SAN resources. Over the last several months we have

> occasionally been getting a situation where the active node slows down and

> becomes almost non-responsive. It doesn't quite stop, so the cluster

> failover doesn't automatically happen. We've set the registry key (I

don't

> remember the KB article) that is supposed to cause the failover to happen

> when the higher level Windows functions stop, but we've one case since

then

> where the failover didn't happen automatically. The event log entries

seem

> to indicate a loss of system resources, so I'm suspecting a memory leak in

a

> driver or program. About two months ago I updated all the drivers, BIOS,

> hotfixes, etc. on the systems. So far we've had one case of a failover,

but

> that was related to a network issue, so I don't know if the updates took

> care of the issue or not.

>

> Since the update, I've been monitoring the system using the MemSnap

utility

> on a scheduled task a few times a day. I've been analyzing the results

and

> there are three "tags" that seem like they might be showing a memory leak.

> I'm trying to find out what they are from. I've used the "memsnap /c"

> function to build the system tag list, and these don't appear on the list,

> so I'm assuming they're from a third party driver, and I'm hoping that

> someone can give me some ideas.

>

> The three tags in question are "CM44" (nonpaged), "CMpa" (nonpaged) and

> "CMpb" (paged). Over the last several weeks they've slowly gone up in

> memory usage. Currently they're at 398K, 796K, and 597K each. Memsnap

> shows them increasing about 800 bytes/hour. Our current system has 4G of

> RAM (I'm NOT using the /3gb switch) and shows about 92M non paged and 260M

> paged in use.

>

> I've done some internet searching, as well as Microsoft Technet, and I

can't

> find anything that would indicate where these are used. If anyone can

give

> me some information on them, I would appreciate it.

>

> Mike O.

>

Guest Mike O.
Posted

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Wi

 

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Wi

 

thank you for the response.

 

I had done some google searching, but I had used words like CMPB and

Memsnap, or CM44, etc. Most of what I found was either posts that had a

memsnap dump, or things like "Center for Molecular Physiology of the Brain

(CMPB)", or the "California Milk Processing Board". I'm sure those are all

useful sites, but didn't really help me..

 

I don't think I actually used the combination "CM44 and Tag".

 

However, I don't know if the information in that article applies to us. I

don't see the RxWriter service (we don't use backup Exec) on our server. We

do use the Symantec Netbackup Enterprise, but even though it's a Symantec

product, as I understand it the Netbackup product is very different "under

the hood" from BackupExec. I don't belive I've ever seen anything like the

"Continuous Protection Server" in the Netbackup software.

 

With all that being said, though, I am going to start looking at the

NetBackup software a little closer. Since Backup Exec seems to use all

three of those tags, and we are using another Symantec/Veritas product, it

does seem very coincidental that those three tags are showing up on our

system. Possibly after Veritas acquired those products there was some

crossover of the programming teams on the software as Veritas (and later

Symantec) modified the programs.

 

thanks again.

Mike O.

 

 

"Edwin vMierlo [MVP]" wrote:

> did you search google ?

> http://www.google.ie/search?hl=en&q=CM44+%2Btag&meta=

>

> second hit will bring you here

> https://forums.symantec.com/syment/board/message?board.id=103&message.id=37954

>

> It is a discussion about the CMpa and CM44 tags, belonging to

> "rxWriterSvc.exe"

>

> Not sure if this is applicable for your situation

>

> rgsd,

> Edwin.

>

>

>

>

> "Mike O" <put_the_spam@the.can> wrote in message

> news:uz0nDGBIIHA.2064@TK2MSFTNGP06.phx.gbl...

> > We currently have a two node, Windows 2003 R2 Enterprise cluster (32 bit)

> > using shared SAN resources. Over the last several months we have

> > occasionally been getting a situation where the active node slows down and

> > becomes almost non-responsive. It doesn't quite stop, so the cluster

> > failover doesn't automatically happen. We've set the registry key (I

> don't

> > remember the KB article) that is supposed to cause the failover to happen

> > when the higher level Windows functions stop, but we've one case since

> then

> > where the failover didn't happen automatically. The event log entries

> seem

> > to indicate a loss of system resources, so I'm suspecting a memory leak in

> a

> > driver or program. About two months ago I updated all the drivers, BIOS,

> > hotfixes, etc. on the systems. So far we've had one case of a failover,

> but

> > that was related to a network issue, so I don't know if the updates took

> > care of the issue or not.

> >

> > Since the update, I've been monitoring the system using the MemSnap

> utility

> > on a scheduled task a few times a day. I've been analyzing the results

> and

> > there are three "tags" that seem like they might be showing a memory leak.

> > I'm trying to find out what they are from. I've used the "memsnap /c"

> > function to build the system tag list, and these don't appear on the list,

> > so I'm assuming they're from a third party driver, and I'm hoping that

> > someone can give me some ideas.

> >

> > The three tags in question are "CM44" (nonpaged), "CMpa" (nonpaged) and

> > "CMpb" (paged). Over the last several weeks they've slowly gone up in

> > memory usage. Currently they're at 398K, 796K, and 597K each. Memsnap

> > shows them increasing about 800 bytes/hour. Our current system has 4G of

> > RAM (I'm NOT using the /3gb switch) and shows about 92M non paged and 260M

> > paged in use.

> >

> > I've done some internet searching, as well as Microsoft Technet, and I

> can't

> > find anything that would indicate where these are used. If anyone can

> give

> > me some information on them, I would appreciate it.

> >

> > Mike O.

> >

>

>

>

Guest Mike O.
Posted

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Wi

 

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Wi

 

I have some additional information:

 

I went to another server used for testing. Using PoolMon, I verifed that

those three memory tags existed. I then set the Symantec/Veritas Netbackup

client service to disabled and rebooted the server. Those three tags still

exist, even with the NetBackup client not activated.

 

I then went into the services and turned off several HP monitoring utility

agents, and again rebooted. The memory tags still were showing up in the

list.

 

So now I'm back where I started. Does anyone have any suggestions on how I

can find out what those three tags are from?

 

Mike O.

 

 

"Mike O." wrote:

> thank you for the response.

>

> I had done some google searching, but I had used words like CMPB and

> Memsnap, or CM44, etc. Most of what I found was either posts that had a

> memsnap dump, or things like "Center for Molecular Physiology of the Brain

> (CMPB)", or the "California Milk Processing Board". I'm sure those are all

> useful sites, but didn't really help me..

>

> I don't think I actually used the combination "CM44 and Tag".

>

> However, I don't know if the information in that article applies to us. I

> don't see the RxWriter service (we don't use backup Exec) on our server. We

> do use the Symantec Netbackup Enterprise, but even though it's a Symantec

> product, as I understand it the Netbackup product is very different "under

> the hood" from BackupExec. I don't belive I've ever seen anything like the

> "Continuous Protection Server" in the Netbackup software.

>

> With all that being said, though, I am going to start looking at the

> NetBackup software a little closer. Since Backup Exec seems to use all

> three of those tags, and we are using another Symantec/Veritas product, it

> does seem very coincidental that those three tags are showing up on our

> system. Possibly after Veritas acquired those products there was some

> crossover of the programming teams on the software as Veritas (and later

> Symantec) modified the programs.

>

> thanks again.

> Mike O.

>

>

> "Edwin vMierlo [MVP]" wrote:

>

> > did you search google ?

> > http://www.google.ie/search?hl=en&q=CM44+%2Btag&meta=

> >

> > second hit will bring you here

> > https://forums.symantec.com/syment/board/message?board.id=103&message.id=37954

> >

> > It is a discussion about the CMpa and CM44 tags, belonging to

> > "rxWriterSvc.exe"

> >

> > Not sure if this is applicable for your situation

> >

> > rgsd,

> > Edwin.

> >

> >

> >

> >

> > "Mike O" <put_the_spam@the.can> wrote in message

> > news:uz0nDGBIIHA.2064@TK2MSFTNGP06.phx.gbl...

> > > We currently have a two node, Windows 2003 R2 Enterprise cluster (32 bit)

> > > using shared SAN resources. Over the last several months we have

> > > occasionally been getting a situation where the active node slows down and

> > > becomes almost non-responsive. It doesn't quite stop, so the cluster

> > > failover doesn't automatically happen. We've set the registry key (I

> > don't

> > > remember the KB article) that is supposed to cause the failover to happen

> > > when the higher level Windows functions stop, but we've one case since

> > then

> > > where the failover didn't happen automatically. The event log entries

> > seem

> > > to indicate a loss of system resources, so I'm suspecting a memory leak in

> > a

> > > driver or program. About two months ago I updated all the drivers, BIOS,

> > > hotfixes, etc. on the systems. So far we've had one case of a failover,

> > but

> > > that was related to a network issue, so I don't know if the updates took

> > > care of the issue or not.

> > >

> > > Since the update, I've been monitoring the system using the MemSnap

> > utility

> > > on a scheduled task a few times a day. I've been analyzing the results

> > and

> > > there are three "tags" that seem like they might be showing a memory leak.

> > > I'm trying to find out what they are from. I've used the "memsnap /c"

> > > function to build the system tag list, and these don't appear on the list,

> > > so I'm assuming they're from a third party driver, and I'm hoping that

> > > someone can give me some ideas.

> > >

> > > The three tags in question are "CM44" (nonpaged), "CMpa" (nonpaged) and

> > > "CMpb" (paged). Over the last several weeks they've slowly gone up in

> > > memory usage. Currently they're at 398K, 796K, and 597K each. Memsnap

> > > shows them increasing about 800 bytes/hour. Our current system has 4G of

> > > RAM (I'm NOT using the /3gb switch) and shows about 92M non paged and 260M

> > > paged in use.

> > >

> > > I've done some internet searching, as well as Microsoft Technet, and I

> > can't

> > > find anything that would indicate where these are used. If anyone can

> > give

> > > me some information on them, I would appreciate it.

> > >

> > > Mike O.

> > >

> >

> >

> >

Guest John Fullbright
Posted

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Windows cluster.

 

Doesn't CM44 belong to rxwritersvc.exe? You should redirect this question

to Symantec.

 

 

"Mike O" <put_the_spam@the.can> wrote in message

news:uz0nDGBIIHA.2064@TK2MSFTNGP06.phx.gbl...

> We currently have a two node, Windows 2003 R2 Enterprise cluster (32 bit)

> using shared SAN resources. Over the last several months we have

> occasionally been getting a situation where the active node slows down and

> becomes almost non-responsive. It doesn't quite stop, so the cluster

> failover doesn't automatically happen. We've set the registry key (I

> don't remember the KB article) that is supposed to cause the failover to

> happen when the higher level Windows functions stop, but we've one case

> since then where the failover didn't happen automatically. The event log

> entries seem to indicate a loss of system resources, so I'm suspecting a

> memory leak in a driver or program. About two months ago I updated all

> the drivers, BIOS, hotfixes, etc. on the systems. So far we've had one

> case of a failover, but that was related to a network issue, so I don't

> know if the updates took care of the issue or not.

>

> Since the update, I've been monitoring the system using the MemSnap

> utility on a scheduled task a few times a day. I've been analyzing the

> results and there are three "tags" that seem like they might be showing a

> memory leak. I'm trying to find out what they are from. I've used the

> "memsnap /c" function to build the system tag list, and these don't appear

> on the list, so I'm assuming they're from a third party driver, and I'm

> hoping that someone can give me some ideas.

>

> The three tags in question are "CM44" (nonpaged), "CMpa" (nonpaged) and

> "CMpb" (paged). Over the last several weeks they've slowly gone up in

> memory usage. Currently they're at 398K, 796K, and 597K each. Memsnap

> shows them increasing about 800 bytes/hour. Our current system has 4G of

> RAM (I'm NOT using the /3gb switch) and shows about 92M non paged and 260M

> paged in use.

>

> I've done some internet searching, as well as Microsoft Technet, and I

> can't find anything that would indicate where these are used. If anyone

> can give me some information on them, I would appreciate it.

>

> Mike O.

Guest John Campbell
Posted

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Wi

 

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Wi

 

Just search your drivers directory for any files containing that text should

give you a clue what driver it is.

Obviously some driver may not be in 'standard' place so you may have to

check out whereever your drivers are for files contain this text.

 

 

"Mike O." <MikeO@discussions.microsoft.com> wrote in message

news:20D6A1EC-674C-486A-BD98-8E8B0407CA00@microsoft.com...

>I have some additional information:

>

> I went to another server used for testing. Using PoolMon, I verifed that

> those three memory tags existed. I then set the Symantec/Veritas

> Netbackup

> client service to disabled and rebooted the server. Those three tags

> still

> exist, even with the NetBackup client not activated.

>

> I then went into the services and turned off several HP monitoring utility

> agents, and again rebooted. The memory tags still were showing up in the

> list.

>

> So now I'm back where I started. Does anyone have any suggestions on how

> I

> can find out what those three tags are from?

>

> Mike O.

>

>

> "Mike O." wrote:

>

>> thank you for the response.

>>

>> I had done some google searching, but I had used words like CMPB and

>> Memsnap, or CM44, etc. Most of what I found was either posts that had a

>> memsnap dump, or things like "Center for Molecular Physiology of the

>> Brain

>> (CMPB)", or the "California Milk Processing Board". I'm sure those are

>> all

>> useful sites, but didn't really help me..

>>

>> I don't think I actually used the combination "CM44 and Tag".

>>

>> However, I don't know if the information in that article applies to us.

>> I

>> don't see the RxWriter service (we don't use backup Exec) on our server.

>> We

>> do use the Symantec Netbackup Enterprise, but even though it's a Symantec

>> product, as I understand it the Netbackup product is very different

>> "under

>> the hood" from BackupExec. I don't belive I've ever seen anything like

>> the

>> "Continuous Protection Server" in the Netbackup software.

>>

>> With all that being said, though, I am going to start looking at the

>> NetBackup software a little closer. Since Backup Exec seems to use all

>> three of those tags, and we are using another Symantec/Veritas product,

>> it

>> does seem very coincidental that those three tags are showing up on our

>> system. Possibly after Veritas acquired those products there was some

>> crossover of the programming teams on the software as Veritas (and later

>> Symantec) modified the programs.

>>

>> thanks again.

>> Mike O.

>>

>>

>> "Edwin vMierlo [MVP]" wrote:

>>

>> > did you search google ?

>> > http://www.google.ie/search?hl=en&q=CM44+%2Btag&meta=

>> >

>> > second hit will bring you here

>> > https://forums.symantec.com/syment/board/message?board.id=103&message.id=37954

>> >

>> > It is a discussion about the CMpa and CM44 tags, belonging to

>> > "rxWriterSvc.exe"

>> >

>> > Not sure if this is applicable for your situation

>> >

>> > rgsd,

>> > Edwin.

>> >

>> >

>> >

>> >

>> > "Mike O" <put_the_spam@the.can> wrote in message

>> > news:uz0nDGBIIHA.2064@TK2MSFTNGP06.phx.gbl...

>> > > We currently have a two node, Windows 2003 R2 Enterprise cluster (32

>> > > bit)

>> > > using shared SAN resources. Over the last several months we have

>> > > occasionally been getting a situation where the active node slows

>> > > down and

>> > > becomes almost non-responsive. It doesn't quite stop, so the

>> > > cluster

>> > > failover doesn't automatically happen. We've set the registry key (I

>> > don't

>> > > remember the KB article) that is supposed to cause the failover to

>> > > happen

>> > > when the higher level Windows functions stop, but we've one case

>> > > since

>> > then

>> > > where the failover didn't happen automatically. The event log

>> > > entries

>> > seem

>> > > to indicate a loss of system resources, so I'm suspecting a memory

>> > > leak in

>> > a

>> > > driver or program. About two months ago I updated all the drivers,

>> > > BIOS,

>> > > hotfixes, etc. on the systems. So far we've had one case of a

>> > > failover,

>> > but

>> > > that was related to a network issue, so I don't know if the updates

>> > > took

>> > > care of the issue or not.

>> > >

>> > > Since the update, I've been monitoring the system using the MemSnap

>> > utility

>> > > on a scheduled task a few times a day. I've been analyzing the

>> > > results

>> > and

>> > > there are three "tags" that seem like they might be showing a memory

>> > > leak.

>> > > I'm trying to find out what they are from. I've used the "memsnap

>> > > /c"

>> > > function to build the system tag list, and these don't appear on the

>> > > list,

>> > > so I'm assuming they're from a third party driver, and I'm hoping

>> > > that

>> > > someone can give me some ideas.

>> > >

>> > > The three tags in question are "CM44" (nonpaged), "CMpa" (nonpaged)

>> > > and

>> > > "CMpb" (paged). Over the last several weeks they've slowly gone up

>> > > in

>> > > memory usage. Currently they're at 398K, 796K, and 597K each.

>> > > Memsnap

>> > > shows them increasing about 800 bytes/hour. Our current system has

>> > > 4G of

>> > > RAM (I'm NOT using the /3gb switch) and shows about 92M non paged and

>> > > 260M

>> > > paged in use.

>> > >

>> > > I've done some internet searching, as well as Microsoft Technet, and

>> > > I

>> > can't

>> > > find anything that would indicate where these are used. If anyone

>> > > can

>> > give

>> > > me some information on them, I would appreciate it.

>> > >

>> > > Mike O.

>> > >

>> >

>> >

>> >

Posted

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Windows cluster.

 

There was another reply that mentioned that tag as being part of

rxwritersvc. However, rxwritersvc is part of Backup Exec, we don't use that

on the our system. We do have NetBackup, another Veritas/Symantec product,

but I disabled the NetBackup services and rebooted the computer. All three

memory tags still showed up in memory.

 

 

"John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

news:OpxdqvXIIHA.4584@TK2MSFTNGP03.phx.gbl...

> Doesn't CM44 belong to rxwritersvc.exe? You should redirect this question

> to Symantec.

>

>

> "Mike O" <put_the_spam@the.can> wrote in message

> news:uz0nDGBIIHA.2064@TK2MSFTNGP06.phx.gbl...

>> We currently have a two node, Windows 2003 R2 Enterprise cluster (32 bit)

>> using shared SAN resources. Over the last several months we have

>> occasionally been getting a situation where the active node slows down

>> and becomes almost non-responsive. It doesn't quite stop, so the

>> cluster failover doesn't automatically happen. We've set the registry

>> key (I don't remember the KB article) that is supposed to cause the

>> failover to happen when the higher level Windows functions stop, but

>> we've one case since then where the failover didn't happen automatically.

>> The event log entries seem to indicate a loss of system resources, so I'm

>> suspecting a memory leak in a driver or program. About two months ago I

>> updated all the drivers, BIOS, hotfixes, etc. on the systems. So far

>> we've had one case of a failover, but that was related to a network

>> issue, so I don't know if the updates took care of the issue or not.

>>

>> Since the update, I've been monitoring the system using the MemSnap

>> utility on a scheduled task a few times a day. I've been analyzing the

>> results and there are three "tags" that seem like they might be showing a

>> memory leak. I'm trying to find out what they are from. I've used the

>> "memsnap /c" function to build the system tag list, and these don't

>> appear on the list, so I'm assuming they're from a third party driver,

>> and I'm hoping that someone can give me some ideas.

>>

>> The three tags in question are "CM44" (nonpaged), "CMpa" (nonpaged) and

>> "CMpb" (paged). Over the last several weeks they've slowly gone up in

>> memory usage. Currently they're at 398K, 796K, and 597K each. Memsnap

>> shows them increasing about 800 bytes/hour. Our current system has 4G of

>> RAM (I'm NOT using the /3gb switch) and shows about 92M non paged and

>> 260M paged in use.

>>

>> I've done some internet searching, as well as Microsoft Technet, and I

>> can't find anything that would indicate where these are used. If anyone

>> can give me some information on them, I would appreciate it.

>>

>> Mike O.

>

>

Posted

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Wi

 

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Wi

 

I did a quick windows "search" for all files on the c: drive that contain

that text, but it didn't find it. I know Windows search doesn't always find

text strings in binary files, so I'm going to search some more using some

other tools.

 

 

"John Campbell" <jc> wrote in message

news:%23HLSaZbIIHA.484@TK2MSFTNGP06.phx.gbl...

> Just search your drivers directory for any files containing that text

> should give you a clue what driver it is.

> Obviously some driver may not be in 'standard' place so you may have to

> check out whereever your drivers are for files contain this text.

>

>

> "Mike O." <MikeO@discussions.microsoft.com> wrote in message

> news:20D6A1EC-674C-486A-BD98-8E8B0407CA00@microsoft.com...

>>I have some additional information:

>>

>> I went to another server used for testing. Using PoolMon, I verifed that

>> those three memory tags existed. I then set the Symantec/Veritas

>> Netbackup

>> client service to disabled and rebooted the server. Those three tags

>> still

>> exist, even with the NetBackup client not activated.

>>

>> I then went into the services and turned off several HP monitoring

>> utility

>> agents, and again rebooted. The memory tags still were showing up in

>> the

>> list.

>>

>> So now I'm back where I started. Does anyone have any suggestions on how

>> I

>> can find out what those three tags are from?

>>

>> Mike O.

>>

>>

>> "Mike O." wrote:

>>

>>> thank you for the response.

>>>

>>> I had done some google searching, but I had used words like CMPB and

>>> Memsnap, or CM44, etc. Most of what I found was either posts that had a

>>> memsnap dump, or things like "Center for Molecular Physiology of the

>>> Brain

>>> (CMPB)", or the "California Milk Processing Board". I'm sure those are

>>> all

>>> useful sites, but didn't really help me..

>>>

>>> I don't think I actually used the combination "CM44 and Tag".

>>>

>>> However, I don't know if the information in that article applies to us.

>>> I

>>> don't see the RxWriter service (we don't use backup Exec) on our server.

>>> We

>>> do use the Symantec Netbackup Enterprise, but even though it's a

>>> Symantec

>>> product, as I understand it the Netbackup product is very different

>>> "under

>>> the hood" from BackupExec. I don't belive I've ever seen anything like

>>> the

>>> "Continuous Protection Server" in the Netbackup software.

>>>

>>> With all that being said, though, I am going to start looking at the

>>> NetBackup software a little closer. Since Backup Exec seems to use all

>>> three of those tags, and we are using another Symantec/Veritas product,

>>> it

>>> does seem very coincidental that those three tags are showing up on our

>>> system. Possibly after Veritas acquired those products there was some

>>> crossover of the programming teams on the software as Veritas (and later

>>> Symantec) modified the programs.

>>>

>>> thanks again.

>>> Mike O.

>>>

>>>

>>> "Edwin vMierlo [MVP]" wrote:

>>>

>>> > did you search google ?

>>> > http://www.google.ie/search?hl=en&q=CM44+%2Btag&meta=

>>> >

>>> > second hit will bring you here

>>> > https://forums.symantec.com/syment/board/message?board.id=103&message.id=37954

>>> >

>>> > It is a discussion about the CMpa and CM44 tags, belonging to

>>> > "rxWriterSvc.exe"

>>> >

>>> > Not sure if this is applicable for your situation

>>> >

>>> > rgsd,

>>> > Edwin.

>>> >

>>> >

>>> >

>>> >

>>> > "Mike O" <put_the_spam@the.can> wrote in message

>>> > news:uz0nDGBIIHA.2064@TK2MSFTNGP06.phx.gbl...

>>> > > We currently have a two node, Windows 2003 R2 Enterprise cluster (32

>>> > > bit)

>>> > > using shared SAN resources. Over the last several months we have

>>> > > occasionally been getting a situation where the active node slows

>>> > > down and

>>> > > becomes almost non-responsive. It doesn't quite stop, so the

>>> > > cluster

>>> > > failover doesn't automatically happen. We've set the registry key

>>> > > (I

>>> > don't

>>> > > remember the KB article) that is supposed to cause the failover to

>>> > > happen

>>> > > when the higher level Windows functions stop, but we've one case

>>> > > since

>>> > then

>>> > > where the failover didn't happen automatically. The event log

>>> > > entries

>>> > seem

>>> > > to indicate a loss of system resources, so I'm suspecting a memory

>>> > > leak in

>>> > a

>>> > > driver or program. About two months ago I updated all the drivers,

>>> > > BIOS,

>>> > > hotfixes, etc. on the systems. So far we've had one case of a

>>> > > failover,

>>> > but

>>> > > that was related to a network issue, so I don't know if the updates

>>> > > took

>>> > > care of the issue or not.

>>> > >

>>> > > Since the update, I've been monitoring the system using the MemSnap

>>> > utility

>>> > > on a scheduled task a few times a day. I've been analyzing the

>>> > > results

>>> > and

>>> > > there are three "tags" that seem like they might be showing a memory

>>> > > leak.

>>> > > I'm trying to find out what they are from. I've used the "memsnap

>>> > > /c"

>>> > > function to build the system tag list, and these don't appear on the

>>> > > list,

>>> > > so I'm assuming they're from a third party driver, and I'm hoping

>>> > > that

>>> > > someone can give me some ideas.

>>> > >

>>> > > The three tags in question are "CM44" (nonpaged), "CMpa" (nonpaged)

>>> > > and

>>> > > "CMpb" (paged). Over the last several weeks they've slowly gone

>>> > > up in

>>> > > memory usage. Currently they're at 398K, 796K, and 597K each.

>>> > > Memsnap

>>> > > shows them increasing about 800 bytes/hour. Our current system has

>>> > > 4G of

>>> > > RAM (I'm NOT using the /3gb switch) and shows about 92M non paged

>>> > > and 260M

>>> > > paged in use.

>>> > >

>>> > > I've done some internet searching, as well as Microsoft Technet, and

>>> > > I

>>> > can't

>>> > > find anything that would indicate where these are used. If anyone

>>> > > can

>>> > give

>>> > > me some information on them, I would appreciate it.

>>> > >

>>> > > Mike O.

>>> > >

>>> >

>>> >

>>> >

>

>

Guest Edwin vMierlo [MVP]
Posted

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Wi

 

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Wi

 

Disabling services and applications might not be disabling kernel drivers

 

have a look at device manager (show hidden devices) and see if there are any

symantec/HP drivers there which are loaded running

 

rgds,

edwin.

 

"Mike O." <MikeO@discussions.microsoft.com> wrote in message

news:20D6A1EC-674C-486A-BD98-8E8B0407CA00@microsoft.com...

> I have some additional information:

>

> I went to another server used for testing. Using PoolMon, I verifed that

> those three memory tags existed. I then set the Symantec/Veritas

Netbackup

> client service to disabled and rebooted the server. Those three tags

still

> exist, even with the NetBackup client not activated.

>

> I then went into the services and turned off several HP monitoring utility

> agents, and again rebooted. The memory tags still were showing up in the

> list.

>

> So now I'm back where I started. Does anyone have any suggestions on how

I

> can find out what those three tags are from?

>

> Mike O.

>

>

> "Mike O." wrote:

>

> > thank you for the response.

> >

> > I had done some google searching, but I had used words like CMPB and

> > Memsnap, or CM44, etc. Most of what I found was either posts that had a

> > memsnap dump, or things like "Center for Molecular Physiology of the

Brain

> > (CMPB)", or the "California Milk Processing Board". I'm sure those are

all

> > useful sites, but didn't really help me..

> >

> > I don't think I actually used the combination "CM44 and Tag".

> >

> > However, I don't know if the information in that article applies to us.

I

> > don't see the RxWriter service (we don't use backup Exec) on our server.

We

> > do use the Symantec Netbackup Enterprise, but even though it's a

Symantec

> > product, as I understand it the Netbackup product is very different

"under

> > the hood" from BackupExec. I don't belive I've ever seen anything like

the

> > "Continuous Protection Server" in the Netbackup software.

> >

> > With all that being said, though, I am going to start looking at the

> > NetBackup software a little closer. Since Backup Exec seems to use all

> > three of those tags, and we are using another Symantec/Veritas product,

it

> > does seem very coincidental that those three tags are showing up on our

> > system. Possibly after Veritas acquired those products there was some

> > crossover of the programming teams on the software as Veritas (and later

> > Symantec) modified the programs.

> >

> > thanks again.

> > Mike O.

> >

> >

> > "Edwin vMierlo [MVP]" wrote:

> >

> > > did you search google ?

> > > http://www.google.ie/search?hl=en&q=CM44+%2Btag&meta=

> > >

> > > second hit will bring you here

> > >

https://forums.symantec.com/syment/board/message?board.id=103&message.id=37954

> > >

> > > It is a discussion about the CMpa and CM44 tags, belonging to

> > > "rxWriterSvc.exe"

> > >

> > > Not sure if this is applicable for your situation

> > >

> > > rgsd,

> > > Edwin.

> > >

> > >

> > >

> > >

> > > "Mike O" <put_the_spam@the.can> wrote in message

> > > news:uz0nDGBIIHA.2064@TK2MSFTNGP06.phx.gbl...

> > > > We currently have a two node, Windows 2003 R2 Enterprise cluster (32

bit)

> > > > using shared SAN resources. Over the last several months we have

> > > > occasionally been getting a situation where the active node slows

down and

> > > > becomes almost non-responsive. It doesn't quite stop, so the

cluster

> > > > failover doesn't automatically happen. We've set the registry key

(I

> > > don't

> > > > remember the KB article) that is supposed to cause the failover to

happen

> > > > when the higher level Windows functions stop, but we've one case

since

> > > then

> > > > where the failover didn't happen automatically. The event log

entries

> > > seem

> > > > to indicate a loss of system resources, so I'm suspecting a memory

leak in

> > > a

> > > > driver or program. About two months ago I updated all the drivers,

BIOS,

> > > > hotfixes, etc. on the systems. So far we've had one case of a

failover,

> > > but

> > > > that was related to a network issue, so I don't know if the updates

took

> > > > care of the issue or not.

> > > >

> > > > Since the update, I've been monitoring the system using the MemSnap

> > > utility

> > > > on a scheduled task a few times a day. I've been analyzing the

results

> > > and

> > > > there are three "tags" that seem like they might be showing a memory

leak.

> > > > I'm trying to find out what they are from. I've used the "memsnap

/c"

> > > > function to build the system tag list, and these don't appear on the

list,

> > > > so I'm assuming they're from a third party driver, and I'm hoping

that

> > > > someone can give me some ideas.

> > > >

> > > > The three tags in question are "CM44" (nonpaged), "CMpa" (nonpaged)

and

> > > > "CMpb" (paged). Over the last several weeks they've slowly gone

up in

> > > > memory usage. Currently they're at 398K, 796K, and 597K each.

Memsnap

> > > > shows them increasing about 800 bytes/hour. Our current system has

4G of

> > > > RAM (I'm NOT using the /3gb switch) and shows about 92M non paged

and 260M

> > > > paged in use.

> > > >

> > > > I've done some internet searching, as well as Microsoft Technet, and

I

> > > can't

> > > > find anything that would indicate where these are used. If anyone

can

> > > give

> > > > me some information on them, I would appreciate it.

> > > >

> > > > Mike O.

> > > >

> > >

> > >

> > >

Guest John Fullbright
Posted

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Windows cluster.

 

Try uninstalling the agent. The rxwritersvc is used by the CPS feature.

 

..

 

 

"Mike O" <put_the_spam@the.can> wrote in message

news:ud%23rwobIIHA.4732@TK2MSFTNGP05.phx.gbl...

> There was another reply that mentioned that tag as being part of

> rxwritersvc. However, rxwritersvc is part of Backup Exec, we don't use

> that on the our system. We do have NetBackup, another Veritas/Symantec

> product, but I disabled the NetBackup services and rebooted the computer.

> All three memory tags still showed up in memory.

>

>

> "John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

> news:OpxdqvXIIHA.4584@TK2MSFTNGP03.phx.gbl...

>> Doesn't CM44 belong to rxwritersvc.exe? You should redirect this

>> question to Symantec.

>>

>>

>> "Mike O" <put_the_spam@the.can> wrote in message

>> news:uz0nDGBIIHA.2064@TK2MSFTNGP06.phx.gbl...

>>> We currently have a two node, Windows 2003 R2 Enterprise cluster (32

>>> bit) using shared SAN resources. Over the last several months we have

>>> occasionally been getting a situation where the active node slows down

>>> and becomes almost non-responsive. It doesn't quite stop, so the

>>> cluster failover doesn't automatically happen. We've set the registry

>>> key (I don't remember the KB article) that is supposed to cause the

>>> failover to happen when the higher level Windows functions stop, but

>>> we've one case since then where the failover didn't happen

>>> automatically. The event log entries seem to indicate a loss of system

>>> resources, so I'm suspecting a memory leak in a driver or program.

>>> About two months ago I updated all the drivers, BIOS, hotfixes, etc. on

>>> the systems. So far we've had one case of a failover, but that was

>>> related to a network issue, so I don't know if the updates took care of

>>> the issue or not.

>>>

>>> Since the update, I've been monitoring the system using the MemSnap

>>> utility on a scheduled task a few times a day. I've been analyzing the

>>> results and there are three "tags" that seem like they might be showing

>>> a memory leak. I'm trying to find out what they are from. I've used the

>>> "memsnap /c" function to build the system tag list, and these don't

>>> appear on the list, so I'm assuming they're from a third party driver,

>>> and I'm hoping that someone can give me some ideas.

>>>

>>> The three tags in question are "CM44" (nonpaged), "CMpa" (nonpaged) and

>>> "CMpb" (paged). Over the last several weeks they've slowly gone up in

>>> memory usage. Currently they're at 398K, 796K, and 597K each.

>>> Memsnap shows them increasing about 800 bytes/hour. Our current system

>>> has 4G of RAM (I'm NOT using the /3gb switch) and shows about 92M non

>>> paged and 260M paged in use.

>>>

>>> I've done some internet searching, as well as Microsoft Technet, and I

>>> can't find anything that would indicate where these are used. If anyone

>>> can give me some information on them, I would appreciate it.

>>>

>>> Mike O.

>>

>>

>

Posted

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Wi

 

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Wi

 

It looks like I've found them. I found a copy of the pooltag file from the

Windows debugging kit.

CMpa is "registry post apcs"

CMpb is "registry Post Blocks"

 

I also used the findstr utility and found that the string CM44 appears in

NTOSKRNL.

 

I'm not sure exactly what they're for, but it it sounds like they're all

Windows system routines so there's not much I can do about them.

 

None of those tags are excessively large, it's just that in every "memsnap"

of three times a day over the last month all three have increased every

time.

 

I'm going to keep monitoring the system and see what happens. In a couple

of months we're hoping to get some new hardware and I will be building a new

cluster using blade servers and x64 Windows. Hopefully we should be OK

until then. I'll just do a cluster failover every now and then and reboot

the inactive node.

 

 

 

"Edwin vMierlo [MVP]" <EdwinvMierlo@discussions.microsoft.com> wrote in

message news:uqJyf1eIIHA.1316@TK2MSFTNGP02.phx.gbl...

> Disabling services and applications might not be disabling kernel drivers

>

> have a look at device manager (show hidden devices) and see if there are

> any

> symantec/HP drivers there which are loaded running

>

> rgds,

> edwin.

>

> "Mike O." <MikeO@discussions.microsoft.com> wrote in message

> news:20D6A1EC-674C-486A-BD98-8E8B0407CA00@microsoft.com...

>> I have some additional information:

>>

>> I went to another server used for testing. Using PoolMon, I verifed that

>> those three memory tags existed. I then set the Symantec/Veritas

> Netbackup

>> client service to disabled and rebooted the server. Those three tags

> still

>> exist, even with the NetBackup client not activated.

>>

>> I then went into the services and turned off several HP monitoring

>> utility

>> agents, and again rebooted. The memory tags still were showing up in

>> the

>> list.

>>

>> So now I'm back where I started. Does anyone have any suggestions on how

> I

>> can find out what those three tags are from?

>>

>> Mike O.

>>

>>

>> "Mike O." wrote:

>>

>> > thank you for the response.

>> >

>> > I had done some google searching, but I had used words like CMPB and

>> > Memsnap, or CM44, etc. Most of what I found was either posts that had

>> > a

>> > memsnap dump, or things like "Center for Molecular Physiology of the

> Brain

>> > (CMPB)", or the "California Milk Processing Board". I'm sure those are

> all

>> > useful sites, but didn't really help me..

>> >

>> > I don't think I actually used the combination "CM44 and Tag".

>> >

>> > However, I don't know if the information in that article applies to us.

> I

>> > don't see the RxWriter service (we don't use backup Exec) on our

>> > server.

> We

>> > do use the Symantec Netbackup Enterprise, but even though it's a

> Symantec

>> > product, as I understand it the Netbackup product is very different

> "under

>> > the hood" from BackupExec. I don't belive I've ever seen anything like

> the

>> > "Continuous Protection Server" in the Netbackup software.

>> >

>> > With all that being said, though, I am going to start looking at the

>> > NetBackup software a little closer. Since Backup Exec seems to use

>> > all

>> > three of those tags, and we are using another Symantec/Veritas product,

> it

>> > does seem very coincidental that those three tags are showing up on our

>> > system. Possibly after Veritas acquired those products there was some

>> > crossover of the programming teams on the software as Veritas (and

>> > later

>> > Symantec) modified the programs.

>> >

>> > thanks again.

>> > Mike O.

>> >

>> >

>> > "Edwin vMierlo [MVP]" wrote:

>> >

>> > > did you search google ?

>> > > http://www.google.ie/search?hl=en&q=CM44+%2Btag&meta=

>> > >

>> > > second hit will bring you here

>> > >

> https://forums.symantec.com/syment/board/message?board.id=103&message.id=37954

>> > >

>> > > It is a discussion about the CMpa and CM44 tags, belonging to

>> > > "rxWriterSvc.exe"

>> > >

>> > > Not sure if this is applicable for your situation

>> > >

>> > > rgsd,

>> > > Edwin.

>> > >

>> > >

>> > >

>> > >

>> > > "Mike O" <put_the_spam@the.can> wrote in message

>> > > news:uz0nDGBIIHA.2064@TK2MSFTNGP06.phx.gbl...

>> > > > We currently have a two node, Windows 2003 R2 Enterprise cluster

>> > > > (32

> bit)

>> > > > using shared SAN resources. Over the last several months we have

>> > > > occasionally been getting a situation where the active node slows

> down and

>> > > > becomes almost non-responsive. It doesn't quite stop, so the

> cluster

>> > > > failover doesn't automatically happen. We've set the registry key

> (I

>> > > don't

>> > > > remember the KB article) that is supposed to cause the failover to

> happen

>> > > > when the higher level Windows functions stop, but we've one case

> since

>> > > then

>> > > > where the failover didn't happen automatically. The event log

> entries

>> > > seem

>> > > > to indicate a loss of system resources, so I'm suspecting a memory

> leak in

>> > > a

>> > > > driver or program. About two months ago I updated all the drivers,

> BIOS,

>> > > > hotfixes, etc. on the systems. So far we've had one case of a

> failover,

>> > > but

>> > > > that was related to a network issue, so I don't know if the updates

> took

>> > > > care of the issue or not.

>> > > >

>> > > > Since the update, I've been monitoring the system using the MemSnap

>> > > utility

>> > > > on a scheduled task a few times a day. I've been analyzing the

> results

>> > > and

>> > > > there are three "tags" that seem like they might be showing a

>> > > > memory

> leak.

>> > > > I'm trying to find out what they are from. I've used the "memsnap

> /c"

>> > > > function to build the system tag list, and these don't appear on

>> > > > the

> list,

>> > > > so I'm assuming they're from a third party driver, and I'm hoping

> that

>> > > > someone can give me some ideas.

>> > > >

>> > > > The three tags in question are "CM44" (nonpaged), "CMpa" (nonpaged)

> and

>> > > > "CMpb" (paged). Over the last several weeks they've slowly gone

> up in

>> > > > memory usage. Currently they're at 398K, 796K, and 597K each.

> Memsnap

>> > > > shows them increasing about 800 bytes/hour. Our current system has

> 4G of

>> > > > RAM (I'm NOT using the /3gb switch) and shows about 92M non paged

> and 260M

>> > > > paged in use.

>> > > >

>> > > > I've done some internet searching, as well as Microsoft Technet,

>> > > > and

> I

>> > > can't

>> > > > find anything that would indicate where these are used. If anyone

> can

>> > > give

>> > > > me some information on them, I would appreciate it.

>> > > >

>> > > > Mike O.

>> > > >

>> > >

>> > >

>> > >

>

>

Posted

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Windows cluster.

 

We aren't using Backup Exec or the CPS function.

 

However, It looks like I've found them. CMpa is "registry post apcs", CMpb

is "registry Post Blocks". CM44 appears to be part of NTOSKRNL.

 

I'm going to keep monitoring the system and see what happens.

 

 

 

"John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

news:ux$BBdjIIHA.6108@TK2MSFTNGP03.phx.gbl...

> Try uninstalling the agent. The rxwritersvc is used by the CPS feature.

>

> .

>

>

> "Mike O" <put_the_spam@the.can> wrote in message

> news:ud%23rwobIIHA.4732@TK2MSFTNGP05.phx.gbl...

>> There was another reply that mentioned that tag as being part of

>> rxwritersvc. However, rxwritersvc is part of Backup Exec, we don't use

>> that on the our system. We do have NetBackup, another Veritas/Symantec

>> product, but I disabled the NetBackup services and rebooted the computer.

>> All three memory tags still showed up in memory.

>>

>>

>> "John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

>> news:OpxdqvXIIHA.4584@TK2MSFTNGP03.phx.gbl...

>>> Doesn't CM44 belong to rxwritersvc.exe? You should redirect this

>>> question to Symantec.

>>>

>>>

>>> "Mike O" <put_the_spam@the.can> wrote in message

>>> news:uz0nDGBIIHA.2064@TK2MSFTNGP06.phx.gbl...

>>>> We currently have a two node, Windows 2003 R2 Enterprise cluster (32

>>>> bit) using shared SAN resources. Over the last several months we have

>>>> occasionally been getting a situation where the active node slows down

>>>> and becomes almost non-responsive. It doesn't quite stop, so the

>>>> cluster failover doesn't automatically happen. We've set the registry

>>>> key (I don't remember the KB article) that is supposed to cause the

>>>> failover to happen when the higher level Windows functions stop, but

>>>> we've one case since then where the failover didn't happen

>>>> automatically. The event log entries seem to indicate a loss of system

>>>> resources, so I'm suspecting a memory leak in a driver or program.

>>>> About two months ago I updated all the drivers, BIOS, hotfixes, etc. on

>>>> the systems. So far we've had one case of a failover, but that was

>>>> related to a network issue, so I don't know if the updates took care of

>>>> the issue or not.

>>>>

>>>> Since the update, I've been monitoring the system using the MemSnap

>>>> utility on a scheduled task a few times a day. I've been analyzing the

>>>> results and there are three "tags" that seem like they might be showing

>>>> a memory leak. I'm trying to find out what they are from. I've used

>>>> the "memsnap /c" function to build the system tag list, and these don't

>>>> appear on the list, so I'm assuming they're from a third party driver,

>>>> and I'm hoping that someone can give me some ideas.

>>>>

>>>> The three tags in question are "CM44" (nonpaged), "CMpa" (nonpaged) and

>>>> "CMpb" (paged). Over the last several weeks they've slowly gone up

>>>> in memory usage. Currently they're at 398K, 796K, and 597K each.

>>>> Memsnap shows them increasing about 800 bytes/hour. Our current system

>>>> has 4G of RAM (I'm NOT using the /3gb switch) and shows about 92M non

>>>> paged and 260M paged in use.

>>>>

>>>> I've done some internet searching, as well as Microsoft Technet, and I

>>>> can't find anything that would indicate where these are used. If

>>>> anyone can give me some information on them, I would appreciate it.

>>>>

>>>> Mike O.

>>>

>>>

>>

>

>

Guest John Fullbright
Posted

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Windows cluster.

 

Backupexec and CPS use these tags because they leverage VSS (and veritas'

software provider). I think if I remember right they technically belong to

the configuration manager.

 

What kind of backups do you take with netbackup? System state?

 

 

"Mike O" <put_the_spam@the.can> wrote in message

news:%23bpfVtoIIHA.2268@TK2MSFTNGP02.phx.gbl...

> We aren't using Backup Exec or the CPS function.

>

> However, It looks like I've found them. CMpa is "registry post apcs",

> CMpb is "registry Post Blocks". CM44 appears to be part of NTOSKRNL.

>

> I'm going to keep monitoring the system and see what happens.

>

>

>

> "John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

> news:ux$BBdjIIHA.6108@TK2MSFTNGP03.phx.gbl...

>> Try uninstalling the agent. The rxwritersvc is used by the CPS feature.

>>

>> .

>>

>>

>> "Mike O" <put_the_spam@the.can> wrote in message

>> news:ud%23rwobIIHA.4732@TK2MSFTNGP05.phx.gbl...

>>> There was another reply that mentioned that tag as being part of

>>> rxwritersvc. However, rxwritersvc is part of Backup Exec, we don't use

>>> that on the our system. We do have NetBackup, another Veritas/Symantec

>>> product, but I disabled the NetBackup services and rebooted the

>>> computer. All three memory tags still showed up in memory.

>>>

>>>

>>> "John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

>>> news:OpxdqvXIIHA.4584@TK2MSFTNGP03.phx.gbl...

>>>> Doesn't CM44 belong to rxwritersvc.exe? You should redirect this

>>>> question to Symantec.

>>>>

>>>>

>>>> "Mike O" <put_the_spam@the.can> wrote in message

>>>> news:uz0nDGBIIHA.2064@TK2MSFTNGP06.phx.gbl...

>>>>> We currently have a two node, Windows 2003 R2 Enterprise cluster (32

>>>>> bit) using shared SAN resources. Over the last several months we

>>>>> have occasionally been getting a situation where the active node slows

>>>>> down and becomes almost non-responsive. It doesn't quite stop, so

>>>>> the cluster failover doesn't automatically happen. We've set the

>>>>> registry key (I don't remember the KB article) that is supposed to

>>>>> cause the failover to happen when the higher level Windows functions

>>>>> stop, but we've one case since then where the failover didn't happen

>>>>> automatically. The event log entries seem to indicate a loss of system

>>>>> resources, so I'm suspecting a memory leak in a driver or program.

>>>>> About two months ago I updated all the drivers, BIOS, hotfixes, etc.

>>>>> on the systems. So far we've had one case of a failover, but that was

>>>>> related to a network issue, so I don't know if the updates took care

>>>>> of the issue or not.

>>>>>

>>>>> Since the update, I've been monitoring the system using the MemSnap

>>>>> utility on a scheduled task a few times a day. I've been analyzing

>>>>> the results and there are three "tags" that seem like they might be

>>>>> showing a memory leak. I'm trying to find out what they are from.

>>>>> I've used the "memsnap /c" function to build the system tag list, and

>>>>> these don't appear on the list, so I'm assuming they're from a third

>>>>> party driver, and I'm hoping that someone can give me some ideas.

>>>>>

>>>>> The three tags in question are "CM44" (nonpaged), "CMpa" (nonpaged)

>>>>> and "CMpb" (paged). Over the last several weeks they've slowly gone

>>>>> up in memory usage. Currently they're at 398K, 796K, and 597K each.

>>>>> Memsnap shows them increasing about 800 bytes/hour. Our current

>>>>> system has 4G of RAM (I'm NOT using the /3gb switch) and shows about

>>>>> 92M non paged and 260M paged in use.

>>>>>

>>>>> I've done some internet searching, as well as Microsoft Technet, and I

>>>>> can't find anything that would indicate where these are used. If

>>>>> anyone can give me some information on them, I would appreciate it.

>>>>>

>>>>> Mike O.

>>>>

>>>>

>>>

>>

>>

>

Posted

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Windows cluster.

 

System state and all local drives. Due to issues we've had on other systems

with the Veritas Snapshot Provider (VSP) service, We don't install VSP and

have the server use the Microsoft Volume Shadow, so that does tie in to what

you mention. Unfortunately, we're at the latest patches and service pack,

so unless we open up something with Microsoft, there's not much I can do

right now. As I said before, these three tags aren't taking up that much

memory, but they're the only ones that keep going up.

 

 

"John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

news:OlXyjSxIIHA.2064@TK2MSFTNGP06.phx.gbl...

> Backupexec and CPS use these tags because they leverage VSS (and veritas'

> software provider). I think if I remember right they technically belong

> to the configuration manager.

>

> What kind of backups do you take with netbackup? System state?

>

>

> "Mike O" <put_the_spam@the.can> wrote in message

> news:%23bpfVtoIIHA.2268@TK2MSFTNGP02.phx.gbl...

>> We aren't using Backup Exec or the CPS function.

>>

>> However, It looks like I've found them. CMpa is "registry post apcs",

>> CMpb is "registry Post Blocks". CM44 appears to be part of NTOSKRNL.

>>

>> I'm going to keep monitoring the system and see what happens.

>>

>>

>>

>> "John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

>> news:ux$BBdjIIHA.6108@TK2MSFTNGP03.phx.gbl...

>>> Try uninstalling the agent. The rxwritersvc is used by the CPS feature.

>>>

>>> .

>>>

>>>

>>> "Mike O" <put_the_spam@the.can> wrote in message

>>> news:ud%23rwobIIHA.4732@TK2MSFTNGP05.phx.gbl...

>>>> There was another reply that mentioned that tag as being part of

>>>> rxwritersvc. However, rxwritersvc is part of Backup Exec, we don't use

>>>> that on the our system. We do have NetBackup, another Veritas/Symantec

>>>> product, but I disabled the NetBackup services and rebooted the

>>>> computer. All three memory tags still showed up in memory.

>>>>

>>>>

>>>> "John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

>>>> news:OpxdqvXIIHA.4584@TK2MSFTNGP03.phx.gbl...

>>>>> Doesn't CM44 belong to rxwritersvc.exe? You should redirect this

>>>>> question to Symantec.

>>>>>

>>>>>

>>>>> "Mike O" <put_the_spam@the.can> wrote in message

>>>>> news:uz0nDGBIIHA.2064@TK2MSFTNGP06.phx.gbl...

>>>>>> We currently have a two node, Windows 2003 R2 Enterprise cluster (32

>>>>>> bit) using shared SAN resources. Over the last several months we

>>>>>> have occasionally been getting a situation where the active node

>>>>>> slows down and becomes almost non-responsive. It doesn't quite

>>>>>> stop, so the cluster failover doesn't automatically happen. We've

>>>>>> set the registry key (I don't remember the KB article) that is

>>>>>> supposed to cause the failover to happen when the higher level

>>>>>> Windows functions stop, but we've one case since then where the

>>>>>> failover didn't happen automatically. The event log entries seem to

>>>>>> indicate a loss of system resources, so I'm suspecting a memory leak

>>>>>> in a driver or program. About two months ago I updated all the

>>>>>> drivers, BIOS, hotfixes, etc. on the systems. So far we've had one

>>>>>> case of a failover, but that was related to a network issue, so I

>>>>>> don't know if the updates took care of the issue or not.

>>>>>>

>>>>>> Since the update, I've been monitoring the system using the MemSnap

>>>>>> utility on a scheduled task a few times a day. I've been analyzing

>>>>>> the results and there are three "tags" that seem like they might be

>>>>>> showing a memory leak. I'm trying to find out what they are from.

>>>>>> I've used the "memsnap /c" function to build the system tag list, and

>>>>>> these don't appear on the list, so I'm assuming they're from a third

>>>>>> party driver, and I'm hoping that someone can give me some ideas.

>>>>>>

>>>>>> The three tags in question are "CM44" (nonpaged), "CMpa" (nonpaged)

>>>>>> and "CMpb" (paged). Over the last several weeks they've slowly

>>>>>> gone up in memory usage. Currently they're at 398K, 796K, and 597K

>>>>>> each. Memsnap shows them increasing about 800 bytes/hour. Our

>>>>>> current system has 4G of RAM (I'm NOT using the /3gb switch) and

>>>>>> shows about 92M non paged and 260M paged in use.

>>>>>>

>>>>>> I've done some internet searching, as well as Microsoft Technet, and

>>>>>> I can't find anything that would indicate where these are used. If

>>>>>> anyone can give me some information on them, I would appreciate it.

>>>>>>

>>>>>> Mike O.

>>>>>

>>>>>

>>>>

>>>

>>>

>>

>

>

Guest John Fullbright
Posted

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Windows cluster.

 

By "latest", do you mean

a.. 898790

b.. 902837

c.. 908249

d.. 912063

e.. 913648

f.. 922772

g.. 923424

h.. 923801

i.. 931300

?

 

 

"Mike O" <put_the_spam@the.can> wrote in message

news:OpLBpQ$IIHA.3672@TK2MSFTNGP02.phx.gbl...

> System state and all local drives. Due to issues we've had on other

> systems with the Veritas Snapshot Provider (VSP) service, We don't install

> VSP and have the server use the Microsoft Volume Shadow, so that does tie

> in to what you mention. Unfortunately, we're at the latest patches and

> service pack, so unless we open up something with Microsoft, there's not

> much I can do right now. As I said before, these three tags aren't taking

> up that much memory, but they're the only ones that keep going up.

>

>

> "John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

> news:OlXyjSxIIHA.2064@TK2MSFTNGP06.phx.gbl...

>> Backupexec and CPS use these tags because they leverage VSS (and veritas'

>> software provider). I think if I remember right they technically belong

>> to the configuration manager.

>>

>> What kind of backups do you take with netbackup? System state?

>>

>>

>> "Mike O" <put_the_spam@the.can> wrote in message

>> news:%23bpfVtoIIHA.2268@TK2MSFTNGP02.phx.gbl...

>>> We aren't using Backup Exec or the CPS function.

>>>

>>> However, It looks like I've found them. CMpa is "registry post apcs",

>>> CMpb is "registry Post Blocks". CM44 appears to be part of NTOSKRNL.

>>>

>>> I'm going to keep monitoring the system and see what happens.

>>>

>>>

>>>

>>> "John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

>>> news:ux$BBdjIIHA.6108@TK2MSFTNGP03.phx.gbl...

>>>> Try uninstalling the agent. The rxwritersvc is used by the CPS

>>>> feature.

>>>>

>>>> .

>>>>

>>>>

>>>> "Mike O" <put_the_spam@the.can> wrote in message

>>>> news:ud%23rwobIIHA.4732@TK2MSFTNGP05.phx.gbl...

>>>>> There was another reply that mentioned that tag as being part of

>>>>> rxwritersvc. However, rxwritersvc is part of Backup Exec, we don't

>>>>> use that on the our system. We do have NetBackup, another

>>>>> Veritas/Symantec product, but I disabled the NetBackup services and

>>>>> rebooted the computer. All three memory tags still showed up in

>>>>> memory.

>>>>>

>>>>>

>>>>> "John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

>>>>> news:OpxdqvXIIHA.4584@TK2MSFTNGP03.phx.gbl...

>>>>>> Doesn't CM44 belong to rxwritersvc.exe? You should redirect this

>>>>>> question to Symantec.

>>>>>>

>>>>>>

>>>>>> "Mike O" <put_the_spam@the.can> wrote in message

>>>>>> news:uz0nDGBIIHA.2064@TK2MSFTNGP06.phx.gbl...

>>>>>>> We currently have a two node, Windows 2003 R2 Enterprise cluster (32

>>>>>>> bit) using shared SAN resources. Over the last several months we

>>>>>>> have occasionally been getting a situation where the active node

>>>>>>> slows down and becomes almost non-responsive. It doesn't quite

>>>>>>> stop, so the cluster failover doesn't automatically happen. We've

>>>>>>> set the registry key (I don't remember the KB article) that is

>>>>>>> supposed to cause the failover to happen when the higher level

>>>>>>> Windows functions stop, but we've one case since then where the

>>>>>>> failover didn't happen automatically. The event log entries seem to

>>>>>>> indicate a loss of system resources, so I'm suspecting a memory leak

>>>>>>> in a driver or program. About two months ago I updated all the

>>>>>>> drivers, BIOS, hotfixes, etc. on the systems. So far we've had one

>>>>>>> case of a failover, but that was related to a network issue, so I

>>>>>>> don't know if the updates took care of the issue or not.

>>>>>>>

>>>>>>> Since the update, I've been monitoring the system using the MemSnap

>>>>>>> utility on a scheduled task a few times a day. I've been analyzing

>>>>>>> the results and there are three "tags" that seem like they might be

>>>>>>> showing a memory leak. I'm trying to find out what they are from.

>>>>>>> I've used the "memsnap /c" function to build the system tag list,

>>>>>>> and these don't appear on the list, so I'm assuming they're from a

>>>>>>> third party driver, and I'm hoping that someone can give me some

>>>>>>> ideas.

>>>>>>>

>>>>>>> The three tags in question are "CM44" (nonpaged), "CMpa" (nonpaged)

>>>>>>> and "CMpb" (paged). Over the last several weeks they've slowly

>>>>>>> gone up in memory usage. Currently they're at 398K, 796K, and 597K

>>>>>>> each. Memsnap shows them increasing about 800 bytes/hour. Our

>>>>>>> current system has 4G of RAM (I'm NOT using the /3gb switch) and

>>>>>>> shows about 92M non paged and 260M paged in use.

>>>>>>>

>>>>>>> I've done some internet searching, as well as Microsoft Technet, and

>>>>>>> I can't find anything that would indicate where these are used. If

>>>>>>> anyone can give me some information on them, I would appreciate it.

>>>>>>>

>>>>>>> Mike O.

>>>>>>

>>>>>>

>>>>>

>>>>

>>>>

>>>

>>

>>

>

Posted

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Windows cluster.

 

By latest, I meant that I had run Windows Update and applied all the

critical, and most of the optional (unless they had nothing to do with our

system). This was as of early October.

 

I have the server doing a "memsnap" three times a day, and every once in a

while start up Process Explorer and check the system info. At this time it

seems to be hovering around 170MB paged memory , out of a 360MB limit. I've

seen it hit around 220MB, but that was while backups were running, and then

it drops back down, so it seems to be stable at this time.

 

In addition to file serving this server run antivirus, backups, hardware

monitor services, etc. so I'm wondering if part of the issues we've had is

that the local services eating up all the kernel memory.

 

As a test, on the inactive node I set the system to "balanced", instead of

file serving. The paged limit went up to 500MB plus (the server has 4G of

RAM). Do you think this might be a better option for this server? How

much would I be hurting file serving to use the "balanced" server setting?

 

 

 

"John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

news:ONlHpjPJIHA.484@TK2MSFTNGP06.phx.gbl...

> By "latest", do you mean

> a.. 898790

> b.. 902837

> c.. 908249

> d.. 912063

> e.. 913648

> f.. 922772

> g.. 923424

> h.. 923801

> i.. 931300

> ?

>

>

> "Mike O" <put_the_spam@the.can> wrote in message

> news:OpLBpQ$IIHA.3672@TK2MSFTNGP02.phx.gbl...

>> System state and all local drives. Due to issues we've had on other

>> systems with the Veritas Snapshot Provider (VSP) service, We don't

>> install VSP and have the server use the Microsoft Volume Shadow, so that

>> does tie in to what you mention. Unfortunately, we're at the latest

>> patches and service pack, so unless we open up something with Microsoft,

>> there's not much I can do right now. As I said before, these three tags

>> aren't taking up that much memory, but they're the only ones that keep

>> going up.

>>

>>

>> "John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

>> news:OlXyjSxIIHA.2064@TK2MSFTNGP06.phx.gbl...

>>> Backupexec and CPS use these tags because they leverage VSS (and

>>> veritas' software provider). I think if I remember right they

>>> technically belong to the configuration manager.

>>>

>>> What kind of backups do you take with netbackup? System state?

>>>

>>>

>>> "Mike O" <put_the_spam@the.can> wrote in message

>>> news:%23bpfVtoIIHA.2268@TK2MSFTNGP02.phx.gbl...

>>>> We aren't using Backup Exec or the CPS function.

>>>>

>>>> However, It looks like I've found them. CMpa is "registry post apcs",

>>>> CMpb is "registry Post Blocks". CM44 appears to be part of NTOSKRNL.

>>>>

>>>> I'm going to keep monitoring the system and see what happens.

>>>>

>>>>

>>>>

>>>> "John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

>>>> news:ux$BBdjIIHA.6108@TK2MSFTNGP03.phx.gbl...

>>>>> Try uninstalling the agent. The rxwritersvc is used by the CPS

>>>>> feature.

>>>>>

>>>>> .

>>>>>

>>>>>

>>>>> "Mike O" <put_the_spam@the.can> wrote in message

>>>>> news:ud%23rwobIIHA.4732@TK2MSFTNGP05.phx.gbl...

>>>>>> There was another reply that mentioned that tag as being part of

>>>>>> rxwritersvc. However, rxwritersvc is part of Backup Exec, we don't

>>>>>> use that on the our system. We do have NetBackup, another

>>>>>> Veritas/Symantec product, but I disabled the NetBackup services and

>>>>>> rebooted the computer. All three memory tags still showed up in

>>>>>> memory.

>>>>>>

>>>>>>

>>>>>> "John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

>>>>>> news:OpxdqvXIIHA.4584@TK2MSFTNGP03.phx.gbl...

>>>>>>> Doesn't CM44 belong to rxwritersvc.exe? You should redirect this

>>>>>>> question to Symantec.

>>>>>>>

>>>>>>>

>>>>>>> "Mike O" <put_the_spam@the.can> wrote in message

>>>>>>> news:uz0nDGBIIHA.2064@TK2MSFTNGP06.phx.gbl...

>>>>>>>> We currently have a two node, Windows 2003 R2 Enterprise cluster

>>>>>>>> (32 bit) using shared SAN resources. Over the last several months

>>>>>>>> we have occasionally been getting a situation where the active node

>>>>>>>> slows down and becomes almost non-responsive. It doesn't quite

>>>>>>>> stop, so the cluster failover doesn't automatically happen. We've

>>>>>>>> set the registry key (I don't remember the KB article) that is

>>>>>>>> supposed to cause the failover to happen when the higher level

>>>>>>>> Windows functions stop, but we've one case since then where the

>>>>>>>> failover didn't happen automatically. The event log entries seem to

>>>>>>>> indicate a loss of system resources, so I'm suspecting a memory

>>>>>>>> leak in a driver or program. About two months ago I updated all the

>>>>>>>> drivers, BIOS, hotfixes, etc. on the systems. So far we've had one

>>>>>>>> case of a failover, but that was related to a network issue, so I

>>>>>>>> don't know if the updates took care of the issue or not.

>>>>>>>>

>>>>>>>> Since the update, I've been monitoring the system using the MemSnap

>>>>>>>> utility on a scheduled task a few times a day. I've been analyzing

>>>>>>>> the results and there are three "tags" that seem like they might be

>>>>>>>> showing a memory leak. I'm trying to find out what they are from.

>>>>>>>> I've used the "memsnap /c" function to build the system tag list,

>>>>>>>> and these don't appear on the list, so I'm assuming they're from a

>>>>>>>> third party driver, and I'm hoping that someone can give me some

>>>>>>>> ideas.

>>>>>>>>

>>>>>>>> The three tags in question are "CM44" (nonpaged), "CMpa" (nonpaged)

>>>>>>>> and "CMpb" (paged). Over the last several weeks they've slowly

>>>>>>>> gone up in memory usage. Currently they're at 398K, 796K, and

>>>>>>>> 597K each. Memsnap shows them increasing about 800 bytes/hour. Our

>>>>>>>> current system has 4G of RAM (I'm NOT using the /3gb switch) and

>>>>>>>> shows about 92M non paged and 260M paged in use.

>>>>>>>>

>>>>>>>> I've done some internet searching, as well as Microsoft Technet,

>>>>>>>> and I can't find anything that would indicate where these are used.

>>>>>>>> If anyone can give me some information on them, I would appreciate

>>>>>>>> it.

>>>>>>>>

>>>>>>>> Mike O.

>>>>>>>

>>>>>>>

>>>>>>

>>>>>

>>>>>

>>>>

>>>

>>>

>>

>

>

Guest John Fullbright
Posted

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Windows cluster.

 

What is the cache architecture of your SAN? Changing to balanced reduces

the amount of RAM reserved for the filesystem cache. This means a greater

likelyhood that a read request would not be in cache and go to storage.

Depending on the architecure of the storage, it may be no big deal; many

have a much larger L1 cache than you could achieve in memory on a 32bit

windows server.

 

For an application server, like Exchange, you want it balanced anyway.

Exchange does not leverage the filesystem cache, but instead manages it's

own cache (the database and log buffers)

 

 

"Mike O" <put_the_spam@the.can> wrote in message

news:%23Giuvz%23JIHA.5400@TK2MSFTNGP04.phx.gbl...

> By latest, I meant that I had run Windows Update and applied all the

> critical, and most of the optional (unless they had nothing to do with our

> system). This was as of early October.

>

> I have the server doing a "memsnap" three times a day, and every once in a

> while start up Process Explorer and check the system info. At this time

> it seems to be hovering around 170MB paged memory , out of a 360MB limit.

> I've seen it hit around 220MB, but that was while backups were running,

> and then it drops back down, so it seems to be stable at this time.

>

> In addition to file serving this server run antivirus, backups, hardware

> monitor services, etc. so I'm wondering if part of the issues we've had is

> that the local services eating up all the kernel memory.

>

> As a test, on the inactive node I set the system to "balanced", instead of

> file serving. The paged limit went up to 500MB plus (the server has 4G of

> RAM). Do you think this might be a better option for this server? How

> much would I be hurting file serving to use the "balanced" server setting?

>

>

>

> "John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

> news:ONlHpjPJIHA.484@TK2MSFTNGP06.phx.gbl...

>> By "latest", do you mean

>> a.. 898790

>> b.. 902837

>> c.. 908249

>> d.. 912063

>> e.. 913648

>> f.. 922772

>> g.. 923424

>> h.. 923801

>> i.. 931300

>> ?

>>

>>

>> "Mike O" <put_the_spam@the.can> wrote in message

>> news:OpLBpQ$IIHA.3672@TK2MSFTNGP02.phx.gbl...

>>> System state and all local drives. Due to issues we've had on other

>>> systems with the Veritas Snapshot Provider (VSP) service, We don't

>>> install VSP and have the server use the Microsoft Volume Shadow, so that

>>> does tie in to what you mention. Unfortunately, we're at the latest

>>> patches and service pack, so unless we open up something with Microsoft,

>>> there's not much I can do right now. As I said before, these three tags

>>> aren't taking up that much memory, but they're the only ones that keep

>>> going up.

>>>

>>>

>>> "John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

>>> news:OlXyjSxIIHA.2064@TK2MSFTNGP06.phx.gbl...

>>>> Backupexec and CPS use these tags because they leverage VSS (and

>>>> veritas' software provider). I think if I remember right they

>>>> technically belong to the configuration manager.

>>>>

>>>> What kind of backups do you take with netbackup? System state?

>>>>

>>>>

>>>> "Mike O" <put_the_spam@the.can> wrote in message

>>>> news:%23bpfVtoIIHA.2268@TK2MSFTNGP02.phx.gbl...

>>>>> We aren't using Backup Exec or the CPS function.

>>>>>

>>>>> However, It looks like I've found them. CMpa is "registry post apcs",

>>>>> CMpb is "registry Post Blocks". CM44 appears to be part of NTOSKRNL.

>>>>>

>>>>> I'm going to keep monitoring the system and see what happens.

>>>>>

>>>>>

>>>>>

>>>>> "John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

>>>>> news:ux$BBdjIIHA.6108@TK2MSFTNGP03.phx.gbl...

>>>>>> Try uninstalling the agent. The rxwritersvc is used by the CPS

>>>>>> feature.

>>>>>>

>>>>>> .

>>>>>>

>>>>>>

>>>>>> "Mike O" <put_the_spam@the.can> wrote in message

>>>>>> news:ud%23rwobIIHA.4732@TK2MSFTNGP05.phx.gbl...

>>>>>>> There was another reply that mentioned that tag as being part of

>>>>>>> rxwritersvc. However, rxwritersvc is part of Backup Exec, we don't

>>>>>>> use that on the our system. We do have NetBackup, another

>>>>>>> Veritas/Symantec product, but I disabled the NetBackup services and

>>>>>>> rebooted the computer. All three memory tags still showed up in

>>>>>>> memory.

>>>>>>>

>>>>>>>

>>>>>>> "John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

>>>>>>> news:OpxdqvXIIHA.4584@TK2MSFTNGP03.phx.gbl...

>>>>>>>> Doesn't CM44 belong to rxwritersvc.exe? You should redirect this

>>>>>>>> question to Symantec.

>>>>>>>>

>>>>>>>>

>>>>>>>> "Mike O" <put_the_spam@the.can> wrote in message

>>>>>>>> news:uz0nDGBIIHA.2064@TK2MSFTNGP06.phx.gbl...

>>>>>>>>> We currently have a two node, Windows 2003 R2 Enterprise cluster

>>>>>>>>> (32 bit) using shared SAN resources. Over the last several

>>>>>>>>> months we have occasionally been getting a situation where the

>>>>>>>>> active node slows down and becomes almost non-responsive. It

>>>>>>>>> doesn't quite stop, so the cluster failover doesn't automatically

>>>>>>>>> happen. We've set the registry key (I don't remember the KB

>>>>>>>>> article) that is supposed to cause the failover to happen when the

>>>>>>>>> higher level Windows functions stop, but we've one case since then

>>>>>>>>> where the failover didn't happen automatically. The event log

>>>>>>>>> entries seem to indicate a loss of system resources, so I'm

>>>>>>>>> suspecting a memory leak in a driver or program. About two months

>>>>>>>>> ago I updated all the drivers, BIOS, hotfixes, etc. on the

>>>>>>>>> systems. So far we've had one case of a failover, but that was

>>>>>>>>> related to a network issue, so I don't know if the updates took

>>>>>>>>> care of the issue or not.

>>>>>>>>>

>>>>>>>>> Since the update, I've been monitoring the system using the

>>>>>>>>> MemSnap utility on a scheduled task a few times a day. I've been

>>>>>>>>> analyzing the results and there are three "tags" that seem like

>>>>>>>>> they might be showing a memory leak. I'm trying to find out what

>>>>>>>>> they are from. I've used the "memsnap /c" function to build the

>>>>>>>>> system tag list, and these don't appear on the list, so I'm

>>>>>>>>> assuming they're from a third party driver, and I'm hoping that

>>>>>>>>> someone can give me some ideas.

>>>>>>>>>

>>>>>>>>> The three tags in question are "CM44" (nonpaged), "CMpa"

>>>>>>>>> (nonpaged) and "CMpb" (paged). Over the last several weeks

>>>>>>>>> they've slowly gone up in memory usage. Currently they're at

>>>>>>>>> 398K, 796K, and 597K each. Memsnap shows them increasing about 800

>>>>>>>>> bytes/hour. Our current system has 4G of RAM (I'm NOT using the

>>>>>>>>> /3gb switch) and shows about 92M non paged and 260M paged in use.

>>>>>>>>>

>>>>>>>>> I've done some internet searching, as well as Microsoft Technet,

>>>>>>>>> and I can't find anything that would indicate where these are

>>>>>>>>> used. If anyone can give me some information on them, I would

>>>>>>>>> appreciate it.

>>>>>>>>>

>>>>>>>>> Mike O.

>>>>>>>>

>>>>>>>>

>>>>>>>

>>>>>>

>>>>>>

>>>>>

>>>>

>>>>

>>>

>>

>>

>

  • 2 weeks later...
Posted

Re: Trying to identify memory pool tags CM44, CMpa, and CMpb in Windows cluster.

 

The SAN is an EMC CX700. It has two processor boards, both with I believe

4G of cache. The connection from the server to the SAN switch is dual 2G

fiber connections (set up in a load sharing mode). It sounds like running

the servers with less file cache won't be too big of a hit.

 

I've been checking the active node every day or two, so far everything is

looking good. Paged and non-paged is running about 50%. The active node is

still running in the "file server" configuration, but if we do fail over,

the other one is set for "balanced". If it looks like we're running short

on resources (or if it freezes up), I'll fail it over. Even if it keeps

running OK, I figure there will be some critical updates soon and I'll have

to fail it over anyway..

 

 

"John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

news:%23%23WQt0iKIHA.1164@TK2MSFTNGP02.phx.gbl...

> What is the cache architecture of your SAN? Changing to balanced reduces

> the amount of RAM reserved for the filesystem cache. This means a greater

> likelyhood that a read request would not be in cache and go to storage.

> Depending on the architecure of the storage, it may be no big deal; many

> have a much larger L1 cache than you could achieve in memory on a 32bit

> windows server.

>

> For an application server, like Exchange, you want it balanced anyway.

> Exchange does not leverage the filesystem cache, but instead manages it's

> own cache (the database and log buffers)

>

>

> "Mike O" <put_the_spam@the.can> wrote in message

> news:%23Giuvz%23JIHA.5400@TK2MSFTNGP04.phx.gbl...

>> By latest, I meant that I had run Windows Update and applied all the

>> critical, and most of the optional (unless they had nothing to do with

>> our system). This was as of early October.

>>

>> I have the server doing a "memsnap" three times a day, and every once in

>> a while start up Process Explorer and check the system info. At this

>> time it seems to be hovering around 170MB paged memory , out of a 360MB

>> limit. I've seen it hit around 220MB, but that was while backups were

>> running, and then it drops back down, so it seems to be stable at this

>> time.

>>

>> In addition to file serving this server run antivirus, backups, hardware

>> monitor services, etc. so I'm wondering if part of the issues we've had

>> is that the local services eating up all the kernel memory.

>>

>> As a test, on the inactive node I set the system to "balanced", instead

>> of file serving. The paged limit went up to 500MB plus (the server has

>> 4G of RAM). Do you think this might be a better option for this

>> server? How much would I be hurting file serving to use the "balanced"

>> server setting?

>>

>>

>>

>> "John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

>> news:ONlHpjPJIHA.484@TK2MSFTNGP06.phx.gbl...

>>> By "latest", do you mean

>>> a.. 898790

>>> b.. 902837

>>> c.. 908249

>>> d.. 912063

>>> e.. 913648

>>> f.. 922772

>>> g.. 923424

>>> h.. 923801

>>> i.. 931300

>>> ?

>>>

>>>

>>> "Mike O" <put_the_spam@the.can> wrote in message

>>> news:OpLBpQ$IIHA.3672@TK2MSFTNGP02.phx.gbl...

>>>> System state and all local drives. Due to issues we've had on other

>>>> systems with the Veritas Snapshot Provider (VSP) service, We don't

>>>> install VSP and have the server use the Microsoft Volume Shadow, so

>>>> that does tie in to what you mention. Unfortunately, we're at the

>>>> latest patches and service pack, so unless we open up something with

>>>> Microsoft, there's not much I can do right now. As I said before,

>>>> these three tags aren't taking up that much memory, but they're the

>>>> only ones that keep going up.

>>>>

>>>>

>>>> "John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

>>>> news:OlXyjSxIIHA.2064@TK2MSFTNGP06.phx.gbl...

>>>>> Backupexec and CPS use these tags because they leverage VSS (and

>>>>> veritas' software provider). I think if I remember right they

>>>>> technically belong to the configuration manager.

>>>>>

>>>>> What kind of backups do you take with netbackup? System state?

>>>>>

>>>>>

>>>>> "Mike O" <put_the_spam@the.can> wrote in message

>>>>> news:%23bpfVtoIIHA.2268@TK2MSFTNGP02.phx.gbl...

>>>>>> We aren't using Backup Exec or the CPS function.

>>>>>>

>>>>>> However, It looks like I've found them. CMpa is "registry post

>>>>>> apcs", CMpb is "registry Post Blocks". CM44 appears to be part of

>>>>>> NTOSKRNL.

>>>>>>

>>>>>> I'm going to keep monitoring the system and see what happens.

>>>>>>

>>>>>>

>>>>>>

>>>>>> "John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

>>>>>> news:ux$BBdjIIHA.6108@TK2MSFTNGP03.phx.gbl...

>>>>>>> Try uninstalling the agent. The rxwritersvc is used by the CPS

>>>>>>> feature.

>>>>>>>

>>>>>>> .

>>>>>>>

>>>>>>>

>>>>>>> "Mike O" <put_the_spam@the.can> wrote in message

>>>>>>> news:ud%23rwobIIHA.4732@TK2MSFTNGP05.phx.gbl...

>>>>>>>> There was another reply that mentioned that tag as being part of

>>>>>>>> rxwritersvc. However, rxwritersvc is part of Backup Exec, we don't

>>>>>>>> use that on the our system. We do have NetBackup, another

>>>>>>>> Veritas/Symantec product, but I disabled the NetBackup services and

>>>>>>>> rebooted the computer. All three memory tags still showed up in

>>>>>>>> memory.

>>>>>>>>

>>>>>>>>

>>>>>>>> "John Fullbright" <fjohn@donotspamnetappdotcom> wrote in message

>>>>>>>> news:OpxdqvXIIHA.4584@TK2MSFTNGP03.phx.gbl...

>>>>>>>>> Doesn't CM44 belong to rxwritersvc.exe? You should redirect this

>>>>>>>>> question to Symantec.

>>>>>>>>>

>>>>>>>>>

>>>>>>>>> "Mike O" <put_the_spam@the.can> wrote in message

>>>>>>>>> news:uz0nDGBIIHA.2064@TK2MSFTNGP06.phx.gbl...

>>>>>>>>>> We currently have a two node, Windows 2003 R2 Enterprise cluster

>>>>>>>>>> (32 bit) using shared SAN resources. Over the last several

>>>>>>>>>> months we have occasionally been getting a situation where the

>>>>>>>>>> active node slows down and becomes almost non-responsive. It

>>>>>>>>>> doesn't quite stop, so the cluster failover doesn't automatically

>>>>>>>>>> happen. We've set the registry key (I don't remember the KB

>>>>>>>>>> article) that is supposed to cause the failover to happen when

>>>>>>>>>> the higher level Windows functions stop, but we've one case since

>>>>>>>>>> then where the failover didn't happen automatically. The event

>>>>>>>>>> log entries seem to indicate a loss of system resources, so I'm

>>>>>>>>>> suspecting a memory leak in a driver or program. About two months

>>>>>>>>>> ago I updated all the drivers, BIOS, hotfixes, etc. on the

>>>>>>>>>> systems. So far we've had one case of a failover, but that was

>>>>>>>>>> related to a network issue, so I don't know if the updates took

>>>>>>>>>> care of the issue or not.

>>>>>>>>>>

>>>>>>>>>> Since the update, I've been monitoring the system using the

>>>>>>>>>> MemSnap utility on a scheduled task a few times a day. I've been

>>>>>>>>>> analyzing the results and there are three "tags" that seem like

>>>>>>>>>> they might be showing a memory leak. I'm trying to find out what

>>>>>>>>>> they are from. I've used the "memsnap /c" function to build the

>>>>>>>>>> system tag list, and these don't appear on the list, so I'm

>>>>>>>>>> assuming they're from a third party driver, and I'm hoping that

>>>>>>>>>> someone can give me some ideas.

>>>>>>>>>>

>>>>>>>>>> The three tags in question are "CM44" (nonpaged), "CMpa"

>>>>>>>>>> (nonpaged) and "CMpb" (paged). Over the last several weeks

>>>>>>>>>> they've slowly gone up in memory usage. Currently they're at

>>>>>>>>>> 398K, 796K, and 597K each. Memsnap shows them increasing about

>>>>>>>>>> 800 bytes/hour. Our current system has 4G of RAM (I'm NOT using

>>>>>>>>>> the /3gb switch) and shows about 92M non paged and 260M paged in

>>>>>>>>>> use.

>>>>>>>>>>

>>>>>>>>>> I've done some internet searching, as well as Microsoft Technet,

>>>>>>>>>> and I can't find anything that would indicate where these are

>>>>>>>>>> used. If anyone can give me some information on them, I would

>>>>>>>>>> appreciate it.

>>>>>>>>>>

>>>>>>>>>> Mike O.

>>>>>>>>>

>>>>>>>>>

>>>>>>>>

>>>>>>>

>>>>>>>

>>>>>>

>>>>>

>>>>>

>>>>

>>>

>>>

>>

>

>

×
×
  • Create New...