Performance: NT4 Paging 'v' Informix 
Author Message
 Performance: NT4 Paging 'v' Informix

Hi,

We're running an Informix server with a large DB (2gb).  I've got one of the
tables (300mb) loaded into memory.  Problem is, our server only has 512mb
RAM, and memory usage = 1.2gb.

Obviously Windows is paging 70% of the memory to the swap file.

My question:  Is performance better if I return all of our tables to
non-memory resident, thus cutting out Window's paging and let Informix work
directly off of the hdd, or is it quicker to leave them how they are?

We're probably going to upgrade to 2gb RAM, with which I assume it's safe to
load my 300mb table into RAM..

Cheers/Thanks,
Ryan Barrett



Fri, 04 Jun 2004 21:50:41 GMT
 Performance: NT4 Paging 'v' Informix

I would think that it would be better to avoid the paging.

By increasing your memory to the point that you are paging, you are decreasing
the performance to the system as a whole instead of just the access to one
table.  It does no good to increase the resident table and then have to wait
while something else is being paged in.  You might discover that by overloading
the physical memory, you are having to do IO multiple times in your code path
instead of only having to do IO only once to access the table that is currently
resident.

Quote:

> Hi,

> We're running an Informix server with a large DB (2gb).  I've got one of the
> tables (300mb) loaded into memory.  Problem is, our server only has 512mb
> RAM, and memory usage = 1.2gb.

> Obviously Windows is paging 70% of the memory to the swap file.

> My question:  Is performance better if I return all of our tables to
> non-memory resident, thus cutting out Window's paging and let Informix work
> directly off of the hdd, or is it quicker to leave them how they are?

> We're probably going to upgrade to 2gb RAM, with which I assume it's safe to
> load my 300mb table into RAM..

> Cheers/Thanks,
> Ryan Barrett

--
---------------------------------------------------------
Madison Pruet
Enterprise Replication Product Development
IBM Informix Dynamic Server


Fri, 04 Jun 2004 22:38:08 GMT
 Performance: NT4 Paging 'v' Informix


Quote:
>We're running an Informix server with a large DB (2gb).  I've got one of the
>tables (300mb) loaded into memory.  Problem is, our server only has 512mb
>RAM, and memory usage = 1.2gb.

>Obviously Windows is paging 70% of the memory to the swap file.

>My question:  Is performance better if I return all of our tables to
>non-memory resident, thus cutting out Window's paging and let Informix work
>directly off of the hdd, or is it quicker to leave them how they are?

>We're probably going to upgrade to 2gb RAM, with which I assume it's safe to
>load my 300mb table into RAM..

I'd make all your tables non-memory-resident, and let the caching
algorithm do the work. How many BUFFERS are in your onconfig? You'll
have to drop that, too.


Fri, 04 Jun 2004 22:41:26 GMT
 Performance: NT4 Paging 'v' Informix
Urm, my colleague (who's leaving) set everything up.  I'm taking over his
position in 6 weeks, so I'm reading all of his course material and books
atm..

The buffers = 200,000 currently, I've included all of the other setting from
that section of the Onconfig here incase they'll be relevant..

# Shared Memory Parameters

LOCKS  100000  # Maximum number of locks
BUFFERS  200000  # Maximum number of shared buffers
NUMAIOVPS 2    # Number of IO vps
PHYSBUFF 64  # Physical log buffer size (Kbytes)
LOGBUFF  32  # Logical log buffer size (Kbytes)
LOGSMAX  20  # Maximum number of logical log files
CLEANERS        127             # Number of buffer cleaner processes
SHMBASE         0xC000000L # Shared memory base address
SHMVIRTSIZE 16384           # initial virtual shared memory segment size
SHMADD          8192            # Size of new shared memory segments
(Kbytes)
SHMTOTAL        0               # Total shared memory (Kbytes). 0=>unlimited
CKPTINTVL       300             # Check point interval (in sec)
LRUS  127  # Number of LRU queues
LRU_MAX_DIRTY 2  # LRU percent dirty begin cleaning limit
LRU_MIN_DIRTY 1  # LRU percent dirty end cleaning limit
LTXHWM  50  # Long transaction high water mark percentage
LTXEHWM  60  # Long transaction high water mark (exclusive)
TXTIMEOUT 300  # Transaction timeout (in sec)
STACKSIZE 32  # Stack size (Kbytes)

Cheers,

--
Ryan Barrett



Quote:


> I'd make all your tables non-memory-resident, and let the caching
> algorithm do the work. How many BUFFERS are in your onconfig? You'll
> have to drop that, too.



Sun, 06 Jun 2004 00:51:46 GMT
 Performance: NT4 Paging 'v' Informix

Quote:

>Urm, my colleague (who's leaving) set everything up.  I'm taking over his
>position in 6 weeks, so I'm reading all of his course material and books
>atm..

>The buffers = 200,000 currently, I've included all of the other setting from
>that section of the Onconfig here incase they'll be relevant..

># Shared Memory Parameters

Best guesses, as I'm not familiar with your system particulars (OS, rev. level, physical setup, etc.)

Quote:
>LOCKS  100000  # Maximum number of locks
>BUFFERS  200000  # Maximum number of shared buffers
>NUMAIOVPS 2    # Number of IO vps

I'm presuming that your platform supports KAIO with this setting this low.

Quote:
>PHYSBUFF 64  # Physical log buffer size (Kbytes)
>LOGBUFF  32  # Logical log buffer size (Kbytes)
>LOGSMAX  20  # Maximum number of logical log files
>CLEANERS        127             # Number of buffer cleaner processes
>SHMBASE         0xC000000L # Shared memory base address
>SHMVIRTSIZE 16384           # initial virtual shared memory segment size

How many segments get added between reboots?  If you have multiple virtual segments, you might have some performance issues
depending on the OS (like HPUX).

Quote:
>SHMADD          8192            # Size of new shared memory segments
>(Kbytes)
>SHMTOTAL        0               # Total shared memory (Kbytes). 0=>unlimited
>CKPTINTVL       300             # Check point interval (in sec)
>LRUS  127  # Number of LRU queues
>LRU_MAX_DIRTY 2  # LRU percent dirty begin cleaning limit
>LRU_MIN_DIRTY 1  # LRU percent dirty end cleaning limit
>LTXHWM  50  # Long transaction high water mark percentage
>LTXEHWM  60  # Long transaction high water mark (exclusive)
>TXTIMEOUT 300  # Transaction timeout (in sec)
>STACKSIZE 32  # Stack size (Kbytes)

--------------------------------
John Carlson
Informix DBA
EDS/WHSmith USA

#include std_disclaimer.h     /* These are my opinions, not my company's opinion  */



Sun, 06 Jun 2004 02:18:21 GMT
 Performance: NT4 Paging 'v' Informix


Quote:
>Urm, my colleague (who's leaving) set everything up.  I'm taking over his
>position in 6 weeks, so I'm reading all of his course material and books
>atm..

Hoo-hah! Lucky you!

Quote:
>The buffers = 200,000 currently, I've included all of the other setting from
>that section of the Onconfig here incase they'll be relevant..

Yeah, you'll want to drop that, that's 800MB of RAM by itself. Try
cutting that down to about 64000 buffers. CLEANERS is way too high,
most likely. Try 4 as a start. (Although it's not really relevant to
your question. :)
Quote:
># Shared Memory Parameters

>LOCKS  100000  # Maximum number of locks
>BUFFERS  200000  # Maximum number of shared buffers
>NUMAIOVPS 2    # Number of IO vps
>PHYSBUFF 64  # Physical log buffer size (Kbytes)
>LOGBUFF  32  # Logical log buffer size (Kbytes)
>LOGSMAX  20  # Maximum number of logical log files
>CLEANERS        127             # Number of buffer cleaner processes
>SHMBASE         0xC000000L # Shared memory base address
>SHMVIRTSIZE 16384           # initial virtual shared memory segment size
>SHMADD          8192            # Size of new shared memory segments
>(Kbytes)
>SHMTOTAL        0               # Total shared memory (Kbytes). 0=>unlimited
>CKPTINTVL       300             # Check point interval (in sec)
>LRUS  127  # Number of LRU queues
>LRU_MAX_DIRTY 2  # LRU percent dirty begin cleaning limit
>LRU_MIN_DIRTY 1  # LRU percent dirty end cleaning limit
>LTXHWM  50  # Long transaction high water mark percentage
>LTXEHWM  60  # Long transaction high water mark (exclusive)
>TXTIMEOUT 300  # Transaction timeout (in sec)
>STACKSIZE 32  # Stack size (Kbytes)

>Cheers,

>--
>Ryan Barrett





>> I'd make all your tables non-memory-resident, and let the caching
>> algorithm do the work. How many BUFFERS are in your onconfig? You'll
>> have to drop that, too.



Sun, 06 Jun 2004 04:07:30 GMT
 Performance: NT4 Paging 'v' Informix

hmmm...  512Mb Ram in your box if I recall right.
200,000 BUFFERS = 800Mb.  I am surprised this works at all on NT.  We have
always had problems if memory was overallocated in this platform
(instability of NT).
SHMVIRTSIZE = 16Mb.  How many V type segments does onstat -g seg show?

MW

Quote:
-----Original Message-----


Sent: Wednesday, 19 December 2001 5:52 a.m.

Subject: Re: Performance: NT4 Paging 'v' Informix

Urm, my colleague (who's leaving) set everything up.  I'm taking over his
position in 6 weeks, so I'm reading all of his course material and books
atm..

The buffers = 200,000 currently, I've included all of the other setting from
that section of the Onconfig here incase they'll be relevant..

# Shared Memory Parameters

LOCKS  100000  # Maximum number of locks
BUFFERS  200000  # Maximum number of shared buffers
NUMAIOVPS 2    # Number of IO vps
PHYSBUFF 64  # Physical log buffer size (Kbytes)
LOGBUFF  32  # Logical log buffer size (Kbytes)
LOGSMAX  20  # Maximum number of logical log files
CLEANERS        127             # Number of buffer cleaner processes
SHMBASE         0xC000000L # Shared memory base address
SHMVIRTSIZE 16384           # initial virtual shared memory segment size
SHMADD          8192            # Size of new shared memory segments
(Kbytes)
SHMTOTAL        0               # Total shared memory (Kbytes). 0=>unlimited
CKPTINTVL       300             # Check point interval (in sec)
LRUS  127  # Number of LRU queues
LRU_MAX_DIRTY 2  # LRU percent dirty begin cleaning limit
LRU_MIN_DIRTY 1  # LRU percent dirty end cleaning limit
LTXHWM  50  # Long transaction high water mark percentage
LTXEHWM  60  # Long transaction high water mark (exclusive)
TXTIMEOUT 300  # Transaction timeout (in sec)
STACKSIZE 32  # Stack size (Kbytes)

Cheers,

--
Ryan Barrett





> I'd make all your tables non-memory-resident, and let the caching
> algorithm do the work. How many BUFFERS are in your onconfig? You'll
> have to drop that, too.



Sun, 06 Jun 2004 04:52:00 GMT
 Performance: NT4 Paging 'v' Informix
Hmmm...first, if performance and/or scalability matters, you need to face facts
and get off Windows.  Any of the major Unixes would be a vast improvement.

Now, with that personal bias out of the way, I'll try to be constructive within
the current situation.   I genuinely doubt Windoze can profitably use 2 GB of
RAM.   It's dirt cheap, so you might as well try it, but don't bank on it.  My
experience has been the performance of any windows machine is a direct function
of the speed of the disk drives.   Practically nothing else matters.  Adding
copious amounts of memory doesn't seem to stop windows from thrashing the hard
drive, so it needs to be as fast as possible.   I've seen 15,000 RPM Seagate
Cheetah SCSI drives  - I'd get as many as I could and stripe liberally.
Include the Windows and app code as well as the swap/page files in the striping
scheme as well, that's where all of the pointless  thrashing is.

I don't know what Windows NT/2000's problem is with thrashing the hard disk is,
but no amount of memory seems to stop it or even slow it down.  I once sat down
for an afternoon and searched every promising looking nook & cranny of the 'net
trying to find a solution.   There isn't one.   Get fast disks.   If that isn't
fast enough, get Unix.

Obnoxio's idea earlier seems worth trying, might as well reduce BUFFERS, because
the {*filter*} NT cache is going to duplicate the data anyhow.  Might as well let
it do the caching work.

2GB isn't a very big DB these days, btw.

Good Luck and welcome to the Informix party.   Sorry you have to experience it
on Windoze.   It's really great on Unix.   Actually, it's probably the best of
the lot on Windows, it's just that Windows is a woeful drag on any DBMS.

Greg

Quote:

> Hi,

> We're running an Informix server with a large DB (2gb).  I've got one of the
> tables (300mb) loaded into memory.  Problem is, our server only has 512mb
> RAM, and memory usage = 1.2gb.

> Obviously Windows is paging 70% of the memory to the swap file.

> My question:  Is performance better if I return all of our tables to
> non-memory resident, thus cutting out Window's paging and let Informix work
> directly off of the hdd, or is it quicker to leave them how they are?

> We're probably going to upgrade to 2gb RAM, with which I assume it's safe to
> load my 300mb table into RAM..

> Cheers/Thanks,
> Ryan Barrett



Sun, 06 Jun 2004 13:02:32 GMT
 Performance: NT4 Paging 'v' Informix
I'm of the mind that Unix is vastly superiour, but it's company policy to
use NT, so convincing them is going to be a BIG problem. Currently we have
2gb, this time next year we'll be expanding to 20gb - still not massive.

We've got four fast SCSI drives in the box am, so disk-IO shouldn't be a
problem.

I'm tempted to disable paging and throw 2gb ram into the box.

--
Ryan Barrett


Quote:
> Hmmm...first, if performance and/or scalability matters, you need to face
facts
> and get off Windows.  Any of the major Unixes would be a vast improvement.

> 2GB isn't a very big DB these days, btw.

> Good Luck and welcome to the Informix party.   Sorry you have to
experience it
> on Windoze.   It's really great on Unix.   Actually, it's probably the
best of
> the lot on Windows, it's just that Windows is a woeful drag on any DBMS.

> Greg



Sun, 06 Jun 2004 19:06:29 GMT
 Performance: NT4 Paging 'v' Informix


Quote:
>I'm of the mind that Unix is vastly superiour, but it's company policy to
>use NT, so convincing them is going to be a BIG problem. Currently we have
>2gb, this time next year we'll be expanding to 20gb - still not massive.

>We've got four fast SCSI drives in the box am, so disk-IO shouldn't be a
>problem.

>I'm tempted to disable paging and throw 2gb ram into the box.

Try at least 4GB. NT seems to hog a lot of RAM, just judging by my
laptop.
Quote:
>--
>Ryan Barrett



>> Hmmm...first, if performance and/or scalability matters, you need to face
>facts
>> and get off Windows.  Any of the major Unixes would be a vast improvement.

>> 2GB isn't a very big DB these days, btw.

>> Good Luck and welcome to the Informix party.   Sorry you have to
>experience it
>> on Windoze.   It's really great on Unix.   Actually, it's probably the
>best of
>> the lot on Windows, it's just that Windows is a woeful drag on any DBMS.

>> Greg



Sun, 06 Jun 2004 20:46:24 GMT
 
 [ 10 post ] 

 Relevant Pages 

1. ADO's Performance vs DAO's Performance

2. **************!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!Help me !!!!!!!!!!!!!!!!!!!!!!!!'''''''''''''''''''''''*************

3. PRESS: SPARCSERVER 1000 RUNNING INFORMIX DELIVERS INDUSTRY'S BEST DATABASE PRICE/PERFORMANCE

4. SELECT permission denied on object 'mytable', database 'mydb', owner 'me'. to run query string from asp web page

5. 'Attempt to fetch logical page' error

6. Johansen Software's TP Programmer's Page

7. multiple pages (do's and don'ts) PDOXWIN45

8. 'return' on web page form

9. VFP5 - Deriving from class 'page'

10. Page 'P' command in Universe

11. Datatype 'Performance'

12. Performance overheads with WHERE x LIKE '%'


 
Powered by phpBB® Forum Software