SyBase, Raw devices Vs. File system devices

Raw devices Vs. File system devices

Author	Message
Nadav Licht #1 / 23	Raw devices Vs. File system devices Hey there! I'm a DBA for a large database project (20+ giga) base on sybase 11.0. We have just finished our Development phase and about to enter Production. The project is set on a Digital Alpha server, and is heavily based on the Sybase server for Storing Processing and Retrieving the information. Until now we have worked with Sybase devices which were file system, and it has been suggested that we switch to raw devices. I have heard raw devices improve performance especialy on large bcp's or select into's but our tests have not shown great increase in perrformance. I would appreciate anyone's point of few on the matter. If you could help me with the pros and cons of Raw devices Vs. File system. Any personal experience would also be apreciated. Thanks a lot in advance. Nadav Lichter ---> Personal e-mail messages on the matter would be greatly apreciated as well.
Sat, 28 Aug 1999 03:00:00 GMT

Tim Azzopard #2 / 23	Raw devices Vs. File system devices My information may be out of date but from what I remember there is NO option for a production environment. Databases on file systems are not guaranteed fully recoverable (but are usually ok for developpement). Only production databases on raw devices are safe. Quote: > I have heard raw devices improve performance especialy on large bcp's or > select into's but our tests have not shown great increase in > perrformance.
Sat, 28 Aug 1999 03:00:00 GMT

Michael Shael O'Neill Selde #3 / 23	Raw devices Vs. File system devices Quote: > Hey there! > I'm a DBA for a large database project (20+ giga) base on sybase 11.0. > We have just finished our Development phase and about to enter > Production. > The project is set on a Digital Alpha server, and is heavily based on the > Sybase server for Storing Processing and Retrieving the information. > Until now we have worked with Sybase devices which were file system, and > it has been suggested that we switch to raw devices. > I have heard raw devices improve performance especialy on large bcp's or > select into's but our tests have not shown great increase in > perrformance. > I would appreciate anyone's point of few on the matter. If you could help > me with the pros and cons of Raw devices Vs. File system. Any personal > experience would also be apreciated. > Thanks a lot in advance. > Nadav Lichter > ---> Personal e-mail messages on the matter would be greatly apreciated > as well. Nadav, It doesn't sound like whoever is giving you SYBASE advice is a qualified SYBASE DBA. The deal with raw devices is as follows. Raw devices are SLOWER than file system based devices due to the laws of physics (the difference in performance between memory writes and disk writes). When SQL Server writes to a raw device it will wait until a device level acknowledgement is received before it considers the write operation valid (i.e. DATA ON DISK). When using file system based devices, however, that acknowledgement will come from the operating system, not the device, and will only take as long as a write to yet another area in system memory, the file system buffer cache. It is for the above reason that Sybase strongly suggests that raw devices be used despite the decrease in performance. If your host computer takes a hit before the operating system has written the contents of the file system buffer cache to disk your data is completely lost. Worse yet, your database transactional consistency is also shot because SYBASE issues COMMITs based on the assumption that the log pages for any given transaction have been written to disk; with log pages on disk SYBASE can recover (roll forward/backward) any given transaction. Good Luck, Michael. -- Michael Shael O'Neill Selden PGP Public Key Available Upon Request
Sat, 28 Aug 1999 03:00:00 GMT

Todd Bo #4 / 23	Raw devices Vs. File system devices Quote: >Nadav, > It doesn't sound like whoever is giving you SYBASE advice >is a qualified SYBASE DBA. The deal with raw devices is as follows. > Raw devices are SLOWER than file system based devices due >to the laws of physics (the difference in performance between memory >writes and disk writes). When SQL Server writes to a raw device it >will wait until a device level acknowledgement is received before it >considers the write operation valid (i.e. DATA ON DISK). When using >file system based devices, however, that acknowledgement will come from >the operating system, not the device, and will only take as long as a >write to yet another area in system memory, the file system buffer >cache. This statement runs counter to several tenets of sql server design that i've taken to heart. By mkfs'ing a raw partition, you add a level of "sequentiality" inherent to Unix file systems and thus eliminate many of the performance advantages built into SCSI devices, thus making file systems slower than raw devices, not vice versa what you state above. Whenver i hear arguments pro-raw devices for sybase devices (save tempdb) there are two points; write buffering causing corruption during crashes, and Ufs {filter} of SCSI performance. perhaps i'm wrong? boss
Sun, 29 Aug 1999 03:00:00 GMT

Rajesh Pilla #5 / 23	Raw devices Vs. File system devices Quote: > Hey there! > I'm a DBA for a large database project (20+ giga) base on sybase 11.0. > We have just finished our Development phase and about to enter > Production. > The project is set on a Digital Alpha server, and is heavily based on the > Sybase server for Storing Processing and Retrieving the information. > Until now we have worked with Sybase devices which were file system, and > it has been suggested that we switch to raw devices. > I have heard raw devices improve performance especialy on large bcp's or > select into's but our tests have not shown great increase in > perrformance. > I would appreciate anyone's point of few on the matter. If you could help > me with the pros and cons of Raw devices Vs. File system. Any personal > experience would also be apreciated. > Thanks a lot in advance. > Nadav Lichter > ---> Personal e-mail messages on the matter would be greatly apreciated > as well. Speaking from personal experience, you should be ok placing tempdb devices on file systems.. This does buy you some perf. gain in most cases and the risk is minimal since tempdb is rebuilt on every reboot.. Even if you get corruption on tempdb device, recovering from that should be fairly simple and straight-forward.. But for all other databases, raw partitions would be the way to go... That is the best way to avoid corruption and guarantee recoverability.. Hope this helps, Rajesh Pillai Perot Systems Corp.
Sun, 29 Aug 1999 03:00:00 GMT

Laurent Doublei #6 / 23	Raw devices Vs. File system devices Quote: > > Hey there! > > I'm a DBA for a large database project (20+ giga) base on sybase 11.0. > > We have just finished our Development phase and about to enter > > Production. > > The project is set on a Digital Alpha server, and is heavily based on the > > Sybase server for Storing Processing and Retrieving the information. > > Until now we have worked with Sybase devices which were file system, and > > it has been suggested that we switch to raw devices. > > I have heard raw devices improve performance especialy on large bcp's or > > select into's but our tests have not shown great increase in > > perrformance. > > I would appreciate anyone's point of few on the matter. If you could help > > me with the pros and cons of Raw devices Vs. File system. Any personal > > experience would also be apreciated. > > Thanks a lot in advance. > > Nadav Lichter > > ---> Personal e-mail messages on the matter would be greatly apreciated > > as well. > Speaking from personal experience, you should be ok placing tempdb > devices on file systems.. This does buy you some perf. gain in most > cases and the risk is minimal since tempdb is rebuilt on every > reboot.. Even if you get corruption on tempdb device, recovering > from that should be fairly simple and straight-forward.. > But for all other databases, raw partitions would be the way to go... > That is the best way to avoid corruption and guarantee recoverability.. > Hope this helps, > Rajesh Pillai > Perot Systems Corp. Hello I think like Rajesh, that it is better to put to put tempdb on File system. I had databases on file syetem and when I put it on raw devices i had a gain of 20%. Check in your errorlog, if at the boot of the sql server it is recognized as a raw device like this 00:96/08/07 15:06:47.93 kernel Initializing virtual device 11, '/dev/vg01/IPF_data1' 00:96/08/07 15:06:47.93 kernel Virtual device 11 started using asynchronous i/o. I hope this help you Laurent Doublein
Mon, 30 Aug 1999 03:00:00 GMT

Michael Shael O'Neill Selde #7 / 23	Raw devices Vs. File system devices Quote: > >Nadav, > > It doesn't sound like whoever is giving you SYBASE advice > >is a qualified SYBASE DBA. The deal with raw devices is as follows. > > Raw devices are SLOWER than file system based devices due > >to the laws of physics (the difference in performance between memory > >writes and disk writes). When SQL Server writes to a raw device it > >will wait until a device level acknowledgement is received before it > >considers the write operation valid (i.e. DATA ON DISK). When using > >file system based devices, however, that acknowledgement will come from > >the operating system, not the device, and will only take as long as a > >write to yet another area in system memory, the file system buffer > >cache. > This statement runs counter to several tenets of sql server design that > i've taken to heart. By mkfs'ing a raw partition, you add a level > of "sequentiality" inherent to Unix file systems and thus eliminate many > of the performance advantages built into SCSI devices, thus making file > systems slower than raw devices, not vice versa what you state above. > Whenver i hear arguments pro-raw devices for sybase devices (save tempdb) > there are two points; write buffering causing corruption during crashes, > and Ufs {filter} of SCSI performance. > perhaps i'm wrong? > boss Hi, I am not discussing the relative merits of raw devices versus file systems in a general sense, rather I am speaking to how SYBASE SQL Server interacts with the OS layer in making writes to configured DBMS devices. In that context SQL Server, when it writes to file system based devices, will wait for a write acknowledgement from an operation which only has to go to memory (the file system buffer cache) to complete, as opposed to actually waiting for the OS to write to the physical disk in the case of a raw device operation. Looking back at my original post I can see how my first statement (raw devices are slower ...) could have been understood to be a general comparison between raw and file system based devices. In fact I was (as stated above) talking about SYBASE devices ___created on___ raw or file system based OS level devices. I hope this clarifies where I am coming from because I don't think that you and I are contradicting each other, and your observation regarding the nature of file system Cheers, Michael. -- Michael Shael O'Neill Selden PGP Public Key Available Upon Request
Tue, 31 Aug 1999 03:00:00 GMT

Michael Shael O'Neill Selde #8 / 23	Raw devices Vs. File system devices Quote: > >Nadav, > > It doesn't sound like whoever is giving you SYBASE advice > >is a qualified SYBASE DBA. The deal with raw devices is as follows. > > Raw devices are SLOWER than file system based devices due > >to the laws of physics (the difference in performance between memory > >writes and disk writes). When SQL Server writes to a raw device it > >will wait until a device level acknowledgement is received before it > >considers the write operation valid (i.e. DATA ON DISK). When using > >file system based devices, however, that acknowledgement will come from > >the operating system, not the device, and will only take as long as a > >write to yet another area in system memory, the file system buffer > >cache. > This statement runs counter to several tenets of sql server design that > i've taken to heart. By mkfs'ing a raw partition, you add a level > of "sequentiality" inherent to Unix file systems and thus eliminate many > of the performance advantages built into SCSI devices, thus making file > systems slower than raw devices, not vice versa what you state above. > Whenver i hear arguments pro-raw devices for sybase devices (save tempdb) > there are two points; write buffering causing corruption during crashes, > and Ufs {filter} of SCSI performance. > perhaps i'm wrong? > boss Hi, I am not discussing the relative merits of raw devices versus file systems in a general sense, rather I am speaking to how SYBASE SQL Server interacts with the OS layer in making writes to configured DBMS devices. In that context SQL Server, when it writes to file system based devices, will wait for a write acknowledgement from an operation which only has to go to memory (the file system buffer cache) to complete, as opposed to actually waiting for the OS to write to the physical disk in the case of a raw device operation. Looking back at my original post I can see how my first statement (raw devices are slower ...) could have been understood to be a general comparison between raw and file system based devices. In fact I was (as stated above) talking about SYBASE devices ___created on___ raw or file system based OS level devices. I hope this clarifies where I am coming from because I don't think that you and I are contradicting each other, and your observation regarding the nature of file system interaction with SCSI device optimization is, I think, quite accurate. Cheers, Michael. -- Michael Shael O'Neill Selden PGP Public Key Available Upon Request
Tue, 31 Aug 1999 03:00:00 GMT

Mike Ma #9 / 23	Raw devices Vs. File system devices Quote: >> >Nadav, >> > It doesn't sound like whoever is giving you SYBASE advice >> >is a qualified SYBASE DBA. The deal with raw devices is as follows. >> > Raw devices are SLOWER than file system based devices due >> >to the laws of physics (the difference in performance between memory >> >writes and disk writes). When SQL Server writes to a raw device it >> >will wait until a device level acknowledgement is received before it >> >considers the write operation valid (i.e. DATA ON DISK). When using >> >file system based devices, however, that acknowledgement will come from >> >the operating system, not the device, and will only take as long as a >> >write to yet another area in system memory, the file system buffer >> >cache. >> This statement runs counter to several tenets of sql server design that >> i've taken to heart. By mkfs'ing a raw partition, you add a level >> of "sequentiality" inherent to Unix file systems and thus eliminate many >> of the performance advantages built into SCSI devices, thus making file >> systems slower than raw devices, not vice versa what you state above. >> Whenver i hear arguments pro-raw devices for sybase devices (save tempdb) >> there are two points; write buffering causing corruption during crashes, >> and Ufs {filter} of SCSI performance. >> perhaps i'm wrong? >> boss >Hi, > I am not discussing the relative merits of raw devices versus file >systems in a general sense, rather I am speaking to how SYBASE SQL >Server >interacts with the OS layer in making writes to configured DBMS devices. >In that context SQL Server, when it writes to file system based devices, >will wait for a write acknowledgement from an operation which only has >to >go to memory (the file system buffer cache) to complete, as opposed to >actually waiting for the OS to write to the physical disk in the case of >a raw device operation. I think another level of granularity will help clarify things a little here. What has been said so far is accurate but at the raw device level there are two methods of writing synchronous and asynchronous. In synchronous operations the server will sit idle waiting for the write ack from the os to the raw device. In asynchronous operations the server will issue the write request then instead of sitting idle go off and service other tasks. The task that requested the write will wait for the ack but the server itself continues to do other work for other tasks. Also, actual writes to disk are only forced for logrecords and not for data records which are written to disk only if the buffer is needed for another task or at checkpoint time. Mike - Hide quoted text - - Show quoted text - Quote: > Looking back at my original post I can see how my first statement >(raw devices are slower ...) could have been understood to be a general >comparison between raw and file system based devices. In fact I was (as >stated above) talking about SYBASE devices ___created on___ raw or file >system based OS level devices. > I hope this clarifies where I am coming from because I don't think >that you and I are contradicting each other, and your observation >regarding >the nature of file system >Cheers, >Michael. >-- >Michael Shael O'Neill Selden >PGP Public Key Available Upon Request
Tue, 31 Aug 1999 03:00:00 GMT

Steve Chowle #10 / 23	Raw devices Vs. File system devices I agree with what has been written in the other posts however, there has been no mention of a disk arrays capability of fast writes. This is where the request in cached in the controller and an acknowledgement is sent back. The controller will then schedule the write when it sees fit. An internal battery in the case of a power failure will ensure no data is lost. At least this is the case in Suns StorageArray. -- -------------------------------- - Steve Chowles - - http://www.scho.demon.co.uk/ - --------------------------------
Wed, 01 Sep 1999 03:00:00 GMT

J.Masino/T.Lars #11 / 23	Raw devices Vs. File system devices : I'm a DBA for a large database project (20+ giga) base on sybase 11.0. : We have just finished our Development phase and about to enter : Production. : The project is set on a Digital Alpha server, and is heavily based on the : Sybase server for Storing Processing and Retrieving the information. : : Until now we have worked with Sybase devices which were file system, and : it has been suggested that we switch to raw devices. : I have heard raw devices improve performance especialy on large bcp's or : select into's but our tests have not shown great increase in : perrformance. They key issue here isn't really performance, it's database integrity. You risk database corruption when you use cooked files. Since the OS buffers the writes to disk, there is an opportunity for corruption. Sybase issues a write command, the OS buffers it up, then system crashes before the write is actually accomplished, poof you're hosed. Using files (cooked devices) is ok in a development environment -- or at least a lot of us are more willing to risk it in that environment. And we have learned over time that it is a good idea to put tempdb on a cooked device -- because it usually improves performance and tempdb is never recovered (it is recreated every time the server is restarted). But Sybase strongly urges us to use raw partitions for data. I think you'll find their discussion of the issue in one of the installation manuals. Do your project a favor and set up the environment on raw partitions. Recovering 20+ gig is no way to be spending your week-ends. Hope this helps Teresa Larson ___________________________________________________________________ / Teresa A. Larson http://www2.ari.net/jmasino/ / / Bell Atlantic Voice: (301) 282-0051 / / 13100 Columbia Pike, A-3-3 Fax: (301) 282-9416 / /__________________________________________________________________/ #include <std_disclaimer>
Thu, 02 Sep 1999 03:00:00 GMT

Al Bleds #12 / 23	Raw devices Vs. File system devices : : I'm a DBA for a large database project (20+ giga) base on sybase 11.0. If large == 20+ gig, How much is 300+ gig? (Using EMC Storage Array as HPUX logical volumes.) (How do you get more than 32 devices per database without using Oracle?)
Fri, 03 Sep 1999 03:00:00 GMT

Al Bleds #13 / 23	Raw devices Vs. File system devices : I agree with what has been written in the other posts however, : there has been no mention of a disk arrays capability of fast : writes. This is where the request in cached in the controller : and an acknowledgement is sent back. The controller will then : schedule the write when it sees fit. An internal battery in : the case of a power failure will ensure no data is lost. At : least this is the case in Suns StorageArray. : -- : -------------------------------- : - Steve Chowles - : - http://www.scho.demon.co.uk/ - : -------------------------------- Ah, yes, we finally get to the battery. This, today, becomes the most important feature for anyone keeping up with drive technology and storage arrays. The AMOUNT of sold-state cache makes a lot of arguments mute doesn't it? Even RAID-5 versus RAID-1, synchronous versus asynchronous, and raw versus vhfs! But, you're only as good as that battery: what kind, how many, etc?
Fri, 03 Sep 1999 03:00:00 GMT

cat.. #14 / 23	Raw devices Vs. File system devices Quote: > Hey there! > I'm a DBA for a large database project (20+ giga) base on sybase 11.0. > We have just finished our Development phase and about to enter > Production. > The project is set on a Digital Alpha server, and is heavily based on the > Sybase server for Storing Processing and Retrieving the information. > Until now we have worked with Sybase devices which were file system, and > it has been suggested that we switch to raw devices. > I have heard raw devices improve performance especialy on large bcp's or > select into's but our tests have not shown great increase in > perrformance. > I would appreciate anyone's point of few on the matter. If you could help > me with the pros and cons of Raw devices Vs. File system. Any personal > experience would also be apreciated. > Thanks a lot in advance. > Nadav Lichter > ---> Personal e-mail messages on the matter would be greatly apreciated > as well. 1. File systems rely on the OS to handle disk writes. This is inherently unsafe for a production system. Since the OS will cache the write request and tell the DB server that the write has finished. This can lead to loss of data in some situations. 2. Some versions OS makers claim their file systems (particularly JFS) are faster than raw devices. I have not seem that demonstrated. However, this still does not alleviate the data loss problem previously mentioned. 3. In order to get the best performance from the raw devices you must use asyncronous i/o. This option is enabled by default for Sybase 11.x. However, you must turn it on in the OS as well. 4. Some people place tempdb on file system space. Since tempdb is recreated when the server is rebooted, and be definition the data in tempdb is temporary, it is an acceptable option. 5. For all of the above reasons Sybase strongly recommends the use of raw devices. This is also true of other database vendors such as oracle. Oh, BTW, large is a relative thing. For most NT systems 20 Gig is still pretty large. However databases are growing exponentially. So, in the UNIX world 20 Gig isn't all that big now days. good luck, Jim Strange Senior Technical Services Analyst BMG Entertainment
Fri, 03 Sep 1999 03:00:00 GMT

Page 1 of 2

[ 23 post ]

Go to page: [1] [2]