bcp input from tape dev? 
Author Message
 bcp input from tape dev?

Hi,

Question says most of it.  Working in a System 10 environment on Sparc hard-
ware, is it possible to load large quantities of data directly from a tape
device?  The bcp description of 'datafile' says nothing of how this might
be done.  There are two reasons.  The most pressing is the need to conserve
disk space with the large quantities of data our Server must absorb.  The
second is that it seems to be clear that a performance hit would be taken to
have to read the data once from the tape with either 'tar' or 'dd', then
read it again with 'bcp'.

There are a few unix variants I can think of, although I've tried neither:

a) use a raw device file for 'datafile'.  This would grant almost no
control over the datastream.

b) if bcp allows a "-" (dash) for input from stdin, then pipe input from
dd through to bcp. I'm not too hopeful of this one.

Any tip appreciated.  TIA.  -Dean
--
Dean Banfield  IDD/Information Svcs
               90 Grove St
               Ridgefield CT 06877



Sat, 01 Feb 1997 04:13:53 GMT
 bcp input from tape dev?


Quote:
>Question says most of it.  Working in a System 10 environment on Sparc hard-
>ware, is it possible to load large quantities of data directly from a tape
>device?  The bcp description of 'datafile' says nothing of how this might
>be done.  There are two reasons.  The most pressing is the need to conserve
>disk space with the large quantities of data our Server must absorb.  The
>second is that it seems to be clear that a performance hit would be taken to
>have to read the data once from the tape with either 'tar' or 'dd', then
>read it again with 'bcp'.

>There are a few unix variants I can think of, although I've tried neither:

>a) use a raw device file for 'datafile'.  This would grant almost no
>control over the datastream.

>b) if bcp allows a "-" (dash) for input from stdin, then pipe input from
>dd through to bcp. I'm not too hopeful of this one.

i did some work with the bcp library (Open Client 4.6, i think),
and quickly discovered that the bcpexec() call attempts an lseek()
on the file you feed it.  this ruled out using a fifo, which was what
i was trying to do.  you can infer the other implications...
deep down, of course, the bcp program is using the same library call.

i don't know if there is a significantly different implementation available
for System 10 (i.e. with the CTlibrary, i think they call it)

-- don kossman, sei information technology



Sun, 02 Feb 1997 23:37:38 GMT
 bcp input from tape dev?

Quote:

> Subject: bcp input from tape dev?

> Hi,

> Question says most of it.  Working in a System 10 environment on Sparc hard-
> ware, is it possible to load large quantities of data directly from a tape
> device?  The bcp description of 'datafile' says nothing of how this might
> be done.  There are two reasons.  The most pressing is the need to conserve
> disk space with the large quantities of data our Server must absorb.  The
> second is that it seems to be clear that a performance hit would be taken to
> have to read the data once from the tape with either 'tar' or 'dd', then
> read it again with 'bcp'.

> There are a few unix variants I can think of, although I've tried neither:

> a) use a raw device file for 'datafile'.  This would grant almost no
> control over the datastream.

> b) if bcp allows a "-" (dash) for input from stdin, then pipe input from
> dd through to bcp. I'm not too hopeful of this one.

> Any tip appreciated.  TIA.  -Dean

We have done bcp's in and out to tapes using System 10 on SCO without any
problems.  It seems that bcp will accept any byte stream file.  We have
also used named pipes to compress or uncompress the file during the bcp
process with no problem.

David



Mon, 03 Feb 1997 08:53:06 GMT
 bcp input from tape dev?

Quote:

>There are a few unix variants I can think of, although I've tried neither:

>a) use a raw device file for 'datafile'.  This would grant almost no
>control over the datastream.

>b) if bcp allows a "-" (dash) for input from stdin, then pipe input from
>dd through to bcp. I'm not too hopeful of this one.

c) use sybperl :-)

as in:
----------- % ------ % ------ % ------- % -----------
#!/usr/local/bin/sybperl

require 'sybperl.pl';

$dbNullIsUndef = 1;

&BCP_SETL($DBTRUE);
$d1 = &dblogin;
open(TAPE, "dd if=/dev/rst0 |") || die "Can't read data from tape: $!\n";
$table = 'mp_test.dbo.com_info';

$count = 0;
while(<TAPE>)
{


    {
        $col = undef if(length($col) == 0);
        $col =~ s/\n$//;
    }
    if(!$count)
    {
        &bcp_init($d1, "$table", undef, "bcp.err", $DB_IN);

    }

    ++$count;

    if(($count % 1000) == 0)
    {
        $ret = &bcp_batch($d1);
        print "Sent $ret rows to the server\n";
    }

Quote:
}

$ret = &bcp_done($d1);
print "$ret rows returned by &bcp_done\n";
----------- % ------ % ------ % ------- % -----------

This code expects the data to be in ascii, with tabs separating fields
and newlines separating records. If your data is in a different format
you'd need to put in some code to parse it right after the while()
statement.

Michael
--

ITF Management SA           |                          |   DBA,
13 Rue de la Fontaine       | Phone: (+4122) 312 1311  | Programmer
CH-1204 Geneva, Switzerland | Fax:   (+4122) 312 1325  | & Trader...



Sun, 09 Feb 1997 17:05:26 GMT
 
 [ 4 post ] 

 Relevant Pages 

1. Setting tape dev to NULL

2. BCP directly to/from a tape?

3. bcp out straight to tape

4. bcp in from tape drive???

5. Using A Pipe as a row delimiter with bcp input

6. direct input vs BCP

7. bcp input and date format

8. bcp and input date format YYYYMMDD

9. ASCII Text Date input to BCP

10. bcp input and date format

11. BCP - Variable Length Input?

12. How to BCP input a quoted CSV file


 
Powered by phpBB® Forum Software