float precision - ALMOST the old same story ! 
Author Message
 float precision - ALMOST the old same story !

Hi,

I'm programming SQL server via OLE-DB using Visual C++ 6.0 and MFC and
OLE-DB interfaces.
I have a double variable (for example 432404.00000000), with the follow
binary rappresentation:

F8 FF FF FF 4F 64 1A 41

So I preapre the query in order to update the column (the column is a
double).

UPDATE ivap01 SET import = 432404.00000000 WHERE etc. etc. etc.

Well, if I reload this row I obtain the double value with the follow
rappresentation:

F7 FF FF FF 4F 64 1A 41

The two binary rappresentation are different......
I know...I know....the float and real column have limited precision.
I suppose SQL server re-convert the value 432404.00000000 into float value
that could have different inner rappresentation from mine.

Well my question is:

is there a way to tell SQL server to use, in the UPDATE query, the binary
rappresentation of the double ?
So I would like to avoide its re-convertion from string to double.

Thank you.
Porlock.



Sun, 27 Feb 2005 18:11:20 GMT
 float precision - ALMOST the old same story !

Hi

In your SQL Table is the column defined as real or float? In SQL 7.0 the
only two options are float (which is the same size as the VB Single (4
bytes)) or real (which is the same size as a VB Double (8 bytes)).

Quote:
> is there a way to tell SQL server to use, in the UPDATE query, the binary
> rappresentation of the double ?

I don't know.

-Dick


Quote:
> Hi,

> I'm programming SQL server via OLE-DB using Visual C++ 6.0 and MFC and
> OLE-DB interfaces.
> I have a double variable (for example 432404.00000000), with the follow
> binary rappresentation:

> F8 FF FF FF 4F 64 1A 41

> So I preapre the query in order to update the column (the column is a
> double).

> UPDATE ivap01 SET import = 432404.00000000 WHERE etc. etc. etc.

> Well, if I reload this row I obtain the double value with the follow
> rappresentation:

> F7 FF FF FF 4F 64 1A 41

> The two binary rappresentation are different......
> I know...I know....the float and real column have limited precision.
> I suppose SQL server re-convert the value 432404.00000000 into float value
> that could have different inner rappresentation from mine.

> Well my question is:

> is there a way to tell SQL server to use, in the UPDATE query, the binary
> rappresentation of the double ?
> So I would like to avoide its re-convertion from string to double.

> Thank you.
> Porlock.



Sun, 27 Feb 2005 20:08:52 GMT
 float precision - ALMOST the old same story !



Quote:
> Hi

> In your SQL Table is the column defined as real or float? In SQL 7.0 the
> only two options are float (which is the same size as the VB Single (4
> bytes)) or real (which is the same size as a VB Double (8 bytes)).

[CUT]

I'm using SQL Server 2000, so float is 8 byte length and real is 4 byte
length.
The column, where I'm having, trouble is float (8 byte)

Porlock.



Sun, 27 Feb 2005 20:46:50 GMT
 float precision - ALMOST the old same story !
Porlock,

books online says(CAST and CONVERT) that the conversion
from binary or varbinary to real or float is not allowed.
I guess the reason is that mssql-server doesn't support
the full IEEE 754 standard as there is no inf or -inf
and maybe no non-normalized mantissa. Thus some valid IEEE
floats are invalid in mssql.
The only way is to save your floats as binary(8) and
maybe in another column as float. then you can order by
or compute in t-sql with float and use the binary as
return value.
cheers,

      robert



Mon, 28 Feb 2005 06:49:32 GMT
 float precision - ALMOST the old same story !
Hi

Rereading BOL I see I had it reversed

What you say is true.

-Dick


Quote:



> > Hi

> > In your SQL Table is the column defined as real or float? In SQL 7.0 the
> > only two options are float (which is the same size as the VB Single (4
> > bytes)) or real (which is the same size as a VB Double (8 bytes)).

> [CUT]

> I'm using SQL Server 2000, so float is 8 byte length and real is 4 byte
> length.
> The column, where I'm having, trouble is float (8 byte)

> Porlock.



Mon, 28 Feb 2005 08:52:45 GMT
 
 [ 5 post ] 

 Relevant Pages 

1. float precision - ALMOST the old same story ! (posted on comp.database.mssqlserver some time ago)

2. JDBC Floating point loss of precision

3. Comparing fixed precision to floating (no anwer)

4. ProC float precision

5. float output precision questions

6. Comparing fixed precision to floating

7. Floating point precision

8. float output precision questions

9. JDBC Floating point loss of precision

10. ODBC can't handle very small single precision floats

11. How to get a certain precision for float type using ct_bind

12. SQL Mail: Old Story.


 
Powered by phpBB® Forum Software