unknownin/out patch (was [HACKERS] PQescapeBytea is 
Author Message
 unknownin/out patch (was [HACKERS] PQescapeBytea is

Quote:

> FWIW, CVS tip with --enable-multibyte builds and passes regression tests
> here (modulo the horology thing).  I concur with Chris' suggestion that
> you may not have done a clean reconfiguration.  If you're not using
> --enable-depend then a "make clean" is certainly needed.

--enable-depend did the trick.

Patch now passes all tests but horology with:

./configure --enable-locale  --enable-debug --enable-cassert
--enable-multibyte --enable-syslog --enable-nls --enable-depend

Thanks!

Joe

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster



Fri, 24 Sep 2004 14:06:42 GMT
 unknownin/out patch (was [HACKERS] PQescapeBytea is

Quote:

> Try a multibyte encoding database. For example,

> $ createdb -E EUC_JP test
> $ psql -c 'SELECT SUBSTRING('1234567890' FROM 3)' test
>  substring
> -----------
>  3456
> (1 row)

> Apparently this is wrong.
> --
> Tatsuo Ishii

This problem exists in CVS tip *without* the unknownin/out patch:

# psql -U postgres testjp
Welcome to psql, the PostgreSQL interactive terminal.

Type:  \copyright for distribution terms
        \h for help with SQL commands
        \? for help on internal slash commands
        \g or terminate with semicolon to execute query
        \q to quit

testjp=# SELECT SUBSTRING('1234567890' FROM 3);
  substring
-----------
  3456
(1 row)

testjp=# select * from pg_type where typname = 'unknown';
  typname | typnamespace | typowner | typlen | typprtlen | typbyval |
typtype | typisdefined | typdelim | typrelid | typelem | typinput |
typoutput | typreceive | typsend | typalign | typstorage | typnotnull |
typbasetype | typtypmod | typndims | typdefaultbin | typdefault
---------+--------------+----------+--------+-----------+----------+---------+--------------+----------+----------+---------+----------+-----------+------------+---------+----------+------------+------------+-------------+-----------+----------+---------------+------------
  unknown |           11 |        1 |     -1 |        -1 | f        | b
       | t            | ,        |        0 |       0 | textin   |
textout   | textin     | textout | i        | p          | f          |
           0 |        -1 |        0 |               |
(1 row)

This is built from source with:
#define CATALOG_VERSION_NO      200204031

./configure --enable-locale  --enable-debug --enable-cassert
--enable-multibyte --enable-syslog --enable-nls --enable-depend

Joe

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/users-lounge/docs/faq.html



Sat, 25 Sep 2004 07:50:19 GMT
 unknownin/out patch (was [HACKERS] PQescapeBytea is

Quote:

> > Try a multibyte encoding database. For example,

> > $ createdb -E EUC_JP test
> > $ psql -c 'SELECT SUBSTRING('1234567890' FROM 3)' test
> >  substring
> > -----------
> >  3456
> > (1 row)

> > Apparently this is wrong.
> > --
> > Tatsuo Ishii

> This problem exists in CVS tip *without* the unknownin/out patch:

Sure. That has been broken for a while.
--
Tatsuo Ishii

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org



Sat, 25 Sep 2004 09:42:34 GMT
 unknownin/out patch (was [HACKERS] PQescapeBytea is

Quote:

> > > Try a multibyte encoding database. For example,

> > > $ createdb -E EUC_JP test
> > > $ psql -c 'SELECT SUBSTRING('1234567890' FROM 3)' test
> > >  substring
> > > -----------
> > >  3456
> > > (1 row)

> > > Apparently this is wrong.
> > > --
> > > Tatsuo Ishii

> > This problem exists in CVS tip *without* the unknownin/out patch:

> Sure. That has been broken for a while.

I guess this actually happened in 1.79 of varlena.c:

---------------------------------------------------------------------------
revision 1.79
date: 2002/03/05 05:33:19;  author: momjian;  state: Exp;  lines: +45 -42
I attach a version of my toast-slicing patch, against current CVS
(current as of a few hours ago.)

This patch:

1. Adds PG_GETARG_xxx_P_SLICE() macros and associated support routines.

2. Adds routines in src/backend/access/tuptoaster.c for fetching only
necessary chunks of a toasted value. (Modelled on latest changes to
assume chunks are returned in order).

3. Amends text_substr and bytea_substr to use new methods. It now
handles multibyte cases -and should still lead to a performance
improvement in the multibyte case where the substring is near the
beginning of the string.

4. Added new command: ALTER TABLE tabname ALTER COLUMN colname SET
STORAGE {PLAIN | EXTERNAL | EXTENDED | MAIN} to parser and documented in
alter-table.sgml. (NB I used ColId as the item type for the storage
mode string, rather than a new production - I hope this makes sense!).
All this does is sets attstorage for the specified column.

4. AlterTableAlterColumnStatistics is now AlterTableAlterColumnFlags and
handles both statistics and storage (it uses the subtype code to
distinguish). The previous version of my patch also re-arranged other
code in backend/commands/command.c but I have dropped that from this
patch.(I plan to return to it separately).

5. Documented new macros (and also the PG_GETARG_xxx_P_COPY macros) in
xfunc.sgml. ref/alter_table.sgml also contains documentation for ALTER
COLUMN SET STORAGE.

John Gray
---------------------------------------------------------------------------

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster



Sat, 25 Sep 2004 13:13:10 GMT
 unknownin/out patch (was [HACKERS] PQescapeBytea is

 >>>
 >>>>
 >>>> Try a multibyte encoding database. For example,
 >>>>
 >>>> $ createdb -E EUC_JP test $ psql -c 'SELECT
 >>>> SUBSTRING('1234567890' FROM 3)' test substring ----------- 3456
 >>>>

 >>>> (1 row)
 >>>>
 >>>> Apparently this is wrong. -- Tatsuo Ishii
 >>>
 >>> This problem exists in CVS tip *without* the unknownin/out
 >>> patch:
 >>
 >> Sure. That has been broken for a while.
 >
 >
 > I guess this actually happened in 1.79 of varlena.c:
 >
Yes, I was just looking at that also. It doesn't consider the case of n
= -1 for MB. See the lines:

#ifdef MULTIBYTE
    eml = pg_database_encoding_max_length ();

    if (eml > 1)
    {
       sm = 0;
       sn = (m + n) * eml + 3;
    }
#endif

When n = -1 this does the wrong thing. And also a few lines later:

#ifdef MULTIBYTE
    len = pg_mbstrlen_with_len (VARDATA (string), sn - 3);

I think both places need to test for n = -1. Do you agree?

Joe

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/users-lounge/docs/faq.html



Sat, 25 Sep 2004 13:38:13 GMT
 unknownin/out patch (was [HACKERS] PQescapeBytea is
This is a multi-part message in MIME format.
--------------080202040002070809010804
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

Quote:



>  >>>> Try a multibyte encoding database. For example,

>  >>>> $ createdb -E EUC_JP test $ psql -c 'SELECT
>  >>>> SUBSTRING('1234567890' FROM 3)' test substring ----------- 3456

>  >>>> (1 row)

>  >>>> Apparently this is wrong. -- Tatsuo Ishii

>  >>> This problem exists in CVS tip *without* the unknownin/out
>  >>> patch:

>  >> Sure. That has been broken for a while.

>  > I guess this actually happened in 1.79 of varlena.c:

> Yes, I was just looking at that also. It doesn't consider the case of n
> = -1 for MB. See the lines:

> #ifdef MULTIBYTE
>    eml = pg_database_encoding_max_length ();

>    if (eml > 1)
>    {
>       sm = 0;
>       sn = (m + n) * eml + 3;
>    }
> #endif

> When n = -1 this does the wrong thing. And also a few lines later:

> #ifdef MULTIBYTE
>    len = pg_mbstrlen_with_len (VARDATA (string), sn - 3);

> I think both places need to test for n = -1. Do you agree?

> Joe

The attached patch should fix the bug reported by Tatsuo.

# psql -U postgres testjp
Welcome to psql, the PostgreSQL interactive terminal.

Type:  \copyright for distribution terms
        \h for help with SQL commands
        \? for help on internal slash commands
        \g or terminate with semicolon to execute query
        \q to quit

testjp=# SELECT SUBSTRING('1234567890' FROM 3);
  substring
------------
  34567890
(1 row)

Joe

--------------080202040002070809010804
Content-Type: text/plain; name="mb_substr.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline; filename="mb_substr.patch"

*** pgsql/src/backend/utils/adt/varlena.c       Sun Apr  7 11:44:54 2002
--- pgsql.orig/src/backend/utils/adt/varlena.c  Mon Apr  8 22:32:16 2002
***************
*** 413,419 ****
        if (eml > 1)
        {
                sm = 0;
!               sn = (m + n) * eml + 3; /* +3 to avoid mb characters overhanging slice end */
        }
  #endif

--- 373,382 ----
        if (eml > 1)
        {
                sm = 0;
!               if (n > -1)
!                       sn = (m + n) * eml + 3; /* +3 to avoid mb characters overhanging slice end */
!               else
!                       sn = n;         /* n < 0 is special-cased by heap_tuple_untoast_attr_slice */
        }
  #endif

***************
*** 427,433 ****
        PG_RETURN_NULL();   /* notreached: suppress compiler warning */
  #endif
  #ifdef MULTIBYTE
!       len = pg_mbstrlen_with_len (VARDATA (string), sn - 3);

        if (m > len)
        {
--- 390,399 ----
        PG_RETURN_NULL();   /* notreached: suppress compiler warning */
  #endif
  #ifdef MULTIBYTE
!       if (n > -1)
!               len = pg_mbstrlen_with_len (VARDATA (string), sn - 3);
!       else    /* n < 0 is special-cased; need full string length */
!               len = pg_mbstrlen (VARDATA (string));

        if (m > len)
        {

--------------080202040002070809010804
Content-Type: text/plain
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command

--------------080202040002070809010804--



Sat, 25 Sep 2004 13:58:05 GMT
 unknownin/out patch (was [HACKERS] PQescapeBytea is
[snipped]

Quote:
> > Yes, I was just looking at that also. It doesn't consider the case of n
> > = -1 for MB. See the lines:

> > #ifdef MULTIBYTE
> >    eml = pg_database_encoding_max_length ();

> >    if (eml > 1)
> >    {
> >       sm = 0;
> >       sn = (m + n) * eml + 3;
> >    }
> > #endif

> > When n = -1 this does the wrong thing. And also a few lines later:

> > #ifdef MULTIBYTE
> >    len = pg_mbstrlen_with_len (VARDATA (string), sn - 3);

> > I think both places need to test for n = -1. Do you agree?

Sorry folks! I hadn't thought through the logic of that in the n = -1
and multibyte case. The patch looks OK to me.

John

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/users-lounge/docs/faq.html



Sat, 25 Sep 2004 18:54:05 GMT
 unknownin/out patch (was [HACKERS] PQescapeBytea is

Quote:
Tom Lane writes:
> FWIW, CVS tip with --enable-multibyte builds and passes regression tests
> here (modulo the horology thing).  I concur with Chris' suggestion that
> you may not have done a clean reconfiguration.  If you're not using
> --enable-depend then a "make clean" is certainly needed.

Maybe we should turn on dependency tracking by default?  This is about the
(enough + 1)th time I'm seeing this.

--

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org



Sun, 26 Sep 2004 02:24:41 GMT
 unknownin/out patch (was [HACKERS] PQescapeBytea is

Quote:

> Tom Lane writes:

> > FWIW, CVS tip with --enable-multibyte builds and passes regression tests
> > here (modulo the horology thing).  I concur with Chris' suggestion that
> > you may not have done a clean reconfiguration.  If you're not using
> > --enable-depend then a "make clean" is certainly needed.

> Maybe we should turn on dependency tracking by default?  This is about the
> (enough + 1)th time I'm seeing this.

What is the downside to turning it on?  I can't think of one.

--
  Bruce Momjian                        |  http://candle.pha.pa.us

  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org



Sun, 26 Sep 2004 02:42:52 GMT
 unknownin/out patch (was [HACKERS] PQescapeBytea is

Quote:


>> Maybe we should turn on dependency tracking by default?  This is about the
>> (enough + 1)th time I'm seeing this.
> What is the downside to turning it on?  I can't think of one.

Well, we'll still see the same kinds of reports from developers using
non-GCC compilers (surely there are some) ... so enable-depend isn't
going to magically make the issue go away.

Personally I tend to rebuild from "make clean" whenever I've done
anything nontrivial, and certainly after a CVS sync; so I have no use
for enable-depend.  But as long as I can turn it off, I don't object
to changing the default.

                        regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster



Sun, 26 Sep 2004 04:26:49 GMT
 
 [ 10 post ] 

 Relevant Pages 

1. unknownin/out patch (was [HACKERS] PQescapeBytea is not

2. unknownin/out patch (was [HACKERS] PQescapeBytea is

3. unknownin/out patch (was [HACKERS] PQescapeBytea is not

4. unknownin/out patch (was [HACKERS] PQescapeBytea is not multibyte aware)

5. error ORA-01855: AM/A.M. or PM/P.M. required

6. Busy Day = Slowdown from 12 AM - 5 AM

7. Use of @am, Am I dumb?

8. I am getting this message when i am tring to export or import anything using

9. I am looking for patch 7.3.4.2 for 7.3.4

10. Patches (current CVS) for changes if index AM tables

11. unknownin/out patch

12. PQescapeBytea documentation patch


 
Powered by phpBB® Forum Software