This problem was picked up by an AI review of my Newsgrouper code, which uses ReTcl.  The review was done by "Billy G" using "Claude".  It's related to my previous ticket about utf-8 data causing a crash - this was resolved by clarifying that it's the client's responsibility to ensure inputs are pure 8-bit data.  My newsgrouper code mostly does this but there were one or two places where I slipped up.  E.g pasting an emoji into one of the search forms would pass that through ReTcl and cause a crash.

However the big issue is that such a crash not only caused that specific request to fail, it left the ReTcl connection in a state where every subsequent request would also fail, making the site unusable until restarted.

Below this I will paste the AI-generated report and patch.  The patch is in three parts.  The middle part tries to fix the encoding, but assumes the input is always utf-8, so I did not use that.  The first and last parts are fixes to the error handling to leave the connection in a usable state, I applied them and verified that an error of this sort no longer leaves the site unusable.  (The line numbers are off by about -60 because I was still using an older ReTcl from before the TLS fix.)



From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: go-while <go-while@users.noreply.github.com>
Date: Sun, 19 Oct 2026 00:00:00 +0000
Subject: [PATCH] retcl: keep one non-Latin-1 character from wedging the shared
 connection

The backend connection (the global 'redis' retcl object in news_code.tcl,
shared by the whole web process) uses a channel configured -translation
binary.  retcl's BuildResp appends each command word to the pipeline as-is
and Flush writes it with [puts].  Under Tcl 9 a word containing a character
above U+00FF (any non-Latin-1 text: CJK, Cyrillic, emoji, a pasted smart
quote) makes that [puts] raise "invalid or incomplete multibyte or wide
character".

Two things then make it permanent rather than a single failed request:
Flush raises before clearing the buffered pipeline, so the poisoned bytes
are re-sent on every later command, and the pending command id is left in
resultsCache, so later replies are misattributed.  The net effect is that
one request from any anonymous visitor returns HTTP 500 for everyone until
the server is restarted.

Fix, all on the send path:
  - BuildResp encodes each word to its UTF-8 bytes and advertises the byte
    length, so the pipeline is always a pure byte string the binary channel
    can write, and the RESP $<len> header matches what is sent.  (Redis
    bulk strings are byte strings, so this is also simply correct.)
  - Flush clears the buffered pipeline before attempting the write, so a
    failed write can never leave poisoned bytes to be re-sent.
  - the command dispatch drops the pending result slot if the send fails,
    so a later reply cannot be misattributed to a command that never went
    out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---
 scripts/retcl.tm | 26 ++++++++++++++---
 1 file changed, 22 insertions(+), 4 deletions(-)

diff --git a/scripts/retcl.tm b/scripts/retcl.tm
--- a/scripts/retcl.tm
+++ b/scripts/retcl.tm
@@ -492,7 +492,16 @@
             dict set resultsCache $cmdId callback $callback
         }
 
-        my Send $args
+        if {[catch {my Send $args} err]} {
+            # The command never reached the wire.  Drop its pending result
+            # slot so a later reply cannot be misattributed to it, then
+            # report the failure.
+            if {$cmdId ne {}} {
+                dict unset resultsCache $cmdId
+            }
+            my Error $err
+            return
+        }
 
         if {$sendAsync || $callback ne {} || $cmdId eq {}} {
             # Asynchronous send, return the command identifier
@@ -721,6 +730,11 @@
     method BuildResp {args} {
         set msg "*[llength $args]\r\n"
         foreach word $args {
+            # Redis bulk strings are byte strings.  Encode each word to its
+            # UTF-8 bytes and advertise the byte length, so a character above
+            # U+00FF cannot make the later [puts] on the binary socket raise,
+            # and the length header matches what is actually sent.
+            set word [encoding convertto utf-8 $word]
             append msg "\$[string length $word]\r\n$word\r\n"
         }
         set msg
@@ -848,13 +862,16 @@
         if {![my connected]} {
             my Error {Disconnected}
         }
-        if {[catch {puts -nonewline $sock $pipeline} err]} {
+        # Clear the buffered pipeline *before* attempting the write, so a
+        # failed write cannot leave poisoned bytes behind to be re-sent on
+        # every later command and wedge the shared connection.
+        set buf $pipeline
+        set pipeline [list]
+        set isPipelined 0
+        if {[catch {puts -nonewline $sock $buf} err]} {
             my Error $err
         }
         chan flush $sock
-
-        set isPipelined 0
-        set pipeline [list]
     }
 
     ##
-- 
2.43.0

Hi!

I fed your reply back to Claude and asked it for a reproducible 
test harness, a version matrix, and a comparison of the fixes.
*duck and cover* 🙂

====================================================================

[Attribution: this follow-up -- the reproduction harness, the testing across
Tcl 8.6 / 9.0 / 9.1, the analysis, and the patches below -- was written by
Claude (Anthropic's Claude Code AI), continuing the "AI review" mentioned in
the original report. I (Billy G.) am relaying it; the technical content and
wording are Claude's, and "I" below refers to that AI author.]

Thanks for looking at this so quickly, and for testing under 9.1.

I went back and reproduced it on your exact stack -- Tcl 9.1.0 built from
source, retcl 0.6.0 (current tip), against a throwaway local Redis -- to
answer your two questions. Short version: you are right that the error is
not recoverable by error handling alone, and the reason is a partial write.

What happens, concretely
------------------------

The write channel is non-blocking and -translation binary. When a word
contains a character above U+00FF, `puts` encodes the leading ASCII bytes
into the channel buffer and only then hits the bad character and raises.
Those already-encoded bytes -- a partial RESP frame -- have reached Redis.
So the server is now mid-parse, waiting for the rest of a command that will
never come, and the next command's bytes are consumed as that command's
tail. The stream is desynced.

The giveaway: after `set foo <U+20AC>` fails, `redis-cli get foo` returns
"*" -- the first byte of the *following* command, swallowed as foo's value.

Which Tcl versions (your 9.0)
-----------------------------

It reproduces on 9.0 and 9.1 identically, so your 9.0 is affected. On 8.6
it does NOT wedge: 8.6 does not raise on the binary-channel write, it
silently truncates the character to its low byte (euro U+20AC -> 0xAC) and
carries on -- silent data corruption instead of a stuck connection. So this
is effectively a Tcl 9 behaviour change: strict binary-channel encoding
turned 8.6's silent truncation into a hard error, and because the pipeline
was not cleared on that error, the hard error became a permanent wedge.
Full matrix in the transcript below. (The guard in (a) also improves 8.6 --
it rejects the character cleanly instead of mangling it to 0xAC.)

How the application "recovers" with the patch (your question 1)
---------------------------------------------------------------

Honestly: it doesn't truly recover. The two error-handling parts I applied
(clear the pipeline before/despite a failed write, and drop the pending
result id) only stop the *re-send loop* -- the old behaviour where the
poisoned pipeline was flushed again on every later command and wedged the
connection permanently. They do NOT restore stream integrity. In the
reproduction, with just those two parts, the next `get k1` comes back as
"OK" (the desynced reply of the failed SET -- a silently wrong answer),
followed by "No request found for response ... ERR unknown command '$3'"
and then timeouts. So those parts are necessary but not sufficient, and
unsafe on their own.

Two things actually make it safe (both verified; transcripts below):

  (a) Reject the bad data before it is ever written -- a guard in
      BuildResp. The command fails cleanly and the connection stays fully
      usable (get/get/ping all succeed right after). This also matches
      retcl's documented "8-bit data only" contract: fail fast and loud
      instead of letting a non-representable string onto the wire.

  (b) Or, drop the socket on a write error, so a possibly-desynced
      connection is discarded rather than reused. Clean "Disconnected",
      nothing corrupted; the app (or retcl's own reconnect) re-establishes.
      Safe, but it throws the connection away on every such event.

(a) is the one I'd suggest as the primary fix; (b) is good defense in depth
in case bytes ever escape another way.

On internal UTF-8 conversion (your question 2)
----------------------------------------------

I'd lean against auto-encoding inside retcl: callers who already pass pure
8-bit data (the documented contract) would get double-encoded. Rejecting
non-8-bit input up front keeps the contract intact and pushes the choice of
encoding to the caller. On the Newsgrouper side that's exactly what I now
do -- `encoding convertto utf-8` before handing text to retcl and
`convertfrom utf-8` on the way back; a euro sign round-trips as 3 bytes and
comes back as U+20AC. But this is your call -- if you'd rather retcl own a
UTF-8 conversion internally, that's internally consistent too, as long as
reads convertfrom symmetrically.

Suggested changes
-----------------

Keep the two error-handling parts (they make a *caught* write error leave
the object consistent), and add the guard. The three methods are shown by
name below; ready-to-apply diffs against pristine 0.6.0 are in Appendix B.

1) Flush -- clear the pipeline before writing, so a failed write is not
   resent, and optionally drop the socket on error:

       method Flush {} {
           if {![my connected]} {
               my Error {Disconnected}
           }
           set buf $pipeline
           set pipeline [list]
           set isPipelined 0
           if {[catch {puts -nonewline $sock $buf} err]} {
               # a partial frame may have reached the server; the stream is
               # desynced, so drop the socket rather than reuse it (option b)
               catch {my disconnect}
               my Error $err
           }
           chan flush $sock
       }

2) unknown -- drop the pending result slot if the send fails, so a caught
   error does not leak a never-to-be-filled cmdId:

       if {[catch {my Send $args} sErr]} {
           if {$cmdId ne {}} {
               dict unset resultsCache $cmdId
           }
           my Error $sErr
       }

3) BuildResp -- the actual fix: reject non-8-bit data before it is built
   or written, so nothing partial can ever reach the socket:

       method BuildResp {args} {
           set msg "*[llength $args]\r\n"
           foreach word $args {
               if {[regexp {[^\u0000-ÿ]} $word]} {
                   my Error "retcl: argument contains a character above\
                             U+00FF; retcl transmits 8-bit data only (encode\
                             text to bytes with \[encoding convertto utf-8 ...]\
                             before sending)"
               }
               append msg "\$[string length $word]\r\n$word\r\n"
           }
           set msg
       }

Reproduction is self-contained: the harness is in Appendix A, the four
variants (stock / the two error-handling parts / guard / reconnect) are in
Appendix B as diffs against pristine 0.6.0, and the raw output of all four
runs is in the transcript immediately below.

(Authorship is stated in the attribution note at the top: this reply and its
patches were produced by Claude.)

Downloads (raw files)
---------------------

    index       https://mbox-mo1.pugleaf.net/retcl/

    tarball      https://mbox-mo1.pugleaf.net/retcl/ng-probe.tar.gz

    harness      https://mbox-mo1.pugleaf.net/retcl/probe2.tcl

    patches      https://mbox-mo1.pugleaf.net/retcl/guard.patch
                 https://mbox-mo1.pugleaf.net/retcl/parts23.patch
                 https://mbox-mo1.pugleaf.net/retcl/reconnect.patch

    variants     https://mbox-mo1.pugleaf.net/retcl/retcl-stock.tm
                 https://mbox-mo1.pugleaf.net/retcl/retcl-parts23.tm
                 https://mbox-mo1.pugleaf.net/retcl/retcl-guard.tm
                 https://mbox-mo1.pugleaf.net/retcl/retcl-reconnect.tm

    transcript   https://mbox-mo1.pugleaf.net/retcl/retcl-repro-transcript.txt

The same files are inlined in the appendices below, so this reply is
self-contained if the links are not reachable.


====================================================================
FULL TRANSCRIPTS
====================================================================
================================================================
 retcl encoding-error reproduction -- full transcripts
 ticket 7cc4bc4416  (AI-assisted, same as the report)
================================================================
Host  : Linux 6.12.38+deb13-amd64 x86_64 / Debian GNU/Linux 13 (trixie)
retcl : 0.6.0 (current tip; package require Tcl relaxed 8.6 -> 8.6- to load under Tcl 9)
Redis : redis-cli 8.0.2 on 127.0.0.1:7777 (throwaway)
Input : one EURO SIGN U+20AC via [format %c 8364], in a SET value
Harness: probe2.tcl (bounded async; 3s per-cmd timeout)

################################################################
# SECTION A -- version matrix (STOCK retcl 0.6.0, unpatched)
################################################################
---------------- Tcl 8.6.16  --  no wedge: euro silently truncated to byte 0xAC ----------------
8.6.16 | baseline set k1=hello | rc=0 | OK
8.6.16 | baseline get k1 | rc=0 | hello
8.6.16 | TRIGGER set foo <euro> | rc=0 | OK
8.6.16 | recover get k1 (#1) | rc=0 | hello
8.6.16 | recover get k1 (#2) | rc=0 | hello
8.6.16 | recover ping  (#3) | rc=0 | PONG
8.6.16 | connected=1
  [server] get foo = "M-,"  strlen foo = 1

---------------- Tcl 9.0.4  --  Colin's version -- WEDGED ----------------
9.0.4 | baseline set k1=hello | rc=0 | OK
9.0.4 | baseline get k1 | rc=0 | hello
9.0.4 | TRIGGER set foo <euro> | SEND-ERR | error writing "sock559b10ebb0e0": invalid or incomplete multibyte or wi
9.0.4 | recover get k1 (#1) | SEND-ERR | error writing "sock559b10ebb0e0": invalid or incomplete multibyte or wi
9.0.4 | recover get k1 (#2) | SEND-ERR | error writing "sock559b10ebb0e0": invalid or incomplete multibyte or wi
9.0.4 | recover ping  (#3) | SEND-ERR | error writing "sock559b10ebb0e0": invalid or incomplete multibyte or wi
9.0.4 | connected=1
  [server] get foo = "*"  strlen foo = 1

---------------- Tcl 9.1.0  --  WEDGED ----------------
9.1.0 | baseline set k1=hello | rc=0 | OK
9.1.0 | baseline get k1 | rc=0 | hello
9.1.0 | TRIGGER set foo <euro> | SEND-ERR | error writing "sock5635d6d155f0": invalid or incomplete multibyte or wi
9.1.0 | recover get k1 (#1) | SEND-ERR | error writing "sock5635d6d155f0": invalid or incomplete multibyte or wi
9.1.0 | recover get k1 (#2) | SEND-ERR | error writing "sock5635d6d155f0": invalid or incomplete multibyte or wi
9.1.0 | recover ping  (#3) | SEND-ERR | error writing "sock5635d6d155f0": invalid or incomplete multibyte or wi
9.1.0 | connected=1
  [server] get foo = "*"  strlen foo = 1

################################################################
# SECTION B -- fix variants (all under Tcl 9.1.0)
################################################################
---------------- STOCK 0.6.0 (unpatched) ----------------
STOCK | baseline set k1=hello | rc=0 | OK
STOCK | baseline get k1 | rc=0 | hello
STOCK | TRIGGER set foo <euro> | SEND-ERR | error writing "sock556c22f305f0": invalid or incomplete multibyte or wi
STOCK | recover get k1 (#1) | SEND-ERR | error writing "sock556c22f305f0": invalid or incomplete multibyte or wi
STOCK | recover get k1 (#2) | SEND-ERR | error writing "sock556c22f305f0": invalid or incomplete multibyte or wi
STOCK | recover ping  (#3) | SEND-ERR | error writing "sock556c22f305f0": invalid or incomplete multibyte or wi
STOCK | connected=1
  [server] get foo = "*"

---------------- PARTS 2+3 only (clear pipeline + drop pending id) ----------------
PARTS23 | baseline set k1=hello | rc=0 | OK
PARTS23 | baseline get k1 | rc=0 | hello
PARTS23 | TRIGGER set foo <euro> | SEND-ERR | error writing "sock55b4663326f0": invalid or incomplete multibyte or wi
PARTS23 | recover get k1 (#1) | rc=0 | OK
No request found for response ERR unknown command '$3', with args beginning with: 
    while executing
"{*}$errorCallback $msg"
    (class "::retcl" method "Error" line 2)
    invoked from within
"my Error "No request found for response $body""
    (class "::retcl" method "HandleResult" line 41)
    invoked from within
"my HandleResult $type $data"
    (class "::retcl" method "readEvent" line 18)
    invoked from within
"::r readEvent"
PARTS23 | recover get k1 (#2) | TIMEOUT (no clean reply)
PARTS23 | recover ping  (#3) | TIMEOUT (no clean reply)
PARTS23 | connected=1
  [server] get foo = "*"

---------------- GUARD (reject >U+00FF before write) + parts 2+3 ----------------
GUARD | baseline set k1=hello | rc=0 | OK
GUARD | baseline get k1 | rc=0 | hello
GUARD | TRIGGER set foo <euro> | SEND-ERR | retcl: argument contains a character above U+00FF; retcl transmits 8-bi
GUARD | recover get k1 (#1) | rc=0 | hello
GUARD | recover get k1 (#2) | rc=0 | hello
GUARD | recover ping  (#3) | rc=0 | PONG
GUARD | connected=1
  [server] get foo = ""

---------------- RECONNECT (drop socket on write error) + parts 2+3 ----------------
RECONNECT | baseline set k1=hello | rc=0 | OK
RECONNECT | baseline get k1 | rc=0 | hello
RECONNECT | TRIGGER set foo <euro> | SEND-ERR | error writing "sock5644e2ecd800": invalid or incomplete multibyte or wi
RECONNECT | recover get k1 (#1) | SEND-ERR | Disconnected
RECONNECT | recover get k1 (#2) | SEND-ERR | Disconnected
RECONNECT | recover ping  (#3) | SEND-ERR | Disconnected
RECONNECT | connected=0
  [server] get foo = ""

================================================================
end of transcript


====================================================================
APPENDIX A -- reproduction harness (probe2.tcl)
====================================================================
How to reproduce the transcript above, end to end:

  0. Prereqs: a working Tcl (8.6, 9.0 or 9.1) with retcl loadable, and a
     local Redis. The harness talks to 127.0.0.1:7777; edit the
     "localhost 7777" line in probe2.tcl for another host/port, or start a
     throwaway instance:
         redis-server --port 7777 --save "" --appendonly no --daemonize yes

  1. Get retcl 0.6.0; save probe2.tcl (below) and the patches (Appendix B)
     beside it.

  2. Build the four variants from a pristine retcl.tm (the patches already
     include the Tcl-9 version-gate relax, 8.6 -> 8.6-):
         cp retcl.tm retcl-stock.tm
         cp retcl.tm retcl-parts23.tm   && patch retcl-parts23.tm   < parts23.patch
         cp retcl.tm retcl-guard.tm     && patch retcl-guard.tm     < guard.patch
         cp retcl.tm retcl-reconnect.tm && patch retcl-reconnect.tm < reconnect.patch
     To run the UNPATCHED stock variant under Tcl 9, relax its gate too:
         sed -i 's/package require Tcl 8.6$/package require Tcl 8.6-/' retcl-stock.tm

  3. Run the harness against each (LABEL is just a tag printed per line):
         tclsh9.1 probe2.tcl ./retcl-stock.tm     STOCK
         tclsh9.1 probe2.tcl ./retcl-parts23.tm   PARTS23
         tclsh9.1 probe2.tcl ./retcl-guard.tm     GUARD
         tclsh9.1 probe2.tcl ./retcl-reconnect.tm RECONNECT
     For the version matrix, run retcl-stock.tm under each interpreter
     (tclsh8.6 / tclsh9.0 / tclsh9.1). Flush Redis between runs:
         redis-cli -p 7777 flushall

  4. After the trigger, check what reached the server:
         redis-cli -p 7777 get foo
     stock/parts23 -> a stray byte ("*"); guard/reconnect -> unset;
     8.6 -> byte 0xAC (the euro, silently truncated).

The harness itself:

cat > probe2.tcl <<'EOF'
# Bounded reproduction harness for retcl ticket 7cc4bc4416.
# Needs a local Redis; edit "localhost 7777" below to point at it.
# Run:  tclsh9.1 probe2.tcl /path/to/retcl.tm LABEL
# Uses the async API + a 3s per-command timeout so a desynced/hung
# connection cannot block the script forever.
lassign $argv retclPath label
source $retclPath
retcl create r localhost 7777
set ns [info object namespace r]

proc trysync {label tag cmd} {
    global ns
    set rc [catch {uplevel #0 $cmd} id]
    if {$rc} { puts "$label | $tag | SEND-ERR | [string range $id 0 70]"; return }
    if {$id eq {}} { puts "$label | $tag | sent(no-id)"; return }
    set wd [after 3000 [list set ${ns}::activity TIMEOUT]]
    while {![r resultReady $id]} {
        if {[catch {vwait ${ns}::activity} e]} {
            puts "$label | $tag | READ-CRASH | [string range $e 0 70]"
            after cancel $wd; return
        }
        if {[set ${ns}::activity] eq {TIMEOUT}} {
            puts "$label | $tag | TIMEOUT (no clean reply)"; return
        }
    }
    after cancel $wd
    set rc [catch {r result -async $id} out]
    catch {r clearResult $id}
    puts "$label | $tag | rc=$rc | [string range $out 0 70]"
}

trysync $label "baseline set k1=hello" {r set k1 hello}
trysync $label "baseline get k1" {r get k1}
set euro [format %c 8364]
trysync $label "TRIGGER set foo <euro>" [list r set foo $euro]
trysync $label "recover get k1 (#1)" {r get k1}
trysync $label "recover get k1 (#2)" {r get k1}
trysync $label "recover ping  (#3)" {r ping}
puts "$label | connected=[r connected]"
catch {r destroy}
EOF


====================================================================
APPENDIX B -- the four variants, as diffs against pristine retcl 0.6.0
====================================================================
"stock" is pristine 0.6.0 with only the Tcl-9 version gate relaxed. Apply a
diff with:  patch retcl.tm < parts23.patch   (then rename/copy as needed).

cat > parts23.patch <<'EOF'
--- retcl.tm (pristine 0.6.0)
+++ retcl.tm (parts23)
@@ -25,7 +25,7 @@
 # SUCH DAMAGE.
 #

-package require Tcl 8.6
+package require Tcl 8.6-
 package require TclOO

 package provide retcl 0.6.0
@@ -552,7 +552,12 @@
             dict set resultsCache $cmdId callback $callback
         }

-        my Send $args
+        if {[catch {my Send $args} sErr]} {
+            if {$cmdId ne {}} {
+                dict unset resultsCache $cmdId
+            }
+            my Error $sErr
+        }

         if {$sendAsync || $callback ne {} || $cmdId eq {}} {
             # Asynchronous send, return the command identifier
@@ -908,13 +913,13 @@
         if {![my connected]} {
             my Error {Disconnected}
         }
-        if {[catch {puts -nonewline $sock $pipeline} err]} {
+        set buf $pipeline
+        set pipeline [list]
+        set isPipelined 0
+        if {[catch {puts -nonewline $sock $buf} err]} {
             my Error $err
         }
         chan flush $sock
-
-        set isPipelined 0
-        set pipeline [list]
     }

     ##
EOF

cat > guard.patch <<'EOF'
--- retcl.tm (pristine 0.6.0)
+++ retcl.tm (guard)
@@ -25,7 +25,7 @@
 # SUCH DAMAGE.
 #

-package require Tcl 8.6
+package require Tcl 8.6-
 package require TclOO

 package provide retcl 0.6.0
@@ -552,7 +552,12 @@
             dict set resultsCache $cmdId callback $callback
         }

-        my Send $args
+        if {[catch {my Send $args} sErr]} {
+            if {$cmdId ne {}} {
+                dict unset resultsCache $cmdId
+            }
+            my Error $sErr
+        }

         if {$sendAsync || $callback ne {} || $cmdId eq {}} {
             # Asynchronous send, return the command identifier
@@ -781,6 +786,11 @@
     method BuildResp {args} {
         set msg "*[llength $args]\r\n"
         foreach word $args {
+            if {[regexp {[^\u0000-ÿ]} $word]} {
+                my Error "retcl: argument contains a character above U+00FF;\
+                          retcl transmits 8-bit data only (encode text to bytes\
+                          with \[encoding convertto utf-8 ...] before sending)"
+            }
             append msg "\$[string length $word]\r\n$word\r\n"
         }
         set msg
@@ -908,13 +918,13 @@
         if {![my connected]} {
             my Error {Disconnected}
         }
-        if {[catch {puts -nonewline $sock $pipeline} err]} {
+        set buf $pipeline
+        set pipeline [list]
+        set isPipelined 0
+        if {[catch {puts -nonewline $sock $buf} err]} {
             my Error $err
         }
         chan flush $sock
-
-        set isPipelined 0
-        set pipeline [list]
     }

     ##
EOF

cat > reconnect.patch <<'EOF'
--- retcl.tm (pristine 0.6.0)
+++ retcl.tm (reconnect)
@@ -25,7 +25,7 @@
 # SUCH DAMAGE.
 #

-package require Tcl 8.6
+package require Tcl 8.6-
 package require TclOO

 package provide retcl 0.6.0
@@ -552,7 +552,12 @@
             dict set resultsCache $cmdId callback $callback
         }

-        my Send $args
+        if {[catch {my Send $args} sErr]} {
+            if {$cmdId ne {}} {
+                dict unset resultsCache $cmdId
+            }
+            my Error $sErr
+        }

         if {$sendAsync || $callback ne {} || $cmdId eq {}} {
             # Asynchronous send, return the command identifier
@@ -908,13 +913,16 @@
         if {![my connected]} {
             my Error {Disconnected}
         }
-        if {[catch {puts -nonewline $sock $pipeline} err]} {
+        set buf $pipeline
+        set pipeline [list]
+        set isPipelined 0
+        if {[catch {puts -nonewline $sock $buf} err]} {
+            # a partial frame may have reached the server: the RESP stream is
+            # now desynced, so drop the socket rather than reuse it.
+            catch {my disconnect}
             my Error $err
         }
         chan flush $sock
-
-        set isPipelined 0
-        set pipeline [list]
     }

     ##
EOF

====================================================================

end of text

Cheers!
Billy G.

Hi again --

A correction to the guard diff in my previous remark. That diff is right
when the error callback throws, which is retcl's default (the [error]
command). It is NOT right for an errorHandler that returns -- a case retcl
supports, and the reason Error is written the way it is:

    method Error {msg} {
        {*}$errorCallback $msg
        return -level 2
    }

[return -level 2] returns from Error's caller; it does not raise. So
[my Error] inside BuildResp never reaches the [catch] in unknown. BuildResp
returns empty, Send still does

    append pipeline "[my BuildResp {*}$cmd]\r\n"

and flushes that stray CRLF, the resultsCache slot stays pending, and since
FindPendingRequest is FIFO the next reply is credited to the command that
was never sent. With -sync, unknown then blocks forever in result's vwait.

So BuildResp and Flush must use bare [error], not [my Error]. The single
[catch] already in unknown is the right and only place to drop the slot and
call [my Error].

Flush should also put [chan flush] inside that same [catch], then disconnect
and raise. puts on the binary channel reports the encoding error with the
RESP prefix still in Tcl's output buffer (0 bytes to the peer at that
instant); a later flush then sends that prefix, which for SET foo <U+20AC>
is the header through "$1\r\n" but not the value -- so the next command's
first byte is consumed as the one-byte bulk. That is the foo="*" in the
earlier transcript, and why clearing the pipeline variable alone (parts 2+3)
desyncs rather than recovers. [close] flushes and sends FIN; Redis discards
an incomplete multibulk. This does not itself reconnect -- the next call
gets {Disconnected} until the app reconnects -- but the stream is no longer
corrupt.

package require Tcl 8.6- is still needed: on 9.0.4 a bare "8.6" is a version
conflict, "8.6-" loads. On 8.6 the guard also changes behaviour -- a
non-Latin-1 character used to be truncated (euro went out as the single byte
0xAC) and now errors -- which matches the contract from the earlier UTF-8
ticket, and is worth a release note.

Still no UTF-8 conversion inside the client: a value that is already 8-bit
would be re-encoded (U+00E9 as the bytes C3 A9 becomes C3 83 C2 A9).

Verification. Measured on Tcl 8.6.16, 9.0.4 and 9.1.0 against a local Redis:
  - default throwing callback: guard rejects before any write, wire
    unchanged, cache empty, next command gets its own reply; a real 0xFF
    byte still round-trips.
  - returning errorHandler: with [my Error] the slot is left pending and a
    bare CRLF is written and -sync hangs; with the patch below, one callback
    invocation, nothing written, slot removed, -sync returns, the next reply
    matches the command actually sent.
  - your own test suite passes on the patched tree (9.0.4 and 9.1.0),
    including test/014-utf8.test (the caller-side convertto/convertfrom
    round-trip, emoji included -- its bytes are <= U+00FF so the guard
    passes them) and the reconnect tests in test/003-connect.test.

Attribution: the hole in the posted guard was surfaced by a cross-review
with Grok; the patch was verified by Claude on Tcl 8.6.16 / 9.0.4 / 9.1.0
against a local Redis, including this test suite. Relayed by go-while.

Patch against 0.6.0 (retcl.tm is at the repo root; apply with patch -p1 or
git apply). Also at https://mbox-mo1.pugleaf.net/retcl/follow-up/consolidated.patch

--- a/retcl.tm
+++ b/retcl.tm
@@ -25,7 +25,7 @@
 # SUCH DAMAGE.
 #
 
-package require Tcl 8.6
+package require Tcl 8.6-
 package require TclOO
 
 package provide retcl 0.6.0
@@ -552,7 +552,12 @@
             dict set resultsCache $cmdId callback $callback
         }
 
-        my Send $args
+        if {[catch {my Send $args} sErr]} {
+            if {$cmdId ne {}} {
+                dict unset resultsCache $cmdId
+            }
+            my Error $sErr
+        }
 
         if {$sendAsync || $callback ne {} || $cmdId eq {}} {
             # Asynchronous send, return the command identifier
@@ -781,6 +786,11 @@
     method BuildResp {args} {
         set msg "*[llength $args]\r\n"
         foreach word $args {
+            if {[regexp {[^\u0000-\u00ff]} $word]} {
+                error "retcl: argument contains a character above U+00FF;\
+                       retcl transmits 8-bit data only (encode text to bytes\
+                       with \[encoding convertto utf-8 ...] before sending)"
+            }
             append msg "\$[string length $word]\r\n$word\r\n"
         }
         set msg
@@ -908,13 +918,16 @@
         if {![my connected]} {
             my Error {Disconnected}
         }
-        if {[catch {puts -nonewline $sock $pipeline} err]} {
-            my Error $err
-        }
-        chan flush $sock
-
-        set isPipelined 0
+        set buf $pipeline
         set pipeline [list]
+        set isPipelined 0
+        if {[catch {
+            puts -nonewline $sock $buf
+            chan flush $sock
+        } err]} {
+            catch {my disconnect}
+            error $err
+        }
     }
 
     ##

Here is a simple script to reproduce:
-------------------------------------------------------
package require retcl
retcl create redis

foreach g [list wooo \U0001F47B scary stuff] {
    puts -nonewline $g
    if {[catch {redis sync lpush ghosts $g} err]} {
        puts " FAILED: $err"
    } else {
        puts " OK"
    }
}
-------------------------------------------------------
With Retcl 0.6.0 the first lpush succeeds, the second fails as expected, but the the third and fourth also fail.  With the patched version the third and fourth operations succeed.

Actually I just noticed I wrote "redis sync lpush ..." instead of "redis -sync lpush".  The version with "sync" appeared to work but when I corrected it to "-sync" I got another strange failure on the patched version. 
Unpatched:
colin@deb2:~/tcl$ tclsh retcl-bug.tcl
wooo OK
👻 FAILED: error writing "sock55682f17c5e0": invalid or incomplete multibyte or wide character
scary FAILED: error writing "sock55682f17c5e0": invalid or incomplete multibyte or wide character
stuff FAILED: error writing "sock55682f17c5e0": invalid or incomplete multibyte or wide character

Patched:
colin@deb2:~/tcl$ tclsh retcl-bug.tcl
wooo OK
👻 FAILED: error writing "sock55e7e1543f00": invalid or incomplete multibyte or wide character
scary OK
No request found for response ERR unknown command '$5', with args beginning with: 
    while executing
"{*}$errorCallback $msg"
    (class "::retcl" method "Error" line 2)
    invoked from within
"my Error "No request found for response $body""
    (class "::retcl" method "HandleResult" line 41)
    invoked from within
"my HandleResult $type $data"
    (class "::retcl" method "readEvent" line 18)
    invoked from within
"::redis readEvent"

(and then hangs, script does not complete)

After some more correspondence with Billy, I understand that it's not valid to just drop the middle part of the original suggested patch.  But for compatibility with the ReTcl spec. it needs to be modified to reject non-8-bit-clean input.  This corresponds to the "guard" version of the latest "Claude" suggestions.  Personally I prefer a more terse error message, so I ended up with the following patch, which works ok with my corrected reproduction script.


--- retcl-0.6.0.tm      2026-10-23 14:46:52.799892887 +0100
+++ retcl-0.6.0.tm.patched      2026-10-23 17:17:39.891629073 +0100
@@ -552,7 +552,15 @@
             dict set resultsCache $cmdId callback $callback
         }
 
-        my Send $args
+        if {[catch {my Send $args} err]} {
+            # The command never reached the wire.  Drop its pending result
+            # slot so a later reply cannot be misattributed to it, then
+            # report the failure.
+            if {$cmdId ne {}} {
+                dict unset resultsCache $cmdId
+            }
+            my Error $err
+        }
 
         if {$sendAsync || $callback ne {} || $cmdId eq {}} {
             # Asynchronous send, return the command identifier
@@ -781,6 +789,9 @@
     method BuildResp {args} {
         set msg "*[llength $args]\r\n"
         foreach word $args {
+            if {[regexp {[^\x00-\xff]} $word]} {
+                my Error "Input outside 8-bit data range: '$word'"
+            }
             append msg "\$[string length $word]\r\n$word\r\n"
         }
         set msg
@@ -908,13 +919,16 @@
         if {![my connected]} {
             my Error {Disconnected}
         }
-        if {[catch {puts -nonewline $sock $pipeline} err]} {
+        # Clear the buffered pipeline *before* attempting the write, so a
+        # failed write cannot leave poisoned bytes behind to be re-sent on
+        # every later command and wedge the shared connection.
+        set buf $pipeline
+        set pipeline [list]
+        set isPipelined 0
+        if {[catch {puts -nonewline $sock $buf} err]} {
             my Error $err
         }
         chan flush $sock
-
-        set isPipelined 0
-        set pipeline [list]
     }
 
     ##

Aargh, I'm still playing catch-up.  The update from "Claude" via Billy at 2026-10-23 01:29:53 picks up another issue - that it's valid for a user-defined error handler to just return, in which case the fix needs further modification, as listed there.

"ready-to-apply: consolidated.patch" the AI said :D

+ 8-bit guard 
+ single catch in dispatch 
+ Flush-clears-before-write.

Applies clean to 0.6.0 and passes your own test suite incl 014-utf8.

Terser guard message:

    error "retcl: non-8-bit character in argument; encode to bytes first\
           (e.g. \[encoding convertto utf-8 ...])"

That sounds good to me :-)

+1 from me too, thanks!

If you have the time, the write-up is worth a read.
This is "real" work done in virtual machines with
real tests on real source code, with real patches and verification.
If you've never tried it, this claude code / grok build is worth putting on your list :)

In future I'll try to nail it down into fewer words, but everything here can be
reproduced. I last touched Tcl 20 years ago... and this crazy Claude helps a lot. 
And Grok found the 2nd error... anyways...

Keep up the good work! a Redis client in Tcl is real commitment!

Cheers!