• docs/zmodem_comparison.md

    From Rob Swindell (on Debian Linux)@1:103/705 to Git commit to main/sbbs/master on Sun Aug 30 03:07:35 2026
    https://gitlab.synchro.net/main/sbbs/-/commit/218c474f6d616d63799bdac5
    Modified Files:
    docs/zmodem_comparison.md
    Log Message:
    docs/zmodem: re-measure everything against the current releases

    The document had accumulated five measuring sessions between 2026-07-23
    and 2026-08-25, comparing components at versions that no longer ship and carrying figures taken against a baseline now known to have been the bottleneck. Re-measure the whole thing in one sitting on one host and
    rewrite around it.

    Structural change: measure the full 20-pair sender x receiver matrix
    rather than holding one end fixed. Fixing an end is what produced the
    old sender table's four-way tie -- every row was scored against lrz,
    which receives for more CPU than any sender in the table spends to send,
    so 1.74x of real difference reported as 1.8%. The matrix has no such
    blind spot, and printing both endpoints' CPU makes the bottleneck visible
    in every cell.

    All twenty pairs interoperate byte-identically and wire overhead is equal
    to two decimals at equal block size, so nothing here is a compatibility problem. zmtx/zmrx 2.04 is the cheapest implementation in both
    directions by roughly a factor of two -- 10.0 instructions per data byte sending and 12.1 receiving, against the next best of 23.5 either way.
    sexyz's receiver is second at 24.6, where the same code cost 126.8 eight
    days earlier, and its error recovery is the strongest measured: 10 of 10
    in both directions where the all-lrzsz pair manages 7 of 10.

    Two findings belong to zmtx/zmrx rather than to us. Its whole-file retry budget resets only on the ZRPOS path, so NAK- and timeout-driven recovery
    still drains it and the sender cannot complete a transfer at or above a
    5e-7 error rate. And its ESCCTL escaping omits carriage return, so no
    receiver that requests escaping can download from it -- the same defect
    our own send path carried until 2026-08-24.

    New sections cover what the old document never measured: feature and
    option support side by side, ESCCTL interoperability, and per-endpoint instruction counts for all eight endpoints.

    Method notes that cost real time to learn are written down. The
    corruption model seeds its PRNG with a constant, so repeated runs at one
    rate are not independent samples and a low enough rate injects nothing at
    all -- a rate sweep is the instrument, not repetition. Error gates are
    far more load-sensitive than throughput runs: a first pass taken while a profiler was still finishing scored the same lrzsz pair 0/3 and 2/3 in
    two identical invocations, where it passes in half a second on an idle
    host. And the first cut of this measurement used a zmtx checkout 32
    commits stale, which is why every version in section 1.1 now carries its release date.
    --- SBBSecho 3.37-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Rob Swindell (on Debian Linux)@1:103/705 to Git commit to main/sbbs/master on Sun Aug 30 03:07:35 2026
    https://gitlab.synchro.net/main/sbbs/-/commit/f8442b0765fd2eab167a8257
    Modified Files:
    docs/zmodem_comparison.md
    Log Message:
    docs/zmodem: add Forsberg's DSZ as a reference for the negotiated options

    The four implementations compared here cannot settle what ZMODEM's option
    bits were meant to do, because for ESC8 none of them agrees with another. Forsberg's commercial DSZ can: he wrote the protocol, and the binary still runs.

    Add section 7.4 covering DSZ.EXE 1997-05-25 under DOSBox, with its COM1
    bridged to TCP. It is deliberately not in the throughput tables -- 16-bit
    DOS, no source, nothing to build or profile -- and appears only for the negotiated options. The base protocol interoperates perfectly: a
    4096-byte random file sent by that 1997 binary arrives byte-identical at
    both sexyz and zmrx.

    Its ZRINIT settles ESC8. "rz -E" sets ZF0 bit 0x80, so ESC8 is the
    negotiation bit for the "8th bit quoting" that DSZ.DOC describes as half
    of ZMODEM-90's 7-bit mode; "rz -e" sets ESCCTL; "rz -P" sets neither.
    CANRLE is set unconditionally, which matches the other half of that 7-bit
    mode being present in Forsberg's free source while the quoting half never
    was.

    And the two implementations of ESC8 that exist do not interoperate. DSZ
    quotes a high-bit byte as 0x0E followed by the byte with bit 7 cleared -- Kermit-style prefixing, and it switches to frame type 0x31, which is not
    among the eight types any published zmodem.h defines -- while zmtx/zmrx
    2.04 sends ZDLE and the byte XOR 0x40. Recovered by known-plaintext
    capture: 0x0E occurs 4,991 times and is followed by a byte below 0x80 in
    100% of them, and decoding on that rule alone recovers 92 contiguous
    bytes of the known plaintext while restoring the stream to 45.6%
    high-bit.

    Neither encoding is wrong, because ZMODEM.DOC names ESC8 once and never
    says how a high-bit byte is escaped, where ESCCTL gets a paragraph. Two implementers 29 years apart read that line and built unrelated mechanisms.

    Also record the method, since the obvious route is the wrong one.
    DSZ.EXE is a plain unpacked MZ image and disassembles fine, but its
    escape table is not in the file -- it is built at runtime into BSS,
    exactly as Forsberg's free zsendline_init() does. Running the binary
    against a deliberately permissive receiver with known plaintext settled
    it in three runs.
    --- SBBSecho 3.37-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Rob Swindell (on Debian Linux)@1:103/705 to Git commit to main/sbbs/master on Sun Aug 30 03:07:35 2026
    https://gitlab.synchro.net/main/sbbs/-/commit/f06bafe5ee11ad2f2056050a
    Modified Files:
    docs/zmodem_comparison.md
    Log Message:
    docs/zmodem: record option-by-option compatibility with DSZ

    Section 7.4 had DSZ answering what the negotiated option bits mean, but
    the only transfers behind it were a pair of defaults. In particular the
    BBS download case -- sexyz sending to DSZ receiving -- had never been run
    once. Run the whole option set in both directions and record it.

    Everything interoperates. Sending to DSZ: default, -8/-4/-2, -o, -s,
    -w4096, -e, -l, -y, -p and -n all transfer an 8 KiB random file
    byte-for-byte or, where the option says not to, correctly decline to.
    Receiving from DSZ: default, -m (MobyTurbo), -Q@ and -e likewise.

    Two results are worth more than the tick in the box. The file-management options added this month produce the documented behaviour at a
    third-party receiver: -p leaves DSZ's existing file alone and -n skips an
    older source. -n only demonstrates itself once the timestamps are forced apart, because FAT's two-second granularity makes a same-minute
    comparison ambiguous -- the first run of it looked like a failure and was
    not. And ESCCTL works in both directions, which is the path that carried
    the carriage-return defect until 2026-08-24.

    DSZ's 7-bit options turn out to be receiver-driven despite their names:
    "sz -E" and "sz -P" sending to sexyz both transfer identically and leave
    46.9% of the wire carrying high-bit bytes, the same as with no option at
    all. They request the mode when DSZ receives and impose nothing when it
    sends.

    State plainly what was not tested rather than letting the table imply
    coverage: -m as a local size limit, the X/YMODEM modes, and -telnet.
    Say too what "we do not support ESC8" actually means -- sexyz no longer
    sets the bit, and it ignores a remote receiver's request rather than
    refusing it, there being no way to decline a ZRINIT capability, which is
    what lrzsz does as well.

    And note a capability we decline unnecessarily: DSZ advertises CANRLE in
    every ZRINIT and implements RLE both ways, while sexyz never advertises
    it, so the compression is never used.
    --- SBBSecho 3.37-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Rob Swindell (on Debian Linux)@1:103/705 to Git commit to main/sbbs/master on Sun Aug 30 03:11:20 2026
    https://gitlab.synchro.net/main/sbbs/-/commit/6a06088bd80f6b71f9681cba
    Modified Files:
    docs/zmodem_comparison.md
    Log Message:
    docs/zmodem: sexyz no longer advertises ESC8, so stop saying it does

    The document was written while the Escape8thBit option still existed and
    kept describing it in the present tense in four places: the feature
    table, the summary, section 7.2's analysis and section 8's findings. The option and the ZRINIT advertisement were both removed the same day the
    last of those was written.

    Put the sexyz half in the past tense and say what the current behaviour
    is -- the bit is never set, and a remote receiver's request is ignored
    rather than refused, there being no way to decline a ZRINIT capability.
    The measurements stay as they were taken; only the claims about what
    sexyz does today change.
    --- SBBSecho 3.37-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Rob Swindell (on Debian Linux)@1:103/705 to Git commit to main/sbbs/master on Sun Aug 30 17:37:54 2026
    https://gitlab.synchro.net/main/sbbs/-/commit/c9fb4003646dcbbaf56bb784
    Modified Files:
    docs/zmodem_comparison.md
    Log Message:
    docs/zmodem: re-measure zmtx/zmrx at 2.05, which fixes both reported defects

    Deuce released 2.05 the day after this document reported two defects
    against 2.04, and fixed both, plus implemented the ESC8 encoding the DSZ investigation had recovered. Re-measure everything the document claims
    about zmtx/zmrx.

    ESCCTL is fixed (82be5ad). CR is folded into the control class --
    action = TX_ESCAPE_CONTROL | TX_ESCAPE_CR -- with the '@' conditional
    kept as a separate test, which is the same shape as our own fix six days earlier. zmtx 2.05 now transfers byte-identically to lrz -e, to sexyz
    -e and to its own zmrx -e, putting 5,245,010 bytes on the wire where
    sexyz puts 5,245,034 and lsz 5,245,032 for the same file. The
    32,400-byte shortfall, which was the unescaped CR count, is gone.

    The retry budget is fixed (6e32e97). The gate at 3e-6 goes from 0 of 5
    to 3 of 5 and the rate sweep now completes where 2.04 failed from 5e-7
    up. Still the weakest of the three -- sexyz manages 10 of 10 and lrzsz
    7 of 10 -- but no longer structurally unable to finish.

    And ESC8 now interoperates (399e3b6). The commit adopts Forsberg's
    encoding rather than inventing one; its comment names DSZ.EXE's
    ZMODEM-90 seven-bit encoder and the code emits SO followed by the byte
    with bit 7 cleared, which is what the known-plaintext capture recovered. Verified against the 1997 binary both ways on a 16 KiB file: zmtx 2.05
    to DSZ rz -E and DSZ sz to zmrx -b both arrive identical. So a de facto normative ESC8 now exists, with two implementations agreeing on it,
    where days ago there were two encodings and no agreement.

    The features cost the sender something: 0.38 CPU-seconds per 256 MiB at
    2.04, 0.48 at the 2.05 tag, 0.44 after dfa97f0 "Recover standard
    transfer performance", which is measured here because that commit lands
    after the tag. Still the cheapest sender measured, but the margin over
    lsz narrowed from 2.5x to 2.2x.

    The 2.02 and 2.04 measurements are kept throughout rather than
    overwritten, since they are how the encoding was established and why the options are the way they are.
    --- SBBSecho 3.37-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Rob Swindell (on Debian Linux)@1:103/705 to Git commit to main/sbbs/master on Sun Aug 30 17:56:37 2026
    https://gitlab.synchro.net/main/sbbs/-/commit/60b70e526213285b61ae11f8
    Modified Files:
    docs/zmodem_comparison.md
    Log Message:
    docs/zmodem: zmtx/zmrx supports CRC-16, it just cannot be asked for

    The feature table listed CRC-16 as "--" for zmtx/zmrx, which reads as
    not supported. That is wrong in a way that matters to anyone pairing it
    with a CRC-16-only peer.

    want_fcs_32 is hard-coded true in zmdm.c and neither program offers a
    CRC flag, so it cannot be asked to prefer 16-bit -- but the machinery is present in both the header and data paths and the selection is
    can_fcs_32 && want_fcs_32, so it falls back whenever the peer omits
    CANFC32. Distinguish "supported but not selectable" from "absent",
    since sexyz and lsz can both force CRC-16 and zmtx/zmrx can only fall
    into it.

    Measured rather than read: zmtx to sexyz -o puts 4,311,016 bytes on the
    wire against 4,312,065 with CRC-32, and sexyz -o to zmrx 4,310,744
    against 4,311,816. A 4 MiB file is 512 subpackets at 8 KiB and two CRC
    bytes saved each is about 1,024, which is what those deltas are. Both
    verified byte-identical, so the fallback is real in both directions.

    Also corrects a stale count in the prose above the table.
    --- SBBSecho 3.37-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Rob Swindell (on Debian Linux)@1:103/705 to Git commit to main/sbbs/master on Sun Aug 30 23:50:56 2026
    https://gitlab.synchro.net/main/sbbs/-/commit/0822029d905accebd45d3265
    Modified Files:
    docs/zmodem_comparison.md
    Log Message:
    docs/zmodem: measure what MobyTurbo is worth

    Rob asked what sexyz would gain by supporting MobyTurbo. zmtx/zmrx 2.05 implements it, so it can be measured rather than estimated.

    It is worth about 2.3% on compressed or otherwise incompressible data and essentially nothing on text. ZMODEM must escape seven byte values --
    ZDLE, 0x10, 0x11, 0x13, 0x90, 0x91 and 0x93 -- at one extra wire byte
    each, which is 7/256 = 2.73% of uniformly distributed data; MobyTurbo
    drops that escaping on links known to be transparent and recovers exactly
    it. Measured on 1 MiB: random data goes from +2.84% overhead to +0.49%,
    while text goes from +0.09% to +0.09%. The split is the escape rate
    itself -- those seven values occupy 2.746% of the random file and 0.000%
    of the text one, none of them occurring in ordinary ASCII.

    Three bounds on what that is worth, all recorded: it is a wire saving, so
    it converts to throughput only where bandwidth binds rather than in this document's CPU-bound regime; it needs a transparent link, the same
    precondition ESC8 exists to work around, inverted; and it is negotiated
    by the receiver, so DSZ sz -m to sexyz puts exactly the same bytes on the
    wire as without it.

    Forsberg measured about the same himself. DSZ.DOC's benchmark table has Pro-YAM at 231 cps standard against 237 with MobyTurbo on a 2400 bps
    link, +2.6%, within half a point of the figure here thirty-six years
    later.

    Also adds feature-table rows for MobyTurbo and Pack-7, both of which
    zmtx/zmrx 2.05 now implements and neither sexyz nor lrzsz does.
    --- SBBSecho 3.37-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)