# Measure line endings before decoding a text export

A text editor can display two files identically even when their bytes differ. This matters when comparing downloaded chat exports, investigating a failed import, or deciding whether a conversion changed a file. Start with a byte-level inventory. Decode a separate copy only after recording what arrived.

# Keep the original download

Work on a copy and keep the original unchanged. A screenshot of an editor does not preserve the encoding, a byte-order mark, or the line separators. Opening and saving a file can change all three. File extensions also describe an intended format; they do not prove that the bytes conform to it.

Python’s usual text mode is convenient for processing lines, but its default universal-newline handling can turn CRLF and CR into LF in the resulting string. That is useful for reading prose. It is unsuitable as the first measurement of the downloaded bytes.

from pathlib import Path

raw = Path("export-copy.txt").read_bytes()
crlf = raw.count(b"\r\n")
bare_lf = raw.count(b"\n") - crlf
bare_cr = raw.count(b"\r") - crlf
has_utf8_bom = raw.startswith(b"\xef\xbb\xbf")

print({
    "bytes": len(raw),
    "crlf": crlf,
    "bare_lf": bare_lf,
    "bare_cr": bare_cr,
    "utf8_bom": has_utf8_bom,
})

The subtraction is deliberate: each CRLF contains one CR and one LF. Reporting all LF bytes as independent LF line endings would count a CRLF twice. An isolated CR is a separator candidate, not proof that the file is corrupt; inspect the format before deciding what it means.

# Check the arithmetic with a synthetic fixture

This sample contains one CRLF pair, one bare LF, one bare CR, and a three-byte UTF-8 BOM. The exact original has eleven bytes.

raw = b"\xef\xbb\xbfA\r\nB\nC\rD"
assert len(raw) == 11
assert raw.count(b"\r\n") == 1
assert raw.count(b"\n") - raw.count(b"\r\n") == 1
assert raw.count(b"\r") - raw.count(b"\r\n") == 1

These counts concern stored separators, not a universal definition of line count. A final separator, an empty file, and an unterminated last line require a convention. Report the counts directly instead of silently adding one and calling it the number of messages or records.

# Decode strictly, without overwriting the evidence

If the format is expected to be UTF-8, test that expectation explicitly:

try:
    text = raw.decode("utf-8-sig", errors="strict")
except UnicodeDecodeError as exc:
    print("Not valid UTF-8 at byte offset", exc.start)
else:
    print("Valid UTF-8; decoded characters:", len(text))

utf-8-sig removes an initial UTF-8 BOM from the decoded string. It does not modify raw, and the initial measurement still includes those three bytes. This codec does not normalize newlines when decoding bytes directly. A BOM elsewhere in the file is not an initial signature and should not be removed indiscriminately.

Avoid errors="ignore" for validation: it can discard bytes and hide the reason an import failed. errors="replace" can be appropriate for an explicitly labelled preview, but a readable preview is not evidence that the original was valid UTF-8. A decoding error also does not identify the correct alternative encoding. Establish that from the exporting application or its documentation.

# Separate file checks from export settings

Valid UTF-8 and consistent separators cannot establish whether an export contains the intended date range, all selected conversations, or referenced attachments. Compare those requirements separately. For the application-level choices, Netzhi’s Chinese Telegram export guide describes the export workflow; the byte inventory here answers a narrower question about the resulting file.

When sharing diagnostics, use a synthetic fixture like the one above. Keep private chat content, names, paths, and attachments out of public troubleshooting reports. For large files, calculate counts in a streaming implementation that handles a CRLF split across chunks; the small example deliberately reads the entire file into memory.

Published by 网知 / Netzhi, which maintains software usage guides and troubleshooting notes. This is a diagnostic procedure, not a claim that any particular export is complete or that changing line endings will repair it.