Help free threading by replacing random.choices

[pauleveritt]

Rather than open a discussion about free threading, I thought I'd focus on one specific part.

When I asked Claude Code to investigate, it flagged these lines in placeholders.py where set them up. Its findings:

🔴 CRITICAL: Random Global State (placeholders.py:9-12)

FRAGMENT_TAG = f"t🐍f-{''.join(random.choices(string.ascii_lowercase, k=4))}-"
_PLACEHOLDER_PREFIX = f"t🐍{''.join(random.choices(string.ascii_lowercase, k=2))}-"
_PLACEHOLDER_SUFFIX = f"-{''.join(random.choices(string.ascii_lowercase, k=2))}🐍t"

Problem: These are module-level constants generated with random values at import time. In free-threaded mode:

1. Multiple threads importing simultaneously could cause race conditions
2. The regex pattern depends on these values - if they're inconsistent, parsing breaks
3. Module initialization isn't guaranteed to be atomic across threads

Impact: HIGH - Could cause parser failures or incorrect template processing

[ianjosephwilson]

Yeah I wasn't sure what the original purpose for these were. Maybe to prevent "gaming" the parser somehow? Since tdom seems to only parse t-strings that were defined by someone hopefully with the knowledge they were going into tdom this seems less of a risk. Although I guess its possible there is tdom, a 2nd party using tdom and a 3rd party library the 2nd party might use then maybe then there could be some trick? This seems like something that could be loaded as a separate configuration param though or even moved into the placeholder tracker class itself:

tdom/tdom/placeholders.py

Line 48 in 3e5b70f

class PlaceholderState:

[davepeck]

The concern was accidental collision with placeholders in real content and, I suppose, third-party provided content somehow slipping through and (maliciously) colliding. Not sure how realistic either case is.

[ianjosephwilson]

I started a draft of moving these into a class. I kind of think we should either finish the anonymous fragments with full tracking like placeholders or take them out which I mentioned in #94 . I'm kind of leaning on just taking them out for now because we keep working around them and we haven't really needed them yet. Then if we need them we can just add them back in.

[davepeck]

@ianjosephwilson 👍

Let's take anonymous fragments out for now.

[ianjosephwilson]

#96 and #95 should resolve this or move it along to the next step

[ianjosephwilson]

I could not find a definitive explanation as to why this is an issue nor a complete understanding of how free threading affects module globals OTHER than constant chatter about mutating globals being bad (which this doesn't seem to be exactly the case here). Overall the consensus seems to be everyone was getting away with things because python normally doesn't thread switch very quickly and now everything needs locking at the python level if you expect to share it between threads. We'll have to figure this out if we start building out the html() to be more pluggable/extendable but still keep it global. Ie. this might not work:

DEFAULT_PROCESSOR = HTMLProcessor() # use defaults
def html(template):
    return DEFAULT_PROCESSOR.process(template)