Optimization idea from a non-coder:
If PM was to check each frame for errors (in this case, C64 multicol mode restrictions, etc) in real-time, using “tiles” may be part of the solution.
Photoshop splits up the document it’s displaying into tiles (I think they’re 512x512) and stores them in the buffer as a static texture until they change, regardless of how many layers are contributing, and it only updates that changed tile in the buffer if the user does something to affect one of the tile’s pixels. Pretty sure it mipmaps these tiles, too - for different zoom levels.
PS does this for the purpose of optimizing display of the document, but PM might use the same basic concept to optimize error checking - slice up the frame into tiles or regions, and only re-check tiles that have a detected change. Rather than, on every check, run an algorithm on every 8x8 chunk of the frame, or whatever the restriction criteria may be.