CSV encoding nightmare, mixed UTF-8 and Latin-1 in the same feed

Any downside to this approach with incremental models?

Read the file as raw bytes line by line, decode each line individually with errors='replace' or a chardet-style per-line detection, then reassemble. You can't fix this with a single file-level encoding parameter.

Document the grain decision, most BI bugs turn out to be grain bugs.

Also push back on the partner if you possibly can. This class of bug tends to resurface indefinitely otherwise, and every workaround just adds more fragility.

How do you handle backfill without duplicating rows?

Start with the execution plan, numbers beat guesses.