Q: Who’s George in the expression “By George!”?

A: The phrase is a mild oath or exclamation that had its beginnings in the late 1500s. The word “George” here is a substitute for “God,” as are words like “golly,” “ginger,” “gosh,” “gum,” and so on in other similar euphemistic oaths.

The expression began life as “fore (or for) George” and “before George,” according to published references in the Oxford English Dictionary. These were milder versions of “before God,” “fore God,” and so on.

https://www.grammarphobia.com/blog/2012/01/by-george.html