I'll keep this short and happy to discuss more details in other threads.Yeah I think we're basically in complete agreement.
You know I cite Godel often. He's far from the only reason I think there will not be a unifying theory but his work is highly important in that respect (pure mathematics). That is, every system, sufficiently powerful, will be incomplete. That is easily explicated as "all systems -- especially the likes of mathematics and physics, but not restricted to these at all (more nuanced treatments of the implications) -- will be incomplete by axiomatic necessity". The point, and I'll try make it simple, because I usually fail there, is that you go to string theory and it explains the incompleteness of general relativity in this or that respect (and quantum too does a bit of this -- as you'll know). But there will be incompleteness in string theory (in every such system by axiomatic necessity). And so you keep (physicists) moving to new or re-imagined (redesigned) systems to explain an incompleteness (some problem) within a different system -- but it is an apriori categorical error insofar as you want a "theory of everything" (and that implies "completeness" -- that cannot happen unless one is to overturn Godel and I see no way of that ever happening because Godel is far more general than any given physics' system).
That's where I cite my "this unifying theory" (even if it exists -- not so much as theory, but as a reality of some kind) will never be had. It is because the only real pure mathematical axioms of sufficient generality forbid it for very good reasons. And you can see, within this critique, and within that video, too, by Carrolls? own admission, why Godel can be used in the positive. I mean they're right, there is an "incompleteness" in general relativity which in this or that specificity quantum and string and so forth do explain only then to run into their own inevitable incompletness. This is why Godel really does matter. And I note you qualifications regarding computational science (functions and transforms re its functional and practical devolopment as to why that will not be a major problem in the foreseeable future) -- but is AI looking for a theory of everything? Not really (unless they task it to and if so it will be governed by the very axiom, just in physics and mathematics now, that Godel teased out).
Now, I have been engaging with LLMs too much recently so my writing style has taken a hit. The above might be complete fecking gibberish to someone who doesn't know what Godel is saying or even if they do, I might not explain it well. So for the sake of clarity I used the anonymous function on chatgpt and entered the above in:
Apologies for the length. I just want to qualify why I'm certain there can be no unifying theory: Godel's injunction, especially within mathematics and physics, is entirely universal. It applies to all formal systems (including every symbolic system I've ever considered and many theoretical which don't really exist except in axiomatic theory). I.e., you can see why GR is "incomplete" and why a new system seems to explain this or that which GR cannot -- but these too have to be incomplete. And that is ad infinitum insofar as they formal systems. The principle, then, is infinite systemic incompleteness which literally forbids a unifying theorem insofar as that could ever be a complete or the complete truth of the universe.
They are checkmated against doing what they seek to do (if that's a unifying theory) by a mathematical eloquence (pure mathematical logic and systemic brilliance) which is higher (formally) than the semantic of their own systemic assertions.This becomes a category error. And that's what a UT is -- or has to be. Unless one can overturn Godel and honestly I do not believe (within any systemic/formal system) that such is in any respect possible. Godel articulates a genuine truth. That's rare. I.e., a universal necessity across all systems. The "checkmate" is not against progress but against any positivist notion that any theory or infinite combination of such can be "complete" -- or "unified".
I do not think that Godel's second law has any implications about AI, and I would speculate that it does not have much implications about a theory of everything. The entire math is not needed for a theory of everything, in fact, only a subset part of it might be needed. And that part might be consistent. So my point is that while any math system cannot be consistent in the true sense by definition, that doesn't stop various physical theories being consistent. Or vice versa, there are parts of math that do not describe our universe at all, which is fine.