It is very important to understand that AI models are not deterministic and they cannot be made deterministic without severely restricting the environment they run in. Fixing seeds doesn’t help. Setting temperature to 0 doesn’t help. Every small floating point rounding error can ultimately lead to drastically different results.

The complex models are, in essence, chaotic systems and should be treated as such.