I had a discussion in class with one of my teachers. He says that AI is and can only be always deterministic because “even a deep learning neural network is a set of equations running on a computer, and the stochastic factor is added at the beginning. But the output of a model is always deterministic, even if it’s not interpretable by humans.”
How would you reply? (Possibly with examples and papers)
Tysm!
Once trained, a models outputs are completely determined by its inputs at a mathematical level. This is easy to prove. You write out the equation for each output variable (it’s large but possible) and observe that each parameter is constant and not a random variable. Thus the output is also constant given some inputs and weights.
Training is arbitrary but also deterministic, since the combination of the initial states, training batch order, and optimization algorithm and parameters, determines the output.
If you feed random parameters into any of these then the output is arbitrary, but not random
For example at inference time, a transformer is not using any random number, not even any pseudo random number.
I think you are mistaking the sampling procedure with the model itself. The sampling procedure is often pseudo random. The model usually is something that produces a probability distribution. That’s deterministic.