consider a function f that is inaccessible to us. We cannot directly access f or compute its gradients. Our only available information is providing an input x and receiving a noisy estimation (or without any noise) of the true output.
Our objective is to optimize the underlying value within the black box function f according to our purpose, maximizing it for example.
In traditional optimization methods, such as grid search or random search, the objective function is evaluated at a predefined set of points. However, these methods can be computationally expensive and inefficient when dealing with high-dimensional or noisy functions.