Why is validation loss / accuracy important for offline serving? Cant we just overfit?
In offline predictions workflow, the model's predictions are saved to a database for an online lookup. ML Engineers optimize for validation accuracy (i.e. accuracy on unseen data) as they want the model to perform well on newer / unseen data as well. Assuming that due to various constraints, we can only generate all (known) possible inputs from historical data, why not just overfit for the same historical data and save the predictions in a database? This makes sense for me as there are plenty of chances to miss generating predictions for every possible input feature combination. And if that's reasonable, why even optimize for validation accuracy besides training accuracy?