Community

Why is validation loss / accuracy important for offline serving? Cant we just overfit?  

  RSS

nithanaroy
(@nithanaroy)
New Member
Joined: 7 months ago
Posts: 3
27/03/2019 12:04 pm  

In offline predictions workflow, the model's predictions are saved to a database for an online lookup. ML Engineers optimize for validation accuracy (i.e. accuracy on unseen data) as they want the model to perform well on newer / unseen data as well. Assuming that due to various constraints, we can only generate all (known) possible inputs from historical data, why not just overfit for the same historical data and save the predictions in a database? This makes sense for me as there are plenty of chances to miss generating predictions for every possible input feature combination. And if that's reasonable, why even optimize for validation accuracy besides training accuracy?


ReplyQuote
Share:
Subscribe to our newsletter

We use cookies to collect information about our website and how users interact with it. We’ll use this information solely to improve the site. You are agreeing to consent to our use of cookies if you click ‘OK’. All information we collect using cookies will be subject to and protected by our Privacy Policy, which you can view here.

OK
  
Working

Please Login or Register