Speed is dependent on many factors.
If your website is slow because of a bad server, bad html, bad caching etc., then by all means, you might deserve that badge.
But if it is because of a high quality content with high resolution content, that would be plain dumb and idiotic. With 100+ Mb connection everywhere I use internet, I don't care if my favorite site with fun interesting media takes 0.5 seconds to load or 3 seconds.
While google will try to push you to use compressed lower quality images and move content to their amp pages effectively serving your content via google.
Now I am wondering, how will they be distinguishing between the two.
For example, will they punish cnn.com? That is a heavy page that takes seconds, not milliseconds. And if they don't, but do that to a smaller guy, then they are waiting for lawsuits.