Some might wonder how many milliseconds are in a second? Well just so you do not have to go and Google the answer yourself it is 1,000 milliseconds to 1 second. If you are making Googlebot take a long or even an extremely long time how beneficial is that to you? I will let you decide for yourself. There are things you can do, caching, browser caching, reducing image file size, minify css, improve server response time and a host of other things that can be done.
No matter what, always talk to an expert like myself before you start playing with things like this as you might over do it or under do it as there are issues you might not think about that could actually hurt your site if you do not look at all aspects of the issues. So to improve the quality of your site with regards to page speed always talk with a growth hacker with a solid SEO background.