I think their point is indulging in intense activity every day does not help with your performance. If they had to work hard out of necessity so be it, but I doubt any paleo people could out lift or out perform modern athletes in any sports, athletes that know how to train and rest properly.
Not true. Some scientists did a recent study of ancient greek triremes and it was found that the ancient greeks(on a Neolithic diet, no less) far outperformed any modern Olympic rowers. Given that palaeolithic peoples were more active physically than settled Neolithic peoples(like the Ancient Greeks), it's certain that palaeo cavemen could have done even better, especially given the fact they had a healthier diet.
And daily physical performance is very relevant. I remember my father poitning out that in his youth there were still some jobs available which involved frequent heavy physical labour, and he would mention how such labourers etc. could easily outperform any modern ahtlete as regards lifting weights etc., simply because they did such activity virtually every day of their lives.
Modern athletes, by contrast, are heavily dependent on performance-enhancing drugs and still can't compete with their forbears.