The time threshold varies based on the expectation of the human in front of the computer.
For a website the application is far more complex than a desktop GUI. It is the web browser, a desktop GUI application, plus the network connection out to the webserver, and the stuff being requested from the webserver. Clearly a website offers a more complex set of possible factors in slowing responses, and the most interesting thing is that the users of websites have a larger threshold of acceptable response than do desktop GUI users.
For example on desktop GUI's the acceptable response time is about 50 milliseconds for a simple action like clicking on a button. But if that button were on a webpage then the acceptable response time shoots up to several seconds.
The Miller Response-Time Test discusses these thresholds and how they apply to website performance. An interesting part of this is that Jakob Nielson, a very respected long-time expert in computer system usability, quite a long time ago set an 8-second threshold. He established this threshold in the 90's when 56kb modems were all the rage, and in todays era of high speed Internet access it doesn't take nearly so long to download data. There was a reason why WWW used to mean World Wide Wait.
Acceptable Response Times is another article by the same author, discussing some specific claims made by yet another article. An interesting point is a study by Akamai and Jupiter Technologies claiming the acceptable threshold for page load times is 4 seconds. The press release announcing this result then goes on to say that a distributed caching system, such as the one operated by Akamai, can greatly reduce page load times. While it seems duplicitous to have a report published by a company who can benefit from the claims in the report (makes you wonder at the accuracy of the claims) 4 seconds is probably a better threshold to aim for anyway.
The List of human-computer-interaction topics on the Wikipedia is a great place to learn more about HCI in general. I haven't yet found on that page any discussion of application responsiveness or website responsiveness. Human computer interaction is clearly a broad topic area.