Take a site offline with .htaccess
To take down a website or part of one, you can just point search engine spiders to the 410 status code.
.htaccess makes it very easy to flag a link as offline or gone. It's what we'll use to tell either google or other search engine spiders that the content that once was there is not anymore and that it is our deliberate choice.
This is how my graph in Google Webmaster Tools, which is a nice tool to monitor your sites health in terms of search engine compatibility and availability, developed after I made a mistake and unintentionally generated a lot of errors on an old project, that Google did not forget about. It had no visitors, but Google used to be able to crawl it.
HTTP status code 410: gone
If you want to take the whole website offline, you can use the following code:
Redirect 410 /
Mostly though, you would only like to take down a sub-directory or an article, that you have deleted:
Redirect 410 /blog-post-you-deleted
Just put the desired code into the .htacces in your httpdocs folder or the root folder of your web hotel or directory on your server and both users and search engines will see that you took something offline intentionally.
What does 410 mean?
The official description of Response Code 410 from the w3c is:
The requested resource is no longer available at the server and no forwarding address is known. This condition is expected to be considered permanent. Clients with link editing capabilities SHOULD delete references to the Request-URI after user approval. If the server does not know, or has no facility to determine, whether or not the condition is permanent, the status code 404 (Not Found) SHOULD be used instead. This response is cacheable unless indicated otherwise.
The 410 response is primarily intended to assist the task of web maintenance by notifying the recipient that the resource is intentionally unavailable and that the server owners desire that remote links to that resource be removed. Such an event is common for limited-time, promotional services and for resources belonging to individuals no longer working at the server's site. It is not necessary to mark all permanently unavailable resources as "gone" or to keep the mark for any length of time -- that is left to the discretion of the server owner.