spider in a >
two, the maximum use of your Robots, this is the website of the door
according to the conventional view, the Robots file is to limit the spider on your site’s scope of activities, the site owner’s rights and better privacy protection. This is a spider reach a website after the first thing to do, if the site does not have the file, the spider is considered to be the site of all the contents of all mining.
generally, spiders can through three ways to reach your site, which is mainly submitted to the site, by connecting to other sites to track your website, a regular free action.
, a spider origin, how is it to your website
a lot of people say that now is not a smart spider submitted to the site, but it is wrong, because the chain you posted on other sites attract spider. These two forms are operational, the last one is special, we usually think of spiders in accordance with 1-0, A-Z in such a way to scan the domain name, enter the website then.
last weekend, a company should be invited to do search engine optimization to them (Shanghai dragon) the relevant training and skills to enhance the effect. That is what I most want to see things, their Shanghai Longfeng personnel everyday work is still in the original and false hair of the chain in two stages. I generally look at the optimization of their effect, domain related domain reached more than 10000, but the final effect is not ideal. After I finish, I just felt that the cost is too high, not worth mentioning.
as a web site optimization practitioners, did not understand the spider is how to do the job, this optimization is certainly effective, should be the first class to understand spider web site optimization practitioners.
in the Robots file, generally only to ban (Disallow). In fact, you can also use the recommended permit (Allow). A website has a lot of spider crawling to the dark corner is not, at this time, you can use Allow to guide the spider, you can do some map page, let the spider in the activities above.
also said, why to use Allow, the direct use of Sitemap what can not. We all know that, in fact, we pay more attention to the Sitemap, Google is no problem, but the love of spiders in Shanghai to understand there is a problem, can not very well play his role.
there are a lot of Shanghai dragon master, all know the chain and content. These not only today we share, spider. Because, this is a very serious problem I found in time to give them training: they have heard of the spider, the spider know what is, but do not know how to work the spider.