TAY posts are no longer found in Google searches

I noticed a few months ago that forum posts here on TAY were no longer appearing in Google searches -- at all. TAY posts used to appear normally in Google searches without any issue whatsoever, but they no longer do. The problem still appears to exist as of today. Have any others here noticed this? I mentioned it to Marcus after NSAS yesterday, but he wasn't sure what might be causing the issue.

I did some testing today to try to isolate the problem. Apparently, TAY posts are currently found by some search engines but not others, and most importantly, NOT by Google. Obviously Google is by far the most popular search engine, with market share of over 80% depending on which survey you look at (see e.g. www.google.com/search?q=search+engine+market+share ), followed by Bing and Yahoo (which uses the Bing engine) with about 10% and others like Ask and AOL with only a tiny percentage of the pie.

To see the problem for yourself, try searching for the title of any TAY post in Google and other search engines. For example, I searched for one of my posts from 2011 that was also cross-posted on CascadeClimbers, which provides a useful cross-check and control to see whether the problem is limited to TAY posts. Here are the results:
[tt]
                                                                      TAY post   CC post
Search Engine URL                                                      found?     found?

www.google.com/search?q=new+Camp+Muir+webcam+now+online            NO        yes
www.bing.com/search?q=new+Camp+Muir+webcam+now+online              yes       yes
search.yahoo.com/search?p=new+Camp+Muir+webcam+now+online          yes       yes    ("Powered by Bing")
www.ask.com/web?q=new+Camp+Muir+webcam+now+online                  NO        yes
search.aol.com/aol/search?q=new+Camp+Muir+webcam+now+online        NO        yes
duckduckgo.com/?q=new+Camp+Muir+webcam+now+online                  NO        yes
search.lycos.com/web?q=new+Camp+Muir+webcam+now+online             yes       yes
www.dogpile.com/search/web?q=new+Camp+Muir+webcam+now+online       yes       yes    ("Results from: Google Yahoo Yandex")
[/tt]

So all of the search engines I checked found my post on CascadeClimbers, but Google and several others failed to find the TAY post.

However, the problem is isolated to only the TAY forums and posts therein. Other portions of the TAY website are found just fine by those search engines which fail to find TAY posts. For example, all of them correctly find this page when you search for its title: www.turns-all-year.com/indexyrs.html
[tt]
Search Engine URL                                                                   TAY found?

www.google.com/search?q=year-round+backc...ers+and+snowboarders           yes
www.ask.com/web?q=year-round+backcountry...ers+and+snowboarders                 yes
search.aol.com/aol/search?q=year-round+b...ers+and+snowboarders       yes
duckduckgo.com/?q=year-round+backcountry...ers+and+snowboarders                 yes
[/tt]

Does anyone know what might cause a portion of a website (such as the forums) to vanish from the listings of some search engines, but not all parts of the site and not from all search engines?

By the way, as a test I re-submitted the TAY forums URL ( www.turns-all-year.com/skiing_snowboardi...ip_reports/index.php ) to Google via www.google.com/webmasters/tools/submit-url to see if that would cause the posts to be re-indexed soon. But my guess is that will probably not work.

This is probably caused by a configuration change in the robots.txt file on the webserver. Search engines use this as a base line on what to 'crawl' for information and index.

Information can be found here on the file: www.webconfs.com/what-is-robots-txt-article-12.php

Without knowing what the current file says on the site it's hard to say what needs to be changed or if this even is the issue, but my assumption is that this would be the case.

I'd also like to see a feature request for the site to add it to 'Tapatalk' so I can browse it with my smart phone a lot easier than viewing the http webpage that's meant for a computer not a mobile device.

www.tapatalk.com/faq.php

Thanks y'all -- I'll take a look at the robots.txt file and put the results in here if appropriate -- managing that is somewhat out of my area of expertise...

As for the Tapatalk option - I love the idea, but don't have the resources/time to look into what it would take to make it happen. Maybe someday... or I'm open to folks that know more about how to do it offering to help.

I could look into it, I know someone who set it up for another forum.

Thanks -- the biggest problem, I think, is going to be that the SMF forum software is way behind the current release and probably doesn't support it, so any big change is going to require an upgrade to the software, which is a not inconsequential task at this point, given all the changes Charles made when he first put it in place...