Quick Login
A webmaster forum specifically catering for Australian site owners. We discuss site development, marketing and management issues.
Please welcome our newest member, yindsoft!
Spice up your web site with the ultimate community message board solution!
Fake Id'sGet fake Id's made right now!
vBSEOSearch Engine Optimization for your VBulletin Forum.
AdminFusion
»
Robots.txt
09-22-2006, 03:03 PM
| #1 |
| | |||||
Title: Forum Junkie
Join Date: Feb 2006 Posts: 3,633 Location: Holmfirth, England    
| Robots.txt Where do i put Robots.txt, is it in /public_html or the location of my forum, which would be, /public_html/vb/forum ? This is so Google, MSN, BoardTracker and other bots will behave, and correctly index the correct things.... Oh, and how accurate does it have to be? I mean, how many errors can there be before it will cease to function, is it like SQL where the slightest wrong character will cause it to crash, and complain on the stupidest things, or is it not as bad, Because i am no good at writing specific things correctly.
__________________  | ||||
| |
|
09-22-2006, 03:07 PM
| #2 |
| | |||||
Title: Apprentice
Join Date: May 2006 Posts: 348 | Put it in your /public_html/ or www And do not make any mistakes !
__________________ Web Directory - Submit your forums! 2007 Timeline - What happened in 2007 ? Weblog - free css templates, ... | ||||
| |
|
09-22-2006, 03:09 PM
| #3 |
| | ||||||
|
Title: Forum Junkie
Join Date: Feb 2006 Posts: 3,633 Location: Holmfirth, England
| Quote:
Well, i will live without one then, Thanks anyway 
__________________ | |||||
| |
|
09-22-2006, 03:17 PM
| #4 |
| | |||||
|
Title: Forum Junkie
Join Date: Feb 2006 Posts: 3,633 Location: Holmfirth, England
| OK, Currently, this is what my Robots.txt looks like, empty except. Code: User-agent: * Disallow: /vb/forum/member.php
__________________ | ||||
| |
|
09-22-2006, 03:18 PM
| #5 |
| | |||||
|
Title: Apprentice
Join Date: May 2006 Posts: 348 | it is good  .... I would leave it without robots.txt Why do you want it anyway ?
__________________ Web Directory - Submit your forums! 2007 Timeline - What happened in 2007 ? Weblog - free css templates, ... | ||||
| |
|
09-22-2006, 03:25 PM
| #6 |
| | |||||
|
Title: Forum Junkie
Join Date: Feb 2006 Posts: 3,633 Location: Holmfirth, England
| Im trying to stop bots from reading certain areas, currently its only set to stop viewing of profiles. But im now trying to get it so it will control unruly BoardTracker Bots from viewing the wrong forums.
__________________ | ||||
| |
|
09-22-2006, 04:09 PM
| #7 |
| | |||||||
 Title: Groupie
Join Date: May 2006 Posts: 47 | We try to make it so that errors in the robots.txt will not "break" things down. However, try to keep the file as correct as you can. Looking at your robots.txt, I see one thing that is not needed... you have something like Quote:
You can do someting like Quote:
| ||||||
| |
|
09-22-2006, 04:43 PM
| #8 |
| | |||||
|
Title: Forum Junkie
Join Date: Feb 2006 Posts: 3,633 Location: Holmfirth, England
| Oh right, I will re edit the robots.txt file now Thanks EDIT: Done 
__________________ | ||||
| |
|
10-19-2006, 12:06 PM
| #9 |
| | |||||
| Title: Lurker
Join Date: Oct 2006 Posts: 9 | You need to go to https://www.google.com/webmasters/ when you have uploaded it then you can let google test it for you and tell you if it correct or not. ITS VERY IMPORTANT | ||||
| |
 |
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
| Thread Tools | |
| Display Modes | |
| |
Similar Threads | ||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| IPB duplicate content and robots.txt | Einari | Search Engine Optimization | 0 | 10-14-2005 08:19 AM |
Linear Mode
