Aug 03, 2011 Automate Robots.txt File Script
Have a lot of directories that you'd like to include in your robots.txt file but don't feel like writing all of them?
Run the script below in your root directory and it will write a robots.txt file with everything in your directory. Change the Disallow: to Allow: if you want the opposite effect.
This script uses the 'glob' function in php which searches for all the path names matching a pattern. The pattern I'm using in this script is '*', which is all. Then I use the optional flag for the function 'GLOB_ONLYDIR' which only looks for directories. So the output is essentially a search and return of names for all directories.
Then I loop through all the returned names using 'foreach' and write the names to the robots.txt file using 'file_put_contents'.
$search = glob('*', GLOB_ONLYDIR);
$hold = '';
foreach($search as $folder):
$hold .= "Disallow: /".$folder."/\n";
endforeach;
$file = 'robots.txt';
$data = "User-agent: *\n";
$data .= $hold;
file_put_contents($file, $data);
The output of the 'robots.txt' will look something like this but with your directories.
Ta-da! Simple and straight to the point. If you have questions or suggestions just comment below.
Nice work mate! Gonna give this a test soon :)
1 comment real quick. Why are you resetting the value of $hold to '' everytime you loop through ur for each statement?
@jeremy You're right. I updated the example by putting it outside of the loop now.
No prob man. Just 1 code gorilla trying to help out a code monkey.
Interesting, very clever. I don't have many directories to disallow at the moment but I'll keep this in mind in case I ever need it
Very nice! Thanks for this :)
Nice code.