Quickly test robots.txt rules against example URLs
Robots.txt rules:
If no User-agent set in rules, we will auto assume `User-agent: *`
`User-agent: *`
Example URLs:
User-agent (optional):
Parse Rules