SEO: Robots.txt

Hi,

Let's add following code at robots.txt to make more SEO compatible.

User-agent: *
Disallow: /fileadmin/_temp_/
Disallow: /typo3/
Disallow: /typo3_src/
Disallow: /typo3conf/
Disallow: /clear.gif
Allow: /typo3/sysext/frontend/Resources/Public/*
Sitemap: https://typo3.org/sitemap.xml?tx_csseo_view=pages

Cheers, Sanjay