Ensure that samples aren't loaded when they're hit by a crawler

[MatthewHawkins]

No description provided.

[StephanWald]

To be more specific: when the documentation is hit by Google or Bing etc. crawler we don't want to load the samples inside the iframe in order to keep server load low.

[hyyan]

https://developers.google.com/search/docs/crawling-indexing/robots/intro

User-agent: Googlebot
Disallow: /webforj*

[bbrennanbasis]

@hyyan, from the same page that you linked:

If other pages point to your page with descriptive text, Google could still index the URL without visiting the page. If you want to block your page from search results, use another method such as password protection or noindex.

Would it be more effective to add the following annotation to the Application class?

@AppMeta(name = "robots", content = "noindex")