Geta SEO Sitemaps for hidden pages (behind login)

Victoria Vardina

Vote:

Hello

We are using an external tool called Siteimprove, who are using sitemaps to crawl the webpage for spelling errors and etc.
Recently we have released a self-service environment, where there are pages behind account authorization, that we wish to crawl. In order to do that, we have to add them to a sitemap.

Using Geta Seo Sitemaps (version 4.0.0), has anyone any experience with adding pages behind a login to a sitemap, which aren't public for search engines?

#292905

Dec 09, 2022 9:52

valdis

Vote:

If you can override this (https://github.com/Geta/SEO.Sitemaps/blob/eebf3007c5004a7351ca5ce59eb6de95c9ff0252/src/Geta.SEO.Sitemaps/Utils/ContentFilter.cs#L18) implementation, then you can control filter yourself and decide whether you want to include or exclude protected pages.

#294988

Jan 19, 2023 17:42

Tomas Hensrud Gulla

Vote:

Even if you add the URLs to sitemap.xml, how are Google supposed to crawl the pages if they are behind account authorization?

#295004

Jan 19, 2023 20:57

valdis - Jan 19, 2023 21:29

Maybe Siteimprove is capable to "login" and check page state behind the "firewall"?! Don't know the tool..

Tomas Hensrud Gulla - Jan 19, 2023 21:33

Valis, you are – as always – correct. The question was about Siteimprove, but I was thinking Google-only.
Looks like Siteimprove can log in, if configured correctly.

JessWade

Vote:

Yes, I've had experience with this. Using Geta SEO Sitemaps of one of our personal project, you can include pages behind a login in your sitemap by creating a separate sitemap for authenticated pages. You would need to ensure this sitemap is only accessible to Siteimprove by providing them with the necessary authentication credentials. Be cautious, though, to avoid exposing sensitive information to public search engines.

Hope this helps!

#326049

Edited, Jul 26, 2024 11:45