Skip to main content
All CollectionsAdvanced GuidesWebsite management
How to exclude pages or PDFs from a website
How to exclude pages or PDFs from a website
Daniel Towers avatar
Written by Daniel Towers
Updated over a week ago

Once you’ve added a website, you can choose to exclude certain pages or PDFs from being tested.

For example, if you tested example.com, you might want to exclude pages inside part of that website, like pages that begin with example.com/archive.

Add a new denied rule

  1. Open the website you want to update.

  2. Select Settings in the top right corner. You will need to have the required account permissions to see this.

  3. Inside the Denied URLs box, enter any web addresses you want to exclude.

Any URLs you enter are excluded from any URL that matches the start of it. So if you exclude say https://example.com/archive, you would also have excluded these:

https://example.com/archive https://example.com/archive/another https://example.com/archive/and/another/example

Note that if you are using the 'URL Starting with' rule that the URLs must be fully qualified, that is to say they must start with http:// or https://. Also note that if you exclude one of these that the other will not be excluded, for example if you exclude https://example.com/archive that will not exclude http://example.com/archive (the same address but with 'http' instead of 'https').

Rule types

Silktide supports a number of page selection rule types, including:

  • URL Equals

  • URL Starting with

  • URL Contains

  • URL Shorter than

  • URL Longer than

  • Regular expressions

Advanced rules

For sections, you can also make use of the advanced rule builder to select pages that contain a particular HTML element, language, page title and so on.

For advanced rules, you can make use of any of the following rules on their own or in combination with other rules:

  • Image

  • Language

  • Link

  • Page title

  • Page type

  • Resource

  • Text

  • URL

  • XPath selector

Excluding pages that contain query parameters

Sometimes you may have tested a selection of duplicate pages or nearly-duplicate pages, for example a website search feature might include numerous category options. These pages usually look something like this: example.com/search?category=news&page=3.

How can query parameters create duplication?

Taking search categories as an example, our crawler might attempt to test each page in the search results multiple times, repeating each time the category query parameter changes.

If you have lots of repeated pages, excluding a query parameter can be useful to reduce the number of pages you are testing for a website and to save on your page allowance.

Contact Silktide support

To exclude a query parameter or selection of parameters, please contact the Silktide support team.

We can also provide further advice on the types of query parameters to select for exclusion. For example with example.com/search?category=news&page=3, we might suggest excluding the 'category' query parameter, while keeping the 'page' query parameter.

Our tooling enables us to exclude the 'category' query parameter, then re-test the remainder of the URL. In our example, the remaining URL to be tested after exclusions would be example.com/search?page=3.

Did this answer your question?