CSS Selector Exclusion

Remove unwanted HTML elements from scraped pages to improve AI response quality. Headers, footers, navigation, and ads don't belong in your knowledge base.

How it works

CSS selectors target specific HTML elements for exclusion. When you scrape a URL, matching elements are filtered out before content enters your knowledge base.

Example scenario:
You scrape a help center article. Without exclusions, your AI ingests the sidebar menu, cookie banner, and footer links. With .sidebar and footer selectors, only the article content is stored.

Setting global defaults

Global defaults apply to every URL you add to your knowledge base.[1]

To configure:

Go to Advanced > Web Scraping
Add selectors in "Default CSS selectors to exclude"
Separate multiple selectors with commas or spaces

Start with common noise elements: header, footer, .sidebar, .navigation.

Overriding for specific URLs

Some pages need different rules. A product page might have a useful sidebar while your docs pages don't.

To customize:

Click "Add Documents" > "Public URL(s)"
Expand "Options"
Edit the CSS selectors field
Add your URLs

These overrides only affect the current import. Your global defaults remain unchanged.

Persistence

Selector configurations persist when content refreshes. Whether you manually sync or rely on automatic updates, your exclusion rules stay active.

Verify which selectors were applied for one document

To access:

Navigate to Knowledge hub
Click the three-dot menu on any document
Select "Details"

The modal will display the excluded CSS Selectors.
View the exact selectors used during scraping. Copy them to reuse on similar documents or troubleshoot extraction issues.

Common selectors

Start with these patterns and refine based on your target sites:

header, footer — Site-wide navigation
.sidebar, #sidebar — Auxiliary content panels
.nav, .navigation, .menu — Navigation elements
.cookie-banner, .modal, .popup — Overlays
.ad, .advertisement, .promo — Marketing content
.social-share, .comments — Engagement widgets

Finding selectors:
Right-click an element in your browser > Inspect > Copy the class name or ID.