How it works? 

Our software continuously crawls eCommerce shop shelves, looking through the customer lens, to capture anything a shopper can potentially see. How it works? We explain that below in 4 steps.

eCommerce data crawler

Trusted by

lg-logo-png-lg-png-5000_2187 logitech-logo 5000 x 1520 Logo Signify Logo garmin 1 gsk
Ahold logo logo_trust_corporate logo Keter logo Denon logo Tiger


Step 1: Web crawlers collect information

Data crawling tools can be used for many purposes. Our software crawls product detail pages of worldwide webshops and marketplaces daily. Why? Because it takes hours of time to manually check hundreds or thousands of product pages online and by gathering this info continuously, insights and opportunities for improvement can be uncovered for e-commerce teams.

A Product Detail Page (PDP) is a web page on an eCommerce site that presents a specific product. On a PDP we usually find a product title, product images and videos, a product description, price, specifications and reviews.


Step 2: Metrics are calculated

SiteLucent daily monitors millions of product pages all over the web. We scrape and organize all data (anything a shopper can potentially see) and calculate over 70 different metrics such as:

icons-calculate metrics

Step 3: Dashboard visualisation

Our software makes the eCommerce data that is being crawled daily, transparent and applicable through visualizations, calculations and reports. This is what you see  in your dashboards and widgets. 

Widgets are small applications that visualize your data and thus make it easy understandable and applicable. SiteLucent currently uses 6 different visualization types: KPI, scatter plot, line chart, bar chart, data table and product overview.

Besides visualization type, a widget can show you different metrics such as content criteria, availability and search rank (as we explained in step 2).

Also the dimension, aggregation, time range and, in case of a data table, the table fields can change per widget.


Step 4: Improve the Customer Experience

A precondition for staying ahead of the competition is to offer the best possible presentation and visibility of products on all online shelves. Touch-point monitoring and analysis uncover powerful insights for brands, as well as opportunities to improve customers' experience on all sales channels.

Do you want to see it in action?

Schedule Live Demo Get started for free

Customer experience




Frequently Asked Questions


How are products and corresponding eCommerce product pages identified, so that I see the correct data in my dashboards?

After you create an account, you have to walk through a product selection . Here you can choose the web shops, brands and product categories you wish to start monitoring. 

To match products to corresponding product pages online, we use Unique Product Identifiers such as EAN, MPN or GTIN codes. In this article you can read more about the product matching process. 

How do I know if my products are available in SiteLucent's database?

We daily monitor over 1.000.000 product detail pages, mainly in the categories consumer electronics, Do-It-Yourself (DIY) and Fast Moving Consumer Goods (FMCG). More product categories, brands and retailers (worldwide) are added weekly.

In case you cannot select the products that are important to you, there is an option to request product categories, brands and retailers in the product selection process when you create a free account.

Do you want to be sure if SiteLucent covers your products, before signing up? Scan your brand by filling out the form below.

Can I customize dashboards and widgets?

With a free account you have access to preset dashboards and widgets. It is possible to download widget data as CSV file. 

With a Premium account, you can create your own dashboards by selecting widgets that are available in the SiteLucent 'library'.

With a Premium Plus account you can create own widgets, together with one of our specialists who will make sure your ideas and wishes become reality. 

How is the completeness of content measured?

To measure the completeness of a products' content, we use the Digital Content Completeness score (DCC score). The DCC score is a weighted average of 3 components. The 3 components' criteria can be adjusted to meet your companies specific requirements and wishes. The default score criteria are as follows:

Minimal Content
  • EAN/UPC or MPN code present
  • Selling price present
  • At least 1 customer review
  • Star rating of at least 3.5
  • At least 1 product image present
  • Specs table present
  • Product description present
  • Brand name present in the product title
Basic Content 
  • At least 3 images present
  • Product title contains 7+ words
  • Description contains 30+ words
  • At least 20 customer reviews
  • Rating of at least 4.0
Extended Content
  • Rich (vendor provided) content present
  • At least 1 product video
  • At least 10 customer reviews
  • Star rating of 4.0 or higher
What metrics do i have access to when using a free account?

SiteLucent daily monitors over 70 different metrics on product detail pages all over the web. With a free account you have access to default dashboards with default widgets. Within these widgets, the following metrics are available:

  • Product listing: Is a product covered (found online) Y/N?
  • Availability: Is the product in stock Y/N?
  • Days out of stock.
  • Stock text e.g. "Attention: almost sold out!" and delivery text e.g.: "order before 23.59, delivered the next day".
  • Buy Box Coverage:  The number of products for which sellers "own" the buy box. In case of other sellers: Who are other sellers?
  • Selling price and an optional original price.
  • Digital Content Completeness score (DCC score) which includes many metrics that you can read more about here.
  • Review count and review score (star rating).
  • Search rank and category rank. Read more about the search dashboard here.

If you want to learn more about the widget types we use in SiteLucent you can read it here.


What can you monitor?


Product listings

eCommerce assortment monitoring

 Selling prices

eCommerce price monitoring


eCommerce availability monitoring





Search ranking

eCommerce site search results


Product content

optimize eCommerce product pages


Product reviews

Track eCommerce customer reviews