DiscoverLinkedURLs Method (crawlUriList, pathMatchesToBeIgnored, pathMatchesToBeIncluded)

Keyoti SearchUnit v6

Crawls all URLs in urlList, and returns list of all newly found URLs (whether they are known to the document index or not, and not including urls in urlList)

Declaration Syntax

C#	Visual Basic

public ArrayList DiscoverLinkedURLs(
	ArrayList crawlUriList,
	ArrayList pathMatchesToBeIgnored,
	ArrayList pathMatchesToBeIncluded
)

Public Function DiscoverLinkedURLs ( 
	crawlUriList As ArrayList,
	pathMatchesToBeIgnored As ArrayList,
	pathMatchesToBeIncluded As ArrayList
) As ArrayList

Parameters

crawlUriList (ArrayList): List of string objects specifying Uris to be crawled.

pathMatchesToBeIgnored (ArrayList): List of strings which specify path matches to be ignored, can be null.

pathMatchesToBeIncluded (ArrayList): List of strings which specify path matches to be included, can be null.

Return Value

ArrayList
ArrayList of string objects with new URLs found through crawling

Remarks

Unlike Crawl, this method doesn't add new URLs to the 'database'. Crawl uses this method, overriding this method will affect Crawl.

Assembly: Keyoti4.SearchEngine.Core (Module: Keyoti4.SearchEngine.Core.dll) Version: 2022.8.22.610