Interface IScraper

A IScraper specifies a type of ISeed and some restrictions on the seed, and uses the value of the seed to return a resultant tree of new seeds.

Inherited Members

IPlugin.Name

IPlugin.Author

IPlugin.Description

IPlugin.Version

System.IDisposable.Dispose()

Namespace: Snowflake.Scraping.Extensibility

Assembly: Snowflake.Framework.Primitives.dll

Syntax

public interface IScraper : IPlugin, IDisposable

Properties

AttachPoint

Gets the seed node where the results of this scraper will attach to the scrape context tree.

Declaration

AttachTarget AttachPoint { get; }

Property Value

Type	Description
AttachTarget

Directives

Gets the list of directives or restrictions that must be fulfilled if this IScraper is to be run.

Declaration

IEnumerable<IScraperDirective> Directives { get; }

Property Value

Type	Description
System.Collections.Generic.IEnumerable<IScraperDirective>

TargetType

Gets the type of seed this IScraper will examine to produce new seeds.

Declaration

string TargetType { get; }

Property Value

Type	Description
System.String

Methods

ScrapeAsync(ISeed, ILookup<String, SeedContent>, ILookup<String, SeedContent>, ILookup<String, SeedContent>, CancellationToken)

Once all Directives have been fulfilled, returns a new tree of seeds that are attached to the specified attach point on the job ISeedRootContext.

If this method throws, the entire resulting enumerable will be ignored.

Declaration

IAsyncEnumerable<SeedTree> ScrapeAsync(ISeed target, ILookup<string, SeedContent> rootSeeds, ILookup<string, SeedContent> childSeeds, ILookup<string, SeedContent> siblingSeeds, CancellationToken cancellationToken = default(CancellationToken))

Parameters

Type	Name	Description
ISeed	target	An instance of the seed with the specified TargetType.
System.Linq.ILookup<System.String, SeedContent>	rootSeeds	Any seeds, keyed on their type, that must exist as children of the root as specified in Directives.
System.Linq.ILookup<System.String, SeedContent>	childSeeds	Any seeds, keyed on their type, that must exist as children of the target seed as specified in Directives.
System.Linq.ILookup<System.String, SeedContent>	siblingSeeds	Any seeds, keyed on their type, that must exist as siblings of the target seed as specified in Directives
System.Threading.CancellationToken	cancellationToken	The cancellation token passed to this scraper if it is to be cancelled.

Returns

Type	Description
System.Collections.Generic.IAsyncEnumerable<SeedTree>	A tree of seeds based on information available in the given target.