Sources

Sources are divided into the groups:

Novel
Interface to be implemented by primary novel content scrapers
MetaData
Interface to be implemented by supplementary metadata scrapers

Novel source interface

class novelsave_sources.Source(*args, **kwargs)[source]

Bases: novelsave_sources.sources.crawler.Crawler

Novel source interface

All novel sources must implement this interface

name

Alternative name for the source, otherwise use the class name Source.__name__ magic attribute.

For example:

name = getattr(Source, 'name', Source.__name__)

Type: Optional[str]

login_viable

Specifies if the source has login functionality implemented.

Type: bool

search_viable

Specifies if the source has the ability to search for novels implemented.

Type: bool

__init__(*args, **kwargs)[source]

When initializing the source,

The source is checked for cookie domains, if there are no cookie domains they are built using the base_urls.

abstract chapter(chapter: novelsave_sources.models.chapter.Chapter)[source]

Download and parse chapter content

The typical implementation of this method retrieves the chapters reading content and updates the paragraph attribute of the provided chapter. It does not return any result.

In rare instances, other attributes of the Chapter are also updated like title.

Parameters: chapter (Chapter) – Chapter object with atleast the url attribute option filled.

login(email: str, password: str)[source]

Login to the source and assign the required cookies

Even though unlike novel and chapter, login is not marked abstract it does not have an implementation. By default, it throws an UnavailableException.

You may specify whether login is implemented using login_viable.

Parameters

email (str) – Email or username credentials
password (str) – password credentials

abstract novel(url: str) → novelsave_sources.models.novel.Novel[source]

Download and parse novel information

The typical implementation of this method is very straight forward. They download and parse the profile page into a novel object. Usually the table of contents would be a part of this. However, In the other instances, additional downloads may be required.

Parameters: url (str) – The url pointing towards the main profile page
Returns: Novel object that contains the parsed data
Return type: Novel

search(keyword: str, *args, **kwargs) → List[novelsave_sources.models.novel.Novel][source]

Search for a novel on the source

Even though unlike novel and chapter, search is not marked abstract it does not have an implementation. By default, it throws an UnavailableException.

You may specify whether search is implemented using search_viable.

Parameters: keyword (str) – The query text to be used in the search. Usually part of title.
Returns: The resulting novels from the search
Return type: List[Novel]

MetaData source interface

class novelsave_sources.MetaSource(*args, **kwargs)[source]

Bases: novelsave_sources.sources.crawler.Crawler

MetaData source interface

All metadata sources must implement this interface.

abstract retrieve(url: str) → List[novelsave_sources.models.metadata.Metadata][source]

Retrieves metadata from url

An implementation might retrieve the metadata by requesting from an api endpoint or from scraping a website.

Parameters: url (str) – Url pointing to the metadata
Returns: List of metadata retrieved for the page.
Return type: List[Metadata]